cars.orchestrator.cluster.mp_wrapper
Contains functions for wrapper disk
Module Contents
Classes
AbstractWrapper |
|
AbstractWrapper |
|
WrapperDisk |
Functions
|
Remove directory from disk |
|
Create a wrapper for functionn running it |
|
Create a wrapper for function |
|
Load args from disk to memory |
|
Load key args from disk to memory |
|
Check if a given object is dumped |
|
Load object from disk |
|
Dump object to disk |
|
Create path where to dump object |
|
Dump results to tmp_dir, according to ids |
Attributes
- cars.orchestrator.cluster.mp_wrapper.DENSE_NAME = DenseDO
- cars.orchestrator.cluster.mp_wrapper.SPARSE_NAME = SparseDO
- class cars.orchestrator.cluster.mp_wrapper.AbstractWrapper
AbstractWrapper
- abstract get_obj(obj)
Get Object
- Parameters
obj – object to transform
- Returns
object
- abstract get_function_and_kwargs(func, kwargs, nout=1)
Get function to apply and overloaded key arguments
- Parameters
func – function to run
kwargs – key arguments of func
nout – number of outputs
- Returns
function to apply, overloaded key arguments
- abstract cleanup()
Cleanup tmp_dir
- abstract cleanup_future_res(future_res)
Cleanup future result
- Parameters
future_res – future result to clean
- class cars.orchestrator.cluster.mp_wrapper.WrapperNone(tmp_dir)
Bases:
AbstractWrapper
AbstractWrapper
- get_obj(obj)
Get Object
- Parameters
obj – object to transform
- Returns
object
- get_function_and_kwargs(func, kwargs, nout=1)
Get function to apply and overloaded key arguments
- Parameters
func – function to run
kwargs – key arguments of func
nout – number of outputs
- Returns
function to apply, overloaded key arguments
- cleanup()
Cleanup tmp_dir
- cleanup_future_res(future_res)
Cleanup future result
- Parameters
future_res – future result to clean
- class cars.orchestrator.cluster.mp_wrapper.WrapperDisk(tmp_dir)
Bases:
AbstractWrapper
WrapperDisk
- cleanup()
Cleanup tmp_dir
- cleanup_future_res(future_res)
Cleanup future result
- Parameters
future_res – future result to clean
- get_function_and_kwargs(func, kwargs, nout=1)
Get function to apply and overloaded key arguments
- Parameters
func – function to run
kwargs – key arguments of func
nout – number of outputs
- Returns
function to apply, overloaded key arguments
- get_obj(obj)
Get Object
- Parameters
obj – object to transform
- Returns
object
- cars.orchestrator.cluster.mp_wrapper.removing_disk_data(path)
Remove directory from disk
- Parameters
path – path to delete
- cars.orchestrator.cluster.mp_wrapper.none_wrapper_fun(*argv, **kwargs)
Create a wrapper for functionn running it
- Parameters
argv – args of func
kwargs – kwargs of func
- Returns
path to results
- cars.orchestrator.cluster.mp_wrapper.disk_wrapper_fun(*argv, **kwargs)
Create a wrapper for function
- Parameters
argv – args of func
kwargs – kwargs of func
- Returns
path to results
- cars.orchestrator.cluster.mp_wrapper.load_args(args)
Load args from disk to memory
- Parameters
argv – args of func
- Returns
new args
- cars.orchestrator.cluster.mp_wrapper.load_kwargs(kwargs)
Load key args from disk to memory
- Parameters
kwargs – keyargs of func
- Returns
new kwargs
- cars.orchestrator.cluster.mp_wrapper.is_dumped_object(obj)
Check if a given object is dumped
- Parameters
obj – object
- Returns
is dumped
- Return type
bool
- cars.orchestrator.cluster.mp_wrapper.load(path)
Load object from disk
- Parameters
path (str) – path
- Returns
object
- cars.orchestrator.cluster.mp_wrapper.dump_single_object(obj, path)
Dump object to disk
- Parameters
path (str) – path
- cars.orchestrator.cluster.mp_wrapper.create_path(obj, tmp_dir, id_num)
Create path where to dump object
- Parameters
tmp_dir – tmp_dir
id_num – id of object
- Returns
path
- cars.orchestrator.cluster.mp_wrapper.dump(res, tmp_dir, id_list)
Dump results to tmp_dir, according to ids
- Parameters
res – objects to dump
tmp_dir – tmp_dir
id_list – list of ids of objects
- Returns
path