cars.orchestrator.cluster.mp_wrapper

Contains functions for wrapper disk

Module Contents

Classes

AbstractWrapper	AbstractWrapper
WrapperNone	AbstractWrapper
WrapperDisk	WrapperDisk

Functions

removing_disk_data(path)	Remove directory from disk
none_wrapper_fun(*argv, **kwargs)	Create a wrapper for functionn running it
disk_wrapper_fun(*argv, **kwargs)	Create a wrapper for function
load_args(args)	Load args from disk to memory
load_kwargs(kwargs)	Load key args from disk to memory
is_dumped_object(obj)	Check if a given object is dumped
load(path)	Load object from disk
dump_single_object(obj, path)	Dump object to disk
create_path(obj, tmp_dir, id_num)	Create path where to dump object
dump(res, tmp_dir, id_list)	Dump results to tmp_dir, according to ids

Attributes

DENSE_NAME
SPARSE_NAME

cars.orchestrator.cluster.mp_wrapper.DENSE_NAME = DenseDO

cars.orchestrator.cluster.mp_wrapper.SPARSE_NAME = SparseDO

class cars.orchestrator.cluster.mp_wrapper.AbstractWrapper

AbstractWrapper

abstract get_obj(obj)

Get Object

Parameters

obj – object to transform

Returns

object

abstract get_function_and_kwargs(func, kwargs, nout=1)

Get function to apply and overloaded key arguments

Parameters

• func – function to run

• kwargs – key arguments of func

• nout – number of outputs

Returns

function to apply, overloaded key arguments

abstract cleanup()

Cleanup tmp_dir

abstract cleanup_future_res(future_res)

Cleanup future result

Parameters

future_res – future result to clean

class cars.orchestrator.cluster.mp_wrapper.WrapperNone(tmp_dir)

Bases: AbstractWrapper

AbstractWrapper

get_obj(obj)

Get Object

Parameters

obj – object to transform

Returns

object

get_function_and_kwargs(func, kwargs, nout=1)

Get function to apply and overloaded key arguments

Parameters

• func – function to run

• kwargs – key arguments of func

• nout – number of outputs

Returns

function to apply, overloaded key arguments

cleanup()

Cleanup tmp_dir

cleanup_future_res(future_res)

Cleanup future result

Parameters

future_res – future result to clean

class cars.orchestrator.cluster.mp_wrapper.WrapperDisk(tmp_dir)

Bases: AbstractWrapper

WrapperDisk

cleanup()

Cleanup tmp_dir

cleanup_future_res(future_res)

Cleanup future result

Parameters

future_res – future result to clean

get_function_and_kwargs(func, kwargs, nout=1)

Get function to apply and overloaded key arguments

Parameters

• func – function to run

• kwargs – key arguments of func

• nout – number of outputs

Returns

function to apply, overloaded key arguments

get_obj(obj)

Get Object

Parameters

obj – object to transform

Returns

object

cars.orchestrator.cluster.mp_wrapper.removing_disk_data(path)

Remove directory from disk

Parameters

path – path to delete

cars.orchestrator.cluster.mp_wrapper.none_wrapper_fun(*argv, **kwargs)

Create a wrapper for functionn running it

Parameters

• argv – args of func

• kwargs – kwargs of func

Returns

path to results

cars.orchestrator.cluster.mp_wrapper.disk_wrapper_fun(*argv, **kwargs)

Create a wrapper for function

Parameters

• argv – args of func

• kwargs – kwargs of func

Returns

path to results

cars.orchestrator.cluster.mp_wrapper.load_args(args)

Load args from disk to memory

Parameters

argv – args of func

Returns

new args

cars.orchestrator.cluster.mp_wrapper.load_kwargs(kwargs)

Load key args from disk to memory

Parameters

kwargs – keyargs of func

Returns

new kwargs

cars.orchestrator.cluster.mp_wrapper.is_dumped_object(obj)

Check if a given object is dumped

Parameters

obj – object

Returns

is dumped

Return type

bool

cars.orchestrator.cluster.mp_wrapper.load(path)

Load object from disk

Parameters

path (str) – path

Returns

object

cars.orchestrator.cluster.mp_wrapper.dump_single_object(obj, path)

Dump object to disk

Parameters

path (str) – path

cars.orchestrator.cluster.mp_wrapper.create_path(obj, tmp_dir, id_num)

Create path where to dump object

Parameters

• tmp_dir – tmp_dir

• id_num – id of object

Returns

path

cars.orchestrator.cluster.mp_wrapper.dump(res, tmp_dir, id_list)

Dump results to tmp_dir, according to ids

Parameters

• res – objects to dump

• tmp_dir – tmp_dir

• id_list – list of ids of objects

Returns

path