cars.orchestrator.cluster.mp_wrapper

Contains functions for wrapper disk

Module Contents

Classes

AbstractWrapper

AbstractWrapper

WrapperNone

AbstractWrapper

WrapperDisk

WrapperDisk

Functions

removing_disk_data(path)

Remove directory from disk

none_wrapper_fun(*argv, **kwargs)

Create a wrapper for functionn running it

disk_wrapper_fun(*argv, **kwargs)

Create a wrapper for function

load_args(args)

Load args from disk to memory

load_kwargs(kwargs)

Load key args from disk to memory

is_dumped_object(obj)

Check if a given object is dumped

load(path)

Load object from disk

dump_single_object(obj, path)

Dump object to disk

create_path(obj, tmp_dir, id_num)

Create path where to dump object

dump(res, tmp_dir, id_list)

Dump results to tmp_dir, according to ids

Attributes

DENSE_NAME

SPARSE_NAME

cars.orchestrator.cluster.mp_wrapper.DENSE_NAME = 'DenseDO'
cars.orchestrator.cluster.mp_wrapper.SPARSE_NAME = 'SparseDO'
class cars.orchestrator.cluster.mp_wrapper.AbstractWrapper

AbstractWrapper

abstract get_obj(obj)

Get Object

Parameters

obj – object to transform

Returns

object

abstract get_function_and_kwargs(func, kwargs, nout=1)

Get function to apply and overloaded key arguments

Parameters
  • func – function to run

  • kwargs – key arguments of func

  • nout – number of outputs

Returns

function to apply, overloaded key arguments

abstract cleanup()

Cleanup tmp_dir

abstract cleanup_future_res(future_res)

Cleanup future result

Parameters

future_res – future result to clean

class cars.orchestrator.cluster.mp_wrapper.WrapperNone(tmp_dir)

Bases: AbstractWrapper

AbstractWrapper

get_obj(obj)

Get Object

Parameters

obj – object to transform

Returns

object

get_function_and_kwargs(func, kwargs, nout=1)

Get function to apply and overloaded key arguments

Parameters
  • func – function to run

  • kwargs – key arguments of func

  • nout – number of outputs

Returns

function to apply, overloaded key arguments

cleanup()

Cleanup tmp_dir

cleanup_future_res(future_res)

Cleanup future result

Parameters

future_res – future result to clean

class cars.orchestrator.cluster.mp_wrapper.WrapperDisk(tmp_dir)

Bases: AbstractWrapper

WrapperDisk

cleanup()

Cleanup tmp_dir

cleanup_future_res(future_res)

Cleanup future result

Parameters

future_res – future result to clean

get_function_and_kwargs(func, kwargs, nout=1)

Get function to apply and overloaded key arguments

Parameters
  • func – function to run

  • kwargs – key arguments of func

  • nout – number of outputs

Returns

function to apply, overloaded key arguments

get_obj(obj)

Get Object

Parameters

obj – object to transform

Returns

object

cars.orchestrator.cluster.mp_wrapper.removing_disk_data(path)

Remove directory from disk

Parameters

path – path to delete

cars.orchestrator.cluster.mp_wrapper.none_wrapper_fun(*argv, **kwargs)

Create a wrapper for functionn running it

Parameters
  • argv – args of func

  • kwargs – kwargs of func

Returns

path to results

cars.orchestrator.cluster.mp_wrapper.disk_wrapper_fun(*argv, **kwargs)

Create a wrapper for function

Parameters
  • argv – args of func

  • kwargs – kwargs of func

Returns

path to results

cars.orchestrator.cluster.mp_wrapper.load_args(args)

Load args from disk to memory

Parameters

argv – args of func

Returns

new args

cars.orchestrator.cluster.mp_wrapper.load_kwargs(kwargs)

Load key args from disk to memory

Parameters

kwargs – keyargs of func

Returns

new kwargs

cars.orchestrator.cluster.mp_wrapper.is_dumped_object(obj)

Check if a given object is dumped

Parameters

obj – object

Returns

is dumped

Return type

bool

cars.orchestrator.cluster.mp_wrapper.load(path)

Load object from disk

Parameters

path (str) – path

Returns

object

cars.orchestrator.cluster.mp_wrapper.dump_single_object(obj, path)

Dump object to disk

Parameters

path (str) – path

cars.orchestrator.cluster.mp_wrapper.create_path(obj, tmp_dir, id_num)

Create path where to dump object

Parameters
  • tmp_dir – tmp_dir

  • id_num – id of object

Returns

path

cars.orchestrator.cluster.mp_wrapper.dump(res, tmp_dir, id_list)

Dump results to tmp_dir, according to ids

Parameters
  • res – objects to dump

  • tmp_dir – tmp_dir

  • id_list – list of ids of objects

Returns

path