cars.orchestrator.cluster.abstract_cluster

Contains abstract function for Abstract Cluster

Module Contents

Classes

AbstractCluster

AbstractCluster
class cars.orchestrator.cluster.abstract_cluster.AbstractCluster(conf_cluster, out_dir, launch_worker=True, data_to_propagate=None)[source]

AbstractCluster

available_modes: Dict
profiling: cars.conf.input_parameters.ConfigType
out_dir: str
classmethod register_subclass(*short_names: str)[source]

Allows to register the subclass with its short name :param short_names: the subclasses to be registered :type short_names: string

abstract get_delayed_type()[source]

Get delayed type

abstract cleanup(**kwargs)[source]

Cleanup cluster

abstract check_conf(conf)[source]

Check configuration

Parameters

conf (dict) – configuration to check

Returns

overloaded configuration

Return type

dict

get_conf()[source]

Get overriden configuration

Returns

overriden configuration

create_task(func, nout=1)[source]

Create task

Parameters

  • func – function

  • nout – number of outputs

abstract create_task_wrapped(func, nout=1)[source]

Create task

Parameters

  • func – function

  • nout – number of outputs

abstract start_tasks(task_list)[source]

Start all tasks

Parameters

task_list – task list

abstract scatter(data, broadcast=True)[source]

Distribute data through workers

Parameters

data – task data

abstract future_iterator(future_list, timeout=None)[source]

Iterator, iterating on computed futures

Parameters

  • future_list – future_list list

  • timeout – time to wait for next job