cars.orchestrator.cluster.local_dask_cluster

Contains functions for local dask Cluster

Classes

LocalDaskCluster

LocalDaskCluster

Functions

`start_local_cluster`(nb_workers[, timeout, ...])	Start a local cluster
`stop_local_cluster`(cluster, client)	Stop a local cluster

Module Contents

class cars.orchestrator.cluster.local_dask_cluster.LocalDaskCluster(conf_cluster, out_dir, log_dir, launch_worker=True, data_to_propagate=None)[source]

Bases: cars.orchestrator.cluster.abstract_dask_cluster.AbstractDaskCluster

LocalDaskCluster

check_conf(conf)[source]

Check configuration

Parameters:: conf (dict) – configuration to check
Returns:: overloaded configuration
Return type:: dict

start_dask_cluster()[source]: Start dask cluster

cleanup(**kwargs)[source]: Cleanup cluster

cars.orchestrator.cluster.local_dask_cluster.start_local_cluster(nb_workers, timeout=600, activate_dashboard=False)[source]

Start a local cluster

Parameters:

nb_workers (int) – Number of dask workers
timeout (int) – Connection timeout

Returns:

Local cluster and Dask client

Return type:

(dask.distributed.LocalCluster, dask.distributed.Client) tuple

cars.orchestrator.cluster.local_dask_cluster.stop_local_cluster(cluster, client)[source]

Stop a local cluster

Parameters:

cluster (dask.distributed.LocalCluster) – Local cluster
client (dask.distributed.Client) – Dask client