cars.orchestrator.cluster.dask_jobqueue_utils

Contains functions for dask jobqueue cluster (PBS, SLURM)

Module Contents

Functions

init_cluster_variables(nb_workers, walltime, out_dir, ...)

Initialize global cluster variables

get_dashboard_link(cluster)

This function returns the dashboard address.

stop_cluster(cluster, client)

This function stops a dask cluster.

cars.orchestrator.cluster.dask_jobqueue_utils.init_cluster_variables(nb_workers, walltime, out_dir, activate_dashboard, python, core_memory, cluster_name)

Initialize global cluster variables :param nb_workers: number of workers :param walltime: workers walltime :param out_dir: ouput result directory :param activate_dashboard: option to activate dashboard mode :param python: target python used by workers (retrun system python if None) :param core_memory: cluster node memory (Mo) :param cluster_name: PBS or SLURM :return: all cluster parameters (python, nb_workers_per_job, memory, nb_cpus, stagger, lifetime_with_margin, scheduler_options, envs, log_directory, local_directory)

This function returns the dashboard address.

Parameters

cluster (dask_jobqueue.PBSCluster) – Dask cluster

Returns

Link to the dashboard

Return type

string

cars.orchestrator.cluster.dask_jobqueue_utils.stop_cluster(cluster, client)

This function stops a dask cluster.

Parameters
  • cluster (dask_jobqueue.PBSCluster) – Dask cluster

  • client (dask.distributed.Client) – Dask client