cars.orchestrator.orchestrator

this module contains the orchestrator class

Module Contents

Classes

Orchestrator

Orchestrator

Functions

flatten_object(cars_ds_list)

Flatten list of CarsDatasets to list of delayed

update_saving_infos(saving_info_left[, row, col])

Update saving infos dict with row and col arguments
class cars.orchestrator.orchestrator.Orchestrator(orchestrator_conf=None, out_dir=None, launch_worker=True, out_json_path=None)

Orchestrator

add_to_clean(tmp_dir)
get_conf()

Get orchestrator conf

Returns

orchestrator conf

add_to_save_lists(file_name, tag, cars_ds, dtype='float32', nodata=0, cars_ds_name=None)

Save file to list in order to be saved later

Parameters

  • file_name – file name

  • tag – tag

  • cars_ds – cars dataset to register

  • cars_ds_name – name corresponding to CarsDataset, for information during logging

add_to_replace_lists(cars_ds, cars_ds_name=None)

Add CarsDataset to replacing Registry

Parameters

  • cars_ds (CarsDataset) – CarsDataset to replace

  • cars_ds_name – name corresponding to CarsDataset, for information during logging

save_out_json()

Check out_json and save it to file

update_out_info(new_dict)

Update self.out_json with new dict

Parameters

new_dict (dict) – dict to merge

get_saving_infos(cars_ds_list)

Get saving infos of given cars datasets

Parameters

cars_ds_list (list[CarsDataset]) – list of cars datasets

:return : list of saving infos :rtype: list[dict]

compute_futures()

Compute all futures from regitries

breakpoint()

Breakpoint : compute all delayed, save and replace data in CarsDatasets

__enter__()

Function run on enter

__exit__(exc_type, exc_value, traceback)

Function run on exit.

Compute cluster tasks, save futures to be saved, and cleanup cluster and files

cars.orchestrator.orchestrator.flatten_object(cars_ds_list)

Flatten list of CarsDatasets to list of delayed

Parameters

cars_ds_list (list[CarsDataset]) – list of cars datasets flatten

Returns

list of delayed

Return type

list[Delayed]

cars.orchestrator.orchestrator.update_saving_infos(saving_info_left, row=None, col=None)

Update saving infos dict with row and col arguments

Parameters

  • saving_info_left (dict) – saving infos

  • row (int) – row

  • col (int) – col

Returns

updated saving infos dict

Return type

dict