cars.orchestrator.orchestrator
this module contains the orchestrator class
Module Contents
Classes
Orchestrator |
Functions
|
Flatten list of CarsDatasets to list of delayed |
|
Update saving infos dict with row and col arguments |
- class cars.orchestrator.orchestrator.Orchestrator(orchestrator_conf=None, out_dir=None, launch_worker=True, out_json_path=None)
Orchestrator
- add_to_clean(tmp_dir)
- add_to_save_lists(file_name, tag, cars_ds, dtype='float32', nodata=0, cars_ds_name=None)
Save file to list in order to be saved later
- Parameters
file_name – file name
tag – tag
cars_ds – cars dataset to register
cars_ds_name – name corresponding to CarsDataset, for information during logging
- add_to_replace_lists(cars_ds, cars_ds_name=None)
Add CarsDataset to replacing Registry
- Parameters
cars_ds (CarsDataset) – CarsDataset to replace
cars_ds_name – name corresponding to CarsDataset, for information during logging
- save_out_json()
Check out_json and save it to file
- update_out_info(new_dict)
Update self.out_json with new dict
- Parameters
new_dict (dict) – dict to merge
- get_saving_infos(cars_ds_list)
Get saving infos of given cars datasets
- Parameters
cars_ds_list (list[CarsDataset]) – list of cars datasets
:return : list of saving infos :rtype: list[dict]
- compute_futures()
Compute all futures from regitries
- breakpoint()
Breakpoint : compute all delayed, save and replace data in CarsDatasets
- __enter__()
Function run on enter
- __exit__(exc_type, exc_value, traceback)
Function run on exit.
Compute cluster tasks, save futures to be saved, and cleanup cluster and files
- cars.orchestrator.orchestrator.flatten_object(cars_ds_list)
Flatten list of CarsDatasets to list of delayed
- Parameters
cars_ds_list (list[CarsDataset]) – list of cars datasets flatten
- Returns
list of delayed
- Return type
list[Delayed]
- cars.orchestrator.orchestrator.update_saving_infos(saving_info_left, row=None, col=None)
Update saving infos dict with row and col arguments
- Parameters
saving_info_left (dict) – saving infos
row (int) – row
col (int) – col
- Returns
updated saving infos dict
- Return type
dict