Pipeline

Pipeline

Overview

A pipeline is a series of steps of CARS 3D reconstruction framework. It generates a new product (example: DSM) from another product (example: sensors) .

It is composed of: * an pipeline factory that register all pipelines (including plugins) * an abstract pipeline template * Some subclass pipelines (surface_modeling, tie_points, merging, formatting, etc)

A pipeline use a global orchestrator to manage all its applications and datasets. A series of applications are chained to process the data from input to output.

Example

Let’s take an example of tie_points pipeline to describe the main steps:

First, we can notice that tie_points derives from PipelineTemplate and is registered with the decorator:

@Pipeline.register(
    "tie_points",
)
class TiePointsPipeline(PipelineTemplate):

Then, pipeline name is defined in a subclass register, by is short_name, of tie_points pipeline.

In the init function, the pipeline is instantiated, with all the checks applied on the configuration file:

def __init__(self, conf, config_dir=None):

# Used conf
self.used_conf = {}

# Transform relative path to absolute path
if config_dir is not None:
    config_dir = os.path.abspath(config_dir)

# Check global conf
self.check_global_schema(conf)
...

In the run function, the pipeline is executed, with all its applications processed in a row:

An orchestrator is instantiated at the beginning of the pipeline, and given to each application.

def run(self, log_dir=None, disp_range_grid=None):
    """
    Run pipeline

    """
    ...
    with orchestrator.Orchestrator(
        orchestrator_conf=self.used_conf[ORCHESTRATOR],
        out_dir=self.out_dir,
        log_dir=log_dir,
        out_yaml_path=os.path.join(
            self.out_dir,
            out_cst.INFO_FILENAME,
        ),
    ) as cars_orchestrator:
        # Run applications