cars.applications.dense_match_filling.fill_disp_tools

This module is responsible for the filling disparity algorithms: thus it fills the disparity map with values estimated according to their neighbourhood.

Module Contents

Functions

fill_central_area_using_plane(disp_map, ...)	Finds central area of invalid region and estimates disparity values
add_surrounding_nodata_to_roi(roi_mask, disp, disp_mask)	Add surounding nodata to invalidity region
define_interpolation_band_width(binary_image, percentage)	Define number of pixel for later erosion operation
plot_function(data, fit)	Displays shape of plane/quadratic function used in region to fill.
calculate_disp_plane(values, mask, central_area[, display])	Estimates disparity values in disparity map which contains invalid
fill_area_borders_using_interpolation(disp_map, ...)	Raster interpolation command
make_raster_interpolation(raster, mask, options)	Raster interpolation (scipy, rasterio or pandora)
fill_disp_pandora(disp, msk_fill_disp, nb_directions)	Interpolation of the left disparity map to fill holes.
find_valid_neighbors(dirs, disp, valid, row, col, ...)	Find valid neighbors along directions
estimate_poly_with_disp(poly[, dmin, dmax])	Estimate new polygone using disparity range
get_corresponding_holes(tile_poly, holes_poly_list)	Get list of holes situated in tile
get_corresponding_tiles(tiles_polygones, ...)	Get list of tiles intersecting with holes
get_polygons_from_cars_ds(cars_ds)	Get the holes envelops computed in holes detection application
merge_intersecting_polygones(list_poly)	Merge polygons that intersects each other
fill_disp_using_plane(→ xarray.Dataset)	Fill disparity map holes
fill_disp_using_zero_padding(→ xarray.Dataset)	Fill disparity map holes
add_empty_filling_band(output_dataset, filling_types)	Add filling attribute to dataset or band to filling attribute
update_filling(output_dataset[, filling, filling_type])	Update filling attribute of dataset with an additional mask

cars.applications.dense_match_filling.fill_disp_tools.fill_central_area_using_plane(disp_map: xarray.Dataset, corresponding_poly, row_min, col_min, ignore_nodata: bool, ignore_zero_fill: bool, ignore_extrema: bool, nb_pix: int, percent_to_erode: float, class_index: list)

Finds central area of invalid region and estimates disparity values in this area according to a plan model estimation. The estimation of this model is done using disparity values at invalid region borders.

Parameters

• disp_map (2D np.array (row, col)) – disparity map with several layers (‘disp’, ‘disp_msk’, ‘msk_invalid_sec’)

• corresponding_poly (list(Polygon)) – list of holes polygons

• row_min (int) – row offset of combined tile

• col_min (int) – col offset of combined tile

• ignore_nodata (bool) – option to activate to ignore nodata values at disp mask borders

• ignore_zero_fill (bool) – option to activate to ignore zero values at disp mask borders

• ignore_extrema (bool) – option to activate to ignore extrema values at disp mask borders

• nb_pix (int) – pixel number used to define disparity values band at invalid region borders that will be considered for disp estimation

• percent_to_erode (float) – percentage to define size of central area

• class_index (list(str)) – list of tag to use

Returns

mask of invalid region that hasn’t been filled yet (original invalid region - central area)

Return type

2D np.array (row, col)

cars.applications.dense_match_filling.fill_disp_tools.add_surrounding_nodata_to_roi(roi_mask, disp, disp_mask)

Add surounding nodata to invalidity region

Parameters

• roi_mask (2D np.array (row, col)) – invalidity mask (values to fill)

• disp (2D np.array (row, col)) – disparity values

• disp_mask (2D np.array (row, col)) – disparity values mask

cars.applications.dense_match_filling.fill_disp_tools.define_interpolation_band_width(binary_image, percentage)

Define number of pixel for later erosion operation

Parameters

• binary_image (2D np.array (row, col)) – invalidity mask (values to fill)

• percentage (dict) – percentage of border compared to center region

Returns

pixel number to erode

Return type

int

cars.applications.dense_match_filling.fill_disp_tools.plot_function(data, fit)

Displays shape of plane/quadratic function used in region to fill.

Parameters

• data (3D np.array (row, col)) – coords and value of valid disparity values

• fit (ndarray) – Least-squares solution.

Returns

plot of function used in region to fill

Return type

matplotlib plot

cars.applications.dense_match_filling.fill_disp_tools.calculate_disp_plane(values, mask, central_area, display=False)

Estimates disparity values in disparity map which contains invalid area using valid data and a plane model.

Parameters

• values (3D np.array (row, col)) – valid disparity values

• mask (3D np.array (row, col)) – validity mask

• central_area (3D np.array (row, col)) – mask of disparity values to fill

• display (boolean) – plot interpolation fct in region to fill

Returns

central interpolated disparity values

Return type

list

cars.applications.dense_match_filling.fill_disp_tools.fill_area_borders_using_interpolation(disp_map, masks_to_fill, options)

Raster interpolation command :param disp_map: disparity values :type disp_map: 2D np.array (row, col) :param masks_to_fill: masks to locate disp values to fill :type masks_to_fill: list(2D np.array (row, col)) :param options: parameters for interpolation methods :type options: dict

cars.applications.dense_match_filling.fill_disp_tools.make_raster_interpolation(raster: numpy.ndarray, mask: numpy.ndarray, options: dict)

Raster interpolation (scipy, rasterio or pandora) :param raster: disparity values :type raster: 2D np.array (row, col) :param mask: invalidity mask (values to fill) :type mask: 2D np.array (row, col) :param options: parameters for interpolation methods :type options: dict :return: interpolated raster :rtype: 2D np.array

cars.applications.dense_match_filling.fill_disp_tools.fill_disp_pandora(disp: numpy.ndarray, msk_fill_disp: numpy.ndarray, nb_directions: int)

Interpolation of the left disparity map to fill holes. Interpolate invalid pixel by finding the nearest correct pixels in 8/16 different directions and use the median of their disparities. ?bontar, J., & LeCun, Y. (2016). Stereo matching by training a convolutional neural network to compare image patches. The journal of machine learning research, 17(1), 2287-2318. HIRSCHMULLER, Heiko. Stereo processing by semiglobal matching and mutual information. IEEE Transactions on pattern analysis and machine intelligence, 2007, vol. 30, no 2, p. 328-341.

Copied/adapted fct from pandora/validation/interpolated_disparity.py

Parameters

• disp (2D np.array (row, col)) – disparity map

• msk_fill_disp (2D np.array (row, col)) – validity mask

• nb_directions (integer) – nb directions to explore

Returns

the interpolate left disparity map, with the validity mask update :

Return type

tuple(2D np.array (row, col), 2D np.array (row, col))

cars.applications.dense_match_filling.fill_disp_tools.find_valid_neighbors(dirs: numpy.ndarray, disp: numpy.ndarray, valid: numpy.ndarray, row: int, col: int, nb_directions: int)

Find valid neighbors along directions

Copied/adapted fct from pandora/validation/interpolated_disparity.py

Parameters

• dirs (2D np.array (row, col)) – directions

• disp (2D np.array (row, col)) – disparity map

• valid (2D np.array (row, col)) – validity mask

• row (int) – row current value

• col (int) – col current value

• nb_directions (int) – nb directions to explore

Returns

valid neighbors

Return type

2D np.array

cars.applications.dense_match_filling.fill_disp_tools.estimate_poly_with_disp(poly, dmin=0, dmax=0)

Estimate new polygone using disparity range

Parameters

• poly (Polygon) – polygone to estimate

• dmin (int) – minimum disparity

• dmax (int) – maximum disparity

Returns

polygon in disparity range

Return type

Polygon

cars.applications.dense_match_filling.fill_disp_tools.get_corresponding_holes(tile_poly, holes_poly_list)

Get list of holes situated in tile

Parameters

• tile_poly (Polygon) – envelop of tile

• holes_poly_list (list(Polygon)) – envelop of holes

Returns

list of holes envelops

Return type

list(Polygon)

cars.applications.dense_match_filling.fill_disp_tools.get_corresponding_tiles(tiles_polygones, corresponding_holes, epi_disp_map)

Get list of tiles intersecting with holes

Parameters

• tiles_polygones (list(Polygon)) – envelop of tiles

• corresponding_holes (list(Polygon)) – envelop of holes

• epi_disp_map (CarsDataset) – disparity map cars dataset

Returns

list of tiles to use (window, overlap, xr.Dataset)

Return type

list(tuple)

cars.applications.dense_match_filling.fill_disp_tools.get_polygons_from_cars_ds(cars_ds)

Get the holes envelops computed in holes detection application cars_ds must contain dicts, and not delayed. This function must be called after an orchestrator.breakpoint()

Parameters

cars_ds (CarsDataset) – holes cars dataset

Returns

list of holes

Return type

list(Polygon)

cars.applications.dense_match_filling.fill_disp_tools.merge_intersecting_polygones(list_poly)

Merge polygons that intersects each other

Parameters

list_poly (list(Polygon)) – list of holes

Returns

list of holes

Return type

list(Polygon)

cars.applications.dense_match_filling.fill_disp_tools.fill_disp_using_plane(disp_map: xarray.Dataset, corresponding_poly, row_min, col_min, ignore_nodata: bool, ignore_zero_fill: bool, ignore_extrema: bool, nb_pix: int, percent_to_erode: float, interp_options: dict, classification) → xarray.Dataset

Fill disparity map holes

Parameters

• disp_map (xr.Dataset) – disparity map

• corresponding_poly (list(Polygon)) – list of holes polygons

• row_min (int) – row offset of combined tile

• col_min (int) – col offset of combined tile

• ignore_nodata (bool) – ingore nodata

• ignore_zero_fill (bool) – ingnore zero fill

• ignore_extrema (bool) – ignore extrema

• nb_pix (int) – margin to use

• percent_to_erode (float) – percent to erode

• interp_options (dict) – interp_options

• classification (list(str)) – list of tag to use

cars.applications.dense_match_filling.fill_disp_tools.fill_disp_using_zero_padding(disp_map: xarray.Dataset, class_index) → xarray.Dataset

Fill disparity map holes

Parameters

• disp_map (xr.Dataset) – disparity map

• class_index (int) – class index according to the classification tag

cars.applications.dense_match_filling.fill_disp_tools.add_empty_filling_band(output_dataset: xarray.Dataset, filling_types: list)

Add filling attribute to dataset or band to filling attribute if it already exists

Parameters

• output_dataset – output dataset

• filling – input mask of filled pixels

• band_filling – type of filling (zero padding or plane)

cars.applications.dense_match_filling.fill_disp_tools.update_filling(output_dataset: xarray.Dataset, filling: numpy.ndarray = None, filling_type: str = None)

Update filling attribute of dataset with an additional mask

Parameters

• output_dataset – output dataset

• filling – input mask of filled pixels

• band_filling – type of filling (zero padding or plane)