cars.applications.rasterization.rasterization_tools

This module is responsible for the rasterization step: - it contains all functions related to 3D representation on a 2D raster grid TODO: refactor in several files and remove too-many-lines

Module Contents

Functions

compute_xy_starts_and_sizes(→ Tuple[float, float, int, ...)

Compute xstart, ystart, xsize and ysize

simple_rasterization_dataset_wrapper(→ xarray.Dataset)

Wrapper of simple_rasterization

compute_values_1d(→ Tuple[numpy.ndarray, numpy.ndarray])

Compute the x and y values as 1d arrays

compute_grid_points(→ numpy.ndarray)

Compute the grid points

flatten_index_list(nd_list)

Converts neighbors indices jagged array into a linear 1d array and

search_neighbors(→ List[List[int]])

Search for neighbors of the grid points in the cloud kdTree

get_flatten_neighbors(→ Tuple[numpy.ndarray, ...)

Get the grid point neighbors of the cloud as flatten array.

compute_vector_raster_and_stats(→ Tuple[numpy.ndarray, ...)

Compute vectorized raster and its statistics.

get_neighbors_from_points_array(→ Union[numpy.ndarray, ...)

Use the outputs of the get_flatten_neighbors function

mask_interp(→ numpy.ndarray)

Interpolates mask data at grid point locations.

gaussian_interp(cloud_points, data_valid, ...)

Interpolates point cloud data at grid point locations and produces

create_raster_dataset(→ xarray.Dataset)

Create final raster xarray dataset

rasterize(→ Union[xarray.Dataset, None])

Rasterize a point cloud with its color bands to a Dataset
cars.applications.rasterization.rasterization_tools.compute_xy_starts_and_sizes(resolution: float, cloud: pandas.DataFrame) → Tuple[float, float, int, int]

Compute xstart, ystart, xsize and ysize of the rasterization grid from a set of points

Parameters

  • resolution – Resolution of rasterized cells, expressed in cloud CRS units

  • cloud – set of points as returned by the create_combined_cloud function

Returns

a tuple (xstart, ystart, xsize, ysize)

cars.applications.rasterization.rasterization_tools.simple_rasterization_dataset_wrapper(cloud: pandas.DataFrame, resolution: float, epsg: int, xstart: float = None, ystart: float = None, xsize: int = None, ysize: int = None, sigma: float = None, radius: int = 1, dsm_no_data: int = np.nan, color_no_data: int = np.nan, msk_no_data: int = 65535, grid_points_division_factor: int = None, list_computed_layers: List[str] = None) → xarray.Dataset

Wrapper of simple_rasterization that has xarray.Dataset as inputs and outputs.

Parameters

  • cloud – cloud to rasterize

  • resolution – Resolution of rasterized cells, expressed in cloud CRS units or None

  • epsg – epsg code for the CRS of the final raster

  • color_list – Additional list of images with bands to rasterize (same size as cloud_list), or None

  • xstart – xstart of the rasterization grid (if None, will be estimated by the function)

  • ystart – ystart of the rasterization grid (if None, will be estimated by the function)

  • xsize – xsize of the rasterization grid (if None, will be estimated by the function)

  • ysize – ysize of the rasterization grid (if None, will be estimated by the function)

  • sigma – sigma for gaussian interpolation. (If None, set to resolution)

  • radius – Radius for hole filling.

  • margin – Margin used to invalidate cells too close to epipolar border. Can only be used if input lists are of size 1.

  • dsm_no_data – no data value to use in the final raster

  • color_no_data – no data value to use in the final colored raster

  • msk_no_data – no data value to use in the final mask image

  • grid_points_division_factor – number of blocks to use to divide the grid points (memory optimization, reduce the highest memory peak). If it is not set, the factor is automatically set to construct 700000 points blocks.

  • list_computed_layers – list of computed output data

Returns

Rasterized cloud

cars.applications.rasterization.rasterization_tools.compute_values_1d(x_start: float, y_start: float, x_size: int, y_size: int, resolution: float) → Tuple[numpy.ndarray, numpy.ndarray]

Compute the x and y values as 1d arrays

Parameters

  • x_start – x start of the rasterization grid

  • y_start – y start of the rasterization grid

  • x_size – x size of the rasterization grid

  • y_size – y size of the rasterization grid

  • resolution – Resolution of rasterized cells, in cloud CRS units or None.

Returns

a tuple composed of the x and y 1d arrays

cars.applications.rasterization.rasterization_tools.compute_grid_points(x_start: float, y_start: float, x_size: int, y_size: int, resolution: float) → numpy.ndarray

Compute the grid points

Parameters

  • x_start – x start of the rasterization grid

  • y_start – y start of the rasterization grid

  • x_size – x size of the rasterization grid

  • y_size – y size of the rasterization grid

  • resolution – Resolution of rasterized cells, expressed in cloud CRS units or None.

Returns

Grid point as a numpy array

cars.applications.rasterization.rasterization_tools.flatten_index_list(nd_list)

Converts neighbors indices jagged array into a linear 1d array and the number of neighbors for each grid point.

Parameters

nd_list (list of list of int.) – indices of each neighbor.

Returns

the flattened neighbors ids list and the list of neighbors count for each grid point.

Return type

a tuple of 2 1d int64 numpy.ndarray.

cars.applications.rasterization.rasterization_tools.search_neighbors(grid_points: numpy.ndarray, cloud_tree: scipy.spatial.cKDTree, radius: int, resolution: float) → List[List[int]]

Search for neighbors of the grid points in the cloud kdTree

Parameters

  • grid_points – Grid points

  • cloud_tree – Points cloud kdTree

  • radius – Radius for hole filling.

  • resolution – Resolution of rasterized cells, expressed in cloud CRS units or None.

Returns

The list of neighbors

cars.applications.rasterization.rasterization_tools.get_flatten_neighbors(grid_points: numpy.ndarray, cloud: pandas.DataFrame, radius: int, resolution: float, worker_logger: logging.Logger, grid_points_division_factor: int = None) → Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray]

Get the grid point neighbors of the cloud as flatten array.

This is done by slicing the grid points by blocks in order to reduce the memory peak induced by the list of neighbors retrieve from the kdTree query done in the search_neighbors function.

Parameters

  • grid_points – Grid points

  • cloud – Combined cloud as returned by the create_combined_cloud function

  • radius – Radius for hole filling.

  • resolution – Resolution of rasterized cells, expressed in cloud CRS units or None.

  • worker_logger – logger

  • grid_points_division_factor – number of blocks to use to divide the grid points (memory optimization, reduce the highest memory peak). If it is not set, the factor is automatically set to construct 700000 points blocks.

Returns

the flattened neighbors ids list, the list start index for each grid point and the list of neighbors count for each grid point.

cars.applications.rasterization.rasterization_tools.compute_vector_raster_and_stats(cloud: pandas.DataFrame, data_valid: numpy.ndarray, x_start: float, y_start: float, x_size: int, y_size: int, resolution: float, sigma: float, radius: int, msk_no_data: int, worker_logger: logging.Logger, grid_points_division_factor: int, list_computed_layers: List[str] = None) → Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray, numpy.ndarray, numpy.ndarray, numpy.ndarray, Union[None, numpy.ndarray]]

Compute vectorized raster and its statistics.

Parameters

  • cloud – Combined cloud as returned by the create_combined_cloud function

  • data_valid – mask of points which are not on the border of its original epipolar image. To compute a cell it has to have at least one data valid, for which case it is considered that no contributing points from other neighbor tiles are missing.

  • x_start – x start of the rasterization grid

  • y_start – y start of the rasterization grid

  • x_size – x size of the rasterization grid

  • y_size – y size of the rasterization grid

  • resolution – Resolution of rasterized cells, expressed in cloud CRS units or None.

  • sigma – Sigma for gaussian interpolation. If None, set to resolution

  • radius – Radius for hole filling.

  • msk_no_data – No data value to use for the rasterized mask

  • worker_logger – Logger

  • grid_points_division_factor – Number of blocks to use to divide the grid points (memory optimization, reduce the highest memory peak). If it is not set, the factor is automatically set to construct 700000 points blocks.

  • list_computed_layers – list of computed output data

Returns

a tuple with rasterization results and statistics.

cars.applications.rasterization.rasterization_tools.get_neighbors_from_points_array(points: numpy.ndarray, data_valid: numpy.ndarray, i_grid: int, neighbors_id: numpy.ndarray, neighbors_start: numpy.ndarray, neighbors_count: numpy.ndarray) → Union[numpy.ndarray, None]

Use the outputs of the get_flatten_neighbors function to get the neighbors of the i_grid point in the points numpy array.

Parameters

  • points – points numpy array (one line = one point)

  • data_valid – valid data mask corresponding to the points

  • i_grid – “get_flatten_neighbors” outputs index function used

  • neighbors_id – the flattened neighbors ids list

  • neighbors_start – the flattened neighbors start indexes

  • neighbors_count – the flattened neighbors counts

Returns

a numpy array containing only the i_grid point neighbors or None if the point has no neighbors (or no valid neighbors)

cars.applications.rasterization.rasterization_tools.mask_interp(mask_points: numpy.ndarray, data_valid: numpy.ndarray, neighbors_id: numpy.ndarray, neighbors_start: numpy.ndarray, neighbors_count: numpy.ndarray, grid_points: numpy.ndarray, sigma: float, no_data_val: int = 65535, undefined_val: int = 65535) → numpy.ndarray

Interpolates mask data at grid point locations.

Each points contained into a terrain cell have a weight depending on its distance to the cell center. For each classes, the weights are accumulated. The class with the higher accumulated score is then used as the terrain cell’s final value.

Parameters

  • mask_points – mask data, one point per row (first column is the x position, second is the y position, last column is the mask value).

  • data_valid – flattened validity mask.

  • neighbors_id – flattened neighboring cloud point indices.

  • neighbors_start – flattened grid point neighbors start indices.

  • neighbors_count – flattened grid point neighbor count.

  • grid_points – grid point location, one per row.

  • sigma – sigma parameter for weights computation.

  • no_data_val – no data value.

  • undefined_val – value in case of score equality.

Returns

The interpolated mask

cars.applications.rasterization.rasterization_tools.gaussian_interp(cloud_points, data_valid, neighbors_id, neighbors_start, neighbors_count, grid_points, resolution, sigma)

Interpolates point cloud data at grid point locations and produces quality statistics.

Parameters

  • cloud_points (float64 numpy.ndarray.) – point cloud data, one point per row.

  • data_valid (bool numpy.ndarray.) – flattened validity mask.

  • neighbors_id (int64 numpy.ndarray.) – flattened neighboring cloud point indices.

  • neighbors_start (int64 numpy.ndarray.) – flattened grid point neighbors start indices.

  • neighbors_count (int64 numpy.ndarray.) – flattened grid point neighbor count.

  • grid_points (float64 numpy.ndarray.) – grid point location, one per row.

  • resolution (float.) – rasterization resolution.

  • sigma (float) – sigma parameter of gaussian interpolation.

Returns

a tuple with rasterization results and statistics.

cars.applications.rasterization.rasterization_tools.create_raster_dataset(raster: numpy.ndarray, x_start: float, y_start: float, x_size: int, y_size: int, resolution: float, hgt_no_data: int, color_no_data: int, epsg: int, mean: numpy.ndarray, stdev: numpy.ndarray, n_pts: numpy.ndarray, n_in_cell: numpy.ndarray, msk: numpy.ndarray = None, ambiguity: numpy.ndarray = None) → xarray.Dataset

Create final raster xarray dataset

Parameters

  • raster – height and colors

  • x_start – x start of the rasterization grid

  • y_start – y start of the rasterization grid

  • x_size – x size of the rasterization grid

  • y_size – y size of the rasterization grid

  • resolution – Resolution of rasterized cells, expressed in cloud CRS units or None.

  • hgt_no_data – no data value to use for height

  • color_no_data – no data value to use for color

  • epsg – epsg code for the CRS of the final raster

  • mean – mean of height and colors

  • stdev – standard deviation of height and colors

  • n_pts – number of points that are stricty in a cell

  • n_in_cell – number of points which contribute to a cell

  • msk – raster msk

Returns

the raster xarray dataset

cars.applications.rasterization.rasterization_tools.rasterize(cloud: pandas.DataFrame, resolution: float, epsg: int, x_start: float, y_start: float, x_size: int, y_size: int, sigma: float = None, radius: int = 1, hgt_no_data: int = -32768, color_no_data: int = 0, msk_no_data: int = 65535, grid_points_division_factor: int = None, list_computed_layers: List[str] = None) → Union[xarray.Dataset, None]

Rasterize a point cloud with its color bands to a Dataset that also contains quality statistics.

Parameters

  • cloud – Combined cloud as returned by the create_combined_cloud function

  • resolution – Resolution of rasterized cells, expressed in cloud CRS units or None.

  • epsg – epsg code for the CRS of the final raster

  • x_start – x start of the rasterization grid

  • y_start – y start of the rasterization grid

  • x_size – x size of the rasterization grid

  • y_size – y size of the rasterization grid

  • sigma – sigma for gaussian interpolation. If None, set to resolution

  • radius – Radius for hole filling.

  • hgt_no_data – no data value to use for height

  • color_no_data – no data value to use for color

  • msk_no_data – no data value to use in the final mask image

  • grid_points_division_factor – number of blocks to use to divide the grid points (memory optimization, reduce the highest memory peak). If it is not set, the factor is automatically set to construct 700000 points blocks.

  • list_computed_layers – list of computed output data

Returns

Rasterized cloud color and statistics.