cars.applications.rasterization.rasterization_tools
¶
This module is responsible for the rasterization step:  it contains all functions related to 3D representation on a 2D raster grid TODO: refactor in several files and remove toomanylines
Module Contents¶
Functions¶

Compute xstart, ystart, xsize and ysize 

Wrapper of simple_rasterization 

Compute the x and y values as 1d arrays 

Compute the grid points 

Converts neighbors indices jagged array into a linear 1d array and 

Search for neighbors of the grid points in the cloud kdTree 

Get the grid point neighbors of the cloud as flatten array. 

Compute vectorized raster and its statistics. 

Use the outputs of the get_flatten_neighbors function 

Interpolates mask data at grid point locations. 

Interpolates point cloud data at grid point locations and produces 

Create final raster xarray dataset 

Rasterize a point cloud with its color bands to a Dataset 
 cars.applications.rasterization.rasterization_tools.compute_xy_starts_and_sizes(resolution: float, cloud: pandas.DataFrame) Tuple[float, float, int, int] ¶
Compute xstart, ystart, xsize and ysize of the rasterization grid from a set of points
 Parameters
resolution – Resolution of rasterized cells, expressed in cloud CRS units
cloud – set of points as returned by the create_combined_cloud function
 Returns
a tuple (xstart, ystart, xsize, ysize)
 cars.applications.rasterization.rasterization_tools.simple_rasterization_dataset_wrapper(cloud: pandas.DataFrame, resolution: float, epsg: int, xstart: float = None, ystart: float = None, xsize: int = None, ysize: int = None, sigma: float = None, radius: int = 1, dsm_no_data: int = np.nan, color_no_data: int = np.nan, msk_no_data: int = 65535, grid_points_division_factor: int = None, list_computed_layers: List[str] = None) xarray.Dataset ¶
Wrapper of simple_rasterization that has xarray.Dataset as inputs and outputs.
 Parameters
cloud – cloud to rasterize
resolution – Resolution of rasterized cells, expressed in cloud CRS units or None
epsg – epsg code for the CRS of the final raster
color_list – Additional list of images with bands to rasterize (same size as cloud_list), or None
xstart – xstart of the rasterization grid (if None, will be estimated by the function)
ystart – ystart of the rasterization grid (if None, will be estimated by the function)
xsize – xsize of the rasterization grid (if None, will be estimated by the function)
ysize – ysize of the rasterization grid (if None, will be estimated by the function)
sigma – sigma for gaussian interpolation. (If None, set to resolution)
radius – Radius for hole filling.
margin – Margin used to invalidate cells too close to epipolar border. Can only be used if input lists are of size 1.
dsm_no_data – no data value to use in the final raster
color_no_data – no data value to use in the final colored raster
msk_no_data – no data value to use in the final mask image
grid_points_division_factor – number of blocks to use to divide the grid points (memory optimization, reduce the highest memory peak). If it is not set, the factor is automatically set to construct 700000 points blocks.
list_computed_layers – list of computed output data
 Returns
Rasterized cloud
 cars.applications.rasterization.rasterization_tools.compute_values_1d(x_start: float, y_start: float, x_size: int, y_size: int, resolution: float) Tuple[numpy.ndarray, numpy.ndarray] ¶
Compute the x and y values as 1d arrays
 Parameters
x_start – x start of the rasterization grid
y_start – y start of the rasterization grid
x_size – x size of the rasterization grid
y_size – y size of the rasterization grid
resolution – Resolution of rasterized cells, in cloud CRS units or None.
 Returns
a tuple composed of the x and y 1d arrays
 cars.applications.rasterization.rasterization_tools.compute_grid_points(x_start: float, y_start: float, x_size: int, y_size: int, resolution: float) numpy.ndarray ¶
Compute the grid points
 Parameters
x_start – x start of the rasterization grid
y_start – y start of the rasterization grid
x_size – x size of the rasterization grid
y_size – y size of the rasterization grid
resolution – Resolution of rasterized cells, expressed in cloud CRS units or None.
 Returns
Grid point as a numpy array
 cars.applications.rasterization.rasterization_tools.flatten_index_list(nd_list)¶
Converts neighbors indices jagged array into a linear 1d array and the number of neighbors for each grid point.
 Parameters
nd_list (list of list of int.) – indices of each neighbor.
 Returns
the flattened neighbors ids list and the list of neighbors count for each grid point.
 Return type
a tuple of 2 1d int64 numpy.ndarray.
 cars.applications.rasterization.rasterization_tools.search_neighbors(grid_points: numpy.ndarray, cloud_tree: scipy.spatial.cKDTree, radius: int, resolution: float) List[List[int]] ¶
Search for neighbors of the grid points in the cloud kdTree
 Parameters
grid_points – Grid points
cloud_tree – Points cloud kdTree
radius – Radius for hole filling.
resolution – Resolution of rasterized cells, expressed in cloud CRS units or None.
 Returns
The list of neighbors
 cars.applications.rasterization.rasterization_tools.get_flatten_neighbors(grid_points: numpy.ndarray, cloud: pandas.DataFrame, radius: int, resolution: float, worker_logger: logging.Logger, grid_points_division_factor: int = None) Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray] ¶
Get the grid point neighbors of the cloud as flatten array.
This is done by slicing the grid points by blocks in order to reduce the memory peak induced by the list of neighbors retrieve from the kdTree query done in the search_neighbors function.
 Parameters
grid_points – Grid points
cloud – Combined cloud as returned by the create_combined_cloud function
radius – Radius for hole filling.
resolution – Resolution of rasterized cells, expressed in cloud CRS units or None.
worker_logger – logger
grid_points_division_factor – number of blocks to use to divide the grid points (memory optimization, reduce the highest memory peak). If it is not set, the factor is automatically set to construct 700000 points blocks.
 Returns
the flattened neighbors ids list, the list start index for each grid point and the list of neighbors count for each grid point.
 cars.applications.rasterization.rasterization_tools.compute_vector_raster_and_stats(cloud: pandas.DataFrame, data_valid: numpy.ndarray, x_start: float, y_start: float, x_size: int, y_size: int, resolution: float, sigma: float, radius: int, msk_no_data: int, worker_logger: logging.Logger, grid_points_division_factor: int, list_computed_layers: List[str] = None) Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray, numpy.ndarray, numpy.ndarray, numpy.ndarray, Union[None, numpy.ndarray]] ¶
Compute vectorized raster and its statistics.
 Parameters
cloud – Combined cloud as returned by the create_combined_cloud function
data_valid – mask of points which are not on the border of its original epipolar image. To compute a cell it has to have at least one data valid, for which case it is considered that no contributing points from other neighbor tiles are missing.
x_start – x start of the rasterization grid
y_start – y start of the rasterization grid
x_size – x size of the rasterization grid
y_size – y size of the rasterization grid
resolution – Resolution of rasterized cells, expressed in cloud CRS units or None.
sigma – Sigma for gaussian interpolation. If None, set to resolution
radius – Radius for hole filling.
msk_no_data – No data value to use for the rasterized mask
worker_logger – Logger
grid_points_division_factor – Number of blocks to use to divide the grid points (memory optimization, reduce the highest memory peak). If it is not set, the factor is automatically set to construct 700000 points blocks.
list_computed_layers – list of computed output data
 Returns
a tuple with rasterization results and statistics.
 cars.applications.rasterization.rasterization_tools.get_neighbors_from_points_array(points: numpy.ndarray, data_valid: numpy.ndarray, i_grid: int, neighbors_id: numpy.ndarray, neighbors_start: numpy.ndarray, neighbors_count: numpy.ndarray) Union[numpy.ndarray, None] ¶
Use the outputs of the get_flatten_neighbors function to get the neighbors of the i_grid point in the points numpy array.
 Parameters
points – points numpy array (one line = one point)
data_valid – valid data mask corresponding to the points
i_grid – “get_flatten_neighbors” outputs index function used
neighbors_id – the flattened neighbors ids list
neighbors_start – the flattened neighbors start indexes
neighbors_count – the flattened neighbors counts
 Returns
a numpy array containing only the i_grid point neighbors or None if the point has no neighbors (or no valid neighbors)
 cars.applications.rasterization.rasterization_tools.mask_interp(mask_points: numpy.ndarray, data_valid: numpy.ndarray, neighbors_id: numpy.ndarray, neighbors_start: numpy.ndarray, neighbors_count: numpy.ndarray, grid_points: numpy.ndarray, sigma: float, no_data_val: int = 65535, undefined_val: int = 65535) numpy.ndarray ¶
Interpolates mask data at grid point locations.
Each points contained into a terrain cell have a weight depending on its distance to the cell center. For each classes, the weights are accumulated. The class with the higher accumulated score is then used as the terrain cell’s final value.
 Parameters
mask_points – mask data, one point per row (first column is the x position, second is the y position, last column is the mask value).
data_valid – flattened validity mask.
neighbors_id – flattened neighboring cloud point indices.
neighbors_start – flattened grid point neighbors start indices.
neighbors_count – flattened grid point neighbor count.
grid_points – grid point location, one per row.
sigma – sigma parameter for weights computation.
no_data_val – no data value.
undefined_val – value in case of score equality.
 Returns
The interpolated mask
 cars.applications.rasterization.rasterization_tools.gaussian_interp(cloud_points, data_valid, neighbors_id, neighbors_start, neighbors_count, grid_points, resolution, sigma)¶
Interpolates point cloud data at grid point locations and produces quality statistics.
 Parameters
cloud_points (float64 numpy.ndarray.) – point cloud data, one point per row.
data_valid (bool numpy.ndarray.) – flattened validity mask.
neighbors_id (int64 numpy.ndarray.) – flattened neighboring cloud point indices.
neighbors_start (int64 numpy.ndarray.) – flattened grid point neighbors start indices.
neighbors_count (int64 numpy.ndarray.) – flattened grid point neighbor count.
grid_points (float64 numpy.ndarray.) – grid point location, one per row.
resolution (float.) – rasterization resolution.
sigma (float) – sigma parameter of gaussian interpolation.
 Returns
a tuple with rasterization results and statistics.
 cars.applications.rasterization.rasterization_tools.create_raster_dataset(raster: numpy.ndarray, x_start: float, y_start: float, x_size: int, y_size: int, resolution: float, hgt_no_data: int, color_no_data: int, epsg: int, mean: numpy.ndarray, stdev: numpy.ndarray, n_pts: numpy.ndarray, n_in_cell: numpy.ndarray, msk: numpy.ndarray = None, ambiguity: numpy.ndarray = None) xarray.Dataset ¶
Create final raster xarray dataset
 Parameters
raster – height and colors
x_start – x start of the rasterization grid
y_start – y start of the rasterization grid
x_size – x size of the rasterization grid
y_size – y size of the rasterization grid
resolution – Resolution of rasterized cells, expressed in cloud CRS units or None.
hgt_no_data – no data value to use for height
color_no_data – no data value to use for color
epsg – epsg code for the CRS of the final raster
mean – mean of height and colors
stdev – standard deviation of height and colors
n_pts – number of points that are stricty in a cell
n_in_cell – number of points which contribute to a cell
msk – raster msk
 Returns
the raster xarray dataset
 cars.applications.rasterization.rasterization_tools.rasterize(cloud: pandas.DataFrame, resolution: float, epsg: int, x_start: float, y_start: float, x_size: int, y_size: int, sigma: float = None, radius: int = 1, hgt_no_data: int = 32768, color_no_data: int = 0, msk_no_data: int = 65535, grid_points_division_factor: int = None, list_computed_layers: List[str] = None) Union[xarray.Dataset, None] ¶
Rasterize a point cloud with its color bands to a Dataset that also contains quality statistics.
 Parameters
cloud – Combined cloud as returned by the create_combined_cloud function
resolution – Resolution of rasterized cells, expressed in cloud CRS units or None.
epsg – epsg code for the CRS of the final raster
x_start – x start of the rasterization grid
y_start – y start of the rasterization grid
x_size – x size of the rasterization grid
y_size – y size of the rasterization grid
sigma – sigma for gaussian interpolation. If None, set to resolution
radius – Radius for hole filling.
hgt_no_data – no data value to use for height
color_no_data – no data value to use for color
msk_no_data – no data value to use in the final mask image
grid_points_division_factor – number of blocks to use to divide the grid points (memory optimization, reduce the highest memory peak). If it is not set, the factor is automatically set to construct 700000 points blocks.
list_computed_layers – list of computed output data
 Returns
Rasterized cloud color and statistics.