CARS overview

Satellite photogrammetry in a nutshell

As its name suggests, satellite photogrammetry works using optical satellite acquisitions to generate a Digital Surface Model (DSM), the 2.5D representation created from observed surface altitude data. We speak of 2.5D because the digital surface model is an array of pixels (raster, 2D) where each pixel (x,y) corresponds to an altitude (z).

Fig. 1 2.5D representation

On the left, an example of an array of pixels, and on the right, a volume representation of altitude values.

Like our eyes, altitude (or depth relative to the satellite, to continue the analogy) is determined from the observed pixels displacement. We therefore need at least two images, acquired from two different viewpoints. This difference in viewpoints between two satellites is quantified by the B over H ratio where B is the distance between the two satellites and H is their altitude.

B over H ratio	Two viewpoints

Every raster readable by GDAL can be given as CARS input. In addition to images, the photogrammetric process requires geometric models. Rational Polynomial Coefficients (RPC) provide a compact representation of a ground-to-image geometry giving a relationship between:

• Image coordinates + altitude and ground coordinates (direct model: image to ground)

• Ground coordinates + altitude and image coordinates (inverse model: ground to image)

These coefficients are classically contained in the RPC*.XML files.

From Satellite Images to Digital Surface Model

Generate a DSM step by step

Images are first resampled in epipolar geometry: by changing viewpoints, objects pixels move along a line.

This enables a one-dimensional search (computer performance + error limitation).

The pixels shifting along other directions are not taken into account : it corresponds to fast moving objects like vehicles.

Pipeline	Resampling

For each point in one image, the software searches the corresponding point in the other image.

The color of the pixels (grayscale) in the image below corresponds to the shift value. Pixels with no match are displayed as transparent pixels in the image below.

The transparent pixels indicate areas where the matching algorithm couldn’t find a reliable correspondence between the two images, highlighting regions of uncertainty in the matching process. These matching errors can occur due to various reasons such as moving objects, shadows, occlusions, or areas with insufficient texture.

Pipeline	Matching

The displacements obtained are transformed into positions in both images.

This allows to deduce lines of sight. The intersection of these lines gives a point in space: longitude, latitude, altitude (see below).

A line of sight is an imaginary straight line from the camera’s perspective through a specific point in the image, extending into 3D space. It represents all possible 3D positions that could have produced that image point.

Pipeline	Triangulation

To obtain a raster image, the final process projects each point into a 2D grid: altitudes and colors (see below).

Pipeline	Rasterization

Initial Input Digital Elevation Model

The user can provide as input a low resolution Digital Elevation Model (DEM). It helps to minimize the disparity intervals to explore. Any geotiff file can be used. If the DEM is not specified by the user, an internal DEM is generated with sparse matches.

To download the low resolution DEM corresponding to your area, see section Get low resolution DEM.

The parameter is dem in initial_elevation as seen in Basic configuration.

Altimetric exploration and geometric inaccuracies

To reduce the search interval (i.e. altimetric exploration) in the matching step and thus save computing time, a faster sparse matching step is typically used. This matching step also enables geometric errors to be corrected, thus ensuring that the epipolar geometry (based on these models) is correct.

Matching can be performed with keypoints like SIFT (Here is an article).

Pipeline	Matching (sparse)

The result is a sparse point cloud…

Pipeline	Triangulation (sparse)

and a sparse digital surface model.

Pipeline	Rasterization (sparse)

Mask and Classification Usage

Photogrammetry is a technique that cannot reproduce altitude on water. This technique also has difficulties for shaded areas, cloudy areas or moving elements such as cars.

For this reason, it is possible to mask out areas or apply ad hoc processing to aid the matching stage.

Mask

CARS can use a mask for each image in order to ignore some image regions. This mask is taken into account during the whole 3D restitution process.

Prerequisites

• The masks are in the same geometry than the satellite images.

• Each mask is a binary file (one band with 1 nbit). Please, see the section Convert image to binary image to make a binary image.

• 1 values are considered as invalid data. 0 values are considered as valid data.

How it works

Masks are resampled in epipolar geometry with the resampling application. The masked values (1 values) are not processed during the computation. They are not taken into account in the matching process (sparse or dense matching method) to avoid mismatch and useless processing. Furthermore, the sparse matching estimation of the disparity range can be enhanced with mask using for the water area typically.

Example

Below, an example of an area close to Nice with its associated water mask (calculated with SLURP) and the DSM calculated by masking the water zones.

Table 1 Example mask usage

Satellite Image	Water Mask	Masked DSM

Classification

CARS can use a classification image for each image in order to apply some processings in the regions. (Please see the section Applications in the Advanced Configuration).

Prerequisites

• The classification images are in the same geometry as the satellite images.

• Each classification image is a multi-band binary file.

  • Binary file: Please, see the section Convert image to binary image to make a binary image.

  • Multi-band file: Each band should have a specific name (Please, see the section Add band name / description in TIF files metadata to add band name / description).

  • 1 values are considered as valid data. 0 values are considered as invalid data.

How it works

By using the classification file, a different application for each band can be applied for the 1 values. See application classification parameter Advanced configuration.

Example

Below, an example of an area close to Nice with its associated classification image (calculated with SLURP). The first band corresponds to the water class and the second band to the vegetation class.

Table 2 Example classification usage

Satellite Image	Classification (Band 1: “Water”)	Classification (Band 2: “Vegetation”)

3D products

CARS produces a geotiff file named dsm.tif that contains the Digital Surface Model in the required cartographic projection and the ground sampling distance defined by the user.

If the user provides an additional input image, an ortho-image color.tif is also produced. The latter is stackable to the DSM (See Getting Started).

If the user saves point clouds as laz format, the point clouds are saved in laz compressed format with colors or graylevel image.

These two products can be visualized with QGIS for example.

dsm.tif	color.tif	QGIS Mix	cloudcompare

CARS pipeline

To summarize, CARS pipeline is organized in sequential steps from input pairs (and metadata) to output data. Each step is performed tile-wise and distributed among workers.

The pipeline will perform the following steps [Michel J. et al, 2020] [Youssefi D. et al, 2020]:

• For each stereo pair:

  1. Create stereo-rectification grids for left and right views.

  2. Resample the both images into epipolar geometry.

  3. Compute sift matches between left and right views in epipolar geometry.

  4. Predict an optimal disparity range from the filtered point cloud resulting from the sift matches triangulation.

  5. Create a bilinear correction model of the right image’s stereo-rectification grid in order to minimize the epipolar error. Apply the estimated correction to the right grid.

  6. Resample again the stereo pair in epipolar geometry (using corrected grid for the right image) by using input DTM (such as SRTM) in order to reduce the disparity intervals to explore.

  7. Compute disparity for each image pair in epipolar geometry.

  8. Fill holes in disparity maps for each image pair in epipolar geometry.

  9. Triangulate the matches and get for each pixel of the reference image a latitude, longitude and altitude coordinate.

• Then

  10. Merge points clouds coming from each stereo pairs.

  11. Filter the resulting 3D points cloud via two consecutive filters: the first removes the small groups of 3D points, the second filters the points which have the most scattered neighbors.

  12. Rasterize: Project these altitudes on a regular grid as well as the associated color.