The ground truth exposed in this modality is the normal vector that comes out of each pixel visible in the image.
This modality consists of the following file:
In this file, we have converted the visual spectrum image into a normal map. The normal map provides the vector coming out of every surface visible in your datapoint.
The file uses the 32-bit floating point version of the OpenEXR file format, which provides room to store extremely accurate measurements in the color data.
A normal map of a human face (left) and its corresponding visual spectrum image (right)
To create the normal map, we have replaced the color value of each pixel with the X, Y, and Z components of the normal vector coming out of that pixel, where the axes are defined as follows:
-1.0 ≤ x ≤ 1.0, where +X is to the right in the camera space, becomes 0 ≤ R ≤ 1.
-1.0 ≤ y ≤ 1.0, where +Y is up in the camera space, becomes 0 ≤ G ≤ 1.
-1.0 ≤ z ≤ 1.0, where +Z is the camera direction, becomes 0 ≥ B ≥ 1 (note the reversed direction).
To retrieve the original components of the normal vector in the camera space, simply reverse the mapping as follows:
x = R*2-1
y = G*2-1
z = 1-(B*2)
Using this ground truth, you can train your model to perform 3D reconstruction of the contours of a person’s face.
To process a normal map, we recommend using code along the following lines:
import cv2 def load_normal(path): normal_map = cv2.cvtColor(cv2.imread(path, cv2.IMREAD_UNCHANGED), cv2.COLOR_BGR2RGB) normal_map = 2 * normal_map - 1 normal_map[..., 2] *= -1 return normal_map path = "normal_maps.exr" normal_map = load_normal(path)
See https://github.com/DatagenTech/dgutils/blob/master/Notebooks/008_normal_depth_maps_exr_format.ipynb for more about how to load and display a normal map using OpenCV.