Normal map#

Overview#

The ground truth exposed in this modality is the normal vector that comes out of each pixel visible in the image.

This modality consists of the following file:

Relevant file

Location

normal_maps.exr

Camera folder

normal_maps.exr#

In this file, we have converted the visual spectrum image into a normal map. The normal map provides the vector coming out of every surface visible in your datapoint.

The file uses the 32-bit floating point version of the OpenEXR file format, which provides room to store extremely accurate measurements in the color data.

Image1 Image2

A normal map of a human face (left) and its corresponding visual spectrum image (right)

To create the normal map, we have replaced the color value of each pixel with the X, Y, and Z components of the normal vector coming out of that pixel, where the axes are defined as follows:

  • -1.0 ≤ x ≤ 1.0, where +X is to the right in the camera space, becomes 0 ≤ R ≤ 1.

  • -1.0 ≤ y ≤ 1.0, where +Y is up in the camera space, becomes 0 ≤ G ≤ 1.

  • -1.0 ≤ z ≤ 1.0, where +Z is the camera direction, becomes 0 ≥ B ≥ 1 (note the reversed direction).

To retrieve the original components of the normal vector in the camera space, simply reverse the mapping as follows:

  • x = R*2-1

  • y = G*2-1

  • z = 1-(B*2)

Using this ground truth, you can train your model to perform 3D reconstruction of the contours of a person’s face.

To process a normal map, we recommend using code along the following lines:

import cv2

def load_normal(path):
    normal_map = cv2.cvtColor(cv2.imread(path, cv2.IMREAD_UNCHANGED), cv2.COLOR_BGR2RGB)
    normal_map = 2 * normal_map - 1
    normal_map[..., 2] *= -1
    return normal_map

path = "normal_maps.exr"
normal_map = load_normal(path)

See https://github.com/DatagenTech/dgutils/blob/master/Notebooks/008_normal_depth_maps_exr_format.ipynb for more about how to load and display a normal map using OpenCV.