Visible spectrum image#

Overview#

The rendered image is the center of the Faces platform. These images will be the datapoints in your model, reflecting the output of real-world cameras; all of the other modalities represent ground truths against which the images are compared.

This modality consists of the following files:

Relevant file	Location
environment.json	Camera folder
infrared_spectrum.png	Camera folder
lights_metadata.json	Scene folder
visible_spectrum_01.png, visible_spectrum_02.png, etc.	Camera folder
visible_spectrum_cross_polarized.png	Camera folder

environment.json#

This file provides a cross-reference between each .png file in the camera folder and the lighting environment that was used when rendering the image in that file.

The file uses the following format:

{
   "version": "1.0.0",
   "environments": [
      {
         "image_name": "visible_spectrum_01.png",
         "environment": "indoor",
         "time_of_day": "morning",
         "background": "hdri"
      },
      {
         "image_name": "visible_spectrum_02.png",
         "environment": "outdoor",
         "time_of_day": "morning",
         "background": "hdri"
      },
   ]
}

Objects and fields:#

version: String. Version tracking for this file. Whenever you access this file in a datapoint, make sure to check that the version matches what you expect it to be; otherwise its format and fields may not be recognized.
environments: Array. Contains a list of the visual spectrum files in this folder and the environment that was used to generate each one. Each object in the array consists of the following:
- image_name: String. Contains the filename of one of the rendered images in this folder. Each visual spectrum image in the folder appears once in this array. Most rendered images are named sequentially according to the pattern “visible_spectrum_01.png”, “visible_spectrum_02.png”, and so on.
  Some files are given unique names if they contain an image with unique properties. For example, an image that simulates cross-polarization photography is named “visible_spectrum_cross_polarized.png”, while an image that simulates near-infrared lighting is named “infrared_spectrum.png”.
- environment: String. Contains the location that is simulated in the image’s lighting. Standard values are “indoor” and “outdoor”.
  Some unique lighting scenarios, such as near-infrared lighting, have the value “N/A”.
- time_of_day: String. Contains the time of day that is simulated in the image’s lighting. Standard values are “morning”, “evening”, “night”.
  Some unique lighting scenarios, such as near-infrared lighting, have the value “N/A”.
- background: String. Contains the type of background used in the image. Standard values are “hdri” and “transparent”.

infrared_spectrum.png#

This file appears only if you selected the near-infrared lighting option when generating your dataset. It displays a simulation of how the subject would look under that spotlight, in standard .png format.

A simulation of a near-infrared spotlight (left) and a visual spectrum image of the same subject (right)

You defined the image resolution and the properties of the near-infrared spotlight when you generated your dataset. The image resolution can be found under the resolution_px object in camera_metadata.json, in the Camera folder. The properties of the near-infrared spotlight can be found in lights_metadata.json, in the Scene folder.

lights_metadata.json#

This file defines any special lighting conditions that were used in rendered images in this dataset. In datasets with no special lighting conditions, the file looks like this:

{
   "version": "1.0.0",
   "lights": {}
}

If your dataset does include one or more special lighting conditions, the “lights” object contains one object for each condition. For example:

{
   "version": "1.0.0",
   "lights": {
      "infra_red_lamp_0001": {
        // Inside this object are the properties of the special lighting condition
      }
      "infra_red_lamp_0002": {
        // Inside this object are the properties of the special lighting condition
      }
   }
}

Objects and fields#

All lighting conditions#

version: String. Version tracking for this file. Whenever you access this file in a datapoint, make sure to check that the version matches what you expect it to be; otherwise its format and fields may not be recognized.

Near-infrared lamp#

The near-infrared lamp is currently the only special lighting condition that is defined in this file. The object representing this lamp is named “infra_red_lamp_0001” and appears under the “lights” object above:

"infra_red_lamp_0001": {
"type": "spot",
   "spectrum": "near_infrared",
   "location": {
      "x": 0.0,
      "y": -1.600000023841858,
      "z": 0.11999999731779099
   },
   "orientation": {
      "look_at_vector": {
         "x": 0.0,
         "y": 1.0,
         "z": 1.4901162614933128e-07
      },
      "up_vector": {
         "x": 0.0,
         "y": -1.4901162614933128e-07,
         "z": 1.0
      }
   }
   "falloff": 1.0,
   "beam_angle": 6.499999896287763,
   "brightness": 300.0,
}

Objects and fields:#

type: String. The type of light that is described in this JSON object. For the near-infrared lamp, the value is “spot”.
spectrum: String. The spectrum of the light that is emitted. For the near-infrared lamp, the value is “near_infrared”.
location: Object. This object contains three Floats named “x”, “y”, and “z”, which give the coordinates of the near-infrared lamp in a global coordinate system measured in meters. The location of the lamp is the same as the location of the camera in the scene, which you defined when you created your dataset. See About our coordinate systems for details.
orientation: Object. This object contains two vectors that describe how the lamp is oriented in the 3D space. The orientation of the lamp is the same as the orientation of the camera in the scene, which you defined through the camera yaw, pitch, and roll controls when you created your dataset.
- look_at_vector: Object. This object contains the normalized x, y, and z components of the vector that defines the direction of the spotlight. This vector is the same as the look-at vector of the camera, whose orientation you defined when you created your dataset.
  - x: Float. At the lamp’s default position and orientation, the x axis runs from right to left. By default, the x component of the lamp’s look-at vector is 0, which means it is pointing neither to the left nor to the right.
    
    The spotlight’s view of the subject at default orientation#
    
    The spotlight’s view of the subject with a 1° rotation in the -x direction#
    
    The spotlight’s view of the subject with a 1° rotation in the +x direction#
  - y: Float. At the lamp’s default position and orientation, the y axis runs from forward to back. By default, the y component of the lamp’s look-at vector is 1, which means it is pointing in the positive y direction - towards the subject’s face, which is located at the origin.
    
    In this default image, the camera and spotlight are facing in the +y direction, and the subject is facing in the -y direction#
    
    In this image, the lamp has been first moved to +1.6 on the y axis, then rotated 180 degrees so that it is facing in the -y direction. The face is also by default pointed in the -y direction, and therefore we are looking at the back of the subject’s head. (If we had not moved the lamp to y=+1.6 before rotating it, the lamp would be shining directly away from the subject, who would not be in the camera frame at all.)#
  - z: Float. At the lamp’s default position and orientation, the z axis runs from bottom to top. By default, the x component of the lamp’s look-at vector is 0, which means it is tilted neither downward nor upward.
    
    The spotlight’s view of the subject at default orientation#
    
    The spotlight’s view of the subject with a 1° rotation in the -z direction#
    
    The spotlight’s view of the subject with a 1° rotation in the +z direction#
- up_vector: Object. This object contains the normalized x, y, and z components of the vector that defines the lamp’s “up” direction, providing you with the lamp’s orientation independent of the direction of the beam. The lamp’s up vector is the same as the up vector of the camera, whose orientation you defined when you created your dataset.
  - x: Float. The x axis runs from left to right from the default camera’s point of view. The default value of the x component of the camera’s up vector (and therefore the lamp’s up vector) is 0, which means the vector is neither tilted to the left nor to the right. If the x component of the look-at vector is lower, this means the lamp has been rotated counterclockwise; if the x component of the look-at vector is higher, this means the lamp has been rotated clockwise. Unless the y and z components are also changed, neither of these differences has an effect on the direction of the beam, because the beam is rotationally symmetric.
    
    The spotlight’s view of the subject at default orientation#
    
    The spotlight’s view of the subject with the up vector tilted 1° in the -x direction from the default orientation#
    
    The spotlight’s view of the subject with the up vector tilted 1° in the +x direction from the default orientation#
  - y: Float. The y axis runs from back to front from the default camera’s point of view. The default y component of the camera’s up vector (and therefore the lamp’s up vector) is 0, because the lamp is upright and its up vector is not tilted toward or away from the subject.
    
    The spotlight’s view of the subject at default orientation#
    
    The spotlight’s view of the subject with the up vector tilted 1° in the -y direction from the default orientation#
    
    The spotlight’s view of the subject with the up vector tilted 1° in the +y direction from the default orientation#
  - z: Float. The z axis runs from bottom to top from the default camera’s point of view. The default z component of the lamp’s up vector is 1, which means the vector is pointing straight up and the spotlight’s beam is parallel to the floor.
    
    The spotlight’s view of the subject at default orientation, with z=1#
    
    The spotlight’s view of the subject when the z component of the lamp’s up vector is -1 and the lamp’s look-at vector is unchanged.#
falloff: Float. A measure of how blurred the edge of the spotlight is. In precise terms, falloff describes how far into the spotlight you have to go, starting from the edge and moving towards the center, before you reach the point where the beam is at full strength.

A falloff of 0 means there is no distance between the full-strength part of the beam and its edge; in other words, the light is at full strength throughout the beam, with a sudden cutoff between light and dark. A falloff of 1 means that the light is at full strength only at a single point in the very center, and dissipates gradually as you approach the edge.

You defined the value of this field as a percentage, between 0% to 100%, when you created your dataset; in this file it is converted to a decimal between 0 and 1.

Examples of falloff#
beam_angle: Float. The width of the near-infrared spotlight, measured in degrees. The spotlight is a point source, and the light coming out of it describes a cone in space; this angle is the angle at the point of that cone. You defined the value of this field when you created your dataset.
brightness: Float. The radiant power of the near-infrared spotlight, measured in watts. Radiant power is the amount of light energy that is output by the lamp. You defined the value of this field when you created your dataset.

Note

The wattage values listed on a commercial light bulb are not the same as the value in this field. Those values generally refer to power draw of one of two types: 1) the amount of electrical power drawn by the bulb itself, or 2) the amount of power that would be drawn by an equivalent incandescent lightbulb. The value in this field refers to light output, not power draw.

visible_spectrum_01.png, visible_spectrum_02.png, etc.#

These are the standard visual spectrum datapoints that are at the center of your dataset. Each one contains an image of the subject under one of the standard types of lighting conditions that Datagen offers. To look up the specific lighting condition that was used in each datapoint, see the environment.json file above.

../_images/Visspec.jpeg — A sample visible spectrum image under normal lighting conditions#

The images are in the standard png format. You defined the image resolution when you generated your dataset; it can be found under the resolution_px object in camera_metadata.json in the Camera folder.

visible_spectrum_cross_polarized.png#

This is a unique name given to a datapoint that depicts an image under cross-polarization photography. Cross-polarization is a technique meant to reduce or eliminate glare and specular reflections. It ensures that the character of the light coming off the subject is entirely based on the subject, and not on the light source.

An image of a subject under cross-polarization photography to reduce glare (left) and a visual spectrum image of the same subject (right)

As with other datapoints, this one is provided in the standard png image format. You defined the image resolution when you generated your dataset; it can be found under the resolution_px object in camera_metadata.json in the Camera folder.