Explore platform

Facial keypoints (MediaPipe)

Overview

The ground truth exposed in this modality is a set of facial landmarks that conforms to the MediaPipe standard.

This modality consists of the following files:

Relevant file	Location
face_dense_key_points.json	key_points folder
all_key_points.json	key_points folder

face_dense_key_points.json

Faces platformHumans in Context platform

This file contains the locations of 468 landmarks on the face of the generated human subject in the scene. These keypoints conform to the MediaPipe facial landmark standard.

The locations (left) and numbering (right) of the 468 facial landmarks developed by MediaPipe. Sources: [1] and [2].

The file has the following format:

{
    "version": "2.0.1",
    "face": {
        "standard": {
            // 468 keypoints defining the contours of the face
        }
    }
}

Objects and fields:

• version: String. Version tracking for this file. Whenever you access this file in a datapoint, make sure to check that the version matches what you expect it to be; otherwise its format and fields may not be recognized.

• standard: Object. This object contains 468 objects, each identifying a specific keypoint on the face. The keypoint objects are named “0” through “467”, and each one has the same format:

  "66": {
     "global_3d": {
        "x": 0.009309491142630577,
        "y": -0.043830640614032745,
        "z": 0.07293474674224854
     },
     "pixel_2d": {
        "x": 692,
        "y": 548
     },
     "is_visible": "true"
  }
  

  • global_3d: Object. Contains a set of three Floats giving the location of the keypoint in global coordinates: “x”, “y”, and “z”. See About our coordinate systems for details.

  • pixel_2d: Object. Contains a set of two Ints giving the x and y coordinates of the keypoint in the rendered image itself. See About our coordinate systems for details.

  • is_visible: Boolean. Indicates whether the keypoint is visible in the images produced by this camera. The value of this field is false if the keypoint is outside the frame or is on the other side of the body from the camera. Otherwise the value is true, even if there is another object between the keypoint and the camera.

Using this ground truth, you can train your model to recognize individual facial landmarks and trace the outline of important parts of the face. See https://github.com/DatagenTech/dgutils/blob/master/Notebooks/005_facial_keypoints_bounding%20_box.ipynb for some examples on how to work with facial landmarks.

all_key_points.json

For convenience, this file includes the same set of keypoints as face_dense_key_points.json above, located at this position in the hierarchy:

{
    "face": {
        "dense": {
            // 468 keypoints defining the contours of the face
        }
    }
}

For details on the structure of the keypoint objects, see face_dense_key_points.json above.