Body keypoints#

Overview#

The ground truth exposed in this modality is a set of body landmarks that are inspired by the MediaPipe and OpenPose standards.

This modality consists of the following files:

Relevant file

Location

body_key_points.json

key_points folder

all_key_points.json

key_points folder

body_key_points.json#

This file contains the locations of 27 landmarks on each body using a standard developed by Datagen, which is inspired by the OpenPose and MediaPipe standards.

The file has the following format:

{
   "version": "2.0.1",
   "body": {
      "heel": {
         "right": {
            // Right heel keypoint
         },
         "left": {
            // Left heel keypoint
         }
      },
      "chest": {
         // Four keypoints describing the extent of the chest: right lower, right upper, left lower, and left upper
      },
      "neck": {
         // Neck keypoint
      },
      "elbow": {
         "right": {
            // Right elbow keypoint
         },
         "left": {
            // Left elbow keypoint
         }
      },
      "wrist": {
         "right": {
            // Right wrist keypoint
         },
         "left": {
            // Left wrist keypoint
         }
      },
      "shoulder": {
         "left": {
            // Left shoulder keypoint
         },
         "right": {
            // Right shoulder keypoint
         }
      },
      "hip": {
         "right": {
            // Right hip keypoint
         },
         "left": {
            // Left hip keypoint
         }
      },
      "pinky": {
         "left": {
            // Left pinky keypoint
         },
         "right": {
            // Right pinky keypoint
         }
      },
      "knee": {
         "right": {
            // Right knee keypoint
         },
         "left": {
            // Left knee keypoint
         }
      },
      "foot": {
         "right": {
            "index": {
               // Keypoint at the edge of the right index toe
            }
         },
         "left": {
            "index": {
               // Keypoint at the edge of the left index toe
            }
         }
      },
      "thumb": {
         "right": {
            // Right thumb keypoint
         },
         "left": {
            // Left thumb keypoint
         }
      },
      "index": {
         "left": {
            // Left index finger keypoint
         },
         "right": {
            // Right index finger keypoint
         }
      },
      "ankle": {
         "right": {
            // Right ankle keypoint
         },
         "left": {
            // Left ankle keypoint
         }
      }
   }
}

Objects and fields:

  • version: String. Version tracking for this file. Whenever you access this file in a datapoint, make sure to check that the version matches what you expect it to be; otherwise its format and fields may not be recognized.

  • body: Object. Contains the set of body keypoints as developed by Datagen.

    Each keypoint has the same format:

    "left": {
    "global_3d": {
        "x": 0.8927506854136785,
        "y": 1.0507536431153615,
        "z": 0.7512611134184731
    },
    "pixel_2d": {
        "x": 585.5965585624375,
        "y": -42.56966529034368
    }
   "is_visible": "true"
},

    • global_3d: Object. Contains a set of three Floats giving the location of the keypoint in global coordinates: “x”, “y”, and “z”. See About our coordinate systems for details.

    • pixel_2d: Object. Contains a set of two Floats giving the x and y coordinates of the keypoint in the rendered image itself. This provides not only which pixel contains the keypoint but also precisely where the keypoint is inside the pixel. See About our coordinate systems for details.

    • is_visible: Boolean. Indicates whether the keypoint is visible in the images produced by this camera. The value of this field is false if the keypoint is outside the frame or is on the other side of the body from the camera. Otherwise the value is true, even if there is another object between the keypoint and the camera.

Using this ground truth, you can train your model to recognize the shape and pose of the human body.

all_key_points.json#

For convenience, this file includes the same set of keypoints as body_key_points.json above, located at this position in the hierarchy:

{
    "body": {
        //The full set of body keypoints according to the hierarchy of the "body" object above
    }
}

For details on the structure of the keypoint objects, see body_key_points.json above.