Hand keypoints#
Overview#
The ground truth exposed in this modality is a set of hand landmarks that conforms to the MediaPipe standard.
This modality consists of the following files:
Relevant file |
Location |
|---|---|
hands_key_points.json |
key_points folder |
all_key_points.json |
key_points folder |
hands_key_points.json#
This file contains the locations of 21 landmarks on each hand. These keypoints conform to the MediaPipe landmark standard.
The locations of the 21 hand landmarks developed by MediaPipe. Source: https://developers.google.com/mediapipe/solutions/vision/hand_landmarker#
The file has the following format:
{
"version": "2.0.1",
"hands": {
"left": {
"finger": {
"thumb": {
"ip": {
// Interphalangeal joint
},
"cmc": {
// Carpometacarpal joint
},
"tip": {
// Tip of the thumb
},
"mcp": {
// Metacarpophalangeal joint
}
},
"middle": {
"mcp": {
// Metacarpophalangeal joint
},
"pip": {
// Proximal Interphalangeal joint
},
"dip": {
// Distal Interphalangeal joint
},
"tip": {
// Tip of the finger
}
},
"ring": {
"mcp": {
// Metacarpophalangeal joint
},
"pip": {
// Proximal Interphalangeal joint
},
"dip": {
// Distal Interphalangeal joint
},
"tip": {
// Tip of the finger
}
},
"index": {
"mcp": {
// Metacarpophalangeal joint
},
"pip": {
// Proximal Interphalangeal joint
},
"dip": {
// Distal Interphalangeal joint
},
"tip": {
// Tip of the finger
}
},
"pinky": {
"mcp": {
// Metacarpophalangeal joint
},
"pip": {
// Proximal Interphalangeal joint
},
"dip": {
// Distal Interphalangeal joint
},
"tip": {
// Tip of the finger
}
}
},
"wrist": {
// Base of the wrist
}
},
"right": {
"finger": {
"thumb": {
"ip": {
// Interphalangeal joint
},
"cmc": {
// Carpometacarpal joint
},
"tip": {
// Tip of the thumb
},
"mcp": {
// Metacarpophalangeal joint
}
},
"middle": {
"mcp": {
// Metacarpophalangeal joint
},
"pip": {
// Proximal Interphalangeal joint
},
"dip": {
// Distal Interphalangeal joint
},
"tip": {
// Tip of the finger
}
},
"ring": {
"mcp": {
// Metacarpophalangeal joint
},
"pip": {
// Proximal Interphalangeal joint
},
"dip": {
// Distal Interphalangeal joint
},
"tip": {
// Tip of the finger
}
},
"index": {
"mcp": {
// Metacarpophalangeal joint
},
"pip": {
// Proximal Interphalangeal joint
},
"dip": {
// Distal Interphalangeal joint
},
"tip": {
// Tip of the finger
}
},
"pinky": {
"mcp": {
// Metacarpophalangeal joint
},
"pip": {
// Proximal Interphalangeal joint
},
"dip": {
// Distal Interphalangeal joint
},
"tip": {
// Tip of the finger
}
}
},
"wrist": {
// Base of the wrist
}
}
}
}
Objects and fields:
version: String. Version tracking for this file. Whenever you access this file in a datapoint, make sure to check that the version matches what you expect it to be; otherwise its format and fields may not be recognized.
left: Object. This object contains 2 objects, finger and wrist.
finger: Object. This object contains 5 keypoint objects, one for each finger on the hand. Each finger contains four keypoint objects, one called “tip” for the tip of the finger and one for each of the three joints.
In the thumb, these joints are:
ip: The interphalangeal. This is the joint in the middle of the thumb.
mcp: The metacarpophalangeal. This is the joint that connects the thumb to the palm.
cmc: The carpometacarpal. This is the ball joint at the base of the thumb, inside the palm, that enables the thumb to move in any direction.
In the other fingers, these joints are:
mcp: The metacarpophalangeal. This is the joint that forms the knuckle of each finger.
pip: The proximal interphalangeal. Of the two joints inside the finger itself, this is the one closer to the wrist.
dip: The distal interphalangeal. Of the two joints inside the finger itself, this is the one closer to the tip.
wrist: Object. A single keypoint object.
Each keypoint object (four for each of the five fingers, plus one in the wrist) has the same format:
"tip": { "global_3d": { "x": 0.0649380311369896, "y": 0.04291728511452675, "z": 0.11093330383300781 }, "pixel_2d": { "x": 91.42204655475659, "y": 31.024237959018187 }, "is_visible": "true" },
global_3d: Object. Contains a set of three Floats giving the location of the keypoint in global coordinates: “x”, “y”, and “z”. See About our coordinate systems for details.
pixel_2d: Object. Contains a set of two Floats giving the x and y coordinates of the keypoint in the rendered image itself. This provides not only which pixel contains the keypoint but also precisely where the keypoint is inside the pixel. See About our coordinate systems for details.
is_visible: Boolean. Indicates whether the keypoint is visible in the images produced by this camera. The value of this field is false if the keypoint is outside the frame or is on the other side of the body from the camera. Otherwise the value is true, even if there is another object between the keypoint and the camera.
right: Object. This object has the same structure as “left” above, except it identifies the 21 keypoints in the right hand.
Using this ground truth, you can train your model to recognize the shape and location of the human hand.
This file contains the locations of 21 landmarks on each hand. These keypoints conform to the MediaPipe landmark standard.
The locations of the 21 hand landmarks developed by MediaPipe. Source: https://developers.google.com/mediapipe/solutions/vision/hand_landmarker#
The file has the following format:
{
"version": "2.0.1",
"hands": {
"left": {
"finger": {
"thumb": {
"ip": {
// Interphalangeal joint
},
"cmc": {
// Carpometacarpal joint
},
"tip": {
// Tip of the thumb
},
"mcp": {
// Metacarpophalangeal joint
}
},
"middle": {
"mcp": {
// Metacarpophalangeal joint
},
"pip": {
// Proximal Interphalangeal joint
},
"dip": {
// Distal Interphalangeal joint
},
"tip": {
// Tip of the finger
}
},
"ring": {
"mcp": {
// Metacarpophalangeal joint
},
"pip": {
// Proximal Interphalangeal joint
},
"dip": {
// Distal Interphalangeal joint
},
"tip": {
// Tip of the finger
}
},
"index": {
"mcp": {
// Metacarpophalangeal joint
},
"pip": {
// Proximal Interphalangeal joint
},
"dip": {
// Distal Interphalangeal joint
},
"tip": {
// Tip of the finger
}
},
"pinky": {
"mcp": {
// Metacarpophalangeal joint
},
"pip": {
// Proximal Interphalangeal joint
},
"dip": {
// Distal Interphalangeal joint
},
"tip": {
// Tip of the finger
}
}
},
"wrist": {
// Base of the wrist
}
},
"right": {
"finger": {
"thumb": {
"ip": {
// Interphalangeal joint
},
"cmc": {
// Carpometacarpal joint
},
"tip": {
// Tip of the thumb
},
"mcp": {
// Metacarpophalangeal joint
}
},
"middle": {
"mcp": {
// Metacarpophalangeal joint
},
"pip": {
// Proximal Interphalangeal joint
},
"dip": {
// Distal Interphalangeal joint
},
"tip": {
// Tip of the finger
}
},
"ring": {
"mcp": {
// Metacarpophalangeal joint
},
"pip": {
// Proximal Interphalangeal joint
},
"dip": {
// Distal Interphalangeal joint
},
"tip": {
// Tip of the finger
}
},
"index": {
"mcp": {
// Metacarpophalangeal joint
},
"pip": {
// Proximal Interphalangeal joint
},
"dip": {
// Distal Interphalangeal joint
},
"tip": {
// Tip of the finger
}
},
"pinky": {
"mcp": {
// Metacarpophalangeal joint
},
"pip": {
// Proximal Interphalangeal joint
},
"dip": {
// Distal Interphalangeal joint
},
"tip": {
// Tip of the finger
}
}
},
"wrist": {
// Base of the wrist
}
}
}
}
Objects and fields:
version: String. Version tracking for this file. Whenever you access this file in a datapoint, make sure to check that the version matches what you expect it to be; otherwise its format and fields may not be recognized.
left: Object. This object contains 2 objects, finger and wrist.
finger: Object. This object contains 5 keypoint objects, one for each finger on the hand. Each finger contains four keypoint objects, one called “tip” for the tip of the finger and one for each of the three joints.
In the thumb, these joints are:
ip: The interphalangeal. This is the joint in the middle of the thumb.
mcp: The metacarpophalangeal. This is the joint that connects the thumb to the palm.
cmc: The carpometacarpal. This is the ball joint at the base of the thumb, inside the palm, that enables the thumb to move in any direction.
In the other fingers, these joints are:
mcp: The metacarpophalangeal. This is the joint that forms the knuckle of each finger.
pip: The proximal interphalangeal. Of the two joints inside the finger itself, this is the one closer to the wrist.
dip: The distal interphalangeal. Of the two joints inside the finger itself, this is the one closer to the tip.
wrist: Object. A single keypoint object.
Each keypoint object (four for each of the five fingers, plus one in the wrist) has the same format:
"tip": { "global_3d": { "x": 0.0649380311369896, "y": 0.04291728511452675, "z": 0.11093330383300781 }, "pixel_2d": { "x": 91.42204655475659, "y": 31.024237959018187 }, },
global_3d: Object. Contains a set of three Floats giving the location of the keypoint in global coordinates: “x”, “y”, and “z”. See About our coordinate systems for details.
pixel_2d: Object. Contains a set of two Floats giving the x and y coordinates of the keypoint in the rendered image itself. This provides not only which pixel contains the keypoint but also precisely where the keypoint is inside the pixel. See About our coordinate systems for details.
right: Object. This object has the same structure as “left” above, except it identifies the 21 keypoints in the right hand.
Using this ground truth, you can train your model to recognize the shape and location of the human hand.
all_key_points.json#
For convenience, this file includes the same set of keypoints as hands_key_points.json above, located at this position in the hierarchy:
{
"hands": {
"left": {
// 21 keypoint objects organized by finger and wrist
}
"right": {
// 21 keypoint objects organized by finger and wrist
}
}
}
For details on the structure of the keypoint objects, see hands_key_points.json above.
Note
all_key_points.json also includes additional hand keypoints that represent part of the Body keypoints modality.