Semantic segmentation#
Overview#
The ground truth exposed in this modality is the parent body part of each pixel visible in the corresponding visual spectrum image.
This modality consists of the following files:
Relevant file |
Location |
---|---|
semantic_segmentation.png |
Camera folder |
semantic_segmentation_metadata.json |
Datapoint folder |
Relevant file |
Location |
---|---|
semantic_segmentation.png |
Camera folder |
semantic_segmentation_metadata.json |
Scene folder |
semantic_segmentation.png#
This file contains a version of the visual spectrum image that has been converted into a semantic segmentation map.
A semantic segmentation map of a human head (left) and the corresponding visual spectrum image (right)
In a semantic segmentation map, the color value of each pixel has been replaced with a color value that indicates which class it belongs to. The color key is found in semantic_segmentation_metadata.json.
Using this ground truth, you can train your model to recognize individual parts of the face.
To process a semantic segmentation map, we recommend using code along the following lines:
import cv2
def load_segmentation(path):
segmentation = cv2.cvtColor(cv2.imread(segmentation_path, cv2.IMREAD_UNCHANGED), cv2.COLOR_BGR2RGB)
return segmentation
path = "semantic_segmentation.png"
segmentation = load_segmentation(path)
We strongly recommend against using PIL
or plt.imread
to load the semantic segmentation file, because the uint16 color will be reduced to uint8 and the color key will no longer be accurate.
See https://github.com/DatagenTech/dgutils/blob/master/Notebooks/003_semantic_segmentation.ipynb for more about how to use semantic segmentation to isolate objects in the datapoint.
semantic_segmentation_metadata.json#
This file contains a lookup table of the RGB values that were assigned to each semantic object in our database. All datasets use the same lookup table, though each individual instance of this file is filtered to list only items present in the scene. For example, if none of the actors in the scene have a beard, the “beard” item will not appear in the file.
The lookup table is constantly being added to and adjusted as we introduce more features into our platform, so we recommend using whichever version accompanied the datapoint you are working with.
At the top of this file is a single field called version, a String that provides version tracking for this file. Whenever you access this file in a datapoint, make sure to check that the version matches what you expect it to be; otherwise its format and fields may not be recognized.
Click here
to view an example of a complete semantic_segmentation_metadata.json file.