Scene parsing

SegFormer-B (ADE20K)

Tag every pixel with one of 150 ADE20K scene classes.

NVIDIA Source Code License (non-commercial)

Explore model

About this model

SegFormer-B is the base-size variant of NVlabs SegFormer, a transformer encoder paired with an MLP decoder. Trained on ADE20K's 150 scene-parsing classes covering buildings, sky, road, vehicles, plants, and indoor objects. The build is a 15 MB float ONNX.

On a 321x400 input, M1 Max with CoreML runs warm in 107 ms. The training license is the NVIDIA Source Code License, which restricts commercial use; review it before shipping.

What you can build

Build a scene parser for street photography or driving footage.
Generate semantic overlays in a photo annotation tool.
Segment outdoor scenes into sky, road, vegetation, and buildings.
Pre-label data before manual review for ADE20K-style tasks.

Variants & downloads

Choose the build for your app

Model files are being prepared.

Explore the model’s examples, source links, and integration notes while download variants are added.

Example inference

A road scene with a stop sign, traffic light, foliage, and sky. — Input

The same scene with a colored mask separating sky, road, vegetation, and signs. — Output

Performance

Results depend on the device, runtime, and test conditions. Each measurement keeps its own context.

M1 Max, macOS 15, ONNX Runtime 1.17

See original catalog hardware notes · CPU

288.9 ms

Warm inference

Catalog reference · variant unspecifiedCold start (ms): 289.97

Reference result from the existing catalog; conversion variant and measurement date were not recorded.

M1 Max, macOS 15, ONNX Runtime 1.17

See original catalog hardware notes · CoreML

107.2 ms

Warm inference

Catalog reference · variant unspecifiedCold start (ms): 260.38

Reference result from the existing catalog; conversion variant and measurement date were not recorded.

RTX 2060, Ubuntu 24.04, ONNX Runtime 1.24.3

See original catalog hardware notes · CUDA

50.1 ms

Warm inference

Catalog reference · variant unspecified

Reference result from the existing catalog; conversion variant and measurement date were not recorded.

Integration examples

Vision SDK integration

cpp

#include "imaged/sdk.hpp"
#include "imaged/families/semantic_segmentation/semantic_segmentation_family.hpp"

imaged::SDK::AI ai;
auto& config = ai.getOptions();
config.set("semantic_segmentation.model", "weights/segformer_base.onnx");
config.set("semantic_segmentation.normalize", "imagenet");

imaged::SDK::SemanticSegmentationFamily family;
family.load(config);

imaged::SDK::Image image;
image.load("scene.png");

auto result = std::get<imaged::SDK::SegmentationResult>(family.run({&image}));
// result.mask is W*H bytes, each pixel a class index 0..149