ofxYolo26

<img src="ofxaddons_thumbnail.png">


Ultralytics YOLO26 ONNX models in openFrameworks.

A C++ port of the inference half of yolo-touchdesigner. It runs the same .onnx exports, but through onnxruntime via ofxOnnxRuntime instead of onnxruntime-web/WebGPU in a browser. The reference project's server half — the Web Server DAT, the WebSocket transport, the binary framing protocol — is deliberately not ported; in openFrameworks you already have the pixels.

Implemented

task	class	model	example
object detection	ofxYolo26::Detector	yolo26n.onnx	example-detect
oriented boxes (OBB)	ofxYolo26::Obb	yolo26n-obb.onnx	example-obb
monocular depth	ofxYolo26::Depth	yolo26n-depth.onnx	example-depth
multi-person pose	ofxYolo26::Pose	yolo26n-pose.onnx	example-pose
instance segmentation	ofxYolo26::Segmentation	yolo26n-seg.onnx	example-seg

Dependencies

• ofxOnnxRuntime, with its

bundled onnxruntime replaced by 1.29.0 — see Upgrading onnxruntime below.

Platforms

platform	backend	build
macOS 14+, Apple Silicon	CoreML (ANE / GPU / CPU)	Xcode or make
Linux aarch64 (Raspberry Pi OS Bookworm 64-bit)	CPU	make

macOS: the onnxruntime 1.29.0 macOS build is arm64 only and requires macOS 14+. Apple Silicon, in other words. If you need Intel or an older macOS, stay on the 1.10.0 that ofxOnnxRuntime ships (universal, CPU-only) — the addon still builds against it apart from the two API calls noted below.

Linux aarch64: build any example with make Release, then run it from bin/, which is where the makefiles put libonnxruntime.so and where the default rpath looks:

cd example-detect
make Release
cd bin && ./example-detect

Nothing else to configure — ofxOnnxRuntime/libs/onnxruntime/lib/linuxaarch64/ already carries the aarch64 .so, and the addon compiles its CoreML paths out on any non-Apple target. The stock aarch64 onnxruntime is CPU-only, so inference runs an order of magnitude slower than an accelerated Mac; see the performance table in ARCHITECTURE.md before planning a live video pipeline around it.

Setup

Copy the model your project needs into its data folder:

bin/data/models/yolo26n.onnx
bin/data/models/yolo26n-obb.onnx
bin/data/models/yolo26n-depth.onnx
bin/data/models/yolo26n-pose.onnx
bin/data/models/yolo26n-seg.onnx

Each example expects its model at that path. Models live in the reference project under reference/yolo-touchdesigner/public/models/.

Usage

Every task follows the same shape: setup() with a model path, update() with pixels, read a result object back.

ofxYolo26::Pose pose;
pose.setup("models/yolo26n-pose.onnx");

const ofxYolo26::PoseResult & result = pose.update(pixels);
for (size_t i = 0; i < result.size(); i++) {
    ofRectangle box = result.getBoxInSource(i);
    glm::vec2 nose = result.getKeypointInSource(i, ofxYolo26::KP_NOSE);
}

Live video, with inference on a background thread — DepthThread, PoseThread and SegmentationThread all work this way:

// setup()
poseThread.setup("models/yolo26n-pose.onnx");
poseThread.start();

// update()
if (grabber.isFrameNew()) poseThread.setInput(grabber.getPixels());
if (poseThread.isFrameNew()) { / poseThread.getPoseResult() / }

// draw()
ofxYolo26::drawPoses(poseThread.getPoseResult(), frameRect);

Frames handed to setInput() while the worker is busy are dropped rather than queued, so results stay as close to live as the model allows.

Examples

All five take space to switch between a bundled still and the webcam, d to dump results to the console, and h to toggle the help line.

• example-detect — boxes and labels. -/= score threshold, f cycles

all / people / vehicles, l labels. Drop an image on the window to use it.

• example-obb — rotated boxes with a heading spur. -/= score

threshold, l labels, o heading. Drop an aerial image on the window; the bundled street photo is a stand-in and will show little or nothing, which the footer says explicitly.

• example-depth — depth map beside the frame, mouse probe reading raw depth

in source coordinates. c colour ramp, i invert, x show/crop padding.

• example-pose — skeletons over the frame. -/= score threshold,

[/] keypoint threshold, b boxes, l labels.

• example-seg — v cycles overlay / mask / cutout, -/= score

threshold, c colour by class or instance, p person-only filter, b boxes. The mouse reports which instance owns the pixel under it.

Architecture

See ARCHITECTURE.md for technical details on backends, coordinate systems, code structure, output formats, and performance.

License

AGPL-3.0, see LICENSE.txt — matching yolo-touchdesigner, which this ports from, and the Ultralytics YOLO models themselves.