ofxAddons
← Back

ofxFaceTracker

github.com/kylemcdonald/ofxFaceTracker · @kylemcdonald
OF 0.11 Releases Computer Vision

CLM face tracking addon for openFrameworks based on Jason Saragih's FaceTracker.

1398 361 Last commit 5y ago Created 15y ago

Forks

Forks pushed to more recently than kylemcdonald/ofxFaceTracker itself, plus any admin-confirmed fork with commits of its own — worth checking if you're hitting issues with the original.

fred-dev/ofxFaceTracker by @fred-dev · ★ 0 · pushed 18d ago

Branches ahead of master

These branches have commits not yet on master — could be unmerged fixes or work in progress worth a look.

opencv4 1 commit ahead · last commit 5y ago
osc-syphon 6 commits ahead · last commit 10y ago

README

ofxFaceTracker is openFrameworks addon for face tracking, based on Jason Saragih's FaceTracker library.



This addon breaks on OF 0.11.0 due OpenCV 4. Consider using ofxFaceTracker2

openFrameworks is an open source toolkit for creative coding.

All ofxFaceTracker code is available under the MIT license, while FaceTracker is provided free for non-commercial use. For commercial use of FaceTracker, please request a quote.

Installation



Required addons: First, download ofxCv.

Then, you need to make a copy of the libs/FaceTracker/model/ directory in example-/bin/data/model/ of each example. You can do this by hand, or python setup.py will take care of this for you.

Then you can generate project files with the OF project generator and run the examples. If you see the error Assertion failed: s.is_open() when running your app, that means you forgot to drop the model files in the right directory.

If you would like to prototype an idea involving face tracking, I encourage you to download FaceOSC. Dan Wilcox has some great FaceOSC templates that will help you get started in environments like Max, pd, Processing, and openFrameworks.

If you're interested in using ofxFaceTracker for face substitution, check out the FaceSubstitution repository.

Examples



iOS



Getting the combination of ofxFaceTracker and ofxCv working on iOS can be complicated and confusing. Try using this ofxFaceTracker-iOS repo as a starting point. Scott Hawley has also made a FaceOSC iOS port to interface with Wekinator that may helpful.

Advanced



Demonstrates how to get the image-space position of the face, and the 3d orientation (collectively, the face "pose") then applies these to the OpenGL context in order to draw an oriented face mesh.

Blink



An advanced example that shows how you might detect blinking from the extracted eye images. This is not necessarily a robust classifier for whether eyes are open or closed, but works well for blinks as events.

Calibrated



Fairly complicated example that demonstrates some advanced/experimental features of ofxCv and ofxFaceTracker for doing AR-style augmentation of the face.

Empty



Provides a minimal example of using an ofxVideoGrabber with an ofxFaceTracker, then extracts and draws the ofPolyline representing the left and right eyes.

Expression



Demonstrates the ExpressionClassifier, which can load, save, and classify expressions into pre-trained categories. Provides basic expressions (eyebrows raised, neutral, smiling) as examples.

Extraction



Demonstrates how to use the mean face mesh to draw pose and expression normalized representation of the face currently being tracked.

FaceOSC



FaceOSC sends as much data as possible from ofxFaceTracker via OSC, including its pose/position and gestural data, and its raw points. It also streams the image over Syphon.

You can download a pre-built binary in the releases page.

OSC information



 Pose center position: /pose/position scale: /pose/scale orientation (which direction you're facing): /pose/orientation Gestures mouth width: /gesture/mouth/width mouth height: /gesture/mouth/height left eyebrow height: /gesture/eyebrow/left right eyebrow height: /gesture/eyebrow/right left eye openness: /gesture/eye/left right eye openness: /gesture/eye/right jaw openness: /gesture/jaw nostril flate: /gesture/nostrils Raw raw points (66 xy-pairs): /raw

Get this addon

HTTPS
SSH
View on GitHub · Open with Github Desktop · Download ZIP · All releases

Latest release

v1.2 FaceOSC published 6y ago
FaceOSC is tested with MacOS 10.15

FaceOSC will track a face and send its pose and gesture data over OSC, as well as the raw tracked points (when selected in the GUI).

A previous version of FaceOSC for MacOS supported streaming of Syphon video, but this feature has been removed.

FaceOSC is developed with openFrameworks and ofxFaceTracker, built on top of Jason Saragih's FaceTracker. The windows build of FaceOSC was prepared by Dan Moore on the CreativeInquiry fork of ofxFaceTracker.

OSC data



By default, sends to localhost, port 8338. If you want to change this, see the below section about editing the settings.xml file.
  • Pose
  • center position: /pose/position
  • scale: /pose/scale
  • orientation (which direction you're facing): /pose/orientation
  • Gestures
  • mouth width: /gesture/mouth/width
  • mouth height: /gesture/mouth/height
  • left eyebrow height: /gesture/eyebrow/left
  • right eyebrow height: /gesture/eyebrow/right
  • left eye openness: /gesture/eye/left
  • right eye openness: /gesture/eye/right
  • jaw openness: /gesture/jaw
  • nostril flate: /gesture/nostrils
  • Raw
  • raw points (66 xy-pairs): /raw

Keyboard controls

  • r - reset the face tracker
  • m - toggle face mesh drawing
  • g - toggle gui's visibility
  • p - pause/unpause (only works with movie source)
  • up/down - increase/decrease movie playback speed (only works with movie source)

Gui controls



"pose" and "gesture" toggle whether to send the pose and gestural stats over OSC, respectively. "raw" toggles whether to send the raw points of the face mesh (66 XY-points = 132 values).

Settings



To change the default settings of FaceOSC, inc