Face replace theatre application

Hi all,

Greatly inspired by the face morphing video a while ago from Kyle McDonald and Arturo Castro, we built a project to replace faces in realtime for a theatre production (http://www.toneelhuis.be/productie.jsp?id=439&lang=en).

We built it using openframeworks, Jason Saragih’s face tracker, and Kyle McDonald and Arturo’s face morphing code, and alot of input from the forums here :slight_smile:

We had some technical challenges; mostly involving cpu constraints (multithreading and openMP), project stability, quality, usability for theatre guys and “cue-based controlling”. Also, it had to be an insert effect in their current setup (pal input/output). To leave as little as possible to chance, a pre-processor was made to analyse video’s and images beforehand, and adjust the mask or create a new one. We also developed new shaders for increased effect control and accuracy around the morphed edges (the-blur-&-clone-shaders), plus some settings like keying the face, replace half the face, effect fade-ins/outs, a “backup path” in case something goes wrong, etc.

Some basic info and screens here: http://www.hangaar.net/#face-replace

A movie of the thing in action during a rehearsal:

Making an actor younger, as viewed on the feedback screen for the technical staff

Making an actress older, as viewed on the feedback screen:

The demo controller software, written in max msp and using osc to control the effect:

Manual adjusting some face points:

The shader enhancement, notice eyes, mouth and face mask edges:

And the face tracker source from Jason: http://web.mac.com/jsaragih/FaceTracker/FaceTracker.html

Our thanks to the OpenFrameworks community, Jason Saragih, Kyle McDonald, Arturo C and Pieter Coussement!