"Face hacking" might not have an artistic ring to it, but it looks stunning in execution. A collaboration between a multimedia artist, a makeup artist and a digital image engineer—Nobumichi Asai, Horoto Kuwahara and Paul Lacroix, respectively—has yielded Project Omote, which combines face-tracking technology with projection mapping to change people's faces in real-time. Here's the inaugural effort, developed last year and using model Yuka Sekimizu:
Enter a caption (optional)
"Technically, it's very difficult to project with real-time tracking," writes Asai.
Nobody has done it before, so we had to start from scratch. The process was complicated, capturing with sensors, processing the data, creating CG renderings, and using projectors. Getting the timing right was also challenging, fixing delays and making sure the projection technology had the maximum precision to get the face and visuals matched.
…We used OptiTrack sensors for [their] speedy response, and developed original programs [in] C++. To [minimize] latency (delay), we devised ways to integrate the whole process into one program. This creates amazing effects.
This year the team pushed the effects further, using members of the J-pop band SMAP: