Publikation
Face It!: A Pipeline for Real-Time Performance-Driven Facial Animation
Jilliam Maria Diaz Barros; Vladislav Golyanik; Kiran Varanasi; Didier Stricker
In: IEEE (Hrsg.). 25th. IEEE International Conference on Image Processing (ICIP-2019), September 22-25, Taipei, Taiwan, Province of China, ISBN 978-1-5386-6249-6, IEEE, 2019.
Zusammenfassung
This paper presents a new lightweight approach for real-time performance-driven facial animation from monocular videos. We transfer facial expressions from 2D images to a 3D virtual character, by estimating the rigid head pose and non-rigid face deformation from detected and tracked 2D facial landmarks. We map the input face into the facial expression space of the 3D head model using blendshape models and formulate a lightweight energy-based optimization problem, which is solved by non-linear least squares at 18 FPS on a single CPU. Our method robustly handles varying head poses and different facial expressions, including moderately asymmetric ones. Compared to related methods, our approach does not require training data, specialised camera setups or graphics cards, and is suitable for embedded systems. We support our claims with several experiments.