Нейросеть (2018): фотореалистичная реанимация портретов

Подробнее We present a novel approach that enables photorealistic reanimation of portrait videos using only an input video. In contrast to existing approaches that are restricted to manipulations of facial expressions only, we are the first to transfer the full 3D head position, head rotation, face expression, eye gaze, and eye blinking from a source actor to a portrait video of a target actor. The core of our approach is a generative neural network with a novel spacetime architecture. The network takes as input synthetic renderings of a parametric face model, based on which it predicts photorealistic video frames for a given target actor. The realism in this renderingtovideo transfer is achieved by careful adversarial training, and as a result, we can create modified target videos that mimic the behavior of the syntheticallycreated input. In order to enable sourcetotarget video reanimation, we render a synthetic target video with the reconstructed hea