Face Swapping Using Convolutional Neural Network and Tiny Facet Primitive

  Objectives  Face swapping technology has important application value in entertainment, virtual reality, film and so on. However, existing methods are limited by face pose consistency and can not overcome the influence of occlusion.

  Methods  We proposes a method of face swapping using convolutional neural network and tiny facet primitive. Firstly, detect the face using the cascade convolutional neural network and segment the face to determine the replacement region using fully convolutional network.Then, the Wallis transform is applied to adjust the skin color of the source image to make it consistent with the skin color of the face in the target image.After that, using facial key points detection method based on an ensemble of regression trees and Delaunay triangulation to construct the face triangulation network, then replacing faces based on tiny facet primitive. Finally, applying Poisson fusion to eliminate splicing traces between different images.

  Results  We evaluate the performance of the proposed method compared with existing method through qualitative and quantitative experiments. Experimental results show that face segment can well solve the problem of occlusion such as cap, glasses, and hair.Moreover, when source image and target image have different face poses, replacing face area using tiny facet primitive separately performs better than using the whole face area.

  Conclusions  Our method can well solve the problem of face pose consistency limitation and occlusion, which has certain practical application value.