利用卷积神经网络和小面元进行人脸图像替换

Face Swapping Using Convolutional Neural Network and Tiny Facet Primitive

  • 摘要: 提出了一种基于卷积神经网络和小面元的人脸替换方法。首先,使用级联卷积神经网络确定人脸位置,并应用全卷积网络对人脸进行分割以确定替换区域;然后,应用回归森林的人脸关键点技术和Delaunay三角剖分建立人脸三角网,对人脸小面元进行替换;最后, 应用泊松融合消除拼接痕迹。定性和定量的试验结果表明,该方法是有效的,并且可较好地解决人脸姿态一致性限制和人脸遮挡的问题。

     

    Abstract:
      Objectives  Face swapping technology has important application value in entertainment, virtual reality, film and so on. However, existing methods are limited by face pose consistency and can not overcome the influence of occlusion.
      Methods  We proposes a method of face swapping using convolutional neural network and tiny facet primitive. Firstly, detect the face using the cascade convolutional neural network and segment the face to determine the replacement region using fully convolutional network.Then, the Wallis transform is applied to adjust the skin color of the source image to make it consistent with the skin color of the face in the target image.After that, using facial key points detection method based on an ensemble of regression trees and Delaunay triangulation to construct the face triangulation network, then replacing faces based on tiny facet primitive. Finally, applying Poisson fusion to eliminate splicing traces between different images.
      Results  We evaluate the performance of the proposed method compared with existing method through qualitative and quantitative experiments. Experimental results show that face segment can well solve the problem of occlusion such as cap, glasses, and hair.Moreover, when source image and target image have different face poses, replacing face area using tiny facet primitive separately performs better than using the whole face area.
      Conclusions  Our method can well solve the problem of face pose consistency limitation and occlusion, which has certain practical application value.

     

/

返回文章
返回