Deepfake Video Detection Using 3DMM Facial Reconstruction Information

Objectives The emergence of deepfake technique leads to a worldwide information security problem. Deepfake videos are used to manipulate and mislead the public. Though there have been a variety of deepfake detection methods, the features extracted generally suffer from poor interpretability. To solve this problem, a deepfake video detection method using 3D morphable model (3DMM) of face is proposed.

Methods The 3DMM is employed to estimate parameters of shape, texture, expression, and gesture of the face frame by frame, constituting the basic information of deepfake detection. The facial behavior feature extraction module and the static face appearance feature extraction module are designed for the construction of feature vectors on a sliding window basis. The facial behavior feature vector is derived from the expression and gesture parameters while the appearance feature vector is calculated with the shape and texture parameters. The consistency measured by cosine distance between the appearance feature vector and the behavior feature vector is the criterion for authentication of the face for each sliding window across the video.

Results The effectiveness of the proposed method is evaluated with three public datasets. The overall half total error rates (HTER) obtained on FF++, DFD and Celeb-DF dataset are 1.33%, 4.93% and 3.92% respectively. For the severely compressed videos, C40 of DFD, the HTER is 7.09%, showing a good robustness against video compression. The model complexity is around 1/4 of that of the most related work.

Conclusions The proposed algorithm has good person pertinence and clear interpret‍ability. Compared with state-of-the-art methods in literature, the proposed algorithm demonstrates lower half total error rates, better resistance to video compression and less computational cost.