#333:
J. K. Lewis,
I. E. Toubal,
H. Chen,
V. Sandesera,
M. Lomnitz,
Z. Hampel-Arias,
C. Prasad, and
K. Palaniappan
Applied Imagery Pattern Recognition Workshop (AIPR),
pgs. 1-9,
2020
Abstract,
Bibtex,
PlainText,
Software,
PDF,
URL,
DOI,
Google Scholar
Authentication of digital media has become an ever-pressing necessity for modern society. Since the introduction of Generative Adversarial Networks (GANs), synthetic media has become increasingly difficult to identify. Synthetic videos that contain altered faces and/or voices of a person are known as deepfakes and threaten trust and privacy in digital media. Deep-fakes can be weaponized for political advantage, slander, and to undermine the reputation of public figures. Despite imperfections of deepfakes, people struggle to distinguish between authentic and manipulated images and videos. Consequently, it is important to have automated systems that accurately and efficiently classify the validity of digital content. Many recent deepfake detection methods use single frames of video and focus on the spatial information in the image to infer the authenticity of the video. Some promising approaches exploit the temporal inconsistencies of manipulated videos; however, research primarily focuses on spatial features. We propose a hybrid deep learning approach that uses spatial, spectral, and temporal content that is coupled in a consistent way to differentiate real and fake videos. We show that the Discrete Cosine transform can improve deepfake detection by capturing spectral features of individual frames. In this work, we build a multimodal network that explores new features to detect deepfake videos, achieving 61.95% accuracy on the Facebook Deepfake Detection Challenge (DFDC) dataset.
@inproceedings{@INPROCEEDINGS{9425167, author={Lewis, John K. and Toubal, Imad Eddine and Chen, Helen and Sandesera, Vishal and Lomnitz, Michael and Hampel-Arias, Zigfried and Prasad, Calyam and Palaniappan, Kannappan}, booktitle={2020 IEEE Applied Imagery Pattern Recognition Workshop (AIPR)}, title={Deepfake Video Detection Based on Spatial, Spectral, and Temporal Inconsistencies Using Multimodal Deep Learning}, year={2020}, volume={}, number={}, pages={1-9}, doi={10.1109/AIPR50011.2020.9425167}},
author = "J. K. Lewis and I. E. Toubal and H. Chen and V. Sandesera and M. Lomnitz and Z. Hampel-Arias and C. Prasad and K. Palaniappan",
title = "Deepfake Video Detection Based on Spatial, Spectral, and Temporal Inconsistencies Using Multimodal Deep Learning",
year = 2020,
journal = "Applied Imagery Pattern Recognition Workshop (AIPR)",
publisher = "IEEE",
pages = "1-9",
month = "Oct",
keywords = "deepfake detection, deep learning, multi-modal, computer vision",
doi = "10.1109/AIPR50011.2020.9425167",
url = "https://ieeexplore.ieee.org/abstract/document/9425167"
}
J. K. Lewis, I. E. Toubal, H. Chen, V. Sandesera, M. Lomnitz, Z. Hampel-Arias, C. Prasad, and K. Palaniappan. Deepfake Video Detection Based on Spatial, Spectral, and Temporal Inconsistencies Using Multimodal Deep Learning. Applied Imagery Pattern Recognition Workshop (AIPR), IEEE, pages 1-9, October 2020.
