Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3531536.3532946acmconferencesArticle/Chapter ViewAbstractPublication Pagesih-n-mmsecConference Proceedingsconference-collections
research-article
Open access

FMFCC-V: An Asian Large-Scale Challenging Dataset for DeepFake Detection

Published: 23 June 2022 Publication History

Abstract

The abuse of DeepFake technique has raised enormous public concerns in recent years. Currently, the existing DeepFake datasets suffer some weaknesses of obvious visual artifacts, minimal Asian proportion, backward synthesis methods and short video length. To make up these weaknesses, we have constructed an Asian large-scale challenging DeepFake dataset to enable the training of DeepFake detection models and organized the accompanying video track of the first Fake Media Forensics Challenge of China Society of Image and Graphics (FMFCC-V). The FMFCC-V dataset is by far the first and the largest public available Asian dataset for DeepFake detection, which contains 38102 DeepFake videos and 44290 pristine videos, corresponding more than 23 million frames. The source videos in the FMFCC-V dataset are carefully collected from 83 paid individuals and all of them are Asians. The DeepFake videos are generated by four of the most popular face swapping methods. Extensive perturbations are applied to obtain a more challenging benchmark of higher diversity. The FMFCC-V dataset can lend powerful support to the development of more effective DeepFake detection methods. We contribute a comprehensive evaluation of six representative DeepFake detection methods to demonstrate the level of challenge posed by FMFCC-V dataset. Meanwhile, we provide a detailed analysis of the top submissions from the FMFCC-V competition.

References

[1]
2019. faceswap-GAN. Retrieved October 4, 2019 from https://github.com/shaoanlu/faceswap-GAN
[2]
2020. CapsuleNet. Retrieved December 15, 2020 from https://github.com/nii-yamagishilab/Capsule-Forensics-v2
[3]
2020. Deepfake Detection Challenge. Retrieved March 3, 2020 from https://www.kaggle.com/c/deepfake-detection-challenge
[4]
2020. DFaker. Retrieved June 19, 2020 from https://github.com/dfaker/df
[5]
2020. DSP-FWA. Retrieved June 25, 2020 from https://github.com/yuezunli/DSP-FWA
[6]
2020. Mixup-DA. Retrieved June 9, 2020 from https://github.com/NTech-Lab/deepfake-detection-challenge
[7]
2020. WS-DA. Retrieved June 14, 2020 from https://github.com/cuihaoleo/kaggle-dfdc
[8]
2020. XceptionNet. Retrieved July 15, 2020 from https://github.com/ondyari/FaceForensics
[9]
2021. DeepFaceLab. Retrieved November 20, 2021 from https://github.com/iperov/DeepFaceLab
[10]
2021. faceswap. Retrieved December 5, 2021 from https://github.com/deepfakes/faceswap
[11]
2021. FakeApp. Retrieved October 2, 2021 from https://www.malavida.com/en/soft/fakeapp/
[12]
2021. Heavy-DA. Retrieved November 9, 2021 from https://github.com/selimsef/dfdc_deepfake_challenge
[13]
Darius Afchar, Vincent Nozick, Junichi Yamagishi, and Isao Echizen. 2018. MesoNet: a Compact Facial Video Forgery Detection Network. In 2018 IEEE International Workshop on Information Forensics and Security. IEEE, 1--7. https://doi.org/10.1109/WIFS.2018.8630761
[14]
Shruti Agarwal, Hany Farid, Yuming Gu, Mingming He, Koki Nagano, and Hao Li. 2019. Protecting World Leaders Against Deep Fakes. In IEEE Conference on Computer Vision and Pattern Recognition Workshops. Computer Vision Foundation / IEEE, 38--45.
[15]
Irene Amerini, Leonardo Galteri, Roberto Caldelli, and Alberto Del Bimbo. 2019. Deepfake Video Detection through Optical Flow Based CNN. In 2019 IEEE/CVF International Conference on Computer Vision Workshops. IEEE, 1205--1207. https://doi.org/10.1109/ICCVW.2019.00152
[16]
Aayush Bansal, Shugao Ma, Deva Ramanan, and Yaser Sheikh. 2018. Recycle-GAN: Unsupervised Video Retargeting. In Computer Vision - ECCV 2018 - 15th European Conference (Lecture Notes in Computer Science, Vol. 11209), Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss (Eds.). Springer, 122--138. https://doi.org/10.1007/978--3-030-01228--1_8
[17]
François Chollet. 2017. Xception: Deep Learning with Depthwise Separable Convolutions. In 2017 IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, 1800--1807. https://doi.org/10.1109/CVPR.2017.195
[18]
Jiankang Deng, Jia Guo, Yuxiang Zhou, Jinke Yu, Irene Kotsia, and Stefanos Zafeiriou. 2019. RetinaFace: Single-stage Dense Face Localisation in the Wild. CoRR abs/1905.00641 (2019). arXiv:1905.00641 http://arxiv.org/abs/1905.00641
[19]
Brian Dolhansky, Joanna Bitton, Ben Pflaum, Jikuo Lu, Russ Howes, Menglin Wang, and Cristian Canton-Ferrer. 2020. The DeepFake Detection Challenge Dataset. CoRR abs/2006.07397 (2020). arXiv:2006.07397 https://arxiv.org/abs/2006.07397
[20]
Haoqi Fan, Bo Xiong, Karttikeya Mangalam, Yanghao Li, Zhicheng Yan, Jitendra Malik, and Christoph Feichtenhofer. 2021. Multiscale Vision Transformers. CoRR abs/2104.11227 (2021). arXiv:2104.11227 https://arxiv.org/abs/2104.11227
[21]
Christoph Feichtenhofer. 2020. X3D: Expanding Architectures for Efficient Video Recognition. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Computer Vision Foundation / IEEE, 200--210. https://doi.org/10.1109/CVPR42600.2020.00028
[22]
Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. 2019. Slow-Fast Networks for Video Recognition. In 2019 IEEE/CVF International Conference on Computer Vision. IEEE, 6201--6210. https://doi.org/10.1109/ICCV.2019.00630
[23]
Steven Lawrence Fernandes, Sunny Raj, Eddy Ortiz, Iustina Vintila, Margaret Salter, Gordana Urosevic, and Sumit Kumar Jha. 2019. Predicting Heart Rate Variations of Deepfake Videos using Neural ODE. In 2019 IEEE/CVF International Conference on Computer Vision Workshops. IEEE, 1721--1729. https://doi.org/10.1109/ICCVW.2019.00213
[24]
Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. 2014. Generative Adversarial Nets. In Advances in Neural Information Processing Systems, Zoubin Ghahramani, Max Welling, Corinna Cortes, Neil D. Lawrence, and Kilian Q. Weinberger (Eds.). 2672--2680.
[25]
Luca Guarnera, Oliver Giudice, and Sebastiano Battiato. 2020. DeepFake Detection by Analyzing Convolutional Traces. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Computer Vision Foundation / IEEE, 2841--2850. https://doi.org/10.1109/CVPRW50498.2020.00341
[26]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37, 9 (2015), 1904--1916. https://doi.org/10.1109/TPAMI.2015.2389824
[27]
Juan Hu, Xin Liao, Wei Wang, and Zheng Qin. 2021. Detecting Compressed Deepfake Videos in Social Networks Using Frame-Temporality Two-Stream Convolutional Network. IEEE Transactions on Circuits and Systems for Video Technology (2021), 1--1. https://doi.org/10.1109/TCSVT.2021.3074259
[28]
Tao Hu and Honggang Qi. 2019. See Better Before Looking Closer: Weakly Supervised Data Augmentation Network for Fine-Grained Visual Classification. CoRR abs/1901.09891 (2019). arXiv:1901.09891 http://arxiv.org/abs/1901.09891
[29]
Liming Jiang, Ren Li, Wayne Wu, Chen Qian, and Chen Change Loy. 2020. DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Computer Vision Foundation / IEEE, 2886--2895. https://doi.org/10.1109/CVPR42600.2020.00296
[30]
Hasam Khalid and Simon S. Woo. 2020. OC-FakeDect: Classifying Deepfakes Using One-class Variational Autoencoder. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Computer Vision Foundation / IEEE, 2794--2803. https://doi.org/10.1109/CVPRW50498.2020.00336
[31]
Diederik P. Kingma and Max Welling. 2014. Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations, Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1312.6114
[32]
Pavel Korshunov and Sébastien Marcel. 2018. DeepFakes: a New Threat to Face Recognition? Assessment and Detection. CoRR abs/1812.08685 (2018). arXiv:1812.08685 http://arxiv.org/abs/1812.08685
[33]
Gen Li, Yun Cao, and Xianfeng Zhao. 2021. Exploiting Facial Symmetry to Expose Deepfakes. In 2021 IEEE International Conference on Image Processing (ICIP). 3587--3591. https://doi.org/10.1109/ICIP42928.2021.9506272
[34]
Lingzhi Li, Jianmin Bao, Ting Zhang, Hao Yang, Dong Chen, Fang Wen, and Baining Guo. 2020. Face X-Ray for More General Face Forgery Detection. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition,. Computer Vision Foundation / IEEE, 5000--5009. https://doi.org/10.1109/CVPR42600.2020.00505
[35]
Yuezun Li, Ming-Ching Chang, and Siwei Lyu. 2018. In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking. In 2018 IEEE International Workshop on Information Forensics and Security. IEEE, 1--7. https://doi.org/10.1109/WIFS.2018.8630787
[36]
Yuezun Li and Siwei Lyu. 2019. Exposing DeepFake Videos By Detecting Face Warping Artifacts. In IEEE Conference on Computer Vision and Pattern Recognition Workshops. Computer Vision Foundation / IEEE, 46--52.
[37]
Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu. 2020. Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Computer Vision Foundation / IEEE, 3204--3213. https://doi.org/10.1109/CVPR42600.2020.00327
[38]
Iacopo Masi, Aditya Killekar, Royston Marian Mascarenhas, Shenoy Pratik Gurudatt, and Wael AbdAlmageed. 2020. Two-Branch Recurrent Network for Isolating Deepfakes in Videos. In Computer Vision - ECCV 2020 - 16th European Conference (Lecture Notes in Computer Science, Vol. 12352), Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (Eds.). Springer, 667--684. https://doi.org/10.1007/978--3-030--58571--6_39
[39]
Falko Matern, Christian Riess, and Marc Stamminger. 2019. Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations. In IEEE Winter Applications of Computer Vision Workshops. IEEE, 83--92. https://doi.org/10.1109/WACVW.2019.00020
[40]
Yisroel Mirsky and Wenke Lee. 2021. The Creation and Detection of Deepfakes: A Survey. ACM Comput. Surv. 54, 1 (2021), 7:1--7:41. https://doi.org/10.1145/3425780
[41]
Huy H. Nguyen, Fuming Fang, Junichi Yamagishi, and Isao Echizen. 2019. Multi-task Learning for Detecting and Segmenting Manipulated Facial Images and Videos. In 10th IEEE International Conference on Biometrics Theory, Applications and Systems. IEEE, 1--8. https://doi.org/10.1109/BTAS46853.2019.9185974
[42]
Huy H. Nguyen, Junichi Yamagishi, and Isao Echizen. 2019. Capsule-forensics: Using Capsule Networks to Detect Forged Images and Videos. In IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, 2307--2311. https://doi.org/10.1109/ICASSP.2019.8682602
[43]
Thanh Thi Nguyen, Cuong M. Nguyen, Dung Tien Nguyen, Duc Thanh Nguyen, and Saeid Nahavandi. 2019. Deep Learning for Deepfakes Creation and Detection. CoRR abs/1909.11573 (2019). arXiv:1909.11573 http://arxiv.org/abs/1909.11573
[44]
Google Research Nick Dufour and Jigsaw Andrew Gully. 2019. Contributing Data to Deepfake Detection Research. Retrieved September 24, 2019 from https://ai.googleblog.com/2019/09/contributing-data-to-deepfake-detection.html
[45]
Yuyang Qian, Guojun Yin, Lu Sheng, Zixuan Chen, and Jing Shao. 2020. Thinking in Frequency: Face Forgery Detection by Mining Frequency-Aware Clues. In Computer Vision - ECCV 2020 - 16th European Conference (Lecture Notes in Computer Science, Vol. 12357), Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (Eds.). Springer, 86--103. https://doi.org/10.1007/978--3-030--58610--2_6
[46]
Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. 2019. FaceForensics++: Learning to Detect Manipulated Facial Images. In 2019 IEEE/CVF International Conference on Computer Vision. IEEE, 1--11. https://doi.org/10.1109/ICCV.2019.00009
[47]
Ekraam Sabir, Jiaxin Cheng, Ayush Jaiswal, Wael AbdAlmageed, Iacopo Masi, and Prem Natarajan. 2019. Recurrent Convolutional Strategies for Face Manipulation Detection in Videos. In IEEE Conference on Computer Vision and Pattern Recognition Workshops. Computer Vision Foundation / IEEE, 80--87.
[48]
Sara Sabour, Nicholas Frosst, and Geoffrey E. Hinton. 2017. Dynamic Routing Between Capsules. In Advances in Neural Information Processing Systems, Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 3856--3866.
[49]
Conrad Sanderson and Brian C. Lovell. 2009. Multi-Region Probabilistic Histograms for Robust and Scalable Identity Inference. In Advances in Biometrics, Third International Conference (Lecture Notes in Computer Science, Vol. 5558), Massimo Tistarelli and Mark S. Nixon (Eds.). Springer, 199--208. https://doi.org/10.1007/978--3--642-01793--3_21
[50]
Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. In 3rd International Conference on Learning Representations, Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1409.1556
[51]
Mingxing Tan and Quoc V. Le. 2019. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. In Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, 6105--6114. http://proceedings.mlr.press/v97/tan19a.html
[52]
Rubén Tolosana, Rubén Vera-Rodríguez, Julian Fiérrez, Aythami Morales, and Javier Ortega-Garcia. 2020. Deepfakes and beyond: A Survey of face manipulation and fake detection. Inf. Fusion 64 (2020), 131--148. https://doi.org/10.1016/j.inffus.2020.06.014
[53]
Luisa Verdoliva. 2020. Media Forensics and DeepFakes: An Overview. IEEE J. Sel. Top. Signal Process. 14, 5 (2020), 910--932. https://doi.org/10.1109/JSTSP.2020.3002101
[54]
Xi Wu, Zhen Xie, YuTao Gao, and Yu Xiao. 2020. SSTNet: Detecting Manipulated Faces Through Spatial, Steganalysis and Temporal Features. In 2020 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, 2952--2956. https://doi.org/10.1109/ICASSP40776.2020.9053969
[55]
Qizhe Xie, Minh-Thang Luong, Eduard H. Hovy, and Quoc V. Le. 2020. Self-Training With Noisy Student Improves ImageNet Classification. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Computer Vision Foundation / IEEE, 10684--10695. https://doi.org/10.1109/CVPR42600.2020.01070
[56]
Xin Yang, Yuezun Li, and Siwei Lyu. 2019. Exposing Deep Fakes Using Inconsistent Head Poses. In IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, 8261--8265. https://doi.org/10.1109/ICASSP.2019.8683164
[57]
Hongyi Zhang, Moustapha Cissé, Yann N. Dauphin, and David Lopez-Paz. 2018. mixup: Beyond Empirical Risk Minimization. In 6th International Conference on Learning Representations. OpenReview.net. https://openreview.net/forum?id=r1Ddp1-Rb
[58]
Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, and Yu Qiao. 2016. Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks. IEEE Signal Process. Lett. 23, 10 (2016), 1499--1503. https://doi.org/10.1109/LSP.2016.2603342
[59]
Hanqing Zhao, Wenbo Zhou, Dongdong Chen, Tianyi Wei, Weiming Zhang, and Nenghai Yu. 2021. Multi-Attentional Deepfake Detection. In IEEE Conference on Computer Vision and Pattern Recognition. Computer Vision Foundation / IEEE, 2185--2194.

Cited By

View all
  • (2025)Revisiting face forgery detection towards generalizationNeural Networks10.1016/j.neunet.2025.107310(107310)Online publication date: Mar-2025
  • (2024)Video and Audio Deepfake Datasets and Open Issues in Deepfake Technology: Being Ahead of the CurveForensic Sciences10.3390/forensicsci40300214:3(289-377)Online publication date: 13-Jul-2024
  • (2024)Using Graph Neural Networks to Improve Generalization Capability of the Models for Deepfake DetectionIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.345135619(8414-8427)Online publication date: 2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
IH&MMSec '22: Proceedings of the 2022 ACM Workshop on Information Hiding and Multimedia Security
June 2022
177 pages
ISBN:9781450393553
DOI:10.1145/3531536
This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 June 2022

Check for updates

Author Tags

  1. DeepFake dataset
  2. DeepFake detection
  3. detection benchmark
  4. forensics competition

Qualifiers

  • Research-article

Funding Sources

  • National Key Technology Reseach and Development Program

Conference

IH&MMSec '22
Sponsor:

Acceptance Rates

Overall Acceptance Rate 128 of 318 submissions, 40%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)386
  • Downloads (Last 6 weeks)51
Reflects downloads up to 05 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Revisiting face forgery detection towards generalizationNeural Networks10.1016/j.neunet.2025.107310(107310)Online publication date: Mar-2025
  • (2024)Video and Audio Deepfake Datasets and Open Issues in Deepfake Technology: Being Ahead of the CurveForensic Sciences10.3390/forensicsci40300214:3(289-377)Online publication date: 13-Jul-2024
  • (2024)Using Graph Neural Networks to Improve Generalization Capability of the Models for Deepfake DetectionIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.345135619(8414-8427)Online publication date: 2024
  • (2024)A Face Forgery Video Detection Model Based on Knowledge Distillation2024 IEEE/ACIS 27th International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD)10.1109/SNPD61259.2024.10673906(50-55)Online publication date: 5-Jul-2024
  • (2023)DeepFake on Face and Expression Swap: A ReviewIEEE Access10.1109/ACCESS.2023.332440311(117865-117906)Online publication date: 2023
  • (2023)Face manipulated deepfake generation and recognition approaches: a surveySmart Science10.1080/23080477.2023.226838012:1(53-73)Online publication date: 30-Oct-2023

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media