research-article

Improving Generalization of Deepfake Detection with Domain Adaptive Batch Normalization

Authors:

Haotong QinAuthors Info & Claims

ADVM '21: Proceedings of the 1st International Workshop on Adversarial Learning for Multimedia

Pages 21 - 27

https://doi.org/10.1145/3475724.3483603

Published: 22 October 2021 Publication History

Abstract

Deepfake, a well-known face forgery technique, has raised serious concerns about personal privacy and social media security. Therefore, a plenty of deepfake detection methods come out and achieve outstanding performance in the single dataset case. However, current deepfake detection methods fail to perform strong generalization ability in cross-dataset case due to the domain gap. To tackle this issue, we propose Domain Adaptive Batch Normalization (DABN) strategy to mitigate the domain distribution gap on different datasets. Specifically, DABN utilizes the distribution statistics of the testing dataset in replace of the original counterparts so as to avoid distribution mismatch and restore the effectiveness of BN layers. Equipped with our DABN, detection method can be more robust when generalized into a broader usage. Note that our method is flexible and can be further employed on most existing deepfake detection methods during testing, which shows a great practical value. Extensive experiments on multiple datasets and models demonstrate the effectiveness of DABN. The proposed method achieves an average accuracy improvement by nearly 20% of existing strategies on Celeb-DF dataset under black-box settings, indicating strong enhancement of generalization ability of the deepfake detection models.

References

[1]

Shruti Agarwal, Hany Farid, Yuming Gu, Mingming He, Koki Nagano, and Hao Li. 2019. Protecting World Leaders Against Deep Fakes. In CVPR workshops, Vol. 1.

[2]

Xiaoyu Cao and Neil Zhenqiang Gong. 2021. Understanding the Security of Deepfake Detection. arXiv preprint arXiv:2107.02045 (2021).

[3]

Joao Carreira and Andrew Zisserman. 2017. Quo vadis, action recognition? a new model and the kinetics dataset. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 6299--6308.

[4]

Yunjey Choi, Minje Choi, Munyoung Kim, Jung-Woo Ha, Sunghun Kim, and Jaegul Choo. 2018. Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 8789--8797.

[5]

Francc ois Chollet. 2017. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition . 1251--1258.

[6]

Roberto Daza, Aythami Morales, Julian Fierrez, and Ruben Tolosana. 2020. MEBAL: A multimodal database for eye blink detection and attention level estimation. In Companion Publication of the 2020 International Conference on Multimodal Interaction. 32--36.

Digital Library

[7]

Muhammad Ghifary, W Bastiaan Kleijn, and Mengjie Zhang. 2014. Domain adaptive neural networks for object recognition. In Pacific Rim international conference on artificial intelligence. Springer, 898--904.

[8]

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. Advances in neural information processing systems, Vol. 27 (2014).

Digital Library

[9]

David Güera and Edward J Delp. 2018. Deepfake video detection using recurrent neural networks. In 2018 15th IEEE international conference on advanced video and signal based surveillance (AVSS). IEEE, 1--6.

[10]

Kensho Hara, Hirokatsu Kataoka, and Yutaka Satoh. 2018. Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet?. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition . 6546--6555.

[11]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.

[12]

Naoto Inoue, Ryosuke Furuta, Toshihiko Yamasaki, and Kiyoharu Aizawa. 2018. Cross-domain weakly-supervised object detection through progressive domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 5001--5009.

[13]

Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning . PMLR, 448--456.

Digital Library

[14]

Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2017. Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196 (2017).

[15]

Tero Karras, Samuli Laine, and Timo Aila. 2019. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 4401--4410.

[16]

Pavel Korshunov and Sébastien Marcel. 2018. Deepfakes: a new threat to face recognition? assessment and detection. arXiv preprint arXiv:1812.08685 (2018).

[17]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, Vol. 25 (2012), 1097--1105.

Digital Library

[18]

Lingzhi Li, Jianmin Bao, Ting Zhang, Hao Yang, Dong Chen, Fang Wen, and Baining Guo. 2020 a. Face x-ray for more general face forgery detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 5001--5010.

[19]

Yuezun Li, Ming-Ching Chang, and Siwei Lyu. 2018. In ictu oculi: Exposing ai created fake videos by detecting eye blinking. In 2018 IEEE International Workshop on Information Forensics and Security (WIFS). IEEE, 1--7.

[20]

Yuezun Li and Siwei Lyu. 2018. Exposing deepfake videos by detecting face warping artifacts. arXiv preprint arXiv:1811.00656 (2018).

[21]

Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu. 2020 b. Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics. In IEEE Conference on Computer Vision and Patten Recognition (CVPR) .

[22]

Honggu Liu, Xiaodan Li, Wenbo Zhou, Yuefeng Chen, Yuan He, Hui Xue, Weiming Zhang, and Nenghai Yu. 2021. Spatial-phase shallow learning: rethinking face forgery detection in frequency domain. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 772--781.

[23]

Falko Matern, Christian Riess, and Marc Stamminger. 2019. Exploiting visual artifacts to expose deepfakes and face manipulations. In 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW). IEEE, 83--92.

[24]

Abdel-rahman Mohamed, George E Dahl, and Geoffrey Hinton. 2011. Acoustic modeling using deep belief networks. IEEE transactions on audio, speech, and language processing, Vol. 20, 1 (2011), 14--22.

Digital Library

[25]

Yunchen Pu, Zhe Gan, Ricardo Henao, Xin Yuan, Chunyuan Li, Andrew Stevens, and Lawrence Carin. 2016. Variational autoencoder for deep learning of images, labels and captions. Advances in neural information processing systems, Vol. 29 (2016), 2352--2360.

Digital Library

[26]

Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. 2019. FaceForensics

[27]

: Learning to Detect Manipulated Facial Images. In International Conference on Computer Vision (ICCV) .

[28]

Ekraam Sabir, Jiaxin Cheng, Ayush Jaiswal, Wael AbdAlmageed, Iacopo Masi, and Prem Natarajan. 2019. Recurrent convolutional strategies for face manipulation detection in videos. Interfaces (GUI), Vol. 3, 1 (2019), 80--87.

[29]

Conrad Sanderson and Brian C Lovell. 2009. Multi-region probabilistic histograms for robust and scalable identity inference. In International conference on biometrics. Springer, 199--208.

Digital Library

[30]

Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. 2020. Improving robustness against common corruptions by covariate shift adaptation. Advances in Neural Information Processing Systems, Vol. 33 (2020).

[31]

Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems. 3104--3112.

Digital Library

[32]

Justus Thies, Michael Zollhöfer, and Matthias Nießner. 2019. Deferred neural rendering: Image synthesis using neural textures. ACM Transactions on Graphics (TOG), Vol. 38, 4 (2019), 1--12.

Digital Library

[33]

Ruben Tolosana, Ruben Vera-Rodriguez, Julian Fierrez, Aythami Morales, and Javier Ortega-Garcia. 2020. Deepfakes and beyond: A survey of face manipulation and fake detection. Information Fusion, Vol. 64 (2020), 131--148.

[34]

Chao Yang and Ser-Nam Lim. 2020. One-shot domain adaptation for face generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5921--5930.

[35]

Xin Yang, Yuezun Li, and Siwei Lyu. 2019. Exposing deep fakes using inconsistent head poses. In ICASSP 2019--2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 8261--8265.

[36]

Ying Zhang, Lilei Zheng, and Vrizlynn L. L. Thing. 2017. Automated face swapping and its detection. In 2017 IEEE 2nd International Conference on Signal and Image Processing (ICSIP). 15--19. https://doi.org/10.1109/SIPROCESS.2017.8124497

[37]

Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision. 2223--2232.

Cited By

Qin HMa XDing YLi XZhang YMa ZWang JLuo JLiu X(2024)BiFSMNv2: Pushing Binary Neural Networks for Keyword Spotting to Real-Network PerformanceIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.324325935:8(10674-10686)Online publication date: Aug-2024
https://doi.org/10.1109/TNNLS.2023.3243259
Yin ZWang JXiao YZhao HLi TZhou WLiu ALiu X(2024)Improving Deepfake Detection Generalization by Invariant Risk MinimizationIEEE Transactions on Multimedia10.1109/TMM.2024.335565126(6785-6798)Online publication date: 2024
https://doi.org/10.1109/TMM.2024.3355651
Yu YNi RYang SZhao YKot A(2024)Narrowing Domain Gaps With Bridging Samples for Generalized Face Forgery DetectionIEEE Transactions on Multimedia10.1109/TMM.2023.331034126(3405-3417)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3310341
Show More Cited By

Index Terms

Improving Generalization of Deepfake Detection with Domain Adaptive Batch Normalization
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
2. Security and privacy
  1. Human and societal aspects of security and privacy
    1. Social aspects of security and privacy

Recommendations

Domain-Conditioned Normalization for Test-Time Domain Generalization
Computer Vision – ECCV 2022 Workshops
Abstract
Domain generalization aims to train a robust model on multiple source domains that generalizes well to unseen target domains. While considerable attention has focused on training domain generalizable models, a few recent studies have shifted the ...
Interpolation Normalization for Contrast Domain Generalization
MM '23: Proceedings of the 31st ACM International Conference on Multimedia

Domain generalization refers to the challenge of training a model from various source domains that can generalize well to unseen target domains. Contrastive learning is a promising solution that aims to learn domain-invariant representations by utilizing ...
Learning to Optimize Domain Specific Normalization for Domain Generalization
Computer Vision – ECCV 2020
Abstract
We propose a simple but effective multi-source domain generalization technique based on deep neural networks by incorporating optimized normalization layers that are specific to individual domains. Our approach employs multiple normalization ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ADVM '21: Proceedings of the 1st International Workshop on Adversarial Learning for Multimedia

October 2021

73 pages

ISBN:9781450386722

DOI:10.1145/3475724

Program Chairs:
Dawn Song
UC Berkeley, USA
,
Dacheng Tao
JD Explore Academy, China
,
Alan Yuille
Johns Hopkins University, USA
,
Anima Anandkumar
California Institute of Technology, USA
,
Aishan Liu
Beihang University, China
,
Xinyun Chen
UC Berkeley, USA
,
Yingwei Li
Johns Hopkins University, USA
,
Chaowei Xiao
NVIDIA Research, USA
,
Xun Yang
National University of Singapore, Singapore
,
Xianglong Liu
Beihang University, China

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 October 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Key Research and Development Plan of China

Conference

MM '21

Sponsor:

SIGMM

MM '21: ACM Multimedia Conference

October 20, 2021

Virtual Event, China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
277
Total Downloads

Downloads (Last 12 months)57
Downloads (Last 6 weeks)10

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Qin HMa XDing YLi XZhang YMa ZWang JLuo JLiu X(2024)BiFSMNv2: Pushing Binary Neural Networks for Keyword Spotting to Real-Network PerformanceIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.324325935:8(10674-10686)Online publication date: Aug-2024
https://doi.org/10.1109/TNNLS.2023.3243259
Yin ZWang JXiao YZhao HLi TZhou WLiu ALiu X(2024)Improving Deepfake Detection Generalization by Invariant Risk MinimizationIEEE Transactions on Multimedia10.1109/TMM.2024.335565126(6785-6798)Online publication date: 2024
https://doi.org/10.1109/TMM.2024.3355651
Yu YNi RYang SZhao YKot A(2024)Narrowing Domain Gaps With Bridging Samples for Generalized Face Forgery DetectionIEEE Transactions on Multimedia10.1109/TMM.2023.331034126(3405-3417)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3310341
Zhao ZZhang JBai HWang YCui YDeng LSun KZhang CLiu JXu S(2023)Deep Convolutional Sparse Coding Networks for Interpretable Image Fusion2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW59228.2023.00234(2369-2377)Online publication date: Jun-2023
https://doi.org/10.1109/CVPRW59228.2023.00234
Wang JYin ZHu PLiu ATao RQin HLiu XTao D(2022)Defensive Patches for Robust Recognition in the Physical World2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52688.2022.00249(2446-2455)Online publication date: Jun-2022
https://doi.org/10.1109/CVPR52688.2022.00249

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents