research-article

Open access

Overview of the Grand Challenge on Detecting Cheapfakes at ACM ICMR 2024

Authors:

Duc-Tien Dang-Nguyen,

Sohail Ahmed Khan,

Michael Riegler,

Pål Halvorsen,

Minh-Triet TranAuthors Info & Claims

ICMR '24: Proceedings of the 2024 International Conference on Multimedia Retrieval

Pages 1275 - 1281

https://doi.org/10.1145/3652583.3657587

Published: 07 June 2024 Publication History

Abstract

Information disorder is one of the most typical challenges in the current era of science and technology. The amount of information on the internet is increasing, but its correctness and authenticity are not always guaranteed, leading to false information, fake news, etc. The mentioned problem negatively affects users' reception and use of information. Unlike deepfake, cheapfake is created using simple techniques and does not rely on AI to produce fake multimedia. Cheapfake is becoming increasingly popular due to its ease of creation. Thus, there is a growing need to develop techniques that can detect cheapfake content. Following previous events, the Grand Challenge on Detecting Cheapfakes at ACM ICMR 2024 continues to seek contributions from researchers on cheapfake detection with the goal of improving effectiveness and creativity in approach, and understanding the limitations of the current dataset. This challenge has accepted 6 new proposed methods from participants with the highest private test accuracies achieved at 72.2% for Task 1 and 54.84% for Task 2. The highest public test accuracies for the two tasks are 95.6% and 93% respectively. These new methods focus on incorporating new AI models such as Stable Diffusion, LLM. These new findings represent the latest advancements in cheapfake detection research and introduce new potential approaches for future research.

References

[1]

Shivangi Aneja, Chris Bregler, and Matthias Nießner. 2023. COSMOS: Catching Out-of-Context Image Misuse Using Self-Supervised Learning. In AAAI Conference on Artificial Intelligence. https://api.semanticscholar.org/CorpusID:259693535

[2]

S. Aneja, C. Midoglu, D. Dang-Nguyen, M. A. Riegler, P. Halvorsen, Ma. Niessner, B. Adsumilli, and C. Bregler. 2021. MMSys'21 Grand Challenge on Detecting Cheapfakes. arxiv: 2107.05297 [cs.MM]

[3]

AP. 2016. Somalia Military Forces. https://tinyurl.com/2tbxjek2 Retrieved April 20, 2024 from

[4]

AFP Australia. 2020. This virtual image was created by an artist in New South Wales, Australia -- it's not a real photo. https://factcheck.afp.com/virtual-image-was-created-artist-new-south-wales-australia-its-not-real-photo Retrieved April 20, 2024 from

[5]

Duc-Tien Dang-Nguyen, Sohail Ahmed Khan, Cise Midoglu, Michael Riegler, Pål Halvorsen, and Minh-Son Dao. 2023. Grand challenge on detecting cheapfakes. arXiv preprint arXiv:2304.01328 (2023).

[6]

Peng Gao, Jiaming Han, Renrui Zhang, Ziyi Lin, Shijie Geng, Aojun Zhou, Wei Zhang, Pan Lu, Conghui He, Xiangyu Yue, et al. 2023. Llama-adapter v2: Parameter-efficient visual instruction model. arXiv preprint arXiv:2304.15010 (2023).

[7]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.

[8]

Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2021. Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing. arXiv preprint arXiv:2111.09543 (2021).

[9]

Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7B. arXiv preprint arXiv:2310.06825 (2023).

[10]

Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. 2024. Mixtral of experts. arXiv preprint arXiv:2401.04088 (2024).

[11]

Jing Yu Koh, Daniel Fried, and Russ R Salakhutdinov. 2024. Generating images with multimodal language models. Advances in Neural Information Processing Systems, Vol. 36 (2024).

[12]

Anh-Thu Le, Minh-Dat Nguyen, Minh-Son Dao, Anh-Duy Tran, and Duc-Tien Dang-Nguyen. 2024. TeGA: A Text-Guided Generative-based Approach in Cheapfake Detection. In Proceedings of the 2024 ACM International Conference on Multimedia Retrieval.

Digital Library

[13]

Wing Lian, Bleys Goodson, Guan Wang, Eugene Pentland, Austin Cook, Chanvichet Vong, and "Teknium". 2023. MistralOrca: Mistral-7B Model Instruct-tuned on Filtered OpenOrcaV1 GPT-4 Dataset.

[14]

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019).

[15]

Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V. Le, Barret Zoph, Jason Wei, and Adam Roberts. 2023. The Flan Collection: Designing Data and Methods for Effective Instruction Tuning. arxiv: 2301.13688 [cs.AI]

[16]

G. Luo, T. Darrell, and A. Rohrbach. 2021. NewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Media. arxiv: 2104.05893 [cs.CV]

[17]

Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. 2023. Orca: Progressive Learning from Complex Explanation Traces of GPT-4. arxiv: 2306.02707 [cs.CL]

[18]

Van-Loc Nguyen, Bao-Tin Nguyen, Thanh-Son Nguyen, Duc-Tien Dang-Nguyen, and Minh-Triet Tran. 2024. A Unified Network for Detecting Out-Of-Context Information Using Generative Synthetic Data. In Proceedings of the 2024 ACM International Conference on Multimedia Retrieval.

Digital Library

[19]

B. Paris and J. Donovan. 2019. Deepfakes and cheapfakes: The manipulation of audio and visual evidence. https://datasociety.net/wp-content/uploads/2019/09/DataSociety_Deepfakes_Cheap_Fakes.pdf Retrieved April 20, 2024 from

[20]

Andrew Peng, Michael Wu, John Allard, Logan Kilpatrick, and Steven Heidel. 2023 b. Deepfakes and cheapfakes: The manipulation of audio and visual evidence. https://openai.com/blog/gpt-3--5-turbo-fine-tuning-and-api-updates Retrieved April 20, 2024 from

[21]

Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. 2023 a. Kosmos-2: Grounding multimodal large language models to the world. arXiv preprint arXiv:2306.14824 (2023).

[22]

Kha-Luan Pham, Minh-Khoi Nguyen-Nhat, Anh-Huy Dinh, Quang-Tri Le, Manh-Thien Nguyen, Anh-Duy Tran, Minh-Triet Tran, and Duc-Tien Dang-Nguyen. 2024 a. Ookpik-A Collection of Out-of-Context Image-Caption Pairs. In International Conference on Multimedia Modeling. Springer, 132--144.

[23]

Long-Khanh Pham, Hoa-Vien Vo-Hoang, and Anh-Duy Tran. 2024 b. A Generative Adaptive Context Learning Framework for Large Language Models in Cheapfake Detection. In Proceedings of the 2024 ACM International Conference on Multimedia Retrieval.

Digital Library

[24]

Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. 2023. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952 (2023).

[25]

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning. PMLR, 8748--8763.

[26]

Stephen Radford. 2015. Photo of burning house. https://unsplash.com/photos/photo-of-burning-house-hLUTRzcVkqg Retrieved April 20, 2024 from

[27]

Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics. http://arxiv.org/abs/1908.10084

[28]

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10684--10695.

[29]

A. Rössler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, and M. Nießner. 2019. FaceForensics: Learning to Detect Manipulated Facial Images. arxiv: 1901.08971 [cs.CV]

[30]

Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural Machine Translation of Rare Words with Subword Units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7--12, 2016, Berlin, Germany, Volume 1: Long Papers. The Association for Computer Linguistics. https://doi.org/10.18653/V1/P16--1162

[31]

Jangwon Seo, Hyo-Seok Hwang, Jiyoung Lee, Wonsuk Lee, Minhyeok; Kim, and Junhee Seok. 2024. A Multi-Stage Deep Learning Approach Incorporating Text-Image and Image-Image Comparisons for Cheapfake Detection. In Proceedings of the 2024 ACM International Conference on Multimedia Retrieval.

Digital Library

[32]

Saranac Hale Spencer. 2019. Woman in Viral Photo Misidentified as Rep. Omar. https://www.factcheck.org/2019/09/woman-in-viral-photo-misidentified-as-rep-omar/ Retrieved April 20, 2024 from

[33]

Ozzie Stern. 2019. Gray Bridge. https://unsplash.com/photos/gray-bridge-lzcDi7-MWL4 Retrieved April 20, 2024 from

[34]

Hoa-Vien Vo-Hoang, Long-Khanh Pham, and Minh-Son Dao. 2024. Detecting Out-of-Context Media with LLaMa-Adapter V2 and RoBERTa: An Effective Method for Cheapfakes Detection. In Proceedings of the 2024 ACM International Conference on Multimedia Retrieval.

Digital Library

[35]

Dang Vu, Quoc-Trung Nguyen, and Minh-Nhat Nguyen. 2024. Enhancing Cheapfake Detection: An Approach Using Prompt Engineering and Interleaved Text-Image Model. In Proceedings of the 2024 ACM International Conference on Multimedia Retrieval.

Digital Library

[36]

Peng Wang, An Yang, Rui Men, Junyang Lin, Shuai Bai, Zhikang Li, Jianxin Ma, Chang Zhou, Jingren Zhou, and Hongxia Yang. 2022. Ofa: Unifying architectures, tasks, and modalities through a simple sequence-to-sequence learning framework. In International Conference on Machine Learning. PMLR, 23318--23340.

[37]

J. Waterson. 2019. Facebook refuses to delete fake Pelosi video spread by Trump supporters. https://www.theguardian.com/technology/2019/may/24/facebook-leaves-fake-nancy-pelosi-video-on-site Retrieved April 20, 2024 from https://www.theguardian.com/technology/2019/may/24/facebook-leaves-fake-nancy-pelosi-video-on-site

Cited By

Nguyen BNguyen VNguyen TDang-Nguyen DDo TTran M(2024)A Hybrid Approach for Cheapfake Detection Using Reputation Checking and End-To-End NetworkProceedings of the 1st Workshop on Security-Centric Strategies for Combating Information Disorder10.1145/3660512.3665521(1-12)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3660512.3665521
Le ANguyen MDao MTran ADang-Nguyen DGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)TeGA: A Text-Guided Generative-based Approach in Cheapfake DetectionProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3657602(1294-1299)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3657602
Vu DNguyen MNguyen QGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)Enhancing Cheapfake Detection: An Approach Using Prompt Engineering and Interleaved Text-Image ModelProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3657600(1306-1311)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3657600
Show More Cited By

Index Terms

Overview of the Grand Challenge on Detecting Cheapfakes at ACM ICMR 2024
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
2. Information systems
  1. Information retrieval
    1. Evaluation of retrieval results
  2. Information systems applications
    1. Multimedia information systems

Recommendations

Multimedia Grand Challenge 2012

The Multimedia Grand Challenge is a recurring event at the ACM Multimedia Conference series. During this event, delegates from various industries define a number of challenges that they consider of interest from both a business and scientific ...
LSC '19: Proceedings of the ACM Workshop on Lifelog Search Challenge
Report from ACM ICMR 2018

Multimedia computing, indexing, and retrieval continue to be one of the most exciting and fastest-growing research areas in the field of multimedia technology. ACM ICMR is the premier international conference that brings together experts and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMR '24: Proceedings of the 2024 International Conference on Multimedia Retrieval

May 2024

1379 pages

ISBN:9798400706196

DOI:10.1145/3652583

General Chairs:
Cathal Gurrin
Dublin City University, Ireland
,
Rachada Kongkachandra
Thammasat University, Thailand
,
Klaus Schoeffmann
Klagenfurt University, Austria
,
Program Chairs:
Duc-Tien Dang-Nguyen
University of Bergen, Norway
,
Luca Rossetto
University of Zurich, Switzerland
,
Shin'ichi Satoh
National Institute of Informatics, Japan
,
Liting Zhou
Dublin City University, Ireland

Copyright © 2024 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 June 2024

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

NORDIS

Conference

ICMR '24

Sponsor:

ICMR '24: International Conference on Multimedia Retrieval

June 10 - 14, 2024

Phuket, Thailand

Acceptance Rates

Overall Acceptance Rate 254 of 830 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
186
Total Downloads

Downloads (Last 12 months)186
Downloads (Last 6 weeks)61

Reflects downloads up to 24 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Nguyen BNguyen VNguyen TDang-Nguyen DDo TTran M(2024)A Hybrid Approach for Cheapfake Detection Using Reputation Checking and End-To-End NetworkProceedings of the 1st Workshop on Security-Centric Strategies for Combating Information Disorder10.1145/3660512.3665521(1-12)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3660512.3665521
Le ANguyen MDao MTran ADang-Nguyen DGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)TeGA: A Text-Guided Generative-based Approach in Cheapfake DetectionProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3657602(1294-1299)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3657602
Vu DNguyen MNguyen QGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)Enhancing Cheapfake Detection: An Approach Using Prompt Engineering and Interleaved Text-Image ModelProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3657600(1306-1311)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3657600
Nguyen VNguyen BNguyen TDang-Nguyen DTran MGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)A Unified Network for Detecting Out-Of-Context Information Using Generative Synthetic DataProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3657599(1300-1305)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3657599
Pham LVo-Hoang HTran AGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)A Generative Adaptive Context Learning Framework for Large Language Models in Cheapfake DetectionProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3657597(1288-1293)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3657597
Vo-Hoang HPham LDao MGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)Detecting Out-of-Context Media with LLaMa-Adapter V2 and RoBERTa: An Effective Method for Cheapfakes DetectionProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3657596(1282-1287)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3657596
Le DTran MLuong VTran MNguyen-Son H(2024)Document Similarity with Bipartite Graph Matching for Cheapfake and Fake News Detection2024 International Conference on Multimedia Analysis and Pattern Recognition (MAPR)10.1109/MAPR63514.2024.10660799(1-6)Online publication date: 15-Aug-2024
https://doi.org/10.1109/MAPR63514.2024.10660799

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents