short-paper

Public Access

Comprehensive Dataset of Synthetic and Manipulated Overhead Imagery for Development and Evaluation of Forensic Tools

Authors:

Brandon B. May,

Kirill Trapeznikov,

Shengbang Fang,

Matthew StammAuthors Info & Claims

IH&MMSec '23: Proceedings of the 2023 ACM Workshop on Information Hiding and Multimedia Security

Pages 145 - 150

https://doi.org/10.1145/3577163.3595101

Published: 28 June 2023 Publication History

Abstract

We present a first of its kind dataset of overhead imagery for development and evaluation of forensic tools. Our dataset consists of real, fully synthetic and partially manipulated overhead imagery generated from a custom diffusion model trained on two sets of different zoom levels and on two sources of pristine data. We developed our model to support controllable generation of multiple manipulation categories including fully synthetic imagery conditioned on real and generated base maps, and location. We also support partial in-painted imagery with same conditioning options and with several types of manipulated content. The data consist of raw images and ground truth annotations describing the manipulation parameters. We also report benchmark performance on several tasks supported by our dataset including detection of fully and partially manipulated imagery, manipulation localization and classification.

References

[1]

L Abady et al. "GAN generation of synthetic multispectral satellite images". In: Image and Signal Processing for Remote Sensing XXVI. Vol. 11533.

[2]

L. Abady et al. "An Overview on the Generation and Detection of Synthetic and Manipulated Satellite Images". In: arXiv preprint arXiv:2209.08984 (2022).

[3]

Bellingcat. url: https://www.bellingcat.com.

[4]

S. Bond. That smiling LinkedIn profile face might be a computer-generated fake. url: https://www.npr.org/2022/03/27/1088140809/fake-linkedin-profiles.

[5]

T. J. de Carvalho et al. "Exposing Digital Image Forgeries by Illumination Color Classification". In: IEEE Trans. on Information Forensics and Security 8.7 (2013), pp. 1182--1194.

Digital Library

[6]

R. Corvi et al. "On the detection of synthetic images generated by diffusion models". In: arXiv preprint arXiv:2211.00680 (2022).

[7]

D. Cozzolino and L. Verdoliva. "Noiseprint: A CNN-based camera model fingerprint". In: IEEE Trans. on Information Forensics and Security 15 ().

[8]

Dataset: deepfake-satellite-images. url:https://github.com/RijulGupta-DM/ deepfake-satellite-images.

[9]

P. Dhariwal and A. Nichol. "Diffusion models beat gans on image synthesis". In: Advances in Neural Information Processing Systems 34 (2021), pp. 8780--8794.

[10]

I. J. Goodfellow et al. "Generative Adversarial Networks". In: Advances in Neural Information Processing Systems, pp. 2672--2680.

[11]

Graphika and Stanford Internet Observatory (2022). Unheard Voice: Evaluating five years of pro-Western covert influence operations. url: https://purl.stanford.edu/nj914nx9540.

[12]

D. M. Greig, B. T. Porteous, and A. H. Seheult. "Exact maximum a posteriori estimation for binary images". In: Journal of the Royal Statistical Society: Series B (Methodological) 51.2 (1989), pp. 271--279.

[13]

R. Gupta et al. "Creating xBD: A dataset for assessing building damage from satellite imagery". In: CVPR workshops. 2019, pp. 10--17.

[14]

J. Ho, A. Jain, and P. Abbeel. "Denoising diffusion probabilistic models". In: Advances in Neural Information Processing Systems 33 (2020), pp. 6840--6851.

[15]

M. Huh et al. "Fighting fake news: Image splice detection via learned self- consistency". In: ECCV. 2018, pp. 101--117.

[16]

T. Karras et al. "Analyzing and improving the image quality of stylegan". In: CVPR. 2020, pp. 8110--8119.

[17]

A. Lugmayr et al. "Repaint: Inpainting using denoising diffusion probabilistic models". In: CVPR. 2022, pp. 11461--11471.

[18]

J. Marín and S. Escalera. "SSSGAN: Satellite Style and Structure Generative Adversarial Networks". In: Remote Sensing 13.19 (2021). issn: 2072--4292.

[19]

O. Mayer and M. C. Stamm. "Exposing fake images with forensic similarity graphs". In: IEEE Journal of Selected Topics in Signal Processing 14.5 ().

[20]

T. -T. Ng, S.-F. Chang, and Q Sun. "A data set of authentic and spliced image blocks". In: Columbia University, ADVENT Technical Report 4 (2004).

[21]

A. Q. Nichol et al. "GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models". In: ICML. 2022, pp. 16784--16804.

[22]

A. Radford et al. "Learning transferable visual models from natural language supervision". In: ICML. PMLR. 2021, pp. 8748--8763.

[23]

A. Ramesh et al. "Hierarchical text-conditional image generation with clip latents". In: arXiv preprint arXiv:2204.06125 (2022).

[24]

J. Ricker et al. "Towards the Detection of Diffusion Model Deepfakes". In: arXiv preprint arXiv:2210.14571 (2022).

[25]

R. Rombach et al. "High-resolution image synthesis with latent diffusion models". In: CVPR. 2022, pp. 10684--10695.

[26]

C. Saharia et al. "Photorealistic Text-to-Image Diffusion Models with Deep Lan-guage Understanding". In: Advances in Neural Information Processing Systems.

[27]

Z. Sha et al. "DE-FAKE: Detection and Attribution of Fake Images Generated by Text-to-Image Diffusion Models". In: arXiv preprint arXiv:2210.06998 (2022).

[28]

This city does not exist. url: http://thiscitydoesnotexist.com/.

[29]

L. Verdoliva. "Media forensics and deepfakes: an overview". In: IEEE Journal of Selected Topics in Signal Processing 14.5 (2020), pp. 910--932.

[30]

J. Viquerat. GitHub - jviquerat/bshapes: A 2D shape generator using Bezier curves. https://github.com/jviquerat/bshapes. 2022.

[31]

S. -Y. Wang et al. "CNN-generated images are surprisingly easy to spot... for now". In: CVPR. 2020, pp. 8695--8704.

[32]

X. Wang et al. "Gan-generated faces detection: A survey and new perspectives". In: arXiv preprint arXiv:2202.07145 (2022).

[33]

M. Zaytar and C. El Amrani. "Satellite image inpainting with deep generative adversarial neural networks". In: IAES IJ-AI 10 (Mar. 2021), p. 121. 11591/ijai.v10.i1.pp121--130.

[34]

B. Zhao et al. "Deep fake geography" When geospatial data encounter Artificial Intelligence". In: (Apr. 2021).

Cited By

Vahdati DNguyen TAzizpour AStamm M(2024)Beyond Deepfake Images: Detecting AI-Generated Videos2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW63382.2024.00443(4397-4408)Online publication date: 17-Jun-2024
https://doi.org/10.1109/CVPRW63382.2024.00443
Azizpour ANguyen TShrestha MXu KKim EStamm M(2024)E3: Ensemble of Expert Embedders for Adapting Synthetic Image Detectors to New Generators Using Limited Data2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW63382.2024.00437(4334-4344)Online publication date: 17-Jun-2024
https://doi.org/10.1109/CVPRW63382.2024.00437

Index Terms

Comprehensive Dataset of Synthetic and Manipulated Overhead Imagery for Development and Evaluation of Forensic Tools
1. Applied computing
  1. Computer forensics
    1. Investigation techniques
2. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Computer graphics
    1. Image manipulation

Recommendations

Augmenting Prostate MRI Dataset with Synthetic Volumetric Images from Zone-Conditioned Diffusion Generative Model
Deep Generative Models
Abstract
The need for artificial intelligence (AI)-driven computer-assist ed diagnosis (CAD) tools drives up the demand for large high-quality datasets in medical imaging. However, collecting the necessary amount of data is often impractical due to patient ...
Synthetic Data and Hierarchical Object Detection in Overhead Imagery
The performance of neural network models is often limited by the availability of big datasets. To treat this problem, we survey and develop novel synthetic data generation and augmentation techniques for enhancing low/zero-sample learning in satellite ...
Layover Analysis in Synthetic Aperture Imagery

Layover is an artifact observed in synthetic aperture radar (SAR) images. This artifact manifests as geometric distortions or positioning errors due to unknown or inaccurate ground topography information. Layover artifact results in erroneous distance ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

IH&MMSec '23: Proceedings of the 2023 ACM Workshop on Information Hiding and Multimedia Security

June 2023

190 pages

ISBN:9798400700545

DOI:10.1145/3577163

General Chair:
Daniel Moreira
Loyola University Chicago, USA
,
Program Chairs:
Aparna Bharati
Lehigh University, USA
,
Cecilia Pasquini
Fondazione Bruno Kessler, Italy
,
Yassine Yousfi
Comma.ai, USA

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 June 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Best Paper

Author Tags

Qualifiers

Short-paper

Funding Sources

DARPA

Conference

IH&MMSec '23

Sponsor:

SIGMM

IH&MMSec '23: ACM Workshop on Information Hiding and Multimedia Security

June 28 - 30, 2023

IL, Chicago, USA

Acceptance Rates

Overall Acceptance Rate 128 of 318 submissions, 40%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
222
Total Downloads

Downloads (Last 12 months)125
Downloads (Last 6 weeks)13

Reflects downloads up to 13 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Vahdati DNguyen TAzizpour AStamm M(2024)Beyond Deepfake Images: Detecting AI-Generated Videos2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW63382.2024.00443(4397-4408)Online publication date: 17-Jun-2024
https://doi.org/10.1109/CVPRW63382.2024.00443
Azizpour ANguyen TShrestha MXu KKim EStamm M(2024)E3: Ensemble of Expert Embedders for Adapting Synthetic Image Detectors to New Generators Using Limited Data2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW63382.2024.00437(4334-4344)Online publication date: 17-Jun-2024
https://doi.org/10.1109/CVPRW63382.2024.00437

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten