research-article

Evaluation of GIST descriptors for web-scale image search

Authors:

Matthijs Douze,

Harsimrat Sandhawalia,

Laurent Amsaleg,

Cordelia SchmidAuthors Info & Claims

CIVR '09: Proceedings of the ACM International Conference on Image and Video Retrieval

Article No.: 19, Pages 1 - 8

https://doi.org/10.1145/1646396.1646421

Published: 08 July 2009 Publication History

Abstract

The GIST descriptor has recently received increasing attention in the context of scene recognition. In this paper we evaluate the search accuracy and complexity of the global GIST descriptor for two applications, for which a local description is usually preferred: same location/object recognition and copy detection. We identify the cases in which a global description can reasonably be used.

The comparison is performed against a state-of-the-art bag-of-features representation. To evaluate the impact of GIST's spatial grid, we compare GIST with a bag-of-features restricted to the same spatial grid as in GIST.

Finally, we propose an indexing strategy for global descriptors that optimizes the trade-off between memory usage and precision. Our scheme provides a reasonable accuracy in some widespread application cases together with very high efficiency: In our experiments, querying an image database of 110 million images takes 0.18 second per image on a single machine. For common copyright attacks, this efficiency is obtained without noticeably sacrificing the search accuracy compared with state-of-the-art approaches.

References

[1]

O. Chum, J. Philbin, J. Sivic, M. Isard, and A. Zisserman. Total recall: Automatic query expansion with a generative feature model for object retrieval. In ICCV, 2007.

[2]

O. Chum, J. Philbin, and A. Zisserman. Near duplicate image detection: min-hash and tf-idf weighting. In BMVC, 2008.

[3]

M. Douze, A. Gaidon, H. Jégou, M. Marszalek, and C. Schmid. INRIA-LEAR's video copy detection system. In TRECVID Workshop, November 2008.

[4]

J. Hayes and A. Efros. Scene completion using millions of photographs. In SIGGRAPH, 2007.

Digital Library

[5]

H. Jégou, M. Douze, and C. Schmid. Hamming embedding and weak geometric consistency for large scale image search. In ECCV, October 2008.

Digital Library

[6]

H. Jégou, M. Douze, and C. Schmid. Hamming embedding and weak geometry consistency for large scale image search - extended version. Technical report, INRIA, RR 6709, October 2008.

[7]

H. Jégou, H. Harzallah, and C. Schmid. A contextual dissimilarity measure for accurate and efficient image search. In CVPR, 2007.

[8]

S. Lazebnik, C. Schmid, and J. Ponce. Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In CVPR, 2006.

Digital Library

[9]

H. Lejsek, F. Ásmundsson, B. Jónsson, and L. Amsaleg. Scalability of local image descriptors: a comparative study. In ACM Multimedia, pages 589--598, 2006.

Digital Library

[10]

X. Li, C. Wu, C. Zach, S. Lazebnik, and J.-M. Frahm. Modeling and recognition of landmark image collections using iconic scene graphs. In ECCV, October 2008.

Digital Library

[11]

D. Lowe. Distinctive image features from scale-invariant keypoints. IJCV, 60(2): 91--110, 2004.

Digital Library

[12]

J. Matas, O. Chum, U. Martin, and T. Pajdla. Robust wide baseline stereo from maximally stable extremal regions. In BMVC, pages 384--393, 2002.

[13]

K. Mikolajczyk. Binaries for affine covariant region descriptors, 2007.

[14]

K. Mikolajczyk and C. Schmid. Scale and affine invariant interest point detectors. IJCV, 60(1): 63--86, 2004.

Digital Library

[15]

D. Nistér and H. Stewénius. Scalable recognition with a vocabulary tree. In CVPR, pages 2161--2168, 2006.

Digital Library

[16]

A. Oliva and A. Torralba. Modeling the shape of the scene: a holistic representation of the spatial envelope. IJCV, 42(3): 145--175, 2001.

Digital Library

[17]

J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman. Object retrieval with large vocabularies and fast spatial matching. In CVPR, 2007.

[18]

J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman. Lost in quantization: Improving particular object retrieval in large scale image databases. In CVPR, 2008.

[19]

G. Salton and C. Buckley. Term-weighting approaches in automatic text retrieval. Information Processing&Management, 24(5): 513--523, 1988.

Digital Library

[20]

J. Sivic and A. Zisserman. Video Google: A text retrieval approach to object matching in videos. In ICCV, pages 1470--1477, 2003.

Digital Library

[21]

A. F. Smeaton, P. Over, and W. Kraaij. Evaluation campaigns and trecvid. In MIR '06: Proceedings of the 8th ACM International Workshop on Multimedia Information Retrieval, pages 321--330, 2006.

Digital Library

[22]

A. Torralba, R. Fergus, and W. T. Freeman. 80 million tiny images: a large database for non-parametric object and scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(11): 1958--1970, November 2008.

Digital Library

[23]

A. Torralba, R. Fergus, and Y. Weiss. Small codes and large databases for recognition. In CVPR, 2008.

[24]

Y. Weiss, A. Torralba, and R. Fergus. Spectral hashing. In Advances in Neural Information Processing Systems, 2009.

[25]

J. Zobel, A. Moffat, and K. Ramamohanarao. Inverted files versus signature files for text indexing. ACM Transactions on Database Systems, 23(4): 453--490, 1998.

Digital Library

Cited By

Omar M(2024)Revolutionizing Malware DetectionInnovations, Securities, and Case Studies Across Healthcare, Business, and Technology10.4018/979-8-3693-1906-2.ch011(196-220)Online publication date: 12-Apr-2024
https://doi.org/10.4018/979-8-3693-1906-2.ch011
Deliege ADondero MD’Armenio E(2024)On the Dynamism of Paintings Through the Distribution of Edge DirectionsJournal of Imaging10.3390/jimaging1011027610:11(276)Online publication date: 1-Nov-2024
https://doi.org/10.3390/jimaging10110276
Su YSun YZhang MWang J(2024)Vexless: A Serverless Vector Data Management System Using Cloud FunctionsProceedings of the ACM on Management of Data10.1145/36549902:3(1-26)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3654990
Show More Cited By

Index Terms

Evaluation of GIST descriptors for web-scale image search
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision representations
        Image representations
2. Information systems
  1. Information retrieval

Recommendations

A Performance Evaluation of Local Descriptors

In this paper, we compare the performance of descriptors computed for local interest regions, as, for example, extracted by the Harris-Affine detector [32]. Many different descriptors have been proposed in the literature. It is unclear which descriptors ...
New color GPHOG descriptors for object and scene image classification

This paper presents a novel set of image descriptors that encodes information from color, shape, spatial and local features of an image to improve upon the popular Pyramid of Histograms of Oriented Gradients (PHOG) descriptor for object and scene image ...
Novel color Gabor-LBP-PHOG (GLP) descriptors for object and scene image classification
ICVGIP '12: Proceedings of the Eighth Indian Conference on Computer Vision, Graphics and Image Processing

This paper presents a novel set of color descriptors for object and scene image classification. We first introduce a new Gabor-PHOG (GPHOG) descriptor by concatenating the Pyramid of Histograms of Oriented Gradients (PHOG) of the local Gabor filtered ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CIVR '09: Proceedings of the ACM International Conference on Image and Video Retrieval

July 2009

383 pages

ISBN:9781605584805

DOI:10.1145/1646396

Conference Chairs:
Yiannis Kompatsiaris
CERTH-ITI, Greece
,
Stephane Marchand-Maillet
Univ. of Geneva, Switzerland
,
Program Chairs:
Yannis Avrithis
NTUA, Greece
,
Noel O Connor
DCU, Ireland
,
Daniel Gatica-Perez
Idiap Research Institute, Switzerland
,
Tat-Seng Chua
National University of Singapore, Singapore

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

In-Cooperation

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 July 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Conference

CIVR '09

Sponsor:

SIGMM

CIVR '09: CIVR '09 - International Conference on Image and Video Retrieval

July 8 - 10, 2009

Santorini, Fira, Greece

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

261
Total Citations
View Citations
2,131
Total Downloads

Downloads (Last 12 months)34
Downloads (Last 6 weeks)5

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Omar M(2024)Revolutionizing Malware DetectionInnovations, Securities, and Case Studies Across Healthcare, Business, and Technology10.4018/979-8-3693-1906-2.ch011(196-220)Online publication date: 12-Apr-2024
https://doi.org/10.4018/979-8-3693-1906-2.ch011
Deliege ADondero MD’Armenio E(2024)On the Dynamism of Paintings Through the Distribution of Edge DirectionsJournal of Imaging10.3390/jimaging1011027610:11(276)Online publication date: 1-Nov-2024
https://doi.org/10.3390/jimaging10110276
Su YSun YZhang MWang J(2024)Vexless: A Serverless Vector Data Management System Using Cloud FunctionsProceedings of the ACM on Management of Data10.1145/36549902:3(1-26)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3654990
Meng XFeng LGuo CWang HWu S(2024)A Unified Framework Based on Graph Consensus Term for Multiview LearningIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.320149835:3(3964-3977)Online publication date: Mar-2024
https://doi.org/10.1109/TNNLS.2022.3201498
Fawzy MTawfik NSaleh S(2024)Cluster-Based Optimization of Training Data Selection for Image Copy Detection Models2024 14th International Conference on Electrical Engineering (ICEENG)10.1109/ICEENG58856.2024.10566392(225-230)Online publication date: 21-May-2024
https://doi.org/10.1109/ICEENG58856.2024.10566392
Pizzi EKordopatis-Zilos GPatel HPostelnicu GNagavara Ravindra SGupta APapadopoulos STolias GDouze M(2024)The 2023 video similarity dataset and challengeComputer Vision and Image Understanding10.1016/j.cviu.2024.103997243(103997)Online publication date: Jun-2024
https://doi.org/10.1016/j.cviu.2024.103997
Wang WSun YYang Y(2024)Pattern-Expandable Image Copy DetectionInternational Journal of Computer Vision10.1007/s11263-024-02140-5132:12(5618-5634)Online publication date: 22-Jun-2024
https://doi.org/10.1007/s11263-024-02140-5
Andrade GBarreiros WRocha LFerreira RTeodoro G(2024)Large-scale response-aware online ANN search in dynamic datasetsCluster Computing10.1007/s10586-023-04159-827:3(3499-3519)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s10586-023-04159-8
Silva TRivera AOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)Representation learning via consistent assignment of views over random partitionsProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3667842(39582-39601)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3667842
Goldblum MSouri HNi RShu MPrabhu VSomepalli GChattopadhyay PIbrahim MBardes AHoffman JChellappa RWilson AGoldstein TOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)Battle of the backbonesProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3667399(29343-29371)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3667399
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents