research-article

Experiments in interactive video search by addition and subtraction

Authors:

Matthew Cooper,

Jeremy PickensAuthors Info & Claims

CIVR '08: Proceedings of the 2008 international conference on Content-based image and video retrieval

Pages 465 - 474

https://doi.org/10.1145/1386352.1386412

Published: 07 July 2008 Publication History

Abstract

We have developed an interactive video search system that allows the searcher to rapidly assess query results and easily pivot off those results to form new queries. The system is intended to maximize the use of the discriminative power of the human searcher. This is accomplished by providing a hierarchical segmentation, streamlined interface, and redundant visual cues throughout. The typical search scenario includes a single searcher with the ability to search with text and content-based queries. In this paper, we evaluate new variations on our basic search system. In particular we test the system using only visual content-based search capabilities, and using paired searchers in a realtime collaboration. We present analysis and conclusions from these experiments.

References

[1]

J. Adcock, A. Girgensohn, M. Cooper, T. Liu, L. Wilcox, and E. Rieffel. Fxpal experiments for trecvid 2004. In Proceedings of the TREC Video Retrieval Evaluation (TRECVID), pages 70--81, Washington D.C., 2004. NIST.

[2]

J. Adcock, A. Girgensohn, M. Cooper, and L. Wilcox. Interactive video search using multilevel indexing. In International Conference on Image and Video Retrieval, pages 205--214, 2005.

Digital Library

[3]

J. A. Aslam and M. Montague. Models for metasearch. In SIGIR '01: Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, pages 276--284, New York, NY, USA, 2001. ACM.

Digital Library

[4]

H. Bay, T. Tuytelaars, and L. V. Gool. Surf: Speeded up robust features. In European Conference on Computer Vision, 2006.

Digital Library

[5]

M. W. Berry, S. T. Dumais, and G. W. O'Brien. Using linear algebra for intelligent information retrieval. SIAM Rev., 37(4):573--595, 1995.

Digital Library

[6]

L. Bottou and Y. Bengio. Convergence properties of the K-means algorithms. In G. Tesauro, D. Touretzky, and T. Leen, editors, Advances in Neural Information Processing Systems, volume 7, pages 585--592. The MIT Press, 1995.

[7]

C.-C. Chang and C.-J. Lin. LIBSVM: a library for support vector machines, 2001. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.

Digital Library

[8]

S.-F. Chang, D. Ellis, W. Jiang, K. Lee, A. Yanagawa, A. C. Loui, and J. Luo. Large-scale multimodal semantic concept detection for consumer video. In MIR'07: Proceedings of the international workshop on Workshop on multimedia information retrieval, pages 255--264, New York, NY, USA, 2007. ACM.

Digital Library

[9]

M. G. Christel and R. Yan. Merging storyboard strategies and automatic retrieval for improving interactive video search. In CIVR '07: Proceedings of the 6th ACM international conference on Image and video retrieval, pages 486--493, New York, NY, USA, 2007. ACM.

Digital Library

[10]

M. Cooper, J. Adcock, and F. Chen. Fxpal at trecvid 2006. In Proceedings of the TREC Video Retrieval Evaluation (TRECVID), 2006.

[11]

M. Cooper, J. Adcock, H. Zhou, and R. Chen. Fxpal at trecvid 2005. In Proceedings of the TREC Video Retrieval Evaluation (TRECVID), 2005.

[12]

P. I. Good. Permutation, Parametric, and Bootstrap Tests of Hypotheses (Springer Series in Statistics). Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2004.

Digital Library

[13]

A. G. Hauptmann, W.-H. Lin, R. Yan, J. Yang, and M.-Y. Chen. Extreme video retrieval: joint maximization of human and computer performance. In MULTIMEDIA '06: Proceedings of the 14th annual ACM international conference on Multimedia, pages 385--394, New York, NY, USA, 2006. ACM.

Digital Library

[14]

J. Huang, S. R. Kumar, M. Mitra, W.-J. Zhu, and R. Zabih. Image indexing using color correlograms. In CVPR '97: Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), page 762, Washington, DC, USA, 1997. IEEE Computer Society.

Digital Library

[15]

M. A. H. Huijbregts, R. J. F. Ordelman, and F. M. G. de Jong. Annotation of heterogeneous multimedia content using automatic speech recognition. In Proceedings of the Second International Conference on Semantic and Digital Media Technologies, SAMT 2007, Genoa, Italy, volume 4816 of Lecture Notes in Computer Science, pages 78--90. Springer Verlag, 2007.

Digital Library

[16]

X. Li, D. Wang, J. Li, and B. Zhang. Video search in concept subspace: a text-like paradigm. In CIVR '07: Proceedings of the 6th ACM international conference on Image and video retrieval, pages 603--610, New York, NY, USA, 2007. ACM.

Digital Library

[17]

H.-B. Luan, S.-X. Lin, S. Tang, S.-Y. Neo, and T.-S. Chua. Interactive spatio-temporal visual map model for web video retrieval. In Proc. IEEE International Conference on Multimedia and Expo, pages 560--563, 2-5 July 2007.

[18]

Lucene. Jakarta lucene. http://jakarta.apache.org/lucene/docs/index.html.

[19]

M. R. Morris. Interfaces for collaborative exploratory web search: Motivations and directions for multi-user designs. In CHI 2007 Workshop on Exploratory Search and HCI, 2007.

[20]

M. R. Naphade, L. Kennedy, J. R. Kender, S.-F. Chang, J. Smith, P.Over, and A.Hauptmann. A light scale concept ontology for multimedia understanding for trecvid 2005. Computer Science Technical Report RC23612 W0505-104, IBM, 2005.

[21]

P. Over, G. Awad, W. Kraiij, and A. Smeaton. Trecvid 2007 an overview. In Proceedings of the TRECVID 2007 Workshop, Nov. 2007.

[22]

C. Petersohn. Fraunhofer hhi at trecvid 2004: Shot boundary detection system. In Proceedings of the TREC Video Retrieval Evaluation (TRECVID), pages 64--69, Washington D.C., 2004. NIST.

[23]

A. F. Smeaton and P. Browne. A usage study of retrieval modalities for video shot retrieval. Inf. Process. Manage., 42(5):1330--1344, 2006.

Digital Library

[24]

A. F. Smeaton, H. Lee, C. Foley, S. McGivney, and C. Gurrin. Físchlár-diamondtouch: collaborative video searching on a table. In SPIE Electronic Imaging - Multimedia Content Analysis, Management, and Retrieval, pages 15--19, 2006.

[25]

A. F. Smeaton, P. Over, and W. Kraaij. Evaluation campaigns and trecvid. In MIR '06: Proceedings of the 8th ACM international workshop on Multimedia information retrieval, pages 321--330, New York, NY, USA, 2006. ACM.

Digital Library

[26]

C. Snoek, M. Worring, D. Koelma, and A. Smeulders. A learned lexicon-driven paradigm for interactive video retrieval. IEEE Trans. on Multimedia, 9(2):280--292, Feb. 2007.

Digital Library

[27]

C. G. M. Snoek, I. Everts, J. C. van Gemert, J.-M. Geusebroek, B. Huurnink, D. C. Koelma, M. van Liempt, O. de Rooij, K. E. A. van de Sande, A. W. M. Smeulders, J. R. R. Uijlings, and M. Worring. The mediamill trecvid 2007 semantic video search engine. In Proceedings of the 5th TRECVID Workshop, November 2007.

[28]

C. G. M. Snoek, M. Worring, J. C. van Gemert, J.-M. Geusebroek, and A. W. M. Smeulders. The challenge problem for automated detection of 101 semantic concepts in multimedia. In MULTIMEDIA '06: Proceedings of the 14th annual ACM international conference on Multimedia, pages 421--430, New York, NY, USA, 2006. ACM.

Digital Library

[29]

D. Tao, X. Tang, X. Li, and X. Wu. Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval. IEEE Trans. Pattern Anal. Mach. Intell., 28(7):1088--1099, 2006.

Digital Library

[30]

K. Wittenburg, C. Forlines, T. Lanning, A. Esenther, S. Harada, and T. Miyachi. Rapid serial visual presentation techniques for consumer digital video devices. In UIST '03: Proceedings of the 16th annual ACM symposium on User interface software and technology, pages 115--124, New York, NY, USA, 2003. ACM.

Digital Library

Cited By

Schoeffmann KHudelist MHuber J(2015)Video Interaction ToolsACM Computing Surveys10.1145/280879648:1(1-34)Online publication date: 29-Sep-2015
https://dl.acm.org/doi/10.1145/2808796
Huang YMa CWang HItoh TBottoni PTakahashi S(2015)Exploring the Benefits of Text and Sketch in Video Retrieval of Complex QueriesProceedings of the 8th International Symposium on Visual Information Communication and Interaction10.1145/2801040.2801063(115-120)Online publication date: 24-Aug-2015
https://dl.acm.org/doi/10.1145/2801040.2801063
Friedland GGottlieb LJanin A(2013)Narrative theme navigation for sitcoms supported by fan-generated scriptsMultimedia Tools and Applications10.1007/s11042-011-0877-z63:2(387-406)Online publication date: 1-Mar-2013
https://dl.acm.org/doi/10.1007/s11042-011-0877-z
Show More Cited By

Index Terms

Experiments in interactive video search by addition and subtraction
1. Information systems
  1. Information retrieval
    1. Information retrieval query processing
    2. Retrieval models and ranking

Recommendations

SQL-Like Interpretable Interactive Video Search
MultiMedia Modeling
Abstract
Concept-free search, which embeds text and video signals in a joint space for retrieval, appears to be a new state-of-the-art. However, this new search paradigm suffers from two limitations. First, the search result is unpredictable and not ...
Contextual Support for Collaborative Information Retrieval
CHIIR '16: Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval

Recent research shows that Collaborative Information Retrieval (CIR), in which two or more users collaborate on the same search task, has become increasingly popular. The presence of both search and collaboration behaviors makes CIR a complex search ...
An Interactive Video Search Tool: A Case Study Using the V3C1 Dataset
MultiMedia Modeling
Abstract
This paper presents a prototype of an interactive video search tool for the preparation of MMM 2021 Video Browser Showdown (VBS). Our tool is tailored to enable searching for the public V3C1 dataset associated with various analysis results ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CIVR '08: Proceedings of the 2008 international conference on Content-based image and video retrieval

July 2008

674 pages

ISBN:9781605580708

DOI:10.1145/1386352

General Chairs:
Jiebo Luo
Kodak Research Laboratories
,
Ling Guan
Ryerson University
,
Program Chairs:
Alan Hanjalic
Delft University of Technology
,
Mohan Kankanhalli
National University of Singapore
,
Ivan Lee
University of South Australia

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CIVR08

Sponsor:

CIVR08: CIVR'08 - International Conference on Content-based Image and Video Retrieval

July 7 - 9, 2008

Niagara Falls, Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
321
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Schoeffmann KHudelist MHuber J(2015)Video Interaction ToolsACM Computing Surveys10.1145/280879648:1(1-34)Online publication date: 29-Sep-2015
https://dl.acm.org/doi/10.1145/2808796
Huang YMa CWang HItoh TBottoni PTakahashi S(2015)Exploring the Benefits of Text and Sketch in Video Retrieval of Complex QueriesProceedings of the 8th International Symposium on Visual Information Communication and Interaction10.1145/2801040.2801063(115-120)Online publication date: 24-Aug-2015
https://dl.acm.org/doi/10.1145/2801040.2801063
Friedland GGottlieb LJanin A(2013)Narrative theme navigation for sitcoms supported by fan-generated scriptsMultimedia Tools and Applications10.1007/s11042-011-0877-z63:2(387-406)Online publication date: 1-Mar-2013
https://dl.acm.org/doi/10.1007/s11042-011-0877-z
Haesen MMeskens JLuyten KConinx KBecker JTuytelaars TPoulisse GPham PMoens M(2013)Finding a needle in a haystackMultimedia Tools and Applications10.1007/s11042-011-0809-y63:2(331-356)Online publication date: 1-Mar-2013
https://dl.acm.org/doi/10.1007/s11042-011-0809-y
Villa RJose J(2012)A study of awareness in multimedia searchInformation Processing and Management: an International Journal10.1016/j.ipm.2011.03.00548:1(32-46)Online publication date: 1-Jan-2012
https://dl.acm.org/doi/10.1016/j.ipm.2011.03.005
Mezaris VSidiropoulos PKompatsiaris I(2011)Improving Interactive Video Retrieval by Exploiting Automatically-Extracted Video Structural SemanticsProceedings of the 2011 IEEE Fifth International Conference on Semantic Computing10.1109/ICSC.2011.29(224-227)Online publication date: 18-Sep-2011
https://dl.acm.org/doi/10.1109/ICSC.2011.29
Schoeffmann K(2010)Video browsing interfaces and applications: a reviewJournal of Photonics for Energy10.1117/6.0000005(018004)Online publication date: 1-Feb-2010
https://doi.org/10.1117/6.0000005
Schreer OFeldmann IMediavilla IConcejero PSadka ASwash MBenini SLeonardi RJanjusevic TIzquierdo E(2010)RUSHES--an annotation and retrieval engine for multimedia semantic unitsMultimedia Tools and Applications10.1007/s11042-009-0375-848:1(23-49)Online publication date: 1-May-2010
https://dl.acm.org/doi/10.1007/s11042-009-0375-8
Snoek CWorring M(2009)Concept-Based Video RetrievalFoundations and Trends in Information Retrieval10.1561/15000000142:4(215-322)Online publication date: 1-Apr-2009
https://dl.acm.org/doi/10.1561/1500000014
Cooper MYan RTian QSmith JSukthankar R(2009)Image categorization combining neighborhood methods and boostingProceedings of the First ACM workshop on Large-scale multimedia retrieval and mining10.1145/1631058.1631063(11-18)Online publication date: 23-Oct-2009
https://dl.acm.org/doi/10.1145/1631058.1631063
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten