Article

Learned lexicon-driven interactive video retrieval

Authors:

Marcel Worring,

Arnold SmeuldersAuthors Info & Claims

CIVR'06: Proceedings of the 5th international conference on Image and Video Retrieval

Pages 11 - 20

https://doi.org/10.1007/11788034_2

Published: 13 July 2006 Publication History

Abstract

We combine in this paper automatic learning of a large lexicon of semantic concepts with traditional video retrieval methods into a novel approach to narrow the semantic gap. The core of the proposed solution is formed by the automatic detection of an unprecedented lexicon of 101 concepts. From there, we explore the combination of query-by-concept, query-by-example, query-by-keyword, and user interaction into the MediaMill semantic video search engine. We evaluate the search engine against the 2005 NIST TRECVID video retrieval benchmark, using an international broadcast news archive of 85 hours. Top ranking results show that the lexicon-driven search engine is highly effective for interactive video retrieval.

References

[1]

Flickner, M., et al.: Query by image and video content: The QBIC system. IEEE Computer 28(9) (1995) 23-32.

[2]

Chang, S.F., Chen, W., Men, H., Sundaram, H., Zhong, D.: A fully automated content-based video search engine supporting spatio-temporal queries. IEEE TCSVT 8(5) (1998) 602-615.

[3]

Rui, Y., Huang, T., Ortega, M., Mehrotra, S.: Relevance feedback: A power tool in interactive content-based image retrieval. IEEE TCSVT 8(5) (1998) 644-655.

[4]

Smeulders, A., Worring, M., Santini, S., Gupta, A., Jain, R.: Content based image retrieval at the end of the early years. IEEE TPAMI 22(12) (2000) 1349-1380.

[5]

Naphade, M., Huang, T.: A probabilistic framework for semantic video indexing, filtering, and retrieval. IEEE Trans. Multimedia 3(1) (2001) 141-151.

[6]

Amir, A., et al.: IBM research TRECVID-2003 video retrieval system. In: Proc. TRECVID Workshop, Gaithersburg, USA (2003).

[7]

Snoek, C., Worring, M., Geusebroek, J., Koelma, D., Seinstra, F., Smeulders, A.: The semantic pathfinder: Using an authoring metaphor for generic multimedia indexing. IEEE TPAMI (2006) in press.

[8]

Snoek, C., et al.: The MediaMill TRECVID 2005 semantic video search engine. In: Proc. TRECVID Workshop, Gaithersburg, USA (2005).

[9]

Rautiainen, M., Ojala, T., Seppänen, T.: Analysing the performance of visual, concept and text features in content-based video retrieval. In: ACM MIR, NY, USA (2004) 197-204.

[10]

Christel, M., Huang, C., Moraveji, N., Papernick, N.: Exploiting multiple modalities for interactive video retrieval. In: IEEE ICASSP. Volume 3., Montreal, CA (2004) 1032-1035.

[11]

Adcock, J., Cooper, M., Girgensohn, A., Wilcox, L.: Interactive video search using multilevel indexing. In: CIVR. Volume 3569 of LNCS., Springer-Verlag (2005) 205-214.

[12]

Smeaton, A.: Large scale evaluations of multimedia information retrieval: The TRECVid experience. In: CIVR. Volume 3569 of LNCS., Springer-Verlag (2005) 19-27.

[13]

Salton, G., McGill, M.: Introduction to Modern Information Retrieval. McGraw-Hill, New York, USA (1983).

[14]

Deerwester, S., Dumais, S., Furnas, G., Landauer, T., Harshman, R.: Indexing by latent semantic analysis. J. American Soc. Inform. Sci. 41(6) (1990) 391-407.

[15]

Lee, J.: Analysis of multiple evidence combination. In: ACM SIGIR. (1997) 267-276.

[16]

Petersohn, C.: Fraunhofer HHI at TRECVID 2004: Shot boundary detection system. In: Proc. TRECVID Workshop, Gaithersburg, USA (2004).

[17]

Naphade, et al.: A light scale concept ontology for multimedia understanding for TRECVID 2005. Technical Report RC23612, IBM T.J. Watson Research Center (2005).

[18]

Fellbaum, C., ed.: WordNet: an electronic lexical database. The MIT Press, Cambridge, USA (1998).

Cited By

Halvey MVallet DHannah DJose J(2014)Supporting exploratory video retrieval tasks with grouping and recommendationInformation Processing and Management: an International Journal10.1016/j.ipm.2014.06.00450:6(876-898)Online publication date: 1-Nov-2014
https://dl.acm.org/doi/10.1016/j.ipm.2014.06.004
Schoeffmann KBoeszoermenyi LDe Natale FDel Bimbo AHanjalic AManjunath BSatoh S(2011)Image and video browsing with a cylindrical 3D storyboardProceedings of the 1st ACM International Conference on Multimedia Retrieval10.1145/1991996.1992059(1-2)Online publication date: 18-Apr-2011
https://dl.acm.org/doi/10.1145/1991996.1992059
Viaud MBuisson OSaulnier AGuenais Cdel Bimbo AChang SSmeulders A(2010)Video explorationProceedings of the 18th ACM international conference on Multimedia10.1145/1873951.1874209(1311-1314)Online publication date: 25-Oct-2010
https://dl.acm.org/doi/10.1145/1873951.1874209
Show More Cited By

Index Terms

Learned lexicon-driven interactive video retrieval
1. Information systems
  1. Information retrieval
    1. Document representation

Recommendations

A Learned Lexicon-Driven Paradigm for Interactive Video Retrieval

Effective video retrieval is the result of interplay between interactive query selection, advanced visualization of results, and a goal-oriented human user. Traditional interactive video retrieval approaches emphasize paradigms, such as query-by-keyword ...
A Lexicon-Guided LSI Method for Semantic News Video Retrieval
Advances in Multimedia Information Processing – PCM 2007
Abstract
Many researchers try to utilize the semantic information extracted from visual feature to directly realize the semantic video retrieval or to supplement the automated speech recognition (ASR) text retrieval. But bridging the gap between the low-...
A lexicon-guided LSI method for semantic news video retrieval
PCM'07: Proceedings of the multimedia 8th Pacific Rim conference on Advances in multimedia information processing

Many researchers try to utilize the semantic information extracted from visual feature to directly realize the semantic video retrieval or to supplement the automated speech recognition (ASR) text retrieval. But bridging the gap between the low-level ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

CIVR'06: Proceedings of the 5th international conference on Image and Video Retrieval

July 2006

546 pages

ISBN:3540360182

Editors:
Hari Sundaram
Arts, Media and Engineering Program, Arizona State University, Tempe, AZ
,
Milind Naphade
Intelligent Information Management Department, IBM T.J. Watson Research Center, 19 Skyline Drive, Hawthorne, NY
,
John R. Smith
Intelligent Information Management Department, IBM T. J. Watson Research Center, 19 Skyline Drive, Hawthorne, NY
,
Yong Rui
Microsoft Corporation, Microsoft China R&D Group, 49 Zhichun Road, Beijing, NY, China

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 13 July 2006

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Halvey MVallet DHannah DJose J(2014)Supporting exploratory video retrieval tasks with grouping and recommendationInformation Processing and Management: an International Journal10.1016/j.ipm.2014.06.00450:6(876-898)Online publication date: 1-Nov-2014
https://dl.acm.org/doi/10.1016/j.ipm.2014.06.004
Schoeffmann KBoeszoermenyi LDe Natale FDel Bimbo AHanjalic AManjunath BSatoh S(2011)Image and video browsing with a cylindrical 3D storyboardProceedings of the 1st ACM International Conference on Multimedia Retrieval10.1145/1991996.1992059(1-2)Online publication date: 18-Apr-2011
https://dl.acm.org/doi/10.1145/1991996.1992059
Viaud MBuisson OSaulnier AGuenais Cdel Bimbo AChang SSmeulders A(2010)Video explorationProceedings of the 18th ACM international conference on Multimedia10.1145/1873951.1874209(1311-1314)Online publication date: 25-Oct-2010
https://dl.acm.org/doi/10.1145/1873951.1874209
Halvey MVallet DHannah DFeng YJose J(2010)An asynchronous collaborative search system for online video searchInformation Processing and Management: an International Journal10.1016/j.ipm.2009.11.00746:6(733-748)Online publication date: 1-Nov-2010
https://dl.acm.org/doi/10.1016/j.ipm.2009.11.007
Halvey MJose JKompatsiaris YMarchand-Maillet SAvrithis YConnor NGatica-Perez DChua T(2009)The role of expertise in aiding video searchProceedings of the ACM International Conference on Image and Video Retrieval10.1145/1646396.1646407(1-8)Online publication date: 8-Jul-2009
https://dl.acm.org/doi/10.1145/1646396.1646407
Halvey MVallet DHannah DJose JHeath FRice-Lively MFuruta R(2009)ViGORProceedings of the 9th ACM/IEEE-CS joint conference on Digital libraries10.1145/1555400.1555415(87-96)Online publication date: 15-Jun-2009
https://dl.acm.org/doi/10.1145/1555400.1555415
Yang YHsu WChen H(2009)Online reranking via ordinal informative concepts for context fusion in concept detection and video searchIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2009.202697819:12(1880-1890)Online publication date: 1-Dec-2009
https://dl.acm.org/doi/10.1109/TCSVT.2009.2026978
Hopfgartner FVallet DHalvey MJose JEL Saddik AVuong SGriwodz CDel Bimbo ACandan KJaimes A(2008)Search trails using user feedback to improve video searchProceedings of the 16th ACM international conference on Multimedia10.1145/1459359.1459405(339-348)Online publication date: 26-Oct-2008
https://dl.acm.org/doi/10.1145/1459359.1459405
Spyrou EAvrithis Y(2007)A region thesaurus approach for high-level concept detection in the natural disaster domainProceedings of the semantic and digital media technologies 2nd international conference on Semantic Multimedia10.5555/1780533.1780543(74-77)Online publication date: 5-Dec-2007
https://dl.acm.org/doi/10.5555/1780533.1780543
Christel MLienhart RPrasad AHanjalic AChoi SBailey BSebe N(2007)Establishing the utility of non-text search for news video retrieval with real world usersProceedings of the 15th ACM international conference on Multimedia10.1145/1291233.1291395(707-716)Online publication date: 29-Sep-2007
https://dl.acm.org/doi/10.1145/1291233.1291395
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Table of Contents