article

Techniques used and open challenges to the analysis, indexing and retrieval of digital video

Author:

Alan F. SmeatonAuthors Info & Claims

Information Systems, Volume 32, Issue 4

Pages 545 - 559

https://doi.org/10.1016/j.is.2006.09.001

Published: 01 June 2007 Publication History

Abstract

Video in digital format is now commonplace and widespread in both professional use, and in domestic consumer products from camcorders to mobile phones. Video content is growing in volume and while we can capture, compress, store, transmit and display video with great facility, editing videos and manipulating them based on their content is still a non-trivial activity. In this paper, we give a brief review of the state of the art of video analysis, indexing and retrieval and we point to research directions which we think are promising and could make searching and browsing of video archives based on video content, as easy as searching and browsing (text) web pages. We conclude the paper with a list of grand challenges for researchers working in the area.

References

[1]

P. Lyman, H.R. Varian, How much information? (last checked June 2006), {http://www2.sims.berkeley.edu/research/projects/how-much-info-2003/}.

[2]

P. Ingwersen, K. Järvelin, The Turn: Integration of Information Seeking and Retrieval in Context, the Kluwer International Series on Information Retrieval, Springer, Berlin, 2005.

[3]

Hanjalic, A., Shot boundary detection: unraveled and resolved. IEEE Trans. Circuits Syst. Video Technol. v12 i2. 90-105.

[4]

B. Manjunath, P. Salembier, T. Sikora (Eds.), Introduction to MPEG-7: Multimedia Content Description Language, Wiley, New York, 2002.

Digital Library

[5]

A.F. Smeaton, Large scale evaluations of multimedia information retrieval: the TRECVid experience, in: W.-K. Leow, M.S. Lew, T.-S. Chua, W.-Y. Ma, L. Chaisorn, E. M. Bakker (Eds.), CIVR 2005-Fourth International Conference on Image and Video Retrieval, Singapore, Lecture Notes in Computer Science, vol. 3569, Springer, Singapore, 2005, pp. 11-17.

[6]

TRECVid video retrieval evaluation (last checked July 2006), {http://www-nlpir.nist.gov/projects/t01v/t01v.html}.

[7]

The Internet archive: moving image archive (last checked June 2006), {http://www.internet.archive/details/movies/}.

[8]

The open video project: a shared video collection (last checked June 2006), {http://www.open-video.org/}.

[9]

H. Lee, A.F. Smeaton, N.E. O'Connor, B. Smyth, User evaluation of Físchlár-News: an automatic broadcast news delivery system, ACM Trans. Inf. Syst. 24 (2).

[10]

The Google video search engine (last checked June 2006), {http://www.google.com/video}.

[11]

T. Joachims, Text categorization with support vector machines: learning with many relevant features, in: C. Nédellec, C. Rouveirol (Eds.), Proceedings of ECML-98, 10th European Conference on Machine Learning, vol. 1398, Springer, Heidelberg, DE, Chemnitz, DE, 1998, pp. 137-142.

[12]

LSCOM lexicon definitions and annotations (last checked June 2006), {http://www.ee.columbia.edu/dvmm/lscom}.

[13]

M. Naphade, J.R. Smith, J. Tesic, S.-F. Chang, W. Hsu, L. Kennedy, A. Hauptmann, J. Curtis, Large-scale concept ontology for multimedia, IEEE Multimedia 13(2).

[14]

M. Rautiainen, T. Ojala, T. Seppänen, Cluster-temporal browsing of large news video databases, in: Proceedings of the 2004 IEEE International Conference on Multimedia and Expo, 2004, pp. 751-754.

[15]

C. Snoek, M. Worring, D. Koelma, A. Smeulders, Learned lexicon-driven interactive video retrieval, in: H.Sundaram, et al., (Eds.), CIVR 2006-International Conference on Image and Video Retrieval, Lecture Notes in Computer Science, vol. 4071, Springer, Tempe, Ariz., 2006, pp. 11-20.

[16]

J. Sivic, F. Schaffalitzky, A. Zisserman, Efficient object retrieval from videos, in: Proceedings of the 12th European Signal Processing Conference (EUSIPCO '04), Vienna, Austria, 2004, pp. 1737-1740.

[17]

S. Sav, G.J.F. Jones, H. Lee, N.E. O'Connor, A.F. Smeaton, Interactive experiments in object-based retrieval, in: H. Sundaram, M.R. Naphade, J.R. Smith, Y. Rui (Eds.), CIVR 2006-International Conference on Image and Video Retrieval, Lecture Notes in Computer Science, vol. 4071, Tempe, Ariz., 2006, pp. 1-10.

[18]

Sadlier, D. and O'Connor, N., Event detection in field sports video using audio-visual features and a support vector machine. IEEE Trans. Circuits Syst. Video Technol. v15 i10. 1225-1233.

[19]

A.G. Hauptmann, Lessons for the future from a decade of informedia video analysis research., in: W.K. Leow, M.S. Lew, T.-S. Chua, W.-Y. Ma, L. Chaisorn, E.M. Bakker (Eds.), CIVR 2005-Fourth International Conference on Image and Video Retrieval, Singapore, Lecture Notes in Computer Science, vol. 3568, Springer, Berlin, 2005, pp. 1-10.

[20]

R. Yan, J. Yang, A. G. Hauptmann, Learning query-class dependent weights in automatic video retrieval, in: MULTIMEDIA '04: Proceedings of the 12th Annual ACM International Conference on Multimedia, ACM Press, New York, NY, USA, 2004, pp. 548-555.

Digital Library

Cited By

Spolaôr NLee HTakaki WEnsina LCoy CWu F(2020)A systematic review on content-based video retrievalEngineering Applications of Artificial Intelligence10.1016/j.engappai.2020.10355790:COnline publication date: 1-Apr-2020
https://dl.acm.org/doi/10.1016/j.engappai.2020.103557
Mazaheri AGong BShah M(2018)Learning a Multi-Concept Video Retrieval Model with Multiple Latent VariablesACM Transactions on Multimedia Computing, Communications, and Applications10.1145/317664714:2(1-21)Online publication date: 25-Apr-2018
https://dl.acm.org/doi/10.1145/3176647
Jacob HPádua FLacerda APereira A(2017)A video summarization approach based on the emulation of bottom-up mechanisms of visual attentionJournal of Intelligent Information Systems10.1007/s10844-016-0441-449:2(193-211)Online publication date: 1-Oct-2017
https://dl.acm.org/doi/10.1007/s10844-016-0441-4
Show More Cited By

Index Terms

Techniques used and open challenges to the analysis, indexing and retrieval of digital video
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
2. Information systems
  1. Information retrieval
    1. Document representation
    2. Search engine architectures and scalability
      1. Search engine indexing

Recommendations

Multimodal Video Retrieval with the 2017 IMOTION System
ICMR '17: Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval

The IMOTION system is a multimodal content-based video search and browsing application offering a rich set of query modes on the basis of a broad range of different features. It is able to scale with the size of the collection due to its underlying ...
Content-based video retrieval: does video's semantic visual feature matter?
SIGIR '06: Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval

A new shot level video browsing method based on semantic visual features (e.g., car, mountain, and fire) is proposed to facilitate content-based retrieval. The video's binary semantic feature vector is utilized to calculate the score of similarity ...
Video parsing, retrieval and browsing: an integrated and content-based solution
Readings in information retrieval

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

Copyright © Elsevier B.V. © 2006.

Publisher

Elsevier Science Ltd.

United Kingdom

Publication History

Published: 01 June 2007

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 26 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Spolaôr NLee HTakaki WEnsina LCoy CWu F(2020)A systematic review on content-based video retrievalEngineering Applications of Artificial Intelligence10.1016/j.engappai.2020.10355790:COnline publication date: 1-Apr-2020
https://dl.acm.org/doi/10.1016/j.engappai.2020.103557
Mazaheri AGong BShah M(2018)Learning a Multi-Concept Video Retrieval Model with Multiple Latent VariablesACM Transactions on Multimedia Computing, Communications, and Applications10.1145/317664714:2(1-21)Online publication date: 25-Apr-2018
https://dl.acm.org/doi/10.1145/3176647
Jacob HPádua FLacerda APereira A(2017)A video summarization approach based on the emulation of bottom-up mechanisms of visual attentionJournal of Intelligent Information Systems10.1007/s10844-016-0441-449:2(193-211)Online publication date: 1-Oct-2017
https://dl.acm.org/doi/10.1007/s10844-016-0441-4
Kurabayashi SHanaoka H(2017)A Large-Scale Visual Check-In System for TV Content-Aware Web with Client-Side Video Analysis OffloadingWeb Information Systems Engineering – WISE 201710.1007/978-3-319-68786-5_13(159-174)Online publication date: 7-Oct-2017
https://dl.acm.org/doi/10.1007/978-3-319-68786-5_13
Holla KShekar B(2016)Video Retrieval based on Patterns of Oriented Edge MagnitudeProceedings of the Third International Symposium on Computer Vision and the Internet10.1145/2983402.2983433(115-120)Online publication date: 21-Sep-2016
https://dl.acm.org/doi/10.1145/2983402.2983433
Fernandez-Beltran RPla F(2016)Latent topics-based relevance feedback for video retrievalPattern Recognition10.1016/j.patcog.2015.09.00751:C(72-84)Online publication date: 1-Mar-2016
https://dl.acm.org/doi/10.1016/j.patcog.2015.09.007
Xian YLi JZhang CLiao ZSirivianos MHui PSastry NChen YLiu H(2015)Video Highlight Shot Extraction with Time-Sync CommentProceedings of the 7th International Workshop on Hot Topics in Planet-scale mObile computing and online Social neTworking10.1145/2757513.2757516(31-36)Online publication date: 22-Jun-2015
https://dl.acm.org/doi/10.1145/2757513.2757516
Pereira MSouza CPádua FSilva GAssis GPereira A(2015)SAPTEMultimedia Tools and Applications10.1007/s11042-014-2311-974:23(10923-10963)Online publication date: 1-Dec-2015
https://dl.acm.org/doi/10.1007/s11042-014-2311-9
Memar SAffendey LMustapha NDoraisamy SEktefa M(2013)An integrated semantic-based approach in concept based video retrievalMultimedia Tools and Applications10.1007/s11042-011-0848-464:1(77-95)Online publication date: 1-May-2013
https://dl.acm.org/doi/10.1007/s11042-011-0848-4
Choroś K(2012)Video structure analysis for content-based indexing and categorisation of TV sports newsInternational Journal of Intelligent Information and Database Systems10.1504/IJIIDS.2012.0493066:5(451-465)Online publication date: 1-Sep-2012
https://dl.acm.org/doi/10.1504/IJIIDS.2012.049306
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents