research-article

Using an Interactive Video Retrieval Tool for LifeLog Data

Authors:

Tomáš Souček,

Gregor KovalčikAuthors Info & Claims

LSC '18: Proceedings of the 2018 ACM Workshop on The Lifelog Search Challenge

Pages 15 - 19

https://doi.org/10.1145/3210539.3210543

Published: 06 June 2018 Publication History

Abstract

Known-item search in multimodal lifelog data represents a challenging task for present search engines. Since sequences of temporally close images represent a significant part of the provided data, an interactive video retrieval tool with few extensions could be confronted at Lifelog Search Challenge in known-item search tasks. We present an update of the SIRET interactive video retrieval tool that recently won the Video Browser Showdown 2018. As the tool relies on frame-based representations and retrieval models, it can be directly used also for images from lifelog cameras. The updates comprise mostly visualization and navigation methods for a high number of visually similar scenes representing repetitive daily activities.

References

[1]

Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. (2015). https://www.tensorflow.org/ Software available from tensorflow.org.

[2]

George Awad, Asad Butt, Jonathan Fiscus, Martial Michel, David Joy, Wessel Kraaij, Alan F. Smeaton, Georges Quénot, Maria Eskevich, Roeland Ordelman, Gareth J. F. Jones, and Benoit Huet. 2017. TRECVID 2017: Evaluating Ad-hoc and Instance Video Search, Events Detection, Video Captioning and Hyperlinking. In Proceedings of TRECVID 2017. NIST, USA.

[3]

Ricardo A. Baeza-Yates and Berthier A. Ribeiro-Neto. 2011. Modern Information Retrieval - the concepts and technology behind search, Second edition. Pearson Education Ltd., Harlow, England.

Digital Library

[4]

Adam Blazek, David Kubon, and Jakub Lokoc. 2016. Known-Item Search in Video Databases with Textual Queries. In Similarity Search and Applications - 9th International Conference, SISAP 2016, Tokyo, Japan, October 24--26, 2016. Proceedings. 117--124.

[5]

Adam Blazek, Jakub Lokoc, Filip Matzner, and Tomás Skopal. 2015. Enhanced Signature-Based Video Browser. In MultiMedia Modeling - 21st International Conference, MMM 2015, Sydney, NSW, Australia, January 5--7, 2015, Proceedings, Part II. 243--248.

[6]

Adam Blazek, Jakub Lokoc, and Tomás Skopal. 2014. Video Retrieval with Feature Signature Sketches. In Similarity Search and Applications - 7th International Conference, SISAP 2014, Los Cabos, Mexico, October 29--31, 2014. Proceedings. 25--36.

[7]

Edgar Chávez, Gonzalo Navarro, Ricardo A. Baeza-Yates, and José L. Marroquín. 2001. Searching in metric spaces. ACM Comput. Surv. 33, 3 (2001), 273--321.

Digital Library

[8]

Claudiu Cobârzan, Klaus Schoeffmann, Werner Bailer, Wolfgang Hürst, Adam Blazek, Jakub Lokoc, Stefanos Vrochidis, Kai Uwe Barthel, and Luca Rossetto. 2017. Interactive video search tools: a detailed analysis of the video browser showdown 2015. Multimedia Tools Appl. 76, 4 (2017), 5539--5571.

Digital Library

[9]

J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. 2009. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR09.

[10]

Cathal Gurrin, Alan F. Smeaton, and Aiden R. Doherty. 2014. LifeLogging: Personal Big Data. Foundations and Trends in Information Retrieval 8, 1 (2014), 1--125.

Digital Library

[11]

Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross B. Girshick. 2017. Mask R-CNN. CoRR abs/1703.06870 (2017). arXiv:1703.06870 http://arxiv.org/abs/1703. 06870

[12]

Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. CoRR abs/1412.6980 (2014). arXiv:1412.6980 http://arxiv.org/abs/1412.6980

[13]

J. Lokoc, W. Bailer, K. Schoeffmann, B. Muenzer, and G. Awad. 2018. On influential trends in interactive video retrieval: Video Browser Showdown 2015--2017. IEEE Transactions on Multimedia (2018), 1--1.

[14]

Jakub Lokoc, Adam Blazek, and Tomás Skopal. 2014. Signature-Based Video Browser. In MultiMedia Modeling - 20th Anniversary International Conference, MMM 2014, Dublin, Ireland, January 6--10, 2014, Proceedings, Part II. 415--418.

Digital Library

[15]

Jakub Lokoc, Gregor Kovalcík, and Tomás Soucek. 2018. Revisiting SIRET Video Retrieval Tool. In MultiMedia Modeling - 24th International Conference, MMM 2018, Bangkok, Thailand, February 5--7, 2018, Proceedings, Part II. 419--424.

[16]

Jakub Lokoc, Phuong Anh Nguyen, Marta Vomlelová, and Chong-Wah Ngo. 2017. Color-Sketch Simulator: A Guide for Color-Based Visual Known-Item Search. In Advanced Data Mining and Applications - 13th International Conference, ADMA 2017, Singapore, November 5--6, 2017, Proceedings. 754--763.

[17]

George A. Miller. 1995. WordNet: A Lexical Database for English. Commun. ACM 38, 11 (Nov. 1995), 39--41.

Digital Library

[18]

Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. 2015. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV) 115, 3 (2015), 211--252.

Digital Library

[19]

Klaus Schoeffmann, David Ahlström, Werner Bailer, Claudiu Cobârzan, Frank Hopfgartner, Kevin McGuinness, Cathal Gurrin, Christian Frisson, Duy-Dinh Le, Manfred del Fabro, Hongliang Bai, and Wolfgang Weiss. 2014. The Video Browser Showdown: a live evaluation of interactive video search tools. IJMIR 3, 2 (2014), 113--127.

[20]

Klaus Schoeffmann, Marco A. Hudelist, and Jochen Huber. 2015. Video Interaction Tools: A Survey of Recent Work. ACM Comput. Surv. 48, 1 (2015), 14:1--14:34.

Digital Library

[21]

Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott E. Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2014. Going Deeper with Convolutions. CoRR abs/1409.4842 (2014). arXiv:1409.4842 http://arxiv.org/abs/1409.4842

[22]

Peng Wang, Lifeng Sun, Shiqiang Yang, Alan F. Smeaton, and Cathal Gurrin. 2016. Characterizing everyday activities from visual lifelogs based on enhancing concept representation. Computer Vision and Image Understanding 148 (2016), 181--192.

Digital Library

[23]

Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V. Le. 2017. Learning Transferable Architectures for Scalable Image Recognition. CoRR abs/1707.07012 (2017). arXiv:1707.07012 http://arxiv.org/abs/1707.07012

Cited By

Ribeiro RTrifan ANeves A(2022)Lifelog Retrieval From Daily Digital Data: Narrative ReviewJMIR mHealth and uHealth10.2196/3051710:5(e30517)Online publication date: 2-May-2022
https://doi.org/10.2196/30517
Zhou PBai CXia JChen S(2022)CMRDF: A Real-Time Food Alerting System Based on Multimodal DataIEEE Internet of Things Journal10.1109/JIOT.2020.29960099:9(6335-6349)Online publication date: 1-May-2022
https://doi.org/10.1109/JIOT.2020.2996009
Rossetto LGasser RLokoc JBailer WSchoeffmann KMuenzer BSoucek TNguyen PBolettieri PLeibetseder AVrochidis S(2021)Interactive Video Retrieval in the Age of Deep Learning – Detailed Evaluation of VBS 2019IEEE Transactions on Multimedia10.1109/TMM.2020.298094423(243-256)Online publication date: 2021
https://doi.org/10.1109/TMM.2020.2980944
Show More Cited By

Index Terms

Using an Interactive Video Retrieval Tool for LifeLog Data
1. Information systems
  1. Information retrieval
    1. Specialized information retrieval
      1. Multimedia and multimodal retrieval
        Video search

Recommendations

MemoriEase: An Interactive Lifelog Retrieval System for LSC’23
LSC '23: Proceedings of the 6th Annual ACM Lifelog Search Challenge

Lifelogging is an activity of recording all events that happen in the daily life of an individual. The events can contain images, audio, health index, etc which are collected through various devices such as wearable cameras, smartwatches, and other ...
Balancing thread based navigation for targeted video search
CIVR '08: Proceedings of the 2008 international conference on Content-based image and video retrieval

Various query methods for video search exist. Because of the semantic gap each method has its limitations. We argue that for effective retrieval query methods need to be combined at retrieval time. However, switching query methods often involves a ...
Myscéal: An Experimental Interactive Lifelog Retrieval System for LSC'20
LSC '20: Proceedings of the Third Annual Workshop on Lifelog Search Challenge

The Lifelog Search Challenge (LSC), is an annual comparative bench-marking activity for comparing approaches to interactive retrieval from multi-modal lifelogs. Being an interactive search challenge, issues such as retrieval accuracy, search speed and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

LSC '18: Proceedings of the 2018 ACM Workshop on The Lifelog Search Challenge

June 2018

43 pages

ISBN:9781450357968

DOI:10.1145/3210539

Program Chairs:
Cathal Gurrin
Dublin City University, Ireland
,
Klaus Schoeffmann
Klagenfurt University, Austria
,
Hideo Joho
University of Tsukuba, Japan
,
Duc-Tien Dang-Nguyen
Dublin City University, Ireland
,
Michael Riegler
Oslo Metropolitan University, Norway
,
Luca Piras
University of Cagliari, Italy

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 June 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Grantová Agentura České Republiky

Conference

ICMR '18

Sponsor:

SIGMM

ICMR '18: International Conference on Multimedia Retrieval

June 11, 2018

Yokohama, Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
244
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)1

Reflects downloads up to 28 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ribeiro RTrifan ANeves A(2022)Lifelog Retrieval From Daily Digital Data: Narrative ReviewJMIR mHealth and uHealth10.2196/3051710:5(e30517)Online publication date: 2-May-2022
https://doi.org/10.2196/30517
Zhou PBai CXia JChen S(2022)CMRDF: A Real-Time Food Alerting System Based on Multimodal DataIEEE Internet of Things Journal10.1109/JIOT.2020.29960099:9(6335-6349)Online publication date: 1-May-2022
https://doi.org/10.1109/JIOT.2020.2996009
Rossetto LGasser RLokoc JBailer WSchoeffmann KMuenzer BSoucek TNguyen PBolettieri PLeibetseder AVrochidis S(2021)Interactive Video Retrieval in the Age of Deep Learning – Detailed Evaluation of VBS 2019IEEE Transactions on Multimedia10.1109/TMM.2020.298094423(243-256)Online publication date: 2021
https://doi.org/10.1109/TMM.2020.2980944
Kovalčík GŠkrhak VSouček TLokoč JGurrin CSchoeffmann KÞór Jónsson BDang-Nguyen DLokoč JTran MHürst W(2020)VIRET Tool with Advanced Visual Browsing and FeedbackProceedings of the Third Annual Workshop on Lifelog Search Challenge10.1145/3379172.3391725(63-66)Online publication date: 9-Jun-2020
https://dl.acm.org/doi/10.1145/3379172.3391725
Gurrin CSchoeffmann KJoho HLeibetseder AZhou LDuane ADang-Nguyen DRiegler MPiras LTran MLokoč JHürst W(2019)[Invited papers] Comparing Approaches to Interactive Lifelog Search at the Lifelog Search Challenge (LSC2018)ITE Transactions on Media Technology and Applications10.3169/mta.7.467:2(46-59)Online publication date: 2019
https://doi.org/10.3169/mta.7.46
Le TNinh VDang-Nguyen DTran MZhou LRedondo PSmyth SGurrin CGurrin CSchoeffmann KJoho HDang-Nguyen DRiegler MPiras L(2019)LifeSeekerProceedings of the ACM Workshop on Lifelog Search Challenge10.1145/3326460.3329162(37-40)Online publication date: 5-Jun-2019
https://dl.acm.org/doi/10.1145/3326460.3329162
Nguyen SLe DPham UCrane MHealy GGurrin CGurrin CSchoeffmann KJoho HDang-Nguyen DRiegler MPiras L(2019)VieLens,Proceedings of the ACM Workshop on Lifelog Search Challenge10.1145/3326460.3329161(33-35)Online publication date: 5-Jun-2019
https://dl.acm.org/doi/10.1145/3326460.3329161
Rossetto LGasser RHeller SAmiri Parian MSchuldt HGurrin CSchoeffmann KJoho HDang-Nguyen DRiegler MPiras L(2019)Retrieval of Structured and Unstructured Data with vitrivrProceedings of the ACM Workshop on Lifelog Search Challenge10.1145/3326460.3329160(27-31)Online publication date: 5-Jun-2019
https://dl.acm.org/doi/10.1145/3326460.3329160
Lokoč JSouček TČech PKovalčík GGurrin CSchoeffmann KJoho HDang-Nguyen DRiegler MPiras L(2019)Enhanced VIRET Tool for Lifelog DataProceedings of the ACM Workshop on Lifelog Search Challenge10.1145/3326460.3329159(25-26)Online publication date: 5-Jun-2019
https://dl.acm.org/doi/10.1145/3326460.3329159
Nguyen Van Khan IShrestha PZhang MLiu YMa SGurrin CSchoeffmann KJoho HDang-Nguyen DRiegler MPiras L(2019)A Two-Level Lifelog Search Engine at the LSC 2019Proceedings of the ACM Workshop on Lifelog Search Challenge10.1145/3326460.3329158(19-23)Online publication date: 5-Jun-2019
https://dl.acm.org/doi/10.1145/3326460.3329158
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents