short-paper

Open access

A Comparison of Video Browsing Performance between Desktop and Virtual Reality Interfaces

Authors:

Florian Spiess,

Luca RossettoAuthors Info & Claims

ICMR '23: Proceedings of the 2023 ACM International Conference on Multimedia Retrieval

Pages 535 - 539

https://doi.org/10.1145/3591106.3592292

Published: 12 June 2023 Publication History

All formats PDF

Abstract

Interactive retrieval with user-friendly and performant interfaces remains a necessity for video retrieval, even in light of significant gains in retrieval performance through multi-modal encoders. In recent years, novel interaction modalities such as virtual reality (VR) and augmented reality (AR) have gained popularity, but the best way to adapt paradigms from traditional retrieval interfaces, especially for result browsing and interaction, remains an open research question. In this paper, we compare two video retrieval interfaces in a controlled setting to gain insight into the differences in video browsing between VR and desktop interfaces. We formulate hypotheses explaining why there might be performance differences between the two interfaces, define metrics to test the hypotheses, and show results based on data gathered at an evaluation campaign. Our results show that VR interfaces can be competitive in browsing performance and indicate that there can even be an advantage when browsing larger result sets in VR.

References

[1]

Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, and Jenia Jitsev. 2022. Reproducible scaling laws for contrastive language-image learning. CoRR abs/2212.07143 (2022). https://doi.org/10.48550/arXiv.2212.07143 arXiv:2212.07143

[2]

Ralph Gasser, Luca Rossetto, Silvan Heller, and Heiko Schuldt. 2020. Cottontail DB: An Open Source Database System for Multimedia Retrieval and Analysis. In MM ’20: The 28th ACM International Conference on Multimedia, Virtual Event / Seattle, WA, USA, October 12-16, 2020. ACM, 4465–4468. https://doi.org/10.1145/3394171.3414538

Digital Library

[3]

Silvan Heller, Viktor Gsteiger, Werner Bailer, Cathal Gurrin, Björn Þór Jónsson, Jakub Lokoč, Andreas Leibetseder, František Mejzlík, Ladislav Peška, Luca Rossetto, Konstantin Schall, Klaus Schoeffmann, Heiko Schuldt, Florian Spiess, Ly-Duyen Tran, Lucia Vadicamo, Patrik Veselý, Stefanos Vrochidis, and Jiaxin Wu. 2022. Interactive Video Retrieval Evaluation at a Distance: Comparing Sixteen Interactive Video Search Systems in a Remote Setting at the 10th Video Browser Showdown. International Journal of Multimedia Information Retrieval 11, 1 (2022), 1–18. https://doi.org/10.1007/s13735-021-00225-2

[4]

Silvan Heller, Florian Spiess, and Heiko Schuldt. 2023. A Tale of Two Interfaces: Vitrivr at the Lifelog Search Challenge. Multimedia Tools and Applications (2023), 1–25. https://doi.org/10.1007/s11042-023-15082-w

Digital Library

[5]

Pascal Knierim, Thomas Kosch, Johannes Groschopp, and Albrecht Schmidt. 2020. Opportunities and Challenges of Text Input in Portable Virtual Reality. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, CHI 2020, Honolulu, HI, USA, April 25-30, 2020. ACM, 1–8. https://doi.org/10.1145/3334480.3382920

Digital Library

[6]

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 139). PMLR, 8748–8763. http://proceedings.mlr.press/v139/radford21a.html

[7]

Luca Rossetto, Ralph Gasser, Loris Sauter, Abraham Bernstein, and Heiko Schuldt. 2021. A System for Interactive Multimedia Retrieval Evaluations. In MultiMedia Modeling (MMM)(Lecture Notes in Computer Science, Vol. 12573). Springer, 385–390.

[8]

Luca Rossetto, Ivan Giangreco, and Heiko Schuldt. 2014. Cineast: A Multi-feature Sketch-Based Video Retrieval Engine. In International Symposium on Multimedia. IEEE Computer Society, 18–23. https://doi.org/10.1109/ISM.2014.38

Digital Library

[9]

Luca Rossetto, Heiko Schuldt, George Awad, and Asad A. Butt. 2019. V3C – A Research Video Collection. In MultiMedia Modeling. Springer International Publishing, Cham, 349–360. https://doi.org/10.1007/978-3-030-05710-7_29

[10]

Loris Sauter, Ralph Gasser, Silvan Heller, Luca Rossetto, Colin Saladin, Florian Spiess, and Heiko Schuldt. 2023. Exploring Effective Interactive Text-Based Video Search in vitrivr. In MultiMedia Modeling - 29th International Conference, MMM 2023, Bergen, Norway, January 9-12, 2023, Proceedings, Part I(Lecture Notes in Computer Science, Vol. 13833), Duc-Tien Dang-Nguyen, Cathal Gurrin, Martha A. Larson, Alan F. Smeaton, Stevan Rudinac, Minh-Son Dao, Christoph Trattner, and Phoebe Chen (Eds.). Springer, 646–651. https://doi.org/10.1007/978-3-031-27077-2_53

Digital Library

[11]

Florian Spiess, Ralph Gasser, Silvan Heller, Mahnaz Parian-Scherb, Luca Rossetto, Loris Sauter, and Heiko Schuldt. 2022. Multi-Modal Video Retrieval in Virtual Reality with Vitrivr-VR. In MultiMedia Modeling. Springer, 499–504. https://doi.org/10.1007/978-3-030-98355-0_45

Digital Library

[12]

Florian Spiess, Ralph Gasser, Heiko Schuldt, and Luca Rossetto. 2023. The Best of Both Worlds: Lifelog Retrieval with a Desktop-Virtual Reality Hybrid System. In Proceedings of the 2023 International Conference on Multimedia Retrieval (Thessaloniki, Greece) (ICMR ’23). Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3592573.3593107

Digital Library

[13]

Florian Spiess, Silvan Heller, Luca Rossetto, Loris Sauter, Philipp Weber, and Heiko Schuldt. 2023. Traceable Asynchronous Workflows in Video Retrieval with vitrivr-VR. In MultiMedia Modeling - 29th International Conference, MMM 2023, Bergen, Norway, January 9-12, 2023, Proceedings, Part I(Lecture Notes in Computer Science, Vol. 13833), Duc-Tien Dang-Nguyen, Cathal Gurrin, Martha A. Larson, Alan F. Smeaton, Stevan Rudinac, Minh-Son Dao, Christoph Trattner, and Phoebe Chen (Eds.). Springer, 622–627. https://doi.org/10.1007/978-3-031-27077-2_49

Digital Library

[14]

Florian Spiess, Philipp Weber, and Heiko Schuldt. 2022. Direct Interaction Word-Gesture Text Input in Virtual Reality. In International Conference on Artificial Intelligence and Virtual Reality. IEEE, 140–144. https://doi.org/10.1109/AIVR56993.2022.00028

[15]

Quang-Trung Truong, Tuan-Anh Vu, Tan-Sang Ha, Jakub Lokoc, Yue Him Wong Tim, Ajay Joneja, and Sai-Kit Yeung. 2023. Marine Video Kit: A New Marine Video Dataset for Content-Based Analysis and Retrieval. In MultiMedia Modeling - 29th International Conference, MMM 2023, Bergen, Norway, January 9-12, 2023, Proceedings, Part I(Lecture Notes in Computer Science, Vol. 13833). Springer, 539–550. https://doi.org/10.1007/978-3-031-27077-2_42

Digital Library

Cited By

Arnold RBailer WGasser RJónsson BKhan OSchuldt HSpiess FVadicamo LCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Multimedia Information Retrieval in XRProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3689176(11285-11286)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3689176
Vadicamo LArnold RBailer WCarrara FGurrin CHezel NLi XLokoc JLubos SMa ZMessina NNguyen TPeska LRossetto LSauter LSchöffmann KSpiess FTran MVrochidis S(2024)Evaluating Performance and Trends in Interactive Video Retrieval: Insights From the 12th VBS CompetitionIEEE Access10.1109/ACCESS.2024.340563812(79342-79366)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3405638
Spiess FRossetto LSchuldt H(2024)Exploring Multimedia Vector Spaces with vitrivr-VRMultiMedia Modeling10.1007/978-3-031-53302-0_27(317-323)Online publication date: 29-Jan-2024
https://dl.acm.org/doi/10.1007/978-3-031-53302-0_27
Show More Cited By

Index Terms

A Comparison of Video Browsing Performance between Desktop and Virtual Reality Interfaces
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Virtual reality
2. Information systems
  1. Information retrieval

Recommendations

An approach for enabling the use of immersive virtual reality in desktop hybrid interfaces
IASTED-HCI '07: Proceedings of the Second IASTED International Conference on Human Computer Interaction

Hybrid User Interfaces, which create a heterogeneous environment providing different interaction forms and devices, may be enhanced by exploring more extensively the mixed reality continuum, which ranges from the real world to the complete virtual world,...
Bringing Video Browsing to Virtual Reality: Empirical Evaluation of a Novel Multimedia Drawer
ICMR '24: Proceedings of the 2024 International Conference on Multimedia Retrieval

Virtual reality (VR) applications are increasingly permeating our lives. The immersion provided by VR enables novel interactions with data that would be impossible in conventional environments. Especially regarding multimedia data, VR could overcome ...
Evaluation of haptic virtual reality user interfaces for medical marking on 3D models
Highlights
- Two haptic VR interfaces are proposed to interact with volumetric medical images for improving current computer-aided medical diagnosis and planning.
- Compared with the traditional 2D interface using a screen-based 2D display and a ...
Abstract
Three-dimensional (3D) visualization has been widely used in computer-aided medical diagnosis and planning. To interact with 3D models, current user interfaces in medical systems mainly rely on the traditional 2D interaction techniques by ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMR '23: Proceedings of the 2023 ACM International Conference on Multimedia Retrieval

June 2023

694 pages

ISBN:9798400701788

DOI:10.1145/3591106

Editors:
Ioannis (Yiannis) Kompatsiaris
Centre for Research and Technology Hellas, Greece
,
Jiebo Luo
University of Rochester,USA
,
Nicu Sebe
University of Trento, Italy
,
Angela Yao
National University of Singapore, Singapore
,
Vasileios Mezaris
Centre for Research and Technology Hellas, Greece
,
Symeon Papadopoulos
Centre for Research and Technology Hellas, Greece
,
Adrian Popescu
CEA LIST, France
,
Zi (Helen) Huang
University of Queensland, Australia

Copyright © 2023 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 June 2023

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Funding Sources

Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung

Conference

ICMR '23

Sponsor:

SIGMM

ICMR '23: International Conference on Multimedia Retrieval

June 12 - 15, 2023

Thessaloniki, Greece

Acceptance Rates

Overall Acceptance Rate 254 of 830 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
255
Total Downloads

Downloads (Last 12 months)172
Downloads (Last 6 weeks)13

Reflects downloads up to 23 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Arnold RBailer WGasser RJónsson BKhan OSchuldt HSpiess FVadicamo LCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Multimedia Information Retrieval in XRProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3689176(11285-11286)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3689176
Vadicamo LArnold RBailer WCarrara FGurrin CHezel NLi XLokoc JLubos SMa ZMessina NNguyen TPeska LRossetto LSauter LSchöffmann KSpiess FTran MVrochidis S(2024)Evaluating Performance and Trends in Interactive Video Retrieval: Insights From the 12th VBS CompetitionIEEE Access10.1109/ACCESS.2024.340563812(79342-79366)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3405638
Spiess FRossetto LSchuldt H(2024)Exploring Multimedia Vector Spaces with vitrivr-VRMultiMedia Modeling10.1007/978-3-031-53302-0_27(317-323)Online publication date: 29-Jan-2024
https://dl.acm.org/doi/10.1007/978-3-031-53302-0_27
Spiess FGasser RSchuldt HRossetto L(2023)The Best of Both Worlds: Lifelog Retrieval with a Desktop-Virtual Reality Hybrid SystemProceedings of the 6th Annual ACM Lifelog Search Challenge10.1145/3592573.3593107(65-68)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3592573.3593107

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents