Enhancing Direct Visual Odometry with Deblurring and Saliency Maps
Authors: 
Magnus Kaufmann Gjerde, Kamal Nasrollahi, Thomas Moeslund, Joakim Bruslund HaurumAuthors Info & Claims
Pages 154 - 161
Abstract
In this paper, we investigate the field of direct visual odometry and specifically the implementation of hybrid approaches between deep learning and classical hand-crafted methods. We introduce a new approach that integrates a deblurring module with a saliency predictor to perform better point sampling which increases trajectory estimation accuracy in blurry frames, often caused by rapid camera movements or long exposure times in dimly lit conditions. Benchmark testing against DSO and SalientDSO on the EuRoC MAV dataset demonstrated consistent improvements, with the proposed system achieving an average Absolute Trajectory Error (ATE) of 0.26m, compared to 0.335m for DSO and 0.303m for SalientDSO.
References
[1]
Michael Burri, Janosch Nikolic, Pascal Gohl, Thomas Schneider, Joern Rehder, Sammy Omari, Markus W Achtelik, and Roland Siegwart. 2016. The EuRoC micro aerial vehicle datasets. The International Journal of Robotics Research (2016). https://doi.org/10.1177/0278364915620033
[2]
C. Cadena, L. Carlone, H. Carrillo, Y. Latif, D. Scaramuzza, J. Neira, I. Reid, and J. J. Leonard. 2016. Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age. IEEE Transactions on Robotics 32, 6 (2016), 1309–1332. https://doi.org/10.1109/TRO.2016.2624754
[3]
Jakob Engel, Vladlen Koltun, and Daniel Cremers. 2016. Direct sparse odometry. IEEE transactions on pattern analysis and machine intelligence 40, 3 (2016), 611–625.
[4]
Andreas Geiger, Philip Lenz, and Raquel Urtasun. 2012. Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite. In Conference on Computer Vision and Pattern Recognition (CVPR).
[5]
Sungchul Hong, Antyanta Bangunharcana, Jae-Min Park, Minseong Choi, and Hyu-Soung Shin. 2021. Visual SLAM-Based Robotic Mapping Method for Planetary Construction. Sensors 21, 22 (2021), 7715. https://doi.org/10.3390/s21227715
[6]
Iman Abaspur Kazerouni, Luke Fitzgerald, Gerard Dooly, and Daniel Toal. 2021. A survey of state-of-the-art on visual SLAM. Expert Systems With Applications 143 (2021), 103778.
[7]
Jongseok Lee, Ribin Balachandran, Konstantin Kondak, Andre Coelho, Marco De Stefano, Matthias Humt, Jianxiang Feng, Tamim Asfour, and Rudolph Triebel. [n. d.]. Virtual Reality via Object Pose Estimation and Active Learning: Realizing Telepresence Robots with Aerial Manipulation Capabilities. 2020 IEEE International Conference on Robotics and Automation (ICRA) ([n. d.]). https://www.youtube.com/watch?v=JRnPIARW8xY
[8]
Huai-Jen Liang, Nitin J. Sanket, Cornelia Fermüller, and Yiannis Aloimonos. 2019. SalientDSO: Bringing Attention to Direct Sparse Odometry. IEEE Transactions on Automation Science and Engineering 16, 4 (2019), 1619–1626. https://doi.org/10.1109/TASE.2019.2900980
[9]
Peidong Liu, Xingxing Zuo, Viktor Larsson, and Marc Pollefeys. 2021. MBA-VO: Motion blur aware visual odometry. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 5550–5559.
[10]
Jianxun Lou, Hanhe Lin, David Marshall, Dietmar Saupe, and Hantao Liu. 2022. TranSalNet: Towards perceptually relevant visual saliency prediction. Neurocomputing 494 (2022), 455–467.
[11]
Raúl Mur-Artal and Juan D. Tardós. 2017. ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras. IEEE Transactions on Robotics 33, 5 (2017), 1255–1262. https://doi.org/10.1109/TRO.2017.2705103
[12]
Navvis. 2023. Reality Capture. https://www.navvis.com/reality-capture Accessed: 2023-05-31.
[13]
Paweł Nowacki and Marek Woda. 2020. Capabilities of ARCore and ARKit Platforms for AR/VR Applications. In Engineering in Dependability of Computer Systems and Networks, Wojciech Zamojski, Jacek Mazurkiewicz, Jarosław Sugier, Tomasz Walkowiak, and Janusz Kacprzyk (Eds.). Springer International Publishing, Cham, 358–370.
[14]
David Scaradozzi, Silvia Zingaretti, and Arianna Ferrari. 2018. Simultaneous localization and mapping (SLAM) robotics techniques: a possible application in surgery. Shanghai Chest 2 (1 2018). Issue 1. https://doi.org/10.21037/SHC.2018.01.01
[15]
D. Schubert, T. Goll, N. Demmel, V. Usenko, J. Stueckler, and D. Cremers. 2018. The TUM VI Benchmark for Evaluating Visual-Inertial Odometry. In International Conference on Intelligent Robots and Systems (IROS).
[16]
Abhishek Singh. 2017. Super Mario Bros Recreated as Life Size Augmented Reality Game. https://www.youtube.com/watch?v=QN95nNDtxjo [Online; accessed on 1-June-2023].
[17]
Rujun Song, Ran Zhu, Zhuoling Xiao, and Bo Yan. 2023. ContextAVO: Local context guided and refining poses for deep visual odometry. Neurocomputing 533 (2023), 86–103.
[18]
Lukas von Stumberg and Daniel Cremers. 2022. DM-VIO: Delayed Marginalization Visual-Inertial Odometry. IEEE Robotics and Automation Letters 7, 2 (2022), 1408–1415. https://doi.org/10.1109/LRA.2021.3140129
[19]
Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan Bovik, and Yinxiao Li. 2022. Maxim: Multi-axis mlp for image processing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5769–5780.
[20]
S. Umeyama. 1991. Least-squares estimation of transformation parameters between two point patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 13, 4 (1991), 376–380. https://doi.org/10.1109/34.88573
[21]
Vladyslav Usenko, Nikolaus Demmel, David Schubert, Jörg Stückler, and Daniel Cremers. 2020. Visual-Inertial Mapping With Non-Linear Factor Recovery. IEEE Robotics and Automation Letters 5, 2 (2020), 422–429. https://doi.org/10.1109/LRA.2019.2961227
[22]
Fazil E. Uslu, Christopher D. Davidson, Erik Mailand, Nikolaos Bouklas, Brendon M. Baker, and Mahmut Selman Sakar. 2021. Engineered Extracellular Matrices with Integrated Wireless Microactuators to Study Mechanobiology. Advanced Materials 33 (10 2021). Issue 40. https://doi.org/10.1002/ADMA.202102641
[23]
Lukas Von Stumberg, Vladyslav Usenko, and Daniel Cremers. 2018. Direct sparse visual-inertial odometry using dynamic marginalization. In 2018 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2510–2517.
[24]
Sen Wang, Ronald Clark, Hongkai Wen, and Niki Trigoni. 2017. Deepvo: Towards end-to-end visual odometry with deep recurrent convolutional neural networks. In 2017 IEEE international conference on robotics and automation (ICRA). IEEE, 2043–2050.
[25]
Wenhai Wang, Jifeng Dai, Zhe Chen, Zhenhang Huang, Zhiqi Li, Xizhou Zhu, Xiaowei Hu, Tong Lu, Lewei Lu, Hongsheng Li, 2023. Internimage: Exploring large-scale vision foundation models with deformable convolutions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 14408–14419.
[26]
Zhelin Yu, Lidong Zhu, and Guoyu Lu. 2021. VINS-Motion. IEEE International Conference on Robotics and Automation (2021). https://ieeexplore-ieee-org.zorac.aub.aau.dk/stamp/stamp.jsp?tp=&arnumber=9562103
Index Terms
- Enhancing Direct Visual Odometry with Deblurring and Saliency Maps
Recommendations
A review of monocular visual odometry
AbstractMonocular visual odometry provides more robust functions on navigation and obstacle avoidance for mobile robots than other visual odometries, such as binocular visual odometry, RGB-D visual odometry and basic odometry. This paper describes the ...
Mesh saliency
Research over the last decade has built a solid mathematical foundation for representation and analysis of 3D meshes in graphics and geometric modeling. Much of this work however does not explicitly incorporate models of low-level human visual ...
Incorporating visual field characteristics into a saliency map
ETRA '12: Proceedings of the Symposium on Eye Tracking Research and ApplicationsCharacteristics of the human visual field are well known to be different in central (fovea) and peripheral areas. Existing computational models of visual saliency, however, do not take into account this biological evidence. The existing models compute ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
March 2024
184 pages
ISBN:9798400716553
DOI:10.1145/3653946
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 21 June 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICMVA 2024
ICMVA 2024: 2024 The 7th International Conference on Machine Vision and Applications
March 12 - 14, 2024
Singapore, Singapore
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 189Total Downloads
- Downloads (Last 12 months)189
- Downloads (Last 6 weeks)35
Reflects downloads up to 15 Feb 2025
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in