Abstract
Visual localization is the task of camera pose estimation, and is crucial for many technologies which involve localization such as mobile robots and augmented reality. Several convolutional neural network models have been proposed for the task against the more accurate geometry based computer vision techniques. However, they have several shortcomings and to our knowledge, this was the first effort that explored the use of an alternative architecture based on capsule-networks for the task. We achieved better results with capsules than with baseline-CNN PoseNet on small NORB dataset, modified for the task of camera pose estimation. Feature visualizations for both the networks produced more insights on their performance and behaviour. We found that there is a scope for improvement and hence propose few directions for future efforts.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Structure from Motion.
- 2.
Convolutional neural networks.
- 3.
2 dimensional.
- 4.
References
Brahmbhatt, S., Gu, J., Kim, K., Hays, J., Kautz, J.: Geometry-aware learning of maps for camera localization. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, 18–22 June 2018, pp. 2616–2625. IEEE Computer Society (2018). https://doi.org/10.1109/CVPR.2018.00277. http://openaccess.thecvf.com/content_cvpr_2018/html/Brahmbhatt_Geometry-Aware_Learning_of_CVPR_2018_paper.html
Graetz, F.M.: How to visualize convolutional features in 40 lines of code, July 2019. https://towardsdatascience.com/how-to-visualize-convolutional-features-in-40-lines-of-code-70b7d87b0030
Hinton, G.E., Sabour, S., Frosst, N.: Matrix capsules with EM routing. In: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30–May 3 2018, Conference Track Proceedings. OpenReview.net (2018). https://openreview.net/forum?id=HJWLfGWRb
Hui, J.: Understanding dynamic routing between capsules (capsule networks), November 2017. https://jhui.github.io/2017/11/03/Dynamic-Routing-Between-Capsules/
Hui, J.: Understanding matrix capsules with EM routing (based on Hinton’s capsule networks), November 2017. https://jhui.github.io/2017/11/14/Matrix-Capsules-with-EM-routing-Capsule-Network/
Kendall, A., Cipolla, R.: Geometric loss functions for camera pose regression with deep learning. CoRR abs/1704.00390 (2017). http://arxiv.org/abs/1704.00390
Kendall, A., Grimes, M., Cipolla, R.: PoseNet: a convolutional network for real-time 6-DOF camera relocalization. In: 2015 IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile, 7–13 December 2015, pp. 2938–2946. IEEE Computer Society (2015). https://doi.org/10.1109/ICCV.2015.336
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2017)
Krizhevsky, A.: Learning multiple layers of features from tiny images. Technical report (2009)
LeCun, Y., Cortes, C.: MNIST handwritten digit database (2010). http://yann.lecun.com/exdb/mnist/
LeCun, Y., Huang, F.J., Bottou, L.: Learning methods for generic object recognition with invariance to pose and lighting. In: 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2004), with CD-ROM, Washington, DC, USA, 27 June–2 July 2004, pp. 97–104. IEEE Computer Society (2004). https://doi.org/10.1109/CVPR.2004.144. http://doi.ieeecomputersociety.org/10.1109/CVPR.2004.144
Paszke, A., et al.: Automatic differentiation in PyTorch (2017)
Sabour, S., Frosst, N., Hinton, G.E.: Dynamic routing between capsules. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, Long Beach, CA, USA, 4–9 December 2017, pp. 3856–3866 (2017). http://papers.nips.cc/paper/6975-dynamic-routing-between-capsules
Sattler, T., Leibe, B., Kobbelt, L.: Improving image-based localization by active correspondence search. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7572, pp. 752–765. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33718-5_54
Sattler, T., et al.: Benchmarking 6DOF outdoor visual localization in changing conditions (2018)
Sattler, T., Zhou, Q., Pollefeys, M., Leal-Taixé, L.: Understanding the limitations of CNN-based absolute camera pose regression. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, 16–20 June 2019, pp. 3302–3312. Computer Vision Foundation/IEEE (2019). https://doi.org/10.1109/CVPR.2019.00342. http://openaccess.thecvf.com/content_CVPR_2019/html/Sattler_Understanding_the_Limitations_of_CNN-Based_Absolute_Camera_Pose_Regression_CVPR_2019_paper.html
Szegedy, C., et al.: Going deeper with convolutions (2014)
Walch, F., Hazirbas, C., Leal-Taixé, L., Sattler, T., Hilsenbeck, S., Cremers, D.: Image-based localization with spatial LSTMs. CoRR abs/1611.07890 (2016). http://arxiv.org/abs/1611.07890
Acknowledgements
This work was done as a part of the dual degree project requirement in Indian Institute of Technology Madras. I would like to acknowledge the informed guidance of Prof. Anurag Mittal for the same.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Patil, O. (2022). Visual Localization Using Capsule Networks. In: Raman, B., Murala, S., Chowdhury, A., Dhall, A., Goyal, P. (eds) Computer Vision and Image Processing. CVIP 2021. Communications in Computer and Information Science, vol 1567. Springer, Cham. https://doi.org/10.1007/978-3-031-11346-8_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-11346-8_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-11345-1
Online ISBN: 978-3-031-11346-8
eBook Packages: Computer ScienceComputer Science (R0)