3D model-based tracking combining edges, keypoints and fiducial markers
Authors: 
David Jurado-Rodriguez, Rafael Muñoz-Salinas, Sergio Garrido-Jurado, Francisco J. Romero-Ramirez, Rafael Medina-CarnicerAuthors Info & Claims
Abstract
Model-based tracking is an essential task in fields such as Augmented Reality. State-of-the-art approaches rely on the model’s edges, sometimes combined with image keypoints and color. Nevertheless, these image features are not considered part of the model but as temporary information discarded every time the tracking process is restarted. This paper proposes a novel approach that employs an enhanced model that combines edges, keypoints, and fiducial markers for robust and real-time tracking. The experiments conducted show that our method outperforms state-of-the-art model-based approaches and suggest that fiducial markers are a good choice for texturing models.
References
[1]
Bouthemy P A maximum likelihood framework for determining moving edges IEEE Trans Pattern Anal Mach Intell 1989 11 5 499-511
[2]
Cavallaro R The foxtrax hockey puck tracking system IEEE Comput Graphics Appl 1997 17 2 6-12
[3]
Chen Y, Li W, Sakaridis C, Dai D, Van Gool L (June 2018) Domain adaptive faster r-cnn for object detection in the wild. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)
[4]
Choi C and Christensen HI Robust 3d visual tracking using particle filtering on the special euclidean group: a combined approach of keypoint and edge features Int J Robot Res 2012 31 4 498-519
[5]
Choi C, Christensen Henrik I (2010) Real-time 3d model-based tracking using edge and keypoint features for robotic manipulation. In: 2010 IEEE international conference on robotics and automation, p 4048–4055
[6]
Collet A, Berenson D, Srinivasa Siddhartha S, Ferguson D (2009) Object recognition and full pose registration from a single image for robotic manipulation. In: 2009 IEEE international conference on robotics and automation, p 48–55. IEEE
[7]
Comport AI, Marchand E, Pressigout M, and Chaumette F Real-time markerless tracking for augmented reality: the virtual visual servoing framework IEEE Trans Visual Comput Graphics 2006 12 4 615-628
[8]
Drummond T, Cipolla R (1999) Real-time tracking of complex structures with on-line camera calibration. In: BMVC, p 1–10. Citeseer
[9]
Fischler MA and Bolles RC Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography Commun ACM 1981 24 6 381-395
[10]
Foxlin E, Altshuler Y, Naimark L, Harrington M (2004) Flighttracker: a novel optical/inertial tracker for cockpit enhanced vision. In: Third IEEE and ACM international symposium on mixed and augmented reality, p 212–221. IEEE
[11]
Gao X, Wang R, Demmel N (2018) and Daniel Cremers. Direct sparse odometry with loop closure, Ldso
[12]
Garrido-Jurado S, Muñoz-Salinas R, Madrid-Cuevas FJ, and Marín-Jiménez MJ Automatic generation and detection of highly reliable fiducial markers under occlusion Pattern Recogn 2014 47 6 2280-2292
[13]
Gidaris S, Komodakis N (December 2015) Object detection via a multi-region and semantic segmentation-aware cnn model. In: Proceedings of the IEEE international conference on computer vision (ICCV)
[14]
Gordon Iryna, Lowe David G (2006) What and where: 3d object recognition with accurate pose. In: Toward category-level object recognition, Springer, p 67–82
[15]
Han P and Zhao G A review of edge-based 3d tracking of rigid objects Virtual Real Intell Hardw 2019 1 6 580-596
[16]
Harris C, Stennett C (1990) Rapid-a video rate object tracker. In: BMVC, pages 1–6
[17]
Horn BKP Closed-form solution of absolute orientation using unit quaternions J Opt Soc Am A 1987 4 4 629-642
[18]
Issac J, Wüthrich M, Cifuentes C, Bohg J, Trimpe S, Schaal S (2016) Depth-based object tracking using a robust gaussian filter. In 2016 IEEE international conference on robotics and automation (ICRA), p 608–615. IEEE
[19]
Jiao L, Zhang F, Liu F, Yang S, Li L, Feng Z, and Rong Q A survey of deep learning-based object detection IEEE Access 2019 7 128837-128868
[20]
Kang K, Li H, Yan J, Zeng X, Yang B, Xiao T, Zhang C, Wang Z, Wang R, Wang X, and Ouyang W T-cnn: Tubelets with convolutional neural networks for object detection from videos IEEE Trans Circuits Syst Video Technol 2018 28 10 2896-2907
[21]
Kyrki V, Kragic D (2005) Integration of model-based and model-free cues for visual object tracking in 3d. In: Proceedings of the 2005 IEEE international conference on robotics and automation, p 1554–1560. IEEE
[22]
Lee G, Billinghurst M, Clark A (2015) A survey of augmented reality. Foundations and Trends® in Human-Computer Interaction 8(2-3):73–272
[23]
Marchand E, Spindler F, and Chaumette F Visp for visual servoing: a generic software platform with a wide class of robot control skills IEEE Robot Automation Mag 2005 12 4 40-52
[24]
Munoz Fernando I Ireta, Comport Andrew I (2016) Point-to-hyperplane rgb-d pose estimation: Fusing photometric and geometric measurements. In: 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS), p 24–29, IEEE
[25]
Muñoz-Salinas R, Marín-Jimenez MJ, and Medina-Carnicer R Spm-slam: Simultaneous localization and mapping with squared planar markers Pattern Recogn 2019 86 156-171
[26]
Muñoz-Salinas R, Medina-Carnicer R (2020) Ucoslam: simultaneous localization and mapping by fusion of keypoints and squared planar markers. Pattern Recognition, 107193
[27]
Mur-Artal R and Tardós JD Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras IEEE Trans Rob 2017 33 5 1255-1262
[28]
Nee AYC, Ong SK, Chryssolouris G, and Mourtzis D Augmented reality applications in design and manufacturing CIRP Ann 2012 61 2 657-679
[29]
Olson E (2011) Apriltag: a robust and flexible visual fiducial system. In: 2011 IEEE international conference on robotics and automation, p 3400–3407. IEEE
[30]
Paterson MS and Yao FF Efficient binary space partitions for hidden-surface removal and solid modeling Discret Comput Geom 1990 5 5 485-503
[31]
Petit A, Marchand E, Kanani K (2014) Combining complementary edge, keypoint and color features in model-based tracking for highly dynamic scenes. In: 2014 IEEE international conference on robotics and automation (ICRA), p 4115–4120. IEEE
[32]
Pfrommer B, Daniilidis K (2019) Tagslam: robust slam with fiducial markers. arXiv preprint arXiv:1910.00679
[33]
Pressigout M and Marchand E Real-time hybrid tracking using edge and texture information Int J Robot Res 2007 26 7 689-713
[34]
Pressigout M, Marchand E (2006) Real-time 3d model-based tracking: combining edge and texture information. In: Proceedings 2006 IEEE international conference on robotics and automation, 2006. ICRA 2006., p 2726–2731. IEEE
[35]
Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: Towards real-time object detection with region proposal networks. In: C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett (eds) Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc.,
[36]
Ren C, Prisacariu V, Kaehler O, Reid I, Murray D (2014) 3d tracking of multiple objects with identical appearance using rgb-d input. In: 2014 2nd International conference on 3D vision, volume 1, p 47–54. IEEE
[37]
Rey D, Neuhäuser M (2011) Wilcoxon-Signed-Rank Test, p 1658–1659. Springer Berlin Heidelberg, Berlin, Heidelberg
[38]
Romero-Ramirez FJ, Muñoz-Salinas R, and Medina-Carnicer R Speeded up detection of squared fiducial markers Image Vis Comput 2018 76 38-47
[39]
Rublee E, Rabaud V, Konolige K, Bradski G (Nov 2011) Orb: an efficient alternative to sift or surf. In 2011 International conference on computer vision, p 2564–2571
[40]
Sun Y, Page DL, Paik JK, Koschan A, Abidi MA (2002) Triangle mesh-based edge detection and its application to surface segmentation and adaptive surface smoothing. In: Proceedings. international conference on image processing, volume 3, pages 825–828
[41]
Sutherland Ivan E (1968) A head-mounted three dimensional display. In: Proceedings of the December 9-11, 1968, fall joint computer conference, Part I, AFIPS ’68 (Fall, part I), pp 757-764, New York, NY, USA. Association for Computing Machinery
[42]
Triggs B, McLauchlan P, Hartley R, Fitzgibbon A (1999) Bundle adjustment – a modern synthesis. In ICCV ’99: Proceedings of the International Workshop on Vision Algorithms., p 298–372. Springer-Verlag
[43]
Trinh S, Spindler F, Marchand E, Chaumette F (2018) A modular framework for model-based visual tracking using edge, texture and depth features. In 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS), p 89–96. IEEE
[44]
Vacchetti L, Lepetit V, Fua P (2004) Combining edge and texture information for real-time accurate 3d camera tracking. In: Third IEEE and ACM international symposium on mixed and augmented reality, p 48–56. IEEE
[45]
Ye E, Althoff M (2019) Model-based offline vehicle tracking in automotive applications using a precise 3d model. In: 2019 IEEE intelligent transportation systems conference (ITSC), p 1128–1135
[46]
Yovcheva Z, Buhalis D, and Gatzidis C Smartphone augmented reality applications for tourism E-Rev Tour Res (ertr) 2012 10 2 63-66
Index Terms
- 3D model-based tracking combining edges, keypoints and fiducial markers
Index terms have been assigned to the content through auto-classification.
Recommendations
Analysing the attributes of fiducial markers for robust tracking in augmented reality applications
Tracking the position and orientation pose of camera is a critical challenge for different modern applications like augmented reality, robot navigation, robot localisation, and 3D modelling and surveillance systems. Marker-based tracking is the most ...
Fiducial Markers for Pose Estimation: Overview, Applications and Experimental Comparison of the ARTag, AprilTag, ArUco and STag Markers
AbstractRobust localization is critical for the navigation and control of mobile robots. Global Navigation Satellite Systems (GNSS), Visual-Inertial Odometry (VIO), and Simultaneous Localization and Mapping (SLAM) offer different methods for achieving ...
Designing Highly Reliable Fiducial Markers
Fiducial markers are artificial landmarks added to a scene to facilitate locating point correspondences between images, or between images and a known model. Reliable fiducials solve the interest point detection and matching problems when adding markers ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
© The Author(s), under exclusive licence to Springer-Verlag London Ltd., part of Springer Nature 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 07 September 2023
Accepted: 06 August 2023
Received: 20 November 2022
Author Tags
Qualifiers
- Research-article
Funding Sources
- Ministerio de Economía y Competitividad
- Universidad de Cordoba
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 22 Nov 2024
Other Metrics
Citations
View Options
View options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in