Abstract
In this paper we present a scalable architecture that integrates image based augmented reality (AR) with face recognition and augmentation over a single camera video stream. To achieve the real time performance required and ensure a proper level of scalability, the proposed solution makes use of two different approaches. First, we identify that the main bottleneck of the integrated process is the feature descriptor matching step. Taking into account the particularities of this task in the context of AR, we perform a comparison of different well known Approximate Nearest Neighbour search algorithms. After the empirical evaluation of several performance metrics we conclude that HNSW is the best candidate. The second approach consists on delegating other demanding tasks such as face descriptor computation as asynchronous processes, taking advantage of multi-core processors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Annoy. https://github.com/spotify/annoy. Accessed 2020
Arcore. https://developers.google.com/ar/. Accessed 2020
Arkit. https://developer.apple.com/augmented-reality/. Accessed 2020
Augment. https://www.augment.com. Accessed 2020
Aumentaty. http://www.aumentaty.com. Accessed 2020
Aurasma. https://www.aurasma.com. Accessed 2020
Kgraph. https://github.com/aaalgo/kgraph. Accessed 2020
Opencv. https://opencv.org. Accessed 2020
Vuforia. https://developer.vuforia.com/. Accessed 2020
Zappar. https://www.zappar.com. Accessed 2020
Bay, H., Ess, A., Tuytelaars, T., Gool, L.V.: Speeded-up robust features (SURF). Comput. Vis. Image Underst. 110(3), 346–359 (2008)
Boytsov, L., Naidan, B.: Engineering efficient and effective non-metric space library. In: Brisaboa, N., Pedreira, O., Zezula, P. (eds.) SISAP 2013. LNCS, vol. 8199, pp. 280–293. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-41062-8_28
Ierache, J., et al.: Development of a catalogs system for augmented reality applications. World Acad. Sci. Eng. Technol. Int. Sci. Index 9(1), 1–7 (2015)
Ierache, J., et al.: Augmented card system based on knowledge for medical emergency assistance. In: 2016 IEEE Congreso Argentino de Ciencias de La Informática y Desarrollos de Investigación, pp. 1–3 (2016). https://doi.org/10.1109/CACIDI.2016.7785979
Iwasaki, M.: Proximity search in metric spaces using approximate k nearest neighbor graph. IPSJ Trans. Database 3(1), 18–28 (2010). (in Japanese)
Iwasaki, M., Miyazaki, D.: Optimization of indexing based on k-nearest neighbor graph for proximity (2018)
Jégou, H., Douze, M., Schmid, C.: Product quantization for nearest neighbor search (2011)
Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004). https://doi.org/10.1023/B:VISI.0000029664.99615.94
Malkov, Y.A., Yashunin, D.A.: Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. CoRR abs/1603.09320 (2016). http://arxiv.org/abs/1603.09320
Mangiarua, N., Ierache, J., Abasolo, M.J.: Implementation of an open source based augmented reality engine for cloud authoring frameworks. J. Comput. Sci. Technol. 19(2), e16 (2019). https://doi.org/10.24215/16666038.19.e16. http://journal.info.unlp.edu.ar/JCST/article/view/1263
Mikolajczyk, K., Matas, J.: Improving descriptors for fast tree matching by optimal linear projection, pp. 1–8 (2007). https://doi.org/10.1109/ICCV.2007.4408871
Muja, M., Lowe, D.G.: Fast approximate nearest neighbors with automatic algorithm configuration. In: VISSAPP 2009, International Conference on Computer Vision Theory and Application, pp. 331–340. INSTICC Press (2009)
Muja, M., Lowe, D.G.: Scalable nearest neighbor algorithms for high dimensional data. IEEE Trans. Pattern Anal. Mach. Intell. 36(11), 2227–2240 (2014). https://doi.org/10.1109/TPAMI.2014.2321376
Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. In: BMVC (2015)
Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to SIFT or SURF. In: Proceedings of the 2011 International Conference on Computer Vision, Washington, DC, USA, pp. 2564–2571 (2011)
Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. CoRR abs/1503.03832 (2015). http://arxiv.org/abs/1503.03832
Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: DeepFace: closing the gap to human-level performance in face verification. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2014), Columbus, OH, USA, 23–28 June 2014, pp. 1701–1708. IEEE Computer Society (2014). https://doi.org/10.1109/CVPR.2014.220
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Mangiarua, N.A., Ierache, J.S., Abásolo, M.J. (2020). Scalable Integration of Image and Face Based Augmented Reality. In: De Paolis, L., Bourdot, P. (eds) Augmented Reality, Virtual Reality, and Computer Graphics. AVR 2020. Lecture Notes in Computer Science(), vol 12242. Springer, Cham. https://doi.org/10.1007/978-3-030-58465-8_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-58465-8_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58464-1
Online ISBN: 978-3-030-58465-8
eBook Packages: Computer ScienceComputer Science (R0)