Authors:
Alexander K. Seewald
1
and
Alexander K. Pfeiffer
2
Affiliations:
1
Seewald Solutions, Lärchenstraße 1, 4616 Weißkirchen a.d. Traun, Austria
;
2
Danube University Krems, Dr. Karl Dorrek Straße 30, 3500 Krems, Austria
Keyword(s):
Augmented Reality, Depth Cameras, Hand Gesture Recognition, People Counting, Shopping Window Rating, Gaze Direction Heat Map, Person Height Measurement.
Abstract:
Since 2012 we have been building the augmented reality system Magic Mirror based on Kinect V1 and V2’s native API. It relies on the magic mirror metaphor, where a large screen shows a mirrored camera view with overlaid graphical elements. In our case, it shows a different face mesh over the person’s face which reliably tracks face poses in real time while leaving the eyes and mouth of the person visible for interaction and to improve immersion; replaces the background with images that may be changed, smoothly zoomed and dragged; and allows to take screenshots which are automatically printed out on photo cards with an unique QR code linking to its digital twin. Control of the system is primarily via easily learned hand gestures very similar to multitouch screen gestures known from mobile phones and tablets. We have demonstrated the system to the public as well as in private over a wide variety of settings, faces and backgrounds. Here, we explain the challenges inherent in creating hig
h-quality face meshes and textures from 2D images, and how we solved them; describe the different versions of the system, how they differ and their limitations; and demonstrate the usefulness of our system in several applications from people counting and tracking to obtaining height measurements without storing or processing personal data.
(More)