Dragon's Path: Synthesizing User-Centered Flying Creature Animation Paths for Outdoor Augmented Reality Experiences
Article No.: 77, Pages 1 - 11
Abstract
Advances in augmented reality promise to deliver highly immersive storytelling experiences by animating virtual characters naturally in the real world. However, creating such realistic animated content for viewing in augmented reality is non-trivial and challenging. In this paper, we present a novel approach to automatically generate user-centered flying creature animation paths for outdoor augmented reality experiences. Given a sequence of storyline actions, our approach finds suitable locations for the character to perform its actions via a location compatibility predictor trained with user preferences, synthesizing a corresponding animation path optimized with respect to the user’s perspective. We applied our approach to synthesize user-centered augmented reality experiences based on different storyline actions and environments. We also conducted user study experiments to validate the efficacy of our approach for synthesizing desirable augmented reality experiences.
Supplemental Material
MP4 File - presentation
presentation
- Download
- 485.40 MB
ZIP File
Supplementary Material
- Download
- 44.85 MB
References
[1]
Zeyad Abd Algfoor, Mohd Shahrizal Sunar, and Hoshang Kolivand. 2015. A comprehensive study on pathfinding techniques for robotics and video games. International Journal of Computer Games Technology 2015 (2015).
[2]
Rawan Alghofaili, Cuong Nguyen, Vojtĕch Krs, Nathan Carr, Radomír Mĕch, and Lap-Fai Yu. 2023. WARPY: Sketching Environment-Aware 3D Curves in Mobile Augmented Reality. In 2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR). https://doi.org/10.1109/VR55154.2023.00052
[3]
Raphael Anderegg, Loïc Ciccone, and Robert W Sumner. 2018. PuppetPhone: puppeteering virtual characters using a smartphone. In Proceedings of the 11th Annual International Conference on Motion, Interaction, and Games. 1–6.
[4]
Mark Billinghurst, Hirokazu Kato, and Ivan Poupyrev. 2001. The magicbook-moving seamlessly between reality and virtuality. IEEE Computer Graphics and Applications 21, 3 (2001), 6–8.
[5]
Mengyu Chen, Andrés Monroy-Hernández, and Misha Sra. 2021. SceneAR: Scene-based Micro Narratives for Sharing and Remixing in Augmented Reality. In 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 294–303.
[6]
Wei-Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Frank Wang, and Jia-Bin Huang. 2019. A closer look at few-shot classification. arXiv preprint arXiv:1904.04232 (2019).
[7]
Guneet S Dhillon, Pratik Chaudhari, Avinash Ravichandran, and Stefano Soatto. 2019. A baseline for few-shot image classification. arXiv preprint arXiv:1909.02729 (2019).
[8]
Steven Feiner, Blair MacIntyre, Tobias Höllerer, and Anthony Webster. 1997. A touring machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment. Personal Technologies 1, 4 (1997), 208–217.
[9]
Ran Gal, Lior Shapira, Eyal Ofek, and Pushmeet Kohli. 2014. FLARE: Fast layout for augmented reality applications. In 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 207–212.
[10]
Maxime Garcia, Rémi Ronfard, and Marie-Paule Cani. 2019. Spatial Motion Doodles: Sketching Animation in VR Using Hand Gestures and Laban Motion Analysis. In Motion, Interaction and Games. 1–10.
[11]
Christoph Gebhardt and Otmar Hilliges. 2021. Optimization-based user support for cinematographic quadrotor camera target framing. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–13.
[12]
Raphaël Grasset, Andreas Dünser, and Mark Billinghurst. 2008. Edutainment with a mixed reality book: a visually augmented illustrative childrens’ book. In Proceedings of the international conference on advances in computer entertainment technology.
[13]
Edmund T Hall and Edward T Hall. 1966. The hidden dimension. Vol. 609. Anchor.
[14]
Peter E Hart, Nils J Nilsson, and Bertram Raphael. 1968. A formal basis for the heuristic determination of minimum cost paths. IEEE transactions on Systems Science and Cybernetics 4, 2 (1968), 100–107.
[15]
W Keith Hastings. 1970. Monte Carlo sampling methods using Markov chains and their applications. (1970).
[16]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.
[17]
Gaoping Huang, Xun Qian, Tianyi Wang, Fagun Patel, Maitreya Sreeram, Yuanzhi Cao, Karthik Ramani, and Alexander J Quinn. 2021. Adaptutar: An adaptive tutoring system for machine tasks in augmented reality. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–15.
[18]
Wei Huang, Min Sun, and Songnian Li. 2016. A 3D GIS-based interactive registration mechanism for outdoor augmented reality system. Expert Systems with Applications 55 (2016), 48–58.
[19]
Ananya Ipsita, Hao Li, Runlin Duan, Yuanzhi Cao, Subramanian Chidambaram, Min Liu, and Karthik Ramani. 2021. VRFromX: From Scanned Reality to Interactive Virtual Experience with Human-in-the-Loop. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI EA ’21). Association for Computing Machinery, New York, NY, USA, Article 289, 7 pages. https://doi.org/10.1145/3411763.3451747
[20]
Niels Joubert, Mike Roberts, Anh Truong, Floraine Berthouzoz, and Pat Hanrahan. 2015. An interactive tool for designing quadrotor camera shots. ACM Transactions on Graphics (TOG) 34, 6 (2015), 1–11.
[21]
Sertac Karaman and Emilio Frazzoli. 2011. Sampling-based algorithms for optimal motion planning. The international journal of robotics research 30, 7 (2011), 846–894.
[22]
Scott Kirkpatrick, C Daniel Gelatt Jr, and Mario P Vecchi. 1983. Optimization by simulated annealing. science 220, 4598 (1983), 671–680.
[23]
Yining Lang, Wei Liang, and Lap-Fai Yu. 2019. Virtual agent positioning driven by scene semantics in mixed reality. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 767–775.
[24]
Gun A Lee, Andreas Dünser, Seungwon Kim, and Mark Billinghurst. 2012. CityViewAR: A mobile outdoor AR application for city visualization. In 2012 IEEE international symposium on mixed and augmented reality-arts, media, and humanities (ISMAR-AMH). IEEE, 57–64.
[25]
Changyang Li, Wanwan Li, Haikun Huang, and Lap-Fai Yu. 2022. Interactive augmented reality storytelling guided by scene semantics. ACM Transactions on Graphics (TOG) 41, 4 (2022), 1–15.
[26]
Wanwan Li, Changyang Li, Minyoung Kim, Haikun Huang, and Lap-Fai Yu. 2023. Location-Aware Adaptation of Augmented Reality Narratives. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems.
[27]
Wei Liang, Xinzhe Yu, Rawan Alghofaili, Yining Lang, and Lap-Fai Yu. 2021. Scene-aware behavior synthesis for virtual pets in mixed reality. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–12.
[28]
Huimin Liu, Minsoo Choi, Liuchuan Yu, Alexandros Koilias, Lap-Fai Yu, and Christos Mousas. 2022. Synthesizing Shared Space Virtual Reality Fire Evacuation Training Drills. In 2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct).
[29]
Nicholas Metropolis, Arianna W Rosenbluth, Marshall N Rosenbluth, Augusta H Teller, and Edward Teller. 1953. Equation of state calculations by fast computing machines. The journal of chemical physics 21, 6 (1953), 1087–1092.
[30]
Rahul Narain, Abhinav Golas, Sean Curtis, and Ming C Lin. 2009. Aggregate dynamics for dense crowd simulation. In ACM SIGGRAPH Asia 2009 papers. 1–8.
[31]
Dariusz Rumiński and Krzysztof Walczak. 2013. Creation of interactive ar content on mobile devices. In International Conference on Business Information Systems. Springer, 258–269.
[32]
José Ricardo Sánchez-Ibáñez, Carlos J Pérez-del Pulgar, and Alfonso García-Cerezo. 2021. Path Planning for Autonomous Mobile Robots: A Review. Sensors 21, 23 (2021), 7898.
[33]
Aditya Sankar and Steven M. Seitz. 2017. Interactive Room Capture on 3D-Aware Mobile Devices. Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (2017).
[34]
Manolis Savva, Angel X Chang, Pat Hanrahan, Matthew Fisher, and Matthias Nießner. 2014. SceneGrok: Inferring action maps in 3D environments. ACM transactions on graphics (TOG) 33, 6 (2014), 1–10.
[35]
Tomu Tahara, Takashi Seno, Gaku Narita, and Tomoya Ishikawa. 2020. Retargetable AR: Context-aware augmented reality in indoor scenes based on 3D scene graph. In 2020 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). IEEE, 249–255.
[36]
Zeyu Wang, Cuong Nguyen, Paul Asente, and Julie Dorsey. 2021. DistanciAR: Authoring Site-Specific Augmented Reality Experiences for Remote Environments. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 411, 12 pages. https://doi.org/10.1145/3411764.3445552
[37]
Jungdam Won, Jongho Park, Kwanyu Kim, and Jehee Lee. 2017. How to train your dragon: example-guided control of flapping flight. ACM Transactions on Graphics (TOG) 36, 6 (2017), 1–13.
[38]
Jungdam Won, Jungnam Park, and Jehee Lee. 2018. Aerobatics control of flying creatures via self-regulated learning. ACM Transactions on Graphics (TOG) 37, 6 (2018), 1–10.
[39]
Ke Xie, Hao Yang, Shengqiu Huang, Dani Lischinski, Marc Christie, Kai Xu, Minglun Gong, Daniel Cohen-Or, and Hui Huang. 2018. Creating and chaining camera moves for quadrotor videography. ACM Transactions on Graphics (TOG) 37, 4 (2018), 1–13.
[40]
Hao Yang, Ke Xie, Shengqiu Huang, and Hui Huang. 2018. Uncut Aerial Video via a Single Sketch. Computer Graphics Forum 37, 7 (2018), 191–199.
[41]
Jackie Yang, Christian Holz, Eyal Ofek, and Andrew D Wilson. 2019. Dreamwalker: Substituting real-world walking experiences with a virtual reality. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology.
[42]
Hui Ye, Kin Chung Kwan, Wanchao Su, and Hongbo Fu. 2020. ARAnimator: in-situ character animation in mobile AR with user-defined motion gestures. ACM Transactions on Graphics (TOG) 39, 4 (2020), 83–1.
[43]
Ya-Ting Yue, Yong-Liang Yang, Gang Ren, and Wenping Wang. 2017. SceneCtrl: Mixed Reality Enhancement via Efficient Scene Editing. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (Québec City, QC, Canada) (UIST ’17). Association for Computing Machinery, New York, NY, USA, 427–436. https://doi.org/10.1145/3126594.3126601
[44]
Han Zhang, Yucong Yao, Ke Xie, Chi-Wing Fu, Hao Zhang, and Hui Huang. 2021. Continuous aerial path planning for 3D urban scene reconstruction.ACM Trans. Graph. 40, 6 (2021), 225–1.
[45]
Yunxiang Zhang, Kenneth Chen, and Qi Sun. 2023. Toward Optimized VR/AR Ergonomics: Modeling And Predicting User Neck Muscle Contraction. In ACM SIGGRAPH 2023 Conference Proceedings.
[46]
Yeyao Zhang, Eleftheria Tsipidi, Sasha Schriber, Mubbasir Kapadia, Markus Gross, and Ashutosh Modi. 2019. Generating animations from screenplays. arXiv preprint arXiv:1904.05440 (2019).
[47]
Zhiying Zhou, Adrian David Cheok, JiunHorng Pan, and Yu Li. 2004. Magic Story Cube: an interactive tangible interface for storytelling. In Proceedings of the International Conference on Advances in Computer Entertainment Technology. 364–365.
Index Terms
- Dragon's Path: Synthesizing User-Centered Flying Creature Animation Paths for Outdoor Augmented Reality Experiences
Recommendations
Multimodal augmented reality: the norm rather than the exception
MVAR '16: Proceedings of the 2016 workshop on Multimodal Virtual and Augmented RealityAugmented reality (AR) is commonly seen as a technology that overlays virtual imagery onto a participant's view of the world. In line with this, most AR research is focused on what we see. In this paper, we challenge this focus on vision and make a case ...
Designing and implementing interactive and realistic augmented reality experiences
In this paper, we propose an approach for supporting the design and implementation of interactive and realistic Augmented Reality (AR). Despite the advances in AR technology, most software applications still fail to support AR experiences where virtual ...
Interactive Augmented Live Virtual Reality Streaming: A Health Care Application
ICMHI '18: Proceedings of the 2nd International Conference on Medical and Health InformaticsVirtual Reality (VR) technology has been around for decades; however, we have probably only begun to realize the practical applications until recent years. With increase of computing power, decrease of cost and physical dimension, a new class of ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
July 2024
1106 pages
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 13 July 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
Conference
SIGGRAPH '24
Sponsor:
SIGGRAPH '24: Special Interest Group on Computer Graphics and Interactive Techniques Conference
July 27 - August 1, 2024
CO, Denver, USA
Acceptance Rates
Overall Acceptance Rate 1,822 of 8,601 submissions, 21%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 409Total Downloads
- Downloads (Last 12 months)409
- Downloads (Last 6 weeks)122
Reflects downloads up to 21 Dec 2024
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in