Learning Crowd Motion Dynamics with Crowds
Article No.: 9, Pages 1 - 17
Abstract
Reinforcement Learning (RL) has become a popular framework for learning desired behaviors for computational agents in graphics and games. In a multi-agent crowd, one major goal is for agents to avoid collisions while navigating in a dynamic environment. Another goal is to simulate natural-looking crowds, which is difficult to define due to the ambiguity as to what is a natural crowd motion. We introduce a novel methodology for simulating crowds, which learns most-preferred crowd simulation behaviors from crowd-sourced votes via Bayesian optimization. Our method uses deep reinforcement learning for simulating crowds, where crowdsourcing is used to select policy hyper-parameters. Training agents with such parameters results in a crowd simulation that is preferred to users. We demonstrate our method's robustness in multiple scenarios and metrics, where we show it is superior compared to alternate policies and prior work.
Supplemental Material
MP4 File
Supplemental video
- Download
- 181.91 MB
References
[1]
Jur van den Berg, Stephen J Guy, Ming Lin, and Dinesh Manocha. 2011. Reciprocal n-body collision avoidance. In Robotics research. Springer, 3--19.
[2]
Eric Brochu, Tyson Brochu, and Nando de Freitas. 2010. A Bayesian interactive optimization approach to procedural animation design. In Symposium on Computer Animation.
[3]
Eric Brochu, Nando de Freitas, and Abhijeet Ghosh. 2007. Active Preference Learning with Discrete Choice Data (NIPS'07). Curran Associates Inc., Red Hook, NY, USA, 409--416.
[4]
Wei Chu and Zoubin Ghahramani. 2005. Preference learning with Gaussian processes. In International Conference on Machine Learning. 137--144.
[5]
Cathy Ennis, Christopher Peters, and Carol O'sullivan. 2011. Perceptual effects of scene context and viewpoint for virtual pedestrian crowds. ACM Transactions on Applied Perception (TAP) 8, 2 (2011), 1--22.
[6]
J.L. Fleiss, B. Levin, and M.C. Paik. 2013. Statistical Methods for Rates and Proportions. Wiley.
[7]
Stewart Greenhill, Santu Rana, Sunil Gupta, Pratibha Vellanki, and Svetha Venkatesh. 2020. Bayesian optimization for adaptive experimental design: A review. IEEE access 8 (2020), 13937--13948.
[8]
Stephen J. Guy, Jatin Chhugani, Sean Curtis, Pradeep K. Dubey, Ming C. Lin, and Dinesh Manocha. 2010. PLEdestrians: a least-effort approach to crowd simulation. In Symposium on Computer Animation.
[9]
Stephen J Guy, Jatin Chhugani, Changkyu Kim, Nadathur Satish, Ming Lin, Dinesh Manocha, and Pradeep Dubey. 2009. Clearpath: highly parallel collision avoidance for multi-agent simulation. In Symposium on Computer Animation. 177--187.
[10]
Xin He, Kaiyong Zhao, and Xiaowen Chu. 2021. AutoML: A survey of the state-of-the-art. Knowledge-Based Systems 212 (2021), 106622.
[11]
Helbing and Molnár. 1995. Social force model for pedestrian dynamics. Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics 51 5 (1995), 4282--4286.
[12]
Dirk Helbing, Illés Farkas, and Tamas Vicsek. 2000. Simulating dynamical features of escape panic. Nature 407, 6803 (2000), 487--490.
[13]
Leroy F Henderson. 1974. On the fluid mechanics of human crowd motion. Transportation research 8, 6 (1974), 509--515.
[14]
Kaidong Hu, Brandon Haworth, Glen Berseth, Vladimir Pavlovic, Petros Faloutsos, and Mubbasir Kapadia. 2021. Heterogeneous crowd simulation using parametric reinforcement learning. IEEE Transactions on Visualization and Computer Graphics (2021).
[15]
Roger L Hughes. 2002. A continuum theory for the flow of pedestrians. Transportation Research Part B: Methodological 36, 6 (2002), 507--535.
[16]
Schulman J., F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017).
[17]
S J. Guy and A-H Olivier. 2014. Parameter estimation and comparative evaluation of crowd simulations. In Computer Graphics Forum, Vol. 33. Wiley Online Library, 303--312.
[18]
Hao Jiang, Zhigang Deng, Mingliang Xu, Xiangjun He, Tianlu Mao, and Zhaoqi Wang. 2018. An emotion evolution based model for collective behavior simulation. In Symposium on Interactive 3D Graphics and Games. 1--6.
[19]
Ioannis Karamouzas, Brian Skinner, and Stephen J Guy. 2014. Universal power law governing pedestrian interactions. Physical Review Letters 113, 23 (2014), 238701.
[20]
Yuki Koyama, Issei Sato, Daisuke Sakamoto, and Takeo Igarashi. 2017. Sequential line search for efficient visual design optimization by crowds. ACM Transactions on Graphics (TOG) 36 (2017), 1--11.
[21]
Ariel Kwiatkowski, Eduardo Alvarado, Vicky Kalogeiton, C Karen Liu, Julien Pettré, Michiel van de Panne, and Marie-Paule Cani. 2022. A survey on reinforcement learning methods in character animation. In Computer Graphics Forum, Vol. 41. Wiley Online Library, 613--639.
[22]
Ariel Kwiatkowski, Vicky Kalogeiton, Julien Pettré, and Marie-Paule Cani. 2023. Understanding reinforcement learned crowds. Computers & Graphics 110 (2023), 28--37.
[23]
Jaedong Lee, Jungdam Won, and Jehee Lee. 2018. Crowd simulation by deep reinforcement learning. In Proceedings of the 11th Annual International Conference on Motion, Interaction, and Games. 1--7.
[24]
Axel López, François Chaumette, Eric Marchand, and Julien Pettré. 2019. Character navigation in dynamic environments based on optical flow. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 181--192.
[25]
Joe Marks, Brad Andalman, Paul A Beardsley, William Freeman, Sarah Gibson, Jessica Hodgins, Thomas Kang, Brian Mirtich, Hanspeter Pfister, Wheeler Ruml, et al. 1997. Design galleries: A general approach to setting parameters for computer graphics and animation. In Proceedings of the 24th annual conference on Computer graphics and interactive techniques. 389--400.
[26]
CD Tharindu Mathew, Bedrich Benes, and Daniel Aliaga. 2020. Interactive inverse spatio-temporal crowd motion design. In Symposium on Interactive 3D Graphics and Games. 1--9.
[27]
Mehdi Moussaïd, Dirk Helbing, and Guy Theraulaz. 2011. How simple rules determine pedestrian behavior and crowd disasters. Proceedings of the National Academy of Sciences 108 (2011), 6884--6888.
[28]
Andreas Panayiotou, Theodoros Kyriakou, Marilena Lemonari, Yiorgos Chrysanthou, and Panayiotis Charalambous. 2022. CCP: Configurable Crowd Profiles. ACM Transactions on Graphics (TOG) (2022).
[29]
Zhiguo Ren, Panayiotis Charalambous, Julien Bruneau, Qunsheng Peng, and Julien Pettré. 2017. Group Modeling: A Unified Velocity-Based Approach. In Computer Graphics Forum, Vol. 36. Wiley Online Library, 45--56.
[30]
Libo Sun, Jinfeng Zhai, and Wenhu Qin. 2019. Crowd navigation in an unknown and dynamic environment based on deep reinforcement learning. IEEE Access 7 (2019), 109544--109554.
[31]
Bilas Talukdar, Yunhao Zhang, and Tomer Weiss. 2023. Learning to Simulate Crowds with Crowds. In ACM SIGGRAPH 2023 Posters (Los Angeles, CA, USA) (SIGGRAPH '23). Association for Computing Machinery, New York, NY, USA, Article 6, 2 pages.
[32]
TensorFlow. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. http://tensorflow.org/ Software available from tensorflow.org. (2015).
[33]
Justin K Terry, Nathaniel Grammel, Ananth Hari, Luis Santos, and Benjamin Black. 2020. Revisiting parameter sharing in multi-agent deep reinforcement learning. arXiv preprint arXiv:2005.13625 (2020).
[34]
Daniel M Tracy, W Randolph Franklin, Barbara Cutler, Franklin T Luk, and Marcus Andrade. 2008. Path planning on a compressed terrain. In Proceedings of the 16th ACM SIGSPATIAL international conference on Advances in geographic information systems. 1--4.
[35]
Wouter van Toll, Fabien Grzeskowiak, Axel López-Gandía, Javad Amirian, Florian Berton, Julien Bruneau, Beatriz Cabrero Daniel, Alberto Jovane, and Julien Pettré. 2020. Generalized Microscropic Crowd Simulation using Costs in Velocity Space. Symposium on Interactive 3D Graphics and Games (2020).
[36]
Oriol Vinyals, Igor Babuschkin, Wojciech M Czarnecki, Michaël Mathieu, Andrew Dudzik, Junyoung Chung, David H Choi, Richard Powell, Timo Ewalds, Petko Georgiev, et al. 2019. Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature 575, 7782 (2019), 350--354.
[37]
He Wang, Jan Ondřej, and Carol O'Sullivan. 2016. Path patterns: Analyzing and comparing real and simulated crowds. In Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D graphics and games. 49--57.
[38]
Benjamin Watson, Rachit Shrivastava, and Ajinkya Gavane. 2019. The Effects of Adaptive Synchronization on Performance and Experience in Gameplay. Proceedings of the ACM on Computer Graphics and Interactive Techniques 2, 1 (2019), 1--13. Tomer Weiss. 2023. Fast Position-based Multi-Agent Group Dynamics. Proceedings of the ACM on Computer Graphics and Interactive Techniques 6, 1 (2023), 1--15.
[39]
Tomer Weiss, Alan Litteneker, Chenfanfu Jiang, and Demetri Terzopoulos. 2019. Position-based real-time simulation of large crowds. Computers & Graphics 78 (2019), 12--22.
[40]
Dong Xu, Xiao Huang, Zhenlong Li, and Xiang Li. 2020. Local motion simulation using deep reinforcement learning. Transactions in GIS 24, 3 (2020), 756--779.
Index Terms
- Learning Crowd Motion Dynamics with Crowds
Recommendations
Learning to Simulate Crowds with Crowds
SIGGRAPH '23: ACM SIGGRAPH 2023 PostersControlling agent behaviors with Reinforcement Learning is of continuing interest in multiple areas. One major focus is to simulate multi-agent crowds that avoid collisions while locomoting to their goals. Although avoiding collisions is important, it ...
GREIL-Crowds: Crowd Simulation with Deep Reinforcement Learning and Examples
Simulating crowds with realistic behaviors is a difficult but very important task for a variety of applications. Quantifying how a person balances between different conflicting criteria such as goal seeking, collision avoidance and moving within a group ...
Crowd simulation by deep reinforcement learning
MIG '18: Proceedings of the 11th ACM SIGGRAPH Conference on Motion, Interaction and GamesSimulating believable virtual crowds has been an important research topic in many research fields such as industry films, computer games, urban engineering, and behavioral science. One of the key capabilities agents should have is navigation, which is ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 13 May 2024
Published in PACMCGIT Volume 7, Issue 1
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed
Data Availability
Supplemental video https://dl.acm.org/doi/10.1145/3651302#3651302-talukdar.mp4
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 253Total Downloads
- Downloads (Last 12 months)253
- Downloads (Last 6 weeks)58
Reflects downloads up to 10 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in