Abstract
The statistical analysis of multi-agent simulations requires a definitive set of benchmarks that represent the wide spectrum of challenging scenarios that agents encounter in dynamic environments, and a scoring method to objectively quantify the performance of a steering algorithm for a particular scenario. In this paper, we first recognize several limitations in prior evaluation methods. Next, we define a measure of normalized effort that penalizes deviation from desired speed, optimal paths, and collisions in a single metric. Finally, we propose a new set of benchmark categories that capture the different situations that agents encounter in dynamic environments and identify truly challenging scenarios for each category. We use our method to objectively evaluate and compare three state of the art steering approaches and one baseline reactive approach. Our proposed scoring mechanism can be used (a) to evaluate a single algorithm on a single scenario, (b) to compare the performance of an algorithm over different benchmarks, and (c) to compare different steering algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
van den Berg, J., Lin, M.C., Manocha, D.: Reciprocal velocity obstacles for real-time multi-agent navigation. In: Proceedings of ICRA, pp. 1928–1935. IEEE (2008)
Guy, S.J., Chhugani, J., Curtis, S., Dubey, P., Lin, M., Manocha, D.: Pledestrians: a least-effort approach to crowd simulation. In: SCA, pp. 119–128 (2010)
Helbing, D., Farkas, I., Vicsek, T.: Simulating dynamical features of escape panic. Nature 407, 487 (2000)
Kapadia, M., Singh, S., Allen, B., Reinman, G., Faloutsos, P.: Steerbug: an interactive framework for specifying and detecting steering behaviors. In: SCA 2009, pp. 209–216. ACM (2009)
Kapadia, M., Singh, S., Hewlett, W., Faloutsos, P.: Egocentric affordance fields in pedestrian steering. In: I3D 2009, pp. 215–223. ACM (2009)
Kapadia, M., Wang, M., Singh, S., Reinman, G., Faloutsos, P.: Scenario space: Characterizing coverage, quality, and failure of steering algorithms. In: SCA (2011)
Pelechano, N., Stocker, C., Allbeck, J., Badler, N.: Being a part of the crowd: towards validating vr crowds using presence. In: AAMAS, pp. 136–142 (2008)
Shao, W., Terzopoulos, D.: Autonomous pedestrians. In: SCA. ACM (2005)
Singh, S., Kapadia, M., Hewlett, W., Faloutsos, P.: A modular framework for adaptive agent-based steering. In: I3D 2011. ACM (2011)
Singh, S., Kapadia, M., Naik, M., Reinman, G., Faloutsos, P.: SteerBench: A Steering Framework for Evaluating Steering Behaviors. In: CAVW (2009)
Singh, S., Kapadia, M., Reinman, G., Faloutsos, P.: Footstep navigation for dynamic crowds. In: CAVW (2011)
Still, K.G.: Crowd Dynamics. Ph.D. thesis, United Kingdon (2000)
Whittle, M.: Gait analysis: An introduction (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kapadia, M., Wang, M., Reinman, G., Faloutsos, P. (2011). Improved Benchmarking for Steering Algorithms. In: Allbeck, J.M., Faloutsos, P. (eds) Motion in Games. MIG 2011. Lecture Notes in Computer Science, vol 7060. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-25090-3_23
Download citation
DOI: https://doi.org/10.1007/978-3-642-25090-3_23
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-25089-7
Online ISBN: 978-3-642-25090-3
eBook Packages: Computer ScienceComputer Science (R0)