A methodology for evaluation of task performance in robotic systems: a case study in vision-based localization

We investigated the performance of an agent that uses visual information in a partially unknown and changing environment in a principled way. We propose a methodology to study and evaluate the performance of autonomous agents. We first analyze the system theoretically to determine the most important system parameters and to predict error bounds and biases. We then conduct an empirical analysis to update and refine the model. The ultimate goal is to develop self-diagnostic procedures. We show that although simple models can successfully predict some major effects, empirically observed performance deviates from theoretical predictions in interesting ways.

Authors and Affiliations

1. GRASP Lab, Computer and Information Science Department, University of Pennsylvania, 3401 Walnut 301, Philadelphia, PA 19104-6228, USA venetian@grip.cis.upenn.edu, large@grip.cis.upenn.edu, bajcsy@central.cis.upenn.edu, , , , , , US

   Péter L. Venetianer, Edward W. Large & Ruzena Bajcsy

Reprints and permissions