Dagstuhl Seminar 98111
Evaluation and Validation of Computer Vision Algorithms
( Mar 16 – Mar 20, 1998 )
Permalink
Please use the following short url to reference this page:
https://www.dagstuhl.de/98111
Organizers
- M. Viergever (Delft)
- R. Haralick (Seattle)
- R. Klette (Auckland)
- S. Stiehl (Hamburg)
Contact
External Homepage
This Seminar addressed a subject which has been under active discussion in computer vision for several years. The evaluation and validation of algorithms is of basic importance for the configuration of computer vision applications. In the ideal case certain "data sheets" should allow to qualify algorithmic solutions in a specific context, e.g. defined by image data, goal of image analysis, or software environment ("edge detection is not equal to edge detection"). There is a lack of methodological fundamentals in the field of performance analysis.
The Seminar follows typical topics in computer vision. There are some fields in computer vision where an extensive literature about evaluation and validation of algorithms is available, e.g. motion analysis, digital geometry, shape reconstruction, or image registration. But still we are quite far away from "final answers" in these fields. "Compression" could play the role of the "good example" because the evaluation of compression methods is well developed based on comparisons of compression rates, behaviour on specific test sequences, and evaluation of image quality after decompression.
The range of fundamental problems encompasses, e.g.,
- the value of synthetic images in experimental computer vision,
- the selection of a representative set of real images related to specific domains and tasks,
- the definition of ground truth given different tasks and applications,
- the definition of experimental test-beds,
- the analysis of algorithms with respect to general features such as computation time, convergence, stability, or domains of admissible input data,
- the definition and analysis of performance measures for classes of algorithms,
- the role of statistics-based performance measures,
- the generation of data sheets with performance measures of algorithms supporting the system engineer in his configuration problem, etc.
The workshop thus attempted to bring together experienced colleagues from the international computer vision community both to discuss the state-of-the-art and to formulate recommendations for future activities.
36 talks, grouped in several topical sessions, were given by 32 speakers from 14 countries. Out of a total of 41 participants, 11 were from Germany; 5 from The Netherlands and the U.S.A. each; 3 from Denmark and United Kingdom each; 2 from Canada, Czech Republic, Japan, New Zealand, and Slovenia each; and 1 participant from Hungary, Israel, Slovakia, and Taiwan each. In addition to the oral presentations, four working groups - partly working during wood-walking tours – were established to discuss issues of common interest in more detail (see appendix for summaries). The participation of a number of younger scientists from Denmark, Germany, The Netherlands, and United Kingdom was rendered possible through financial support from the TMR (Training and Mobility of Researchers) programme of the European Union which is gratefully acknowledged. Moreover we are pleased to state that the presentations at our meeting were of such a high quality that a refereed proceedings book is planned to be published soon by Kluwer Academic Publishers in the computational Imaging and Vision series.
We are also grateful to the administration of the Dagstuhl enterprise for creating such an inspiring and free-of-duty environment as well as for providing excellent facilities which significantly contributed to the success of our meeting.
Eventually the workshop has stimulated different future activities, ranging from the establishment of an algorithmic web site for the international computer vision community, to the recommendation of organising a similar meeting at Schloß Dagstuhl in Y2K on the subject of theory, methodology, and practice of performance measures.
- M. Viergever (Delft)
- R. Haralick (Seattle)
- R. Klette (Auckland)
- S. Stiehl (Hamburg)
Related Seminars
- Dagstuhl Seminar 9411: Theoretical Foundations of Computer Vision (1994-03-14 - 1994-03-18) (Details)
- Dagstuhl Seminar 9612: Theoretical Foundations of Computer Vision (1996-03-18 - 1996-03-22) (Details)
- Dagstuhl Seminar 00111: Multi-Image Search, Filtering, Reasoning and Visualisation (2000-03-12 - 2000-03-17) (Details)
- Dagstuhl Seminar 02151: Theoretical Foundations of Computer Vision -- Geometry, Morphology, and Computational Imaging (2002-04-07 - 2002-04-12) (Details)
- Dagstuhl Seminar 04251: Imaging Beyond the Pin-hole Camera. 12th Seminar on Theoretical Foundations of Computer Vision (2004-06-13 - 2004-06-18) (Details)
- Dagstuhl Seminar 06241: Human Motion - Understanding, Modeling, Capture and Animation. 13th Workshop on Theoretical Foundations of Computer Vision (2006-06-11 - 2006-06-16) (Details)
- Dagstuhl Seminar 08291: Statistical and Geometrical Approaches to Visual Motion Analysis. 14th Workshop “Theoretic Foundations of Computer Vision” (2008-07-13 - 2008-07-18) (Details)
- Dagstuhl Seminar 11261: Outdoor and Large-Scale Real-World Scene Analysis. 15th Workshop "Theoretic Foundations of Computer Vision" (2011-06-26 - 2011-07-01) (Details)
- Dagstuhl Seminar 15081: Holistic Scene Understanding (2015-02-15 - 2015-02-20) (Details)