Non-Parametric Sequential Frame Decimation for Scene Reconstruction in Low-Memory Streaming Environments

This paper introduces a non-parametric sequential frame decimation algorithm for image sequences in low-memory streaming environments. Frame decimation reduces the number of input frames to increase pose and structure robustness in Structure and Motion (SaM) applications. The main contribution of this paper is the introduction of a sequential low-memory work-flow for frame decimation in embedded systems where memory and memory traffic come at a premium. This approach acts as an online preprocessing filter by removing frames that are ill-posed for reconstruction before streaming. The introduced sequential approach reduces the number of needed frames in memory to three in contrast to global frame decimation approaches that use at least ten frames in memory and is therefore suitable for low-memory streaming environments. This is moreover important in emerging systems with large format cameras which acquire data over several hours and therefore render global approaches impossible.

In this paper a new decimation metric is designed which facilitates sequential keyframe extraction fit for reconstruction purposes, based on factors such as a correspondence-to-feature ratio and residual error relationships between epipolar geometry and homography estimation. The specific design of the error metric allows a local sequential decimation metric evaluation and can therefore be used on the fly. The approach has been tested with various types of input sequences and results in reliable low-memory frame decimation robust to different frame sampling frequencies and independent of any thresholds, scene assumptions or global frame analysis.

Authors and Affiliations

1. University of California, San Diego, USA

   Daniel Knoblauch & Falko Kuester

2. University of California, Davis, USA

   Mauricio Hess-Flores & Kenneth I. Joy

3. Lawrence Livermore National Laboratory, Livermore, USA

   Mark A. Duchaineau

Editors and Affiliations

1. University of Nevada, 89557, Reno, NV, USA

   George Bebis

2. NASA Ames Research Center, 94035, Moffett Field, CA, USA

   Richard Boyle

3. Lawrence Berkeley National Laboratory, 94720, Berkeley, CA, USA

   Bahram Parvin

4. Desert Research Institute, 89512, Reno, NV, USA

   Darko Koracin

5. Department of Computer Science and Engineering, University of South Carolina, 29208, Columbia, SC, USA

   Song Wang

6. HRL Laboratories, 3011 Malibu Canyon Road, 90265-4797, Malibu, CA, USA

   Kim Kyungnam

7. Purdue University, 47907-2021, West Lafayette, IN, USA

   Bedrich Benes

8. Sandia National Laboratory, 87185, Albuquerque, NM, USA

   Kenneth Moreland

9. University of Louisiana, 70504, Lafayette, LA, USA

   Christoph Borst

10. Adobe Systems Incorporated, San Francisco, CA, USA

    Stephen DiVerdi

11. Polytechnic Institute of NYU, 11201, Brooklyn, NY, USA

    Chiang Yi-Jen

12. Lawrence Livermore National Laboratory, 94551-0808, Livermore, CA, USA

    Jiang Ming

Reprints and permissions

© 2011 Springer-Verlag Berlin Heidelberg

Policies and ethics