Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

The PANOPTIC Camera: A Plenoptic Sensor with Real-Time Omnidirectional Capability

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

A new biologically-inspired vision sensor made of one hundred “eyes” is presented, which is suitable for real-time acquisition and processing of 3-D image sequences. This device, named the Panoptic camera, consists of a layered arrangement of approximately 100 classical CMOS imagers, distributed over a hemisphere of 13 cm in diameter. The Panoptic camera is a polydioptric system where all imagers have their own vision of the world, each with a distinct focal point, which is a specific feature of the Panoptic system. This enables 3-D information recording such as omnidirectional stereoscopy or depth estimation, applying specific signal processing. The algorithms dictating the image reconstruction of an omnidirectional observer located at any point inside the hemisphere are presented. A hardware architecture which has the capability of handling these algorithms, and the flexibility to support additional image processing in real time, has been developed as a two-layer system based on FPGAs. The detail of the hardware architecture, its internal blocks, the mapping of the algorithms onto the latter elements, and the device calibration procedure are presented, along with imaging results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13
Figure 14
Figure 15
Figure 16
Figure 17
Figure 18
Figure 19
Figure 20
Figure 21

Similar content being viewed by others

Notes

  1. Notice that this choice is arbitrary, and some alternate spherical pixelization could have been selected (e.g., HEALPix [16]).

References

  1. Richte, I. A. (1980). The notebooks of Leonardo Da Vinci. Oxford University Press.

  2. Adelson, E. H., & Bergen, J. R. (1991). Computational models of visual processing (pp. 3–20). MIT Press.

  3. Neumann, J., Fermuller, C., & Aloimonos, Y. (2002). In 3D data processing visualization and transmission, 2002. Proceedings. First international symposium on (pp. 2–11).

  4. Zbikowski, R. (2005). Spectrum. IEEE, 42(11), 46.

    Google Scholar 

  5. Geyer, C., & Daniilidis, K. (2001). International Journal of Computer Vision, 45(3), 223.

    Article  MATH  Google Scholar 

  6. Adelson, E., & Wang, J. (2002). Single lens stereo with a plenoptic camera. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2), 99.

    Article  Google Scholar 

  7. Cianci, C., Raemy, X., Pugh, J., & Martinoli, A. (2006). In Proceedings of the 2nd international conference on swarm robotics (pp. 103–115). Springer-Verlag.

  8. ViewPLUS. ASTRO Sensor Series (SOS:Stereo Omnidirectional System, BallCam). Available online at http://www.viewplus.co.jp/product/09/05.html. Accessed on 7 December 2011.

  9. Lee, L. P., & Szema, R. (2005). Inspirations from biological optics for advanced photonic systems. Science, 310(5751), 1148.

    Article  Google Scholar 

  10. Jeong, K. H., Kim, J., & Lee, L. P. (2006). Biologically inspired artificial compound eyes. Science, 312(5773), 557.

    Article  Google Scholar 

  11. Spherical trigonometry. Available online at http://en.wikipedia.org/wiki/Spherical_trigonometry. Accessed on 7 December 2011.

  12. Hartley, R. I., & Zisserman, A. (2004). Multiple view geometry in computer vision (2nd Edn.). Cambridge University Press, ISBN: 0521540518.

  13. Levoy, M., & Hanrahan, P. (1996). In SIGGRAPH ’96, proceedings of the 23rd annual conference on computer graphics and interactive techniques (pp. 31–42). ACM.

  14. Gortler, S., Grzeszczuk, R., Szeliski, R., & Cohen, M. (1996). In SIGGRAPH ’96, proceedings of the 23rd annual conference on computer graphics and interactive techniques (pp. 43–54). ACM.

  15. McMillan, L., & Bishop, G. (1995). In SIGGRAPH ’95, proceedings of the 22rd annual conference on computer graphics and interactive techniques (pp. 39–46). ACM.

  16. Górski, K. M., Hivon, E., Banday, A., Wandelt, B., Hansen, F., Reinecke, M., et al. (2005). HEALPix: A Framework for high-resolution discretization and fast analysis of data distributed on the sphere.. The Astrophysical Journal, 622, 759.

    Article  Google Scholar 

  17. Raboud, D. (2009). The panoptic camera—plenoptic interpolation in an omnidirectional polydioptric camera. Master’s thesis, EPFL.

  18. Aurenhammer, F. (1991). Computational geometry: Algorithms and applications. ACM Computing Surveys, 23, 345.

    Article  Google Scholar 

  19. de Berg, M., Cheong, O., van Kreveld, M., & Overmars, M. (2008). Computational geometry (3rd Edn.). Springer-Verlag, ISBN: 978-3-540-77973-5.

  20. Kilts, S. (2007). Advanced FPGA design: Architecture, implementation, and optimization. Wiley-IEEE Press.

  21. Xilinx. Application note 136. Available online at http://www.xilinx.com/support/documentation/application_notes/xapp136.pdf. Accessed on 7 December 2011.

  22. Tierney, J., Rader, C., & Gold, B. (1971). A digital frequency synthesizer. IEEE Transactions on Audio and Electroacoustics, 19(1), 48–57.

    Article  Google Scholar 

  23. Volder, J. E. (1959). The CORDIC trigonometric computing technique. IRE Transactions on Electronic Computers, EC-8(3), 330.

    Article  Google Scholar 

  24. Walther, J. S. (1971). In Proceedings of the 18–20 May 1971, Spring joint computer conference, AFIPS ’71 (Spring) (pp. 379–385). New York, NY, U.S.A.: ACM.

    Chapter  Google Scholar 

  25. Meyer-Baese, U. (2007). Digital signal processing with field programmable gate arrays (3rd Edn.). Springer.

  26. Bouguet, J. (2011). Camera calibration toolbox for matlab. Available online at http://www.vision.caltech.edu/bouguetj/calib_doc. Accessed on 7 December 2011.

Download references

Acknowledgements

The authors thank A. Cevrero, P. Athanasopoulos, S. Hauser, P. Bruehlmeier, D. Raboud, B. Devrim, A. Amini and the staff at the mechanical and electrical workshops of EPFL. The authors gratefully acknowledge the support of Xilinx, Inc., through the Xilinx University Program.

Author information

Authors and Affiliations

  1. EPFL-STI-IEL-LSM, Bldg ELD, Station 11, 1015, Lausanne, Switzerland

    Hossein Afshari, Laurent Jacques, Luigi Bagnato, Alexandre Schmid, Pierre Vandergheynst & Yusuf Leblebici

Authors
  1. Hossein Afshari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Laurent Jacques
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luigi Bagnato
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alexandre Schmid
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Pierre Vandergheynst
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yusuf Leblebici
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hossein Afshari.

Additional information

This research has been partly conducted with the support of the Swiss NSF under grant number 200021-125651.

H. Afshari, A. Schmid and Y. Leblebici are with the Microelectronic Systems Laboratory, L. Bagnato and P. Vandergheynst are with the Signal Processing Laboratory of the Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland. L. Jacques is with the Institute of Information and Communication Technologies, Electronics and Applied Mathematics (ICTEAM), Electrical Engineering Department (ELEN), University of Louvain (UCL), Louvain-la-Neuve, Belgium. He is funded by the Belgian Science Policy (Return Grant, BELSPO) joined to the Belgian Interuniversity Attraction Pole IAP-VI BCRYPT.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Afshari, H., Jacques, L., Bagnato, L. et al. The PANOPTIC Camera: A Plenoptic Sensor with Real-Time Omnidirectional Capability. J Sign Process Syst 70, 305–328 (2013). https://doi.org/10.1007/s11265-012-0668-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-012-0668-4

Keywords

Search

Navigation