Abstract
In this paper, we augment multi-frame super-resolution with the concept of guided filtering for simultaneous upsampling of 3-D range data and complementary photometric information in hybrid range imaging. Our guided super-resolution algorithm is formulated as joint maximum a-posteriori estimation to reconstruct high-resolution range and photometric data. In order to exploit local correlations between both modalities, guided filtering is employed for regularization of the proposed joint energy function. For fast and robust image reconstruction, we employ iteratively re-weighted least square minimization embedded into a cyclic coordinate descent scheme. The proposed method was evaluated on synthetic datasets and real range data acquired with Microsoft’s Kinect. Our experimental evaluation demonstrates that our approach outperforms state-of-the-art range super-resolution algorithms while it also provides super-resolved photometric data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Supplementary material is available at http://www5.cs.fau.de/research/data/.
References
Babacan, S.D., Molina, R., Katsaggelos, A.K.: Variational Bayesian super resolution. IEEE Trans. Image Process. 20(4), 984–999 (2011)
Bauer, S., Seitel, A., Hofmann, H., Blum, T., Wasza, J., Balda, M., Meinzer, H.-P., Navab, N., Hornegger, J., Maier-Hein, L.: Real-time range imaging in health care: a survey. In: Grzegorzek, M., Theobalt, C., Koch, R., Kolb, A. (eds.) Time-of-Flight and Depth Imaging. LNCS, vol. 8200, pp. 228–254. Springer, Heidelberg (2013)
Beder, C., Bartczak, B., Koch, R.: A comparison of PMD-cameras and stereo-vision for the task of surface reconstruction using patchlets. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)
Bhavsar, A.V., Rajagopalan, A.N.: Range map superresolution-inpainting, and reconstruction from sparse data. Comput. Vis. Image Underst. 116(4), 572–591 (2012)
Cui, Y., Schuon, S., Chan, D., Thrun, S., Theobalt, C.: 3D shape scanning with a time-of-flight camera. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1173–1180 (2010)
Elad, M., Feuer, A.: Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images. IEEE Trans. Image Process. 6(12), 1646–1658 (1997)
Farsiu, S., Robinson, D., Elad, M., Milanfar, P.: Advances and challenges in super-resolution. Int. J. Imaging Syst. Technol. 14, 47–57 (2004)
Farsiu, S., Robinson, D., Elad, M., Milanfar, P.: Fast and robust multiframe super resolution. IEEE Trans. Image Process. 13(10), 1327–1344 (2004)
He, K., Sun, J., Tang, X.: Guided image filtering. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 1–14. Springer, Heidelberg (2010)
Köhler, T., Haase, S., Bauer, S., Wasza, J., Kilgus, T., Maier-Hein, L., Feubner, H., Hornegger, J.: ToF meets RGB: novel multi-sensor super-resolution for hybrid 3-D endoscopy. Med. Image Comput. Comput. Assist. Interv. 16, 139–146 (2013)
Köhler, T., Haase, S., Bauer, S., Wasza, J., Kilgus, T., Maier-Hein, L., Feuner, H., Hornegger, J.: Outlier detection for multi-sensor super-resolution in hybrid 3D endoscopy. In: Deserno, T.M., Handels, H., Meinzer, H.-P., Tolxdorff, T. (eds.) Bildverarbeitung für die Medizin 2014. Informatik aktuell, pp. 84–89. Springer, Heidelberg (2014)
Kurmankhojayev, D., Hasler, N., Theobalt, C.: Monocular pose capture with a depth camera using a sums-of-gaussians body model. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 415–424. Springer, Heidelberg (2013)
Liu, C.: Beyond pixels: exploring new representations and applications for motion analysis. Ph.D. thesis, Massachusetts Institute of Technology (2009)
Milanfar, P.: Super-Resolution Imaging. CRC Press, Boca Raton (2010)
Nabney, I.T.: NETLAB: Algorithms for Pattern Recognition. Advances in Pattern Recognition, 1st edn. Springer, Heidelberg (2002)
Park, J., Kim, H., Tai, Y., Brown, M., Kweon, I.: High quality depth map upsampling for 3D-TOF cameras. In: International Conference on Computer Vision, pp. 1623–1630 (2011)
Rajagopalan, A.N., Bhavsar, A., Wallhoff, F., Rigoll, G.: Resolution enhancement of PMD range maps. In: Rigoll, G. (ed.) DAGM 2008. LNCS, vol. 5096, pp. 304–313. Springer, Heidelberg (2008)
Schultz, R.R., Stevenson, R.L.: Extraction of high-resolution frames from video sequences. IEEE Trans. Image Process. 5, 996–1011 (1996)
Schuon, S., Theobalt, C., Davis, J., Thrun, S.: High-quality scanning using time-of-flight depth superresolution. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 1–7 (2008)
Schuon, S., Theobalt, C., Davis, J., Thrun, S.: LidarBoost: depth superresolution for ToF 3D shape scanning. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 343–350 (2009)
Schwarz, S., Sjostrom, M., Olsson, R.: A weighted optimization approach to time-of-flight sensor fusion. IEEE Trans. Image Process. 23(1), 214–225 (2014)
Shotton, J., Girshick, R., Fitzgibbon, A., Sharp, T., Cook, M., Finocchio, M., Moore, R., Kohli, P., Criminisi, A., Kipman, A., Blake, A.: Efficient human pose estimation from single depth images. Pattern Anal. Mach. Intell. 35(12), 2821–2840 (2013)
Wasza, J., Bauer, S., Haase, S., Schmid, M., Reichert, S., Hornegger, J.: RITK: the range imaging toolkit - a framework for 3-D range image stream processing. In: VMV, pp. 57–64. Eurographics Association (2011)
Acknowledgments
The authors gratefully acknowledge funding of the Erlangen Graduate School in Advanced Optical Technologies (SAOT) by the German National Science Foundation (DFG) in the framework of the excellence initiative and the support by the DFG under Grant No. HO 1791/7-1.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Ghesu, F.C., Köhler, T., Haase, S., Hornegger, J. (2014). Guided Image Super-Resolution: A New Technique for Photogeometric Super-Resolution in Hybrid 3-D Range Imaging. In: Jiang, X., Hornegger, J., Koch, R. (eds) Pattern Recognition. GCPR 2014. Lecture Notes in Computer Science(), vol 8753. Springer, Cham. https://doi.org/10.1007/978-3-319-11752-2_18
Download citation
DOI: https://doi.org/10.1007/978-3-319-11752-2_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11751-5
Online ISBN: 978-3-319-11752-2
eBook Packages: Computer ScienceComputer Science (R0)