Abstract
This paper presents a novel approach that is able to resize stereoscopic video to fit various display environments with different aspect-ratios, while preserving the prominent content, keeping temporally consistent, adapting depth, as well as increasing the resolution. Our proposed approach can address retargeting and super-resolution problems simultaneously via replacing the down-sampling matrix appearing in super-resolution algorithm with a novel one, named as content-aware-sampling matrix, derived from retargeting method. The new matrix can sample the image into any resolution while preserving its important information as much as possible. Our approach can be roughly subdivided into three steps. In the first step, we calculate the overall saliency map for a shot, while considering the conspicuous information from still image and the motion information from video. In the second step, given the target resolution, we compute the retargeting parameters by a global optimization and formulate them into a matrix. Finally, we substitute the matrix into the objective function used for super-resolution, and optimize it iteratively to achieve high visual quality outcome. The experimental results based on user studies verify the effectiveness of our approach.
Chapter PDF
Similar content being viewed by others
References
Avidan, S., Shamir, A.: Seam carving for content-aware image resizing. In: ACM Transactions on Graphics (TOG) - Proceedings of ACM SIGGRAPH 2007, vol. 26 (2007)
Brust, H., Tech, G., Muller, K.: Report on generation of mixed spatial resolution stereo data base. Tech. rep., MOBILE3DTV project (2009)
Chang, C.H., Liang, C.K., Chuang, Y.Y.: Content-aware display adaptation and interactive editing for stereoscopic images. IEEE Transactions on Multimedia 13, 589–601 (2011)
Farsiu, S., Robinson, M.D., Elad, M., Milanfar, P.: Fast and robust multiframe super resolution. IEEE Transactions on Image Processing 13, 1327–1344 (2004)
Garcia, D.C., Dorea, C., de Queiroz, R.L.: Super resolution for multiview images using depth information. IEEE Transactions on Circuits and Systems for Video Technology 22, 1249–1256 (2012)
Guthier, B., Kiess, J., Kopf, S., Effelsberg, W.: Seam carving for stereoscopic video. In: IEEE IVMSP Workshop (2013)
Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: Advances in Neural Information Processing Systems (2007)
He, L., Qi, H., Zaretzki, R.: Beta process joint dictionary learning for coupled feature spaces with application to single image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013)
Kopf, S., Guthier, B., Hipp, C., Kiess, J., Effelsberg, W.: Warping-based video retargeting for stereoscopic video. In: IEEE International Conference on Image Processing (ICIP) (2014)
Li, X., Orchard, M.T.: New edge-directed interpolation. IEEE Transactions on Image Processing 10, 1521–1527 (2001)
Liu, C.: Beyond Pixels: Exploring New Representations and Applications for Motion Analysis. Ph.D. thesis, Massachusetts Institute of Technology (2009)
Liu, C., Sun, D.: A bayesian approach to adaptive video super resolution. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2011)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60, 91–110 (2004)
Niu, Y., Liu, F., Feng, W.C., Jin, H.: Aesthetics-based stereoscopic photo cropping for heterogeneous displays. IEEE Transactions on Multimedia 14, 783–796 (2012)
Rubinstein, M., Shamir, A., Avidan, S.: Improved seam carving for video retargeting. In: ACM Transactions on Graphics (TOG) - Proceedings of ACM SIGGRAPH 2008, vol. 27 (2008)
Timofte, R., Smet, V.D., Gool, L.V.: Anchored neighborhood regression for fast example-based super-resolution. In: IEEE International Conference on Computer Vision (ICCV) (2013)
Utsugi, K., Shibahara, T., Koike, T., Takahashi, K., Naemura, T.: Seam carving for stereo images. In: 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON) (2010)
Villena, S., Vega, M., Molina, R., Katsaggelos, A.K.: Bayesian super-resolution image reconstruction using an l1 prior. In: Proceedings of 6th International Symposium on Image and Signal Processing and Analysis (2009)
Wang, Y.S., Tai, C.L., Sorkine, O., Lee, T.Y.: Optimized scale-and-stretch for image resizing. In: ACM Transactions on Graphics (TOG) - Proceedings of ACM SIGGRAPH Asia 2008, vol. 27 (2008)
Yang, C.Y., Yang, M.H.: Fast direct super-resolution by simple functions. In: IEEE International Conference on Computer Vision (ICCV) (2013)
Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution via sparse representation. IEEE Transactions on Image Processing 19, 2861–2873 (2010)
Zhang, J., Cao, Y., Wang, Z.: A simultaneous method for 3d video super-resolution and high-quality depth estimation. In: IEEE International Conference on Image Processing (ICIP) (2013)
Zhang, J., Cao, Y., Zha, Z.J., Zheng, Z., Chen, C.W., Wang, Z.: A unified scheme for super-resolution and depth estimation from asymmetric stereoscopic video. IEEE Transactions on Circuits and Systems for Video Technology (2014). doi:10.1109/TCSVT.2014.2367356
Zhang, J., Cao, Y., Zheng, Z., Chen, C., Wang, Z.: A new closed loop method of super-resolution for multi-view images. Machine Vision and Applications 25, 1685–1695 (2014)
Zhu, Y., Zhang, Y., Yuille, A.L.: Single image super-resolution using deformable patches. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kang, K., Zhang, J., Cao, Y., Wang, Z. (2015). Simultaneously Retargeting and Super-Resolution for Stereoscopic Video. In: Zha, H., Chen, X., Wang, L., Miao, Q. (eds) Computer Vision. CCCV 2015. Communications in Computer and Information Science, vol 547. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-48570-5_4
Download citation
DOI: https://doi.org/10.1007/978-3-662-48570-5_4
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-48569-9
Online ISBN: 978-3-662-48570-5
eBook Packages: Computer ScienceComputer Science (R0)