Nothing Special   »   [go: up one dir, main page]

skip to main content
10.5555/2788890.2788896guideproceedingsArticle/Chapter ViewAbstractPublication PagesgiConference Proceedingsconference-collections
research-article
Free access

Model-driven indoor scenes modeling from a single image

Published: 03 June 2015 Publication History

Abstract

In this paper, we present a new approach of 3D indoor scenes modeling on single image. With a single input indoor image (including sofa, tea table, etc.), a 3D scene can be reconstructed using existing model library in two stages: image analysis and model retrieval. In the image analysis stage, we obtain the object information from input image using geometric reasoning technology combined with image segmentation method. In the model retrieval stage, line drawings are extracted from 2D objects and 3D models by using different line rendering methods. We exploit various tokens to represent local features and then organize them together as a star-graph to show a global description. Finally, by comparing similarity among the encoded line drawings, models are retrieved from the model library and then the scene is reconstructed. Experimental results show that, driven by the given model library, indoor scenes modeling from a single image could be achieved automatically and efficiently.

Supplementary Material

ZIP File (p25-liu.zip)
Supplemental material.

References

[1]
M. Aubry, D. Maturana, A. A. Efros, B. C. Russell, and J. Sivic. Seeing 3d chairs: exemplar part-based 2d-3d alignment using a large dataset of cad models. In Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, pages 3762--3769. IEEE, 2014.
[2]
S. Chaudhuri, E. Kalogerakis, L. Guibas, and V. Koltun. Probabilistic reasoning for assembly-based 3d modeling. In ACM Transactions on Graphics (TOG), volume 30, page 35. ACM, 2011.
[3]
S. Chaudhuri and V. Koltun. Data-driven suggestions for creativity support in 3d modeling. In ACM Transactions on Graphics (TOG), volume 29, page 183. ACM, 2010.
[4]
K. Chen, Y.-K. Lai, Y.-X. Wu, R. Martin, and S.-M. Hu. Automatic semantic modeling of indoor scenes from low-quality rgb-d data using contextual information. ACM Transactions on Graphics (TOG), 33(6):208, 2014.
[5]
T. Chen, Z. Zhu, A. Shamir, S.-M. Hu, and D. Cohen-Or. 3-sweep: extracting editable objects from a single photo. ACM Transactions on Graphics (TOG), 32(6):195, 2013.
[6]
D. Comaniciu and P. Meer. Mean shift: A robust approach toward feature space analysis. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 24(5):603--619, 2002.
[7]
A. Criminisi, I. Reid, and A. Zisserman. Single view metrology. International Journal of Computer Vision, 40(2):123--148, 2000.
[8]
G. Csurka, C. Dance, L. Fan, J. Willamowski, and C. Bray. Visual categorization with bags of keypoints. In Workshop on statistical learning in computer vision, ECCV, volume 1, pages 1--2, 2004.
[9]
N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, volume 1, pages 886--893. IEEE, 2005.
[10]
D. DeCarlo, A. Finkelstein, S. Rusinkiewicz, and A. Santella. Suggestive contours for conveying shape. ACM Transactions on Graphics (TOG), 22(3):848--855, 2003.
[11]
M. Eitz, R. Richter, T. Boubekeur, K. Hildebrand, and M. Alexa. Sketch-based shape retrieval. ACM Trans. Graph., 31(4):31, 2012.
[12]
L. Fan, R. Wang, L. Xu, J. Deng, and L. Liu. Modeling by drawing with shadow guidance. In Computer Graphics Forum, volume 32, pages 157--166. Wiley Online Library, 2013.
[13]
T. Funkhouser, M. Kazhdan, P. Shilane, P. Min, W. Kiefer, A. Tal, S. Rusinkiewicz, and D. Dobkin. Modeling by example. In ACM Transactions on Graphics (TOG), volume 23, pages 652--663. ACM, 2004.
[14]
A. Gupta, M. Hebert, T. Kanade, and D. M. Blei. Estimating spatial layout of rooms using volumetric reasoning about objects and surfaces. In Advances in Neural Information Processing Systems, pages 1288--1296, 2010.
[15]
S.-M. Hu, F.-L. Zhang, M. Wang, R. R. Martin, and J. Wang. Patchnet: a patch-based image representation for interactive library-driven image editing. ACM Transactions on Graphics (TOG), 32(6):196, 2013.
[16]
S.-S. Huang, A. Shamir, C.-H. Shen, H. Zhang, A. Sheffer, S.-M. Hu, and D. Cohen-Or. Qualitative organization of collections of shapes via quartet analysis. ACM Transactions on Graphics (TOG), 32(4):71, 2013.
[17]
H. Kang, S. Lee, and C. K. Chui. Coherent line drawing. In Proceedings of the 5th international symposium on Non-photorealistic animation and rendering, pages 43--50. ACM, 2007.
[18]
A. Khotanzad and Y. H. Hong. Invariant image recognition by zernike moments. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 12(5):489--497, 1990.
[19]
D. C. Lee, M. Hebert, and T. Kanade. Geometric reasoning for single image structure recovery. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pages 2136--2143. IEEE, 2009.
[20]
J. Lee and T. Funkhouser. Sketch-based search and composition of 3d models. In Proceedings of the fifth eurographics conference on sketch-based interfaces and modeling, pages 97--104. Eurographics Association, 2008.
[21]
B. Li, Y. Lu, A. Godil, T. Schreck, B. Bustos, A. Ferreira, T. Furuya, M. J. Fonseca, H. Johan, T. Matsuda, et al. A comparison of methods for sketch-based 3d shape retrieval. Computer Vision and Image Understanding, 119:57--80, 2014.
[22]
Y. Li, Y.-Z. Song, and S. Gong. Sketch recognition by ensemble matching of structured features. 2013.
[23]
J. J. Lim, C. L. Zitnick, and P. Dollár. Sketch tokens: A learned mid-level representation for contour and object detection. In Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, pages 3158--3165. IEEE, 2013.
[24]
D. G. Lowe. Distinctive image features from scale-invariant keypoints. International journal of computer vision, 60(2):91--110, 2004.
[25]
L. Nan, K. Xie, and A. Sharf. A search-classify approach for cluttered indoor scene understanding. ACM Transactions on Graphics (TOG), 31(6):137, 2012.
[26]
D. Scharstein and R. Szeliski. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International journal of computer vision, 47(1-3):7--42, 2002.
[27]
T. Shao, W. Xu, K. Yin, J. Wang, K. Zhou, and B. Guo. Discriminative sketch-based 3d model retrieval via robust shape matching. In Computer Graphics Forum, volume 30. Wiley Online Library, 2011.
[28]
T. Shao, W. Xu, K. Zhou, J. Wang, D. Li, and B. Guo. An interactive approach to semantic modeling of indoor scenes with an rgbd camera. ACM Transactions on Graphics (TOG), 31(6):136, 2012.
[29]
C.-H. Shen, H. Fu, K. Chen, and S.-M. Hu. Structure recovery by part assembly. ACM Transactions on Graphics (TOG), 31(6):180, 2012.
[30]
H. Su, Q. Huang, N. J. Mitra, Y. Li, and L. Guibas. Estimating image depth using shape collections. ACM Transactions on Graphics (TOG), 33(4):37, 2014.
[31]
Y. Wang, M. Gong, T. Wang, D. Cohen-Or, H. Zhang, and B. Chen. Projective analysis for 3d shape segmentation. ACM Transactions on Graphics (TOG), 32(6):192, 2013.
[32]
S. Winder, G. Hua, and M. Brown. Picking the best daisy. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pages 178--185. IEEE, 2009.
[33]
J. Xiao, B. Russell, and A. Torralba. Localizing 3d cuboids in single-view images. In Advances in Neural Information Processing Systems, pages 746--754, 2012.
[34]
X. Xie, K. Xu, N. J. Mitra, D. Cohen-Or, W. Gong, Q. Su, and B. Chen. Sketch-to-design: Context-based part assembly. In Computer Graphics Forum, volume 32, pages 233--245.Wiley Online Library, 2013.
[35]
K. Xu, K. Chen, H. Fu, W.-L. Sun, and S.-M. Hu. Sketch2scene: Sketch-based co-retrieval and co-placement of 3d models. ACM Transactions on Graphics (TOG), 32(4):123, 2013.
[36]
K. Xu, H. Zheng, H. Zhang, D. Cohen-Or, L. Liu, and Y. Xiong. Photo-inspired model-driven 3d object modeling. In ACM Transactions on Graphics (TOG), volume 30, page 80. ACM, 2011.
[37]
D. Zhang, G. Lu, et al. A comparative study on shape retrieval using fourier descriptors with different shape signatures. In Proc. of international conference on intelligent multimedia and distance education (ICIMADE01), pages 1--9, 2001.
[38]
Y. Zheng, X. Chen, M.-M. Cheng, K. Zhou, S.-M. Hu, and N. J. Mitra. Interactive images: cuboid proxies for smart image manipulation. ACM Trans. Graph., 31(4):99, 2012.

Cited By

View all
  • (2018)3D Reconstruction of Indoor Scenes via Image RegistrationNeural Processing Letters10.1007/s11063-018-9781-048:3(1281-1304)Online publication date: 1-Dec-2018
  • (2017)An interactive system for efficient 3D furniture arrangementProceedings of the Computer Graphics International Conference10.1145/3095140.3095169(1-6)Online publication date: 27-Jun-2017
  • (2016)Scene structure inference through scene map estimationProceedings of the Conference on Vision, Modeling and Visualization10.5555/3056901.3056909(45-52)Online publication date: 10-Oct-2016
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings
GI '15: Proceedings of the 41st Graphics Interface Conference
June 2015
297 pages
ISBN:9780994786807

Sponsors

  • The Canadian Human-Computer Communications Society / Société Canadienne du Dialogue Humaine Machine (CHCCS/SCDHM)

Publisher

Canadian Information Processing Society

Canada

Publication History

Published: 03 June 2015

Qualifiers

  • Research-article

Acceptance Rates

Overall Acceptance Rate 206 of 508 submissions, 41%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)69
  • Downloads (Last 6 weeks)10
Reflects downloads up to 18 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2018)3D Reconstruction of Indoor Scenes via Image RegistrationNeural Processing Letters10.1007/s11063-018-9781-048:3(1281-1304)Online publication date: 1-Dec-2018
  • (2017)An interactive system for efficient 3D furniture arrangementProceedings of the Computer Graphics International Conference10.1145/3095140.3095169(1-6)Online publication date: 27-Jun-2017
  • (2016)Scene structure inference through scene map estimationProceedings of the Conference on Vision, Modeling and Visualization10.5555/3056901.3056909(45-52)Online publication date: 10-Oct-2016
  • (2016)Action-driven 3D indoor scene evolutionACM Transactions on Graphics10.1145/2980179.298022335:6(1-13)Online publication date: 5-Dec-2016
  • (2016)Learning 3D scene synthesis from annotated RGB-D imagesProceedings of the Symposium on Geometry Processing10.1111/cgf.12976(197-206)Online publication date: 20-Jun-2016
  • (2016)Modeling interactive furniture from a single imageComputers and Graphics10.1016/j.cag.2016.05.00658:C(102-108)Online publication date: 1-Aug-2016
  • (2015)Singe image-based data-driven indoor scenes modelingComputers and Graphics10.1016/j.cag.2015.10.00453:PB(210-223)Online publication date: 1-Dec-2015

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media