research-article

Open access

OpenSurfaces: a richly annotated catalog of surface appearance

Authors:

Kavita BalaAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 32, Issue 4

Article No.: 111, Pages 1 - 17

https://doi.org/10.1145/2461912.2462002

Published: 21 July 2013 Publication History

Abstract

The appearance of surfaces in real-world scenes is determined by the materials, textures, and context in which the surfaces appear. However, the datasets we have for visualizing and modeling rich surface appearance in context, in applications such as home remodeling, are quite limited. To help address this need, we present OpenSurfaces, a rich, labeled database consisting of thousands of examples of surfaces segmented from consumer photographs of interiors, and annotated with material parameters (reflectance, material names), texture information (surface normals, rectified textures), and contextual information (scene category, and object names).

Retrieving usable surface information from uncalibrated Internet photo collections is challenging. We use human annotations and present a new methodology for segmenting and annotating materials in Internet photo collections suitable for crowdsourcing (e.g., through Amazon's Mechanical Turk). Because of the noise and variability inherent in Internet photos and novice annotators, designing this annotation engine was a key challenge; we present a multi-stage set of annotation tasks with quality checks and validation. We demonstrate the use of this database in proof-of-concept applications including surface retexturing and material and image browsing, and discuss future uses. OpenSurfaces is a public resource available at http://opensurfaces.cs.cornell.edu/.

Supplementary Material

ZIP File (a111-bell.zip)

Supplemental material.

Download
73.22 MB

MP4 File (tp057.mp4)

Download
13.60 MB

References

[1]

Adelson, E. H. 2001. On seeing stuff: the perception of materials by humans and machines. Proc. SPIE Human Vision and Electronic Imaging 4299.

[2]

Ben-Artzi, A., Overbeck, R., and Ramamoorthi, R. 2006. Real-time BRDF editing in complex lighting. In SIGGRAPH Conf. Proc.

Digital Library

[3]

Brainard, D. H., Brunt, W., and Speigle, J. 1997. Color constancy in the nearly natural image. J. of the Optical Society of America 14, 9.

[4]

Cgal, Computational Geometry Algorithms Library. http://www.cgal.org/.

[5]

Chen, X., Golovinskiy, A., and Funkhouser, T. 2009. A benchmark for 3D mesh segmentation. In SIGGRAPH Conf. Proc.

Digital Library

[6]

Cole, F., Sanik, K., DeCarlo, D., Finkelstein, A., Funkhouser, T., Rusinkiewicz, S., and Singh, M. 2009. How well do line drawings depict shape? In SIGGRAPH Conf. Proc.

Digital Library

[7]

Dana, K., Van-Ginneken, B., Nayar, S., and Koenderink, J. 1999. Reflectance and texture of real world surfaces. ACM Transactions on Graphics 18, 1.

Digital Library

[8]

Debevec, P. 1998. Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography. In SIGGRAPH Conf. Proc.

Digital Library

[9]

Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and FeiFei, L. 2009. ImageNet: A large-scale hierarchical image database. In Proc. Comp. Vision and Pattern Recognition.

[10]

Dror, R., Adelson, E. H., and Willsky, A. 2001. Estimating surface reflectance properties from images under unknown illumination. In Proc. SPIE Human Vision and Electronic Imaging.

[11]

Endres, I., Farhadi, A., Hoiem, D., and Forsyth, D. 2010. The benefits and challenges of collecting richer object annotations. In Workshop on Advancing Computer Vision with Humans in the Loop.

[12]

Feng, C., Deng, F., and Kamat, V. R. 2010. Semi-automatic 3D reconstruction of piecewise planar building models from single image. In Int. Conf. on Construction Appl. of Virtual Reality.

[13]

Fleming, R. W., Dror, R. O., and Adelson, E. H. 2003. Real-world illumination and the perception of surface reflectance properties. J. of Vision 3, 5.

[14]

Fleming, R. W., Torralba, A., and Adelson, E. H. 2004. Specular reflections and the perception of shape. J. of Vision 4, 9.

[15]

Geisler-Moroder, D., and Dür, A. 2010. A new Ward BRDF model with bounded albedo. In Proc. Eurographics Symp. on Rendering.

Digital Library

[16]

Gingold, Y., Shamir, A., and Cohen-Or, D. 2012. Micro perceptual human computation. ACM Transactions on Graphics 31, 5.

Digital Library

[17]

Hays, J., and Efros, A. A. 2007. Scene completion using millions of photographs. In SIGGRAPH Conf. Proc., 4:1--4:7.

Digital Library

[18]

Hu, D., Bo, L., and Ren, X. 2011. Toward robust material recognition for everyday objects. In Proc. British Machine Vision Conf.

[19]

Juracek, J. 1996. Surfaces: Visual Research for Artists and Designers. Norton.

[20]

Karsch, K., Hedau, V., Forsyth, D., and Hoiem, D. 2011. Rendering synthetic objects into legacy photographs. In SIGGRAPH Asia Conf. Proc.

Digital Library

[21]

Kerr, W. B., and Pellacini, F. 2010. Toward evaluating material design interface paradigms for novice users. ACM Transactions on Graphics 29, 4.

Digital Library

[22]

Koenderink, J. J., Doorn, A. J. V., and Kappers, A. M. L. 1992. Surface perception in pictures. Perception & Psychophysics.

[23]

Lalonde, J.-F., Hoiem, D., Efros, A. A., Rother, C., Winn, J., and Criminisi, A. 2007. Photo clip art. In SIGGRAPH Conf. Proc.

Digital Library

[24]

Liu, Y., Lin, W.-C., and Hays, J. 2004. Near regular texture analysis and manipulation. In SIGGRAPH Conf. Proc.

Digital Library

[25]

Liu, C., Sharan, L., Adelson, E., and Rosenholtz, R. 2010. Exploring features in a Bayesian framework for material recognition. In Proc. Comp. Vision and Pattern Recognition.

[26]

Marge, M., Banerjee, S., and Rudnicky, A. I. 2010. Using the Amazon Mechanical Turk for transcription of spoken language. In Int. Conf. on Acoustics, Speech, and Signal Processing.

[27]

Matusik, W., Pfister, H., Brand, M., and McMillan, L. 2003. A data-driven reflectance model. ACM Transactions on Graphics 22, 3.

Digital Library

[28]

Ngan, A., Durand, F., and Matusik, W. 2005. Experimental analysis of BRDF models. In Proc. Eurographics Symp. on Rendering.

Digital Library

[29]

Pellacini, F., Ferwerda, J. A., and Greenberg, D. P. 2000. Toward a psychophysically-based light reflection model for image synthesis. In SIGGRAPH Conf. Proc.

Digital Library

[30]

Ramanarayanan, G., Ferwerda, J., Walter, B., and Bala, K. 2007. Visual equivalence: Towards a new standard for image fidelity. In SIGGRAPH Conf. Proc.

Digital Library

[31]

Reinhard, E., Stark, M., Shirley, P., and Ferwerda, J. 2002. Photographic tone reproduction for digital images. In SIGGRAPH Conf. Proc.

Digital Library

[32]

Ren, P., Wang, J., Snyder, J., Tong, X., and Guo, B. 2011. Pocket reflectometry. In SIGGRAPH Conf. Proc.

Digital Library

[33]

Romeiro, F., and Zickler, T. 2010. Blind reflectometry. In Proc. European Conf. on Comp. Vision.

Digital Library

[34]

Rubinstein, M., Gutierrez, D., Sorkine, O., and Shamir, A. 2010. A comparative study of image retargeting. In SIGGRAPH Asia Conf. Proc.

Digital Library

[35]

Russell, B. C., Torralba, A., Murphy, K. P., and Freeman, W. T. 2008. LabelMe: A database and web-based tool for image annotation. Int. J. of Computer Vision 77, 1--3.

Digital Library

[36]

Sharan, L., Rosenholtz, R., and Adelson, E. H. 2009. Material perception: What can you see in a brief glance? J. of Vision 9, 8.

[37]

Tardif, J.-P. 2009. Non-iterative approach for fast and accurate vanishing point detection. In Proc. Int. Conf. on Comp. Vision.

[38]

Toldo, R., and Fusiello, A. 2008. Robust multiple structures estimation with J-linkage. In Proc. European Conf. on Comp. Vision.

Digital Library

[39]

Torralba, A., Fergus, R., and Freeman, W. T. 2008. 80 million tiny images: A large data set for nonparametric object and scene recognition. Trans. on Pattern Analysis and Machine Intelligence 30, 11.

Digital Library

[40]

Vangorp, P., Laurijssen, J., and Dutré, P. 2007. The influence of shape on the perception of material reflectance. ACM Transactions on Graphics 26, 3.

Digital Library

[41]

von Gioi, R. G., Jakubowicz, J., Morel, J.-M., and Randall, G. 2010. LSD: A fast line segment detector with a false detection control. Trans. on Pattern Analysis and Machine Intelligence 32, 4.

Digital Library

[42]

Walter, B., Khungurn, P., and Bala, K. 2012. Bidirectional lightcuts. In SIGGRAPH Conf. Proc.

[43]

Ward, G. 1992. Measuring and modeling anisotropic reflection. In SIGGRAPH Conf. Proc.

Digital Library

[44]

Welinder, P., Branson, S., Belongie, S., and Perona, P. 2010. The multidimensional wisdom of crowds. In Proc. Neural Information Processing Systems.

[45]

Weyrich, T., Lawrence, J., Lensch, H. P. A., Rusinkiewicz, S., and Zickler, T. 2009. Principles of appearance acquisition and representation. Foundations and Trends in Computer Graphics and Vision 4, 2.

Digital Library

[46]

Xiao, J., Hays, J., Ehinger, K. A., Oliva, A., and Torralba, A. 2010. SUN database: Large-scale scene recognition from abbey to zoo. In Proc. Comp. Vision and Pattern Recognition.

Cited By

Li ZGan RLuo CWang YLiu JZhu ZLi QYin XZhang MZhang ZPeng JCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)MaterialSeg3D: Segmenting Dense Materials from 2D Priors for 3D AssetsProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680757(370-379)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680757
Oppenlaender JAbbas TGadiraju U(2024)The State of Pilot Study Reporting in Crowdsourcing: A Reflection on Best Practices and GuidelinesProceedings of the ACM on Human-Computer Interaction10.1145/36410238:CSCW1(1-45)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3641023
Liang HYeoh JChua D(2024)Material augmented semantic segmentation of point clouds for building elementsComputer-Aided Civil and Infrastructure Engineering10.1111/mice.13198Online publication date: 17-Apr-2024
https://doi.org/10.1111/mice.13198
Show More Cited By

Index Terms

OpenSurfaces: a richly annotated catalog of surface appearance
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision

Recommendations

Relief Texture from Specularities

In vision and graphics, advanced object models require not only 3D shape, but also surface detail. While several scanning devices exist to capture the global shape of an object, few methods concentrate on capturing the fine-scale detail. Fine-scale ...
Surface Name Errors in Wikipedia
CODS-COMAD '23: Proceedings of the 6th Joint International Conference on Data Science & Management of Data (10th ACM IKDD CODS and 28th COMAD)

Surface name is the string used to refer to an entity in a text corpus. Crowd-sourced knowledge repositories such as Wikipedia can have multiple types of errors, including surface name errors. This paper focuses on identifying and correcting surface ...
YaLi: a crowdsourcing plug-in for NERD
SIGIR '13: Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval

We demonstrate the YaLi browser plug-in which discovers named entities in Web pages and provides background knowledge about them. The plug-in is implemented with two purposes. From a user perspective, it enriches the browsing experience with entities, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 32, Issue 4

July 2013

1215 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2461912

Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 July 2013

Published in TOG Volume 32, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

161
Total Citations
View Citations
3,432
Total Downloads

Downloads (Last 12 months)752
Downloads (Last 6 weeks)94

Reflects downloads up to 18 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li ZGan RLuo CWang YLiu JZhu ZLi QYin XZhang MZhang ZPeng JCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)MaterialSeg3D: Segmenting Dense Materials from 2D Priors for 3D AssetsProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680757(370-379)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680757
Oppenlaender JAbbas TGadiraju U(2024)The State of Pilot Study Reporting in Crowdsourcing: A Reflection on Best Practices and GuidelinesProceedings of the ACM on Human-Computer Interaction10.1145/36410238:CSCW1(1-45)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3641023
Liang HYeoh JChua D(2024)Material augmented semantic segmentation of point clouds for building elementsComputer-Aided Civil and Infrastructure Engineering10.1111/mice.13198Online publication date: 17-Apr-2024
https://doi.org/10.1111/mice.13198
Guerrero‐Viu JSubias JSerrano AStorrs KFleming RMasia BGutierrez D(2024)Predicting Perceived Gloss: Do Weak Labels Suffice?Computer Graphics Forum10.1111/cgf.1503743:2Online publication date: 27-Apr-2024
https://doi.org/10.1111/cgf.15037
Perez FRueda-Chacón H(2024)Beyond Appearances: Material Segmentation with Embedded Spectral Information from RGB-D imagery2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW63382.2024.00034(293-301)Online publication date: 17-Jun-2024
https://doi.org/10.1109/CVPRW63382.2024.00034
Lopes IPizzati Fde Charette R(2024)Material Palette: Extraction of Materials from a Single Image2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00419(4379-4388)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00419
Kim SPark J(2024)Multi-Level Segmentation Data Generation Based on a Scene-Specific Word TreeIEEE Access10.1109/ACCESS.2024.341851512(88202-88215)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3418515
Singh VSharma SCuzzolin F(2024)Feature boosting with efficient attention for scene parsingNeurocomputing10.1016/j.neucom.2024.128222601(128222)Online publication date: Oct-2024
https://doi.org/10.1016/j.neucom.2024.128222
Nath APatel DNiyogi R(2024)Mass Prediction and Analysis of an Object’s Mass from Its Image Using Deep LearningSN Computer Science10.1007/s42979-024-03050-65:6Online publication date: 15-Jul-2024
https://dl.acm.org/doi/10.1007/s42979-024-03050-6
Armeni IRaghu DDe Wolf C(2024)Artificial Intelligence for Predicting Reuse PatternsA Circular Built Environment in the Digital Age10.1007/978-3-031-39675-5_4(57-78)Online publication date: 4-Jan-2024
https://doi.org/10.1007/978-3-031-39675-5_4
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents