research-article

A holistic approach to aesthetic enhancement of photographs

Authors:

Subhabrata Bhattacharya,

Rahul Sukthankar,

Mubarak ShahAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 7S, Issue 1

Article No.: 21, Pages 1 - 21

https://doi.org/10.1145/2037676.2037678

Published: 04 November 2011 Publication History

Abstract

This article presents an interactive application that enables users to improve the visual aesthetics of their digital photographs using several novel spatial recompositing techniques. This work differs from earlier efforts in two important aspects: (1) it focuses on both photo quality assessment and improvement in an integrated fashion, (2) it enables the user to make informed decisions about improving the composition of a photograph. The tool facilitates interactive selection of one or more than one foreground objects present in a given composition, and the system presents recommendations for where it can be relocated in a manner that optimizes a learned aesthetic metric while obeying semantic constraints. For photographic compositions that lack a distinct foreground object, the tool provides the user with crop or expansion recommendations that improve the aesthetic appeal by equalizing the distribution of visual weights between semantically different regions. The recomposition techniques presented in the article emphasize learning support vector regression models that capture visual aesthetics from user data and seek to optimize this metric iteratively to increase the image appeal. The tool demonstrates promising aesthetic assessment and enhancement results on variety of images and provides insightful directions towards future research.

References

[1]

Avidan, S. and Shamir, A. 2007. Seam carving for content-aware image resizing. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques.

Digital Library

[2]

Boutell, M. and Luo, J. 2004. Bayesian fusion of camera metadata cues in semantic scene classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.

Digital Library

[3]

Cho, T. S., Butman, M., Avidan, S., and Freeman, W. T. 2008. The patch transform and its applications to image editing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.

[4]

Datta, R., Joshi, D., Li, J., and Wang, J. Z. 2006. Studying aesthetics in photographic images using a computational approach. In Proceedings of the European Conference on Computer Vision.

Digital Library

[5]

Datta, R., Li, J., and Wang, J. Z. 2007. Learning the consensus on visual quality for next-generation image management. In Proceedings of the ACM Multimedia Conference.

Digital Library

[6]

Hartley, R. and Zisserman, A. 2004. Multiple View Geometry in Computer Vision, 2nd Ed. Cambridge University Press.

Digital Library

[7]

Hoiem, D., Efros, A., and Hebert, M. 2007. Recovering surface layout from an image. Int. J.Comput Vis. 75, 1.

Digital Library

[8]

Joachims, T. 1999. Making large-scale SVM learning practical. In Advances in Kernel Methods: Support Vector Learning. MIT Press.

Digital Library

[9]

Jonas, P. 1976. Photographic Composition Simplified. Amphoto Publishers. 2, 6, 15.

[10]

Ke, Y., Tang, X., and Jing, F. 2006. The design of high-level features for photo quality assessment. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.

Digital Library

[11]

Leyvand, T., Cohen-Or, D., Dror, G., and Lischinski, D. 2008. Data-driven enhancement of facial attractiveness. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques.

Digital Library

[12]

Liu, C., Szeliski, R., Kang, S. B., Zitnick, C. L., and Freeman, W. T. 2008. Automatic estimation and removal of noise from a single image. IEEE Trans. Patt. Anal. Mach. Intel. 30, 299--314.

Digital Library

[13]

Livio, M. 2002. The golden ratio and aesthetics. Plus Mag. Living Math.

[14]

Luo, Y. and Tang, X. 2008. Photo and video quality evaluation: Focusing on the subject. In Proceedings of the European Conference on Computer Vision.

Digital Library

[15]

Mansoor, A., Haider, M., Mian, A., and Khan, S. 2009. A hybrid image quality measure for automatic image quality assessment. In Proceedings of the Scandinavian Conference on Image Analysis.

Digital Library

[16]

Nishiyama, M., Okabe, T., Sato, Y., and Sato, I. 2009. Sensation-based photo cropping. In Proceedings of the ACM Multimedia Conference. 669--672.

Digital Library

[17]

Sun, X., Yao, H., Ji, R., and Liu, S. 2009. Photo assessment based on computational visual attention model. In Proceedings of the ACM Multimedia Conference. 541--544.

Digital Library

[18]

Venkata, N. D., Kite, T. D., Geisler, W. S., Evans, B. L., and Bovik, A. C. 2000. Image quality assessment based on a degradation model. IEEE Trans. Image Process. 9, 2.

Digital Library

[19]

Walther, D. and Koch, C. 2006. Modeling attention to salient proto-objects. Neural Networks 19, 4.

Digital Library

[20]

You, J., Perkis, A., Hannuksela, M., and Gabbouj, M. 2009. Perceptual quality assessment based on visual attention analysis. In Proceedings of the ACM Multimedia Conference.

Digital Library

[21]

Zhang, Y., Xiao, J., and Shah, M. 2004. Region completion in single image. In Proceedings of EUROGRAPHICS.

Cited By

Zhang XJiang XSong QZhang P(2023)A Visual Enhancement Network with Feature Fusion for Image Aesthetic AssessmentElectronics10.3390/electronics1211252612:11(2526)Online publication date: 3-Jun-2023
https://doi.org/10.3390/electronics12112526
Gao YWang SSo H(2023)A Reconfigurable Architecture for Real-time Event-based Multi-Object TrackingACM Transactions on Reconfigurable Technology and Systems10.1145/359358716:4(1-26)Online publication date: 1-Sep-2023
https://dl.acm.org/doi/10.1145/3593587
Simard VRönnqvist MLebel LLehoux N(2023)A Method to Classify Data Quality for Decision Making Under UncertaintyJournal of Data and Information Quality10.1145/359253415:2(1-27)Online publication date: 21-Apr-2023
https://dl.acm.org/doi/10.1145/3592534
Show More Cited By

Index Terms

A holistic approach to aesthetic enhancement of photographs
1. Information systems
  1. Information systems applications

Recommendations

A framework for photo-quality assessment and enhancement based on visual aesthetics
MM '10: Proceedings of the 18th ACM international conference on Multimedia

We present an interactive application that enables users to improve the visual aesthetics of their digital photographs using spatial recomposition. Unlike earlier work that focuses either on photo quality assessment or interactive tools for photo ...
Designing for Aesthetic Experience
C&C '21: Proceedings of the 13th Conference on Creativity and Cognition

Digital ways of representing fine art are significantly changing the way art is put on display. New exhibition formats including digital projections of fine art allow us to rethink how we can use technology to create art experiences. However, ...
Towards category-based aesthetic models of photographs
MMM'12: Proceedings of the 18th international conference on Advances in Multimedia Modeling

We present a novel data-driven category-based approach to automatically assess the aesthetic appeal of photographs. In order to tackle this problem, a novel set of image segmentation methods based on <em>feature contrast</em> are introduced, such that <...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 7S, Issue 1

Special section on ACM multimedia 2010 best paper candidates, and issue on social media

October 2011

246 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/2037676

Issue’s Table of Contents

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 November 2011

Accepted: 01 September 2011

Revised: 01 August 2011

Received: 01 March 2011

Published in TOMM Volume 7S, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

33
Total Citations
View Citations
683
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)1

Reflects downloads up to 21 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang XJiang XSong QZhang P(2023)A Visual Enhancement Network with Feature Fusion for Image Aesthetic AssessmentElectronics10.3390/electronics1211252612:11(2526)Online publication date: 3-Jun-2023
https://doi.org/10.3390/electronics12112526
Gao YWang SSo H(2023)A Reconfigurable Architecture for Real-time Event-based Multi-Object TrackingACM Transactions on Reconfigurable Technology and Systems10.1145/359358716:4(1-26)Online publication date: 1-Sep-2023
https://dl.acm.org/doi/10.1145/3593587
Simard VRönnqvist MLebel LLehoux N(2023)A Method to Classify Data Quality for Decision Making Under UncertaintyJournal of Data and Information Quality10.1145/359253415:2(1-27)Online publication date: 21-Apr-2023
https://dl.acm.org/doi/10.1145/3592534
Gao YWang PLiu LZhang CMa H(2023)Configure Your Federation: Hierarchical Attention-enhanced Meta-Learning Network for Personalized Federated LearningACM Transactions on Intelligent Systems and Technology10.1145/359136214:4(1-24)Online publication date: 15-Jun-2023
https://dl.acm.org/doi/10.1145/3591362
Hoh WZhang FDodgson N(2023)Salient-Centeredness and Saliency Size in Computational AestheticsACM Transactions on Applied Perception10.1145/358831720:2(1-23)Online publication date: 21-Apr-2023
https://dl.acm.org/doi/10.1145/3588317
Surace LWernikowski MTursun CMyszkowski KMantiuk RDidyk P(2023)Learning GAN-Based Foveated Reconstruction to Recover Perceptually Important Image FeaturesACM Transactions on Applied Perception10.1145/358307220:2(1-23)Online publication date: 21-Apr-2023
https://dl.acm.org/doi/10.1145/3583072
Ni SShao FChai XChen HHo Y(2023)Composition-Guided Neural Network for Image Cropping Aesthetic AssessmentIEEE Transactions on Multimedia10.1109/TMM.2022.321500325(6836-6851)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TMM.2022.3215003
Wu X(2022)Interpretable Aesthetic Analysis Model for Intelligent Photography Guidance SystemsProceedings of the 27th International Conference on Intelligent User Interfaces10.1145/3490099.3511155(661-671)Online publication date: 22-Mar-2022
https://dl.acm.org/doi/10.1145/3490099.3511155
Farhat FKamani MWang J(2022)CAPTAIN: Comprehensive Composition Assistance for Photo TakingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/346276218:1(1-24)Online publication date: 27-Jan-2022
https://dl.acm.org/doi/10.1145/3462762
Zhong LLi FHuang HZhang YLu SWang J(2021)Aesthetic-guided outward image croppingACM Transactions on Graphics10.1145/3478513.348056640:6(1-13)Online publication date: 10-Dec-2021
https://dl.acm.org/doi/10.1145/3478513.3480566
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents