research-article

Predicting Rating Distributions of Website Aesthetics with Deep Learning for AI-Based Research

Authors:

Meinald T. Thielsch,

Matthias Böhmer,

Guido HertelAuthors Info & Claims

ACM Transactions on Computer-Human Interaction, Volume 30, Issue 3

Article No.: 37, Pages 1 - 28

https://doi.org/10.1145/3569889

Published: 10 June 2023 Publication History

Abstract

The aesthetic appeal of a website has strong effects on users’ reactions, appraisals, and even behaviors. However, evaluating website aesthetics through user ratings is resource intensive, and extant models to predict website aesthetics are limited in performance and ability. We contribute a novel and more precise approach to predict website aesthetics that considers rating distributions. Moreover, we use this approach as a baseline model to illustrate how future research might be conducted using predictions instead of participants. Our approach is based on a deep convolutional neural network model and uses innovations in the field of image aesthetic prediction. It was trained with the dataset from Reinecke and Gajos [2014] and was validated using two independent large datasets. The final model reached an unprecedented cross-validated correlation between the ground truth and predicted rating of LCC = 0.752. We then used the model to successfully replicate prior findings and conduct original research as an illustration for AI-based research.

References

[1]

Ahamed Altaboli and Yingzi Lin. 2011. Objective and subjective measures of visual aesthetics of website interface design: The two sides of the coin. In Human-Computer Interaction, Julie A. Jacko (Ed.). Springer, 35–44.

[2]

Rudolf Arnheim. 1974. Art and Visual Perception: A Psychology of the Creative Eye (2nd rev. ed.). University of California Press.

[3]

Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador Garcia, Sergio Gil-Lopez, Daniel Molina, Richard Benjamins, Raja Chatila, and Francisco Herrera. 2020. Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion 58, C (2020), 82–115. DOI:

Digital Library

[4]

Xiaojie Bao, Ping Shi, Da Pan, and Jian You. 2019. Image aesthetic prediction model based on deep convolutional neural network. In Proceedings of the 2019 2nd International Conference on Information Systems and Computer Aided Education. IEEE, 122–125. DOI:

[5]

Daniel Berlyne. 1971. Aesthetics and Psychobiology. Appleton-Century-Crofts.

[6]

George David Birkhoff. 1933. Aesthetic Measure. Havard University Press.

[7]

Jens Bölte, Thomas Hösker, Gerrit Hirschfeld, and Meinald Thielsch. 2017. Electrophysiological correlates of aesthetic processing of webpages: A comparison of experts and laypersons. PeerJ 5, e3440 (June 2017). DOI:

[8]

Dianne Cyr, Milena Head, Hector Larios, and Bing Pan. 2009. Exploring human images in website design: A multi-method approach. MIS Quarterly 33, 3 (September 2009), 539–566. DOI:

[9]

Sagnik Dhar, Vicente Ordonez, and Tamara L. Berg. 2011. High level describable attributes for predicting aesthetics and interestingness. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 1657–1664. DOI:

Digital Library

[10]

Soussan Djamasbi, Marisa Siegel, and Thomas Tullis. 2012. Faces and viewing behavior: An exploratory investigation. AIS Transactions on Human-Computer Interaction 4, 3 (September 2012), 190–211. DOI:

[11]

Soussan Djamasbi, Marisa Siegel, and Tom Tullis. 2010. Generation Y, web design, and eye tracking. International Journal of Human-Computer Studies 68, 5 (May 2010), 307–323. DOI:

Digital Library

[12]

Soussan Djamasbi, Marisa Siegel, and Tom Tullis. 2014. Can fixation on main images predict visual appeal of homepages? In Proceedings of the 2014 47th Hawaii International Conference on System Sciences. 371–375. DOI:

Digital Library

[13]

Yadolah Dodge. 1987. An introduction to L1-norm based statistical data analysis. Computational Statistics & Data Analysis 5, 4 (1987), 239–253. DOI:

Digital Library

[14]

Qi Dou, Xianjun Sam Zheng, Tongfang Sun, and Pheng-Ann Heng. 2019. Webthetics: Quantifying webpage aesthetics with deep learning. International Journal of Human-Computer Studies 124, (2019), 56–66. DOI:

[15]

Maria Douneva, Rafael Jaron, and Meinald T. Thielsch. 2016. Effects of different website designs on first impressions, aesthetic judgements and memory performance after short presentation. Interacting with Computers 28, 4 (2016), 552–567. DOI:

[16]

Ida Engholm. 2002. Digital style history: The development of graphic design on the internet. Digital Creativity 13, 4 (December 2002), 193–211. DOI:

[17]

Gustav Theodor Fechner. 1876. Vorschule der Ästhetik. Breitkopf und Härtel.

[18]

Marshal F. Folstein, Lee N. Robins, and John E. Helzer. 1983. The mini-mental state examination. Archives of General Psychiatry 40, 7 (1983), 812. DOI:

[19]

Anne Frenzel and Ulrike Nett. 2008. Berliner Intelligenzstrukturtest für Jugendliche : Begabungs- und Hochbegabungsdiagnostik (BIS-HB) von A. O. Jäger et al. ; [Rezension]. Zuerst ersch. in: Diagnostica 54, 4 (January 2008), 221–225.

[20]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 9 (2015), 1904–1916. DOI:

Digital Library

[21]

Vlad Hosu, Bastian Goldlucke, and Dietmar Saupe. 2019. Effective aesthetics prediction with multi-level spatially pooled features. In Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9367–9375. DOI:

[22]

Le Hou, Chen-Ping Yu, and Dimitris Samaras. 2017. Squared earth mover's distance-based loss for training deep neural networks. arXiv:1611.05916. Retrieved from https://arxiv.org/abs/1611.05916.

[23]

Andrew Howard, Menglong Zhu, Bo Chen, D. Kalenichenko, W. Wang, Tobias Weyand, M. Andreetto, and H. Adam. 2017. MobileNets: Efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861. Retrieved from https://arxiv.org/abs/1704.04861.

[24]

Glena H. Iten, Antonin Troendle, and Klaus Opwis. 2018. Aesthetics in context: The role of aesthetics and usage mode for a website's success. Interacting with Computers 30, 2 (2018), 133–149. DOI:

[25]

Melody Y. Ivory and Rodrick Megraw. 2005. Evolution of web site design patterns. ACM Transactions on Information Systems 23, 4 (October 2005), 463–497. DOI:

Digital Library

[26]

Thomas Jacobsen, Ricarda I. Schubotz, Lea Höfel, and D. Yves von Cramon. 2006. Brain correlates of aesthetic judgment of beauty. Neuroimage 29, 1 (2006), 276–285. DOI:

[27]

Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. Caffe: Convolutional architecture for fast feature embedding. In Proceedings of the ACM Conference on Multimedia. 675–678. Retrieved from https://arxiv.org/abs/1408.5093.

[28]

Bin Jin, Maria V. Ortiz Segovia, and Sabine Susstrunk. 2016. Image aesthetic predictors based on weighted CNNs. In Proceedings of the 2016 IEEE International Conference on Image Processing. IEEE, Piscataway, NJ, 2291–2295. DOI:

[29]

Yan Ke, Xiaoou Tang, and Feng Jing. 2006. The design of high-level features for photo quality assessment. In Proceedings of the Conference on Computer Vision and Pattern Recognition. IEEE, Piscataway, NJ, 419–426. DOI:

Digital Library

[30]

Masoud Ganj Khani, Mohammad Reza Mazinani, Mohsen Fayyaz, and Mojtaba Hoseini. 2016. A novel approach for website aesthetic evaluation based on convolutional neural networks. In Proceedings of the 2016 2nd International Conference on Web Research. IEEE, Piscataway, NJ, 48–53. DOI:

[31]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems. Curran Associates Inc, Red Hook, NY, 1097–1105.

[32]

Talia Lavie and Noam Tractinsky. 2004. Assessing dimensions of perceived visual aesthetics of web sites. International Journal of Human-Computer Studies 60, 3 (2004), 269–298. DOI:

Digital Library

[33]

Helmut Leder, Benno Belke, Andries Oeberst, and Dorothee Augustin. 2004. A model of aesthetic appreciation and aesthetic judgments. British Journal of Psychology 95, 4 (2004), 489–508. DOI:

[34]

Elizabeth Levina and Peter Bickel. 2001. The earth mover's distance is the Mallows distance: Some insights from statistics. In Proceedings of the 8th IEEE International Conference on Computer Vision. IEEE Computer Society, Los Alamitos, Calif, 251–256. DOI:

[35]

Gitte Lindgaard, Cathy Dudek, Devjani Sen, Livia Sumegi, and Patrick Noonan. 2011. An exploration of relations between visual appeal, trustworthiness and perceived usability of homepages. ACM Transactions on Computer-Human Interaction 18, 1 (2011), 1–30. DOI:

Digital Library

[36]

Gitte Lindgaard, Gary Fernandes, Cathy Dudek, and J. Brown. 2006. Attention web designers: You have 50 milliseconds to make a good first impression! Behaviour & Information Technology 25, 2 (2006), 115–126. DOI:

[37]

Dong Liu, Rohit Puri, Nagendra Kamath, and Subhabrata Bhattacharya. 2020. Composition-aware image aesthetics assessment. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 3558–3567. DOI:

[38]

Xin Lu, Zhe Lin, Hailin Jin, Jianchao Yang, and James Z. Wang. 2014. RAPID. In Proceedings of the 2014 ACM Conference on Multimedia. ACM, New York, NY, 457–466. DOI:

Digital Library

[39]

Long Mai, Hailin Jin, and Feng Liu. 2016. Composition-preserving deep photo aesthetics assessment. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.

[40]

Aliaksei Miniukovich and Antonella de Angeli. 2015. Computation of interface aesthetics. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 1163–1172. DOI:

Digital Library

[41]

Morten Moshagen and Meinald T. Thielsch. 2010. Facets of visual aesthetics. International Journal of Human-Computer Studies 68, 10 (2010), 689–709.

Digital Library

[42]

Morten Moshagen and Meinald T. Thielsch. 2013. A short version of the visual aesthetics of websites inventory. Behaviour & Information Technology 32, 12 (2013), 1305–1311.

Digital Library

[43]

Naila Murray and Albert Gordo. 2017. A deep architecture for unified aesthetic prediction. arXiv:1708.04890. Retrieved from https://arxiv.org/abs/1708.04890.

[44]

David Chek Ling Ngo, Lian Seng Teo, and John G. Byrne. 2003. Modelling interface aesthetics. Information Sciences 152, 1 (June 2003), 25–46. DOI:

Digital Library

[45]

Masashi Nishiyama, Takahiro Okabe, Imari Sato, and Yoichi Sato. 2011. Aesthetic quality classification of photographs based on color harmony. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, Piscataway, NJ, 33–40. DOI:

Digital Library

[46]

PsyWeb. 2020. PsyWeb. Retrieved May 18, 2020 from https://psyweb.uni-muenster.de/.

[47]

Katharina Reinecke and Krzysztof Z. Gajos. 2014. Quantifying visual preferences around the world. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 11–20. DOI:

Digital Library

[48]

Katharina Reinecke, Tom Yeh, Luke Miratrix, Rahmatri Mardiko, Yuechen Zhao, Jenny Liu, and Krzysztof Z. Gajos. 2013. Predicting users’ first impressions of website aesthetics with a quantification of perceived visual complexity and colorfulness. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2049–2058.

Digital Library

[49]

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. Model-agnostic interpretability of machine learning. In Proceedings of the 2016 ICML Workshop on Human Interpretability in Machine Learning. DOI:

[50]

David Robins and Jason Holmes. 2008. Aesthetics and credibility in web site design. Information Processing & Management 44, 1 (2008), 386–399. DOI:

Digital Library

[51]

Sandra P. Roth, Peter Schmutz, Stefan L. Pauwels, Javier A. Bargas-Avila, and Klaus Opwis. 2010. Mental models for web objects: Where do users expect to find the most frequent objects in online shops, news portals, and company web pages? Interacting with Computers 22, 2 (March 2010), 140–152. DOI:

Digital Library

[52]

Mirjam Seckler, Klaus Opwis, and Alexandre N. Tuch. 2015. Linking objective design factors with subjective aesthetics: An experimental study on how structure and color of websites affect the facets of users’ visual aesthetic perception. Computers in Human Behavior 49, C (2015), 375–389. DOI:

Digital Library

[53]

Karen Simonyan and Andrew Zisserman. 2015. Very deep convolutional networks for large-scale image recognition. In Proceedings of the 3rd International Conference on Learning Representations. 1–14. Retrieved from http://arxiv.org/pdf/1409.1556v6.

[54]

Christian Szegedy, V. Vanhoucke, S. Ioffe, Jon Shlens, and Z. Wojna. 2016. Rethinking the inception architecture for computer vision. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. DOI:

[55]

Hossein Talebi and Peyman Milanfar. 2018. NIMA: Neural Image Assessment. IEEE Transactions on Image Processing : A Publication of the IEEE Signal Processing Society 27, 8 (2018), 3998–4011. DOI:

[56]

Jan Theeuwes and Stefan Van der Stigchel. 2006. Faces capture attention: Evidence from inhibition of return. Visual Cognition 13, 6 (April 2006), 657–665. DOI:

[57]

Meinald T. Thielsch, I. Blotenberg, and R. Jaron. 2014. User evaluation of websites: From first impression to recommendation. Interacting with Computers 26, 1 (2014), 89–102. DOI:

[58]

Meinald T. Thielsch, Russell Haines, and Leonie Flacke. 2019. Experimental investigation on the effects of interface aesthetics on user performance in different virtual tasks. PeerJ 7 (2019), e6516. DOI:

[59]

Meinald T. Thielsch and Gerrit Hirschfeld. 2019. Facets of website content. Human–Computer Interaction 34, 4 (2019), 279–327. DOI:

[60]

Meinald T. Thielsch and Gerrit Hirschfeld. 2021. Quick assessment of web content perceptions. International Journal of Human-Computer Interaction 37, 1 (2021), 68–80. DOI:

[61]

Meinald T. Thielsch, Jana Scharfen, Ehsan Masoudi, and Meike Reuter. 2019. Visual aesthetics and performance. In Proceedings of the Mensch und Computer 2019. ACM, New York, NY, 199–210. DOI:

Digital Library

[62]

Noam Tractinsky. 2013. Visual aesthetics. In The Encyclopedia of Human-Computer Interaction (2nd ed.). M. Soegaard and R. F. Dam (Eds.). The Interaction Design Foundation.

[63]

Noam Tractinsky, Avivit Cokhavi, Moti Kirschenbaum, and Tal Sharfi. 2006. Evaluating the consistency of immediate aesthetic perceptions of web pages. International Journal of Human-Computer Studies 64, 11 (2006), 1071–1083. DOI:

Digital Library

[64]

Alexandre N. Tuch, Eva E. Presslaber, Markus Stöcklin, Klaus Opwis, and Javier A. Bargas-Avila. 2012. The role of visual complexity and prototypicality regarding first impression of websites: Working towards understanding aesthetic judgments. International Journal of Human-Computer Studies 70, 11 (November 2012), 794–811. DOI:

Digital Library

[65]

Webwiki. 2020. Webwiki - Das Bewertungsportal für Websites. Retrieved May 18, 2020 from https://www.webwiki.de/.

[66]

Ou Wu, Weiming Hu, and Jun Gao. 2011. Learning to predict the perceived visual quality of photos. In Proceedings of the IEEE International Conference on Computer Vision. IEEE, Piscataway, NJ, 225–232. DOI:

Digital Library

[67]

Tal Yarkoni and Jacob Westfall. 2017. Choosing prediction over explanation in psychology: Lessons from machine learning. Perspectives on Psychological Science 12, 6 (2017), 1100–1122. DOI:

[68]

Xianjun Sam Zheng, Ishani Chakraborty, James Jeng-Weei Lin, and Robert Rauschenberger. 2009. Correlating low-level image statistics with users - rapid aesthetic and affective judgments of web pages. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 1. DOI:

Digital Library

Cited By

Wang XTong MSong YXue C(2024)Utilizing Multiple Regression Analysis and Entropy Method for Automated Aesthetic Evaluation of Interface LayoutsSymmetry10.3390/sym1605052316:5(523)Online publication date: 26-Apr-2024
https://doi.org/10.3390/sym16050523
Zhu H(2024)Application Exploration of Visual Recognition Technology Based on Deep Learning Algorithm in Website DevelopmentProcedia Computer Science10.1016/j.procs.2024.10.052247(438-444)Online publication date: 2024
https://doi.org/10.1016/j.procs.2024.10.052
Arkanov IBakaev M(2024)As AI Like It: Neural Network Models for Recognizing Website AestheticsAdvances in Neural Computation, Machine Learning, and Cognitive Research VIII10.1007/978-3-031-73691-9_41(438-447)Online publication date: 20-Oct-2024
https://doi.org/10.1007/978-3-031-73691-9_41

Index Terms

Predicting Rating Distributions of Website Aesthetics with Deep Learning for AI-Based Research
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Neural networks
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
  2. Interaction design
    1. Interaction design process and methods
      1. User interface design

Recommendations

Predicting users' first impressions of website aesthetics with a quantification of perceived visual complexity and colorfulness
CHI '13: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Users make lasting judgments about a website's appeal within a split second of seeing it for the first time. This first impression is influential enough to later affect their opinions of a site's usability and trustworthiness. In this paper, we ...
Rating prediction of recommended item based on review deep learning and rating probability matrix factorization
Highlights
- Combines the Deep Learning for review texts and the Probability Matrix Factorization method for rating data to predict the rating of the recommended items ...
Abstract
With a sharp improvement in E-commerce and data, the precise rating prediction of recommended items under user preferences has been a hot research topic in the EC intelligence domain. The rating data Matrix Factorization based methods ...
The effects of web site aesthetics and shopping task on consumer online purchasing behavior
CHI EA '08: CHI '08 Extended Abstracts on Human Factors in Computing Systems

This research-in-progress investigates the effects of website aesthetics and shopping task type on consumer attitude towards electronic shopping sites. The conceptualization of website aesthetics is based on the two-dimensional structure, namely, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Computer-Human Interaction

ACM Transactions on Computer-Human Interaction Volume 30, Issue 3

June 2023

544 pages

ISSN:1073-0516

EISSN:1557-7325

DOI:10.1145/3604411

Editors:
Kristina Höök
KTH Royal Institute of Technology, Sweden
,
Kasper Hornbæk
University of Copenhagen, Denmark

Issue’s Table of Contents

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 June 2023

Online AM: 29 October 2022

Accepted: 31 August 2022

Revised: 17 August 2022

Received: 30 June 2021

Published in TOCHI Volume 30, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
806
Total Downloads

Downloads (Last 12 months)307
Downloads (Last 6 weeks)36

Reflects downloads up to 14 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang XTong MSong YXue C(2024)Utilizing Multiple Regression Analysis and Entropy Method for Automated Aesthetic Evaluation of Interface LayoutsSymmetry10.3390/sym1605052316:5(523)Online publication date: 26-Apr-2024
https://doi.org/10.3390/sym16050523
Zhu H(2024)Application Exploration of Visual Recognition Technology Based on Deep Learning Algorithm in Website DevelopmentProcedia Computer Science10.1016/j.procs.2024.10.052247(438-444)Online publication date: 2024
https://doi.org/10.1016/j.procs.2024.10.052
Arkanov IBakaev M(2024)As AI Like It: Neural Network Models for Recognizing Website AestheticsAdvances in Neural Computation, Machine Learning, and Cognitive Research VIII10.1007/978-3-031-73691-9_41(438-447)Online publication date: 20-Oct-2024
https://doi.org/10.1007/978-3-031-73691-9_41

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents