research-article

Public Access

Recognizing perspective accuracy: an intelligent user interface for assisting novices

Authors:

Blake Williford,

Matthew Runyon,

Tracy HammondAuthors Info & Claims

IUI '20: Proceedings of the 25th International Conference on Intelligent User Interfaces

Pages 231 - 242

https://doi.org/10.1145/3377325.3377511

Published: 17 March 2020 Publication History

Abstract

Sketching in perspective is a valuable skill for art, and for professional disciplines like industrial design, architecture, and engineering. However, it tends to be a difficult skill to grasp for novices. We have developed an algorithm that can classify rectilinear perspective strokes, as well as classify which of those strokes are accurate on a stroke-by-stroke basis. We also developed an intelligent user interface which can provide real-time accuracy feedback on a user's free-hand digital perspective sketch. To evaluate the system, we conducted a between-subjects user study with 40 novice participants which involved sketching city street corners in 2-point perspective. We discovered that the participants who received real-time intelligent feedback improved their perspective accuracy significantly (p < 0.0005) in a subsequent unassisted sketch, suggesting a mild learning effect and knowledge transfer. Through qualitative feedback we also derived principles for how intelligent real-time feedback can best assist novices in learning perspective.

References

[1]

Danilo Avola, Andrea Del Buono, Giorgio Gianforme, Stefano Paolozzi, and Rui Wang. 2009. SketchML a representation language for novel sketch recognition approach. In Proceedings of the 2^nd International Conference on PErvasive Technologies Related to Assistive Environments (PETRA). ACM, Corfu, Greece, 31.

Digital Library

[2]

Albert Bandura. 1997. Self-efficacy: The exercise of control (1^st ed.). Worth Publishers, New York, New York, USA.

[3]

Danielle Cummings, Francisco Vides, and Tracy Hammond. 2012. I Don't Believe My Eyes!: Geometric Sketch Recognition for a Computer Art Tutorial. In Proceedings of the International Symposium on Sketch-Based Interfaces and Modeling (SBIM '12). Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 97--106.

Digital Library

[4]

Karen Ernst daSilva. 2001. Drawing on Experience: Connecting Art and Language. Primary Voices K-6 10, 2 (2001), 2--9.

[5]

Daniel Dixon, Manoj Prasad, and Tracy Hammond. 2010. iCanDraw: Using Sketch Recognition and Corrective Feedback to Assist a User in Drawing Human Faces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 10). ACM, New York, New York, USA, 897--906.

Digital Library

[6]

Julie Dorsey, Songhua Xu, Gabe Smedresman, Holly Rushmeier, and Leonard McMillan. 2007. The mental canvas: A tool for conceptual architectural design and analysis. In 15^th Pacific Conference on Computer Graphics and Applications (PG'07). IEEE, Maui, Hawaii, 201--210.

Digital Library

[7]

Betty Edwards. 1999. The New Drawing on the Right Side of the Brain (2^nd ed.). Tarcher, New York, New York, USA.

[8]

Jennifer Fernquist, Tovi Grossman, and George Fitzmaurice. 2011. Sketch-Sketch Revolution: an engaging tutorial system for guided sketching and application learning. In Proceedings of the 24^th annual ACM symposium on User interface software and technology. ACM, Santa Barbara, CA, 373--382.

Digital Library

[9]

Marsha E Fonteyn, Benjamin Kuipers, and Susan J Grobe. 1993. A description of think aloud method and protocol analysis. Qualitative health research 3, 4 (1993), 430--441.

[10]

Tracy Hammond, Shalini Priya Ashok Kumar, Matthew Runyon, Josh Cherian, Blake Williford, Swarna Keshavabhotla, Stephanie Valentine, Wayne Li, and Julie Linsey. 2018. ItâĂ&Zacute;s Not Just about Accuracy: Metrics That Matter When Modeling Expert Sketching Ability. ACM Transactions on Interactive Intelligent Systems (TiiS) 8, 3 (2018), 19.

[11]

James W. Hennessey, Han Liu, Holger Winnemöller, Mira Dontcheva, and Niloy J. Mitra. 2017. How2Sketch: Generating Easy-to-follow Tutorials for Sketching 3D Objects. In Proceedings of the 21^st ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (I3D '17). ACM, New York, NY, USA, Article 8, 11 pages.

Digital Library

[12]

Ethan Hilton, Wayne Li, Sunni H Newton, Meltem Alemdar, Raghuram Pucha, and Julie Linsey. 2016. The Development and Effects of Teaching Perspective Free-Hand Sketching in Engineering Design. In ASME 2016 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. American Society of Mechanical Engineers (ASME), Charlotte, North Carolina, V003T04A013--V003T04A013.

[13]

Ethan Hilton, Myela Paige, Blake Williford, Wayne Li, Tracy Hammond, and Julie Linsey. 2017. Improving the sketching ability of engineering design students. In International Conference on Engineering Design (ICED), Vol. 9: Design Education. The Design Society, Vancouver, BC, Canada, 217--224. ISBN: 978-1-904670-97-1.

[14]

Shari L Jackson, Joseph Krajcik, and Elliot Soloway. 1998. The design of guided learner-adaptable scaffolding in interactive learning environments. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM Press/Addison-Wesley Publishing Co., Los Angeles, CA, 187--194.

Digital Library

[15]

Swarna Keshavabhotla, Blake Williford, Ethan Hilton, Paul Taele, Wayne Li, Julie Linsey, and Tracy Hammond. 2017. Conquering the Cube: Learning to Sketch Primitives in Perspective with an Intelligent Tutoring System. In SBIM '17 Proceedings of the Symposium on Sketch-Based Interfaces and Modeling, Vol. 2. ACM, Los Angeles, CA, USA, 1--11. ISBN: 978-1-4503-5079-2

Digital Library

[16]

Joseph A. Koncelik and Kevin Reeder. 2008. Conceptual Drawing (1^st ed.). Course Technology, Boston, Massachusetts, USA.

[17]

Yuki Kuribayashi, Eito Yamaga, and Takeshi Sasaki. 2019. Handwritten Cube Recognition and Evaluation Based on Perspectives for Sketch Training System. In 2019 IEEE/SICE International Symposium on System Integration (SII). IEEE, Paris, France, 34--39.

[18]

Yong Jae Lee, C. Lawrence Zitnick, and Michael F. Cohen. 2011. ShadowDraw: Real-time User Guidance for Freehand Drawing. In ACM Transactions on Graphics, Vol. 30:4. ACM, New York, NY, USA, Article 27, 10 pages.

Digital Library

[19]

Andy Liaw, Matthew Wiener, et al. 2002. Classification and regression by randomForest. R news 2, 3 (2002), 18--22.

[20]

A Chris Long Jr, James A Landay, Lawrence A Rowe, and Joseph Michiels. 2000. Visual similarity of pen gestures. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. ACM, The Hague, The Netherlands, 360--367.

[21]

Henry J Perkinson. 1979. Learning from our mistakes. Et cetera 36 (1979), 37.

[22]

Paul R. Pintrich, Christopher A. Wolters, and Gail P. Baxter. 2000. Assessing Metacognition and Self-Regulated Learning. In Metacognitive Assessment, G. Schraw (Ed.). University of Nebraska Press, Lincoln, Nebraska, USA.

[23]

Scott Robertson and Thomas Bertling. 2013. How to Draw: Drawing and Sketching Objects and Environments from your Imagination (1^st ed.). Design Studio Press, Culver City, California, USA.

[24]

Dean Rubine. 1991. Specifying Gestures by Example. SIGGRAPH Comput. Graph. 25, 4 (July 1991), 329--337.

Digital Library

[25]

Ben Shneiderman. 2002. Creativity support tools. Commun. ACM 45, 10(2002), 116--120.

Digital Library

[26]

Kathy G. Short, Gloria Kauffman, and Leslie H. Kahn. 2000. "I Just Need To Draw:" Responding to Literature across Multiple Sign Systems. The Reading Teacher 54, 2 (Oct 2000), 160--171.

[27]

Derek Sleeman and John Seely Brown. 1982. Intelligent tutoring systems. Academic Press, London.

[28]

Lev Vygotsky. 1987. Zone of proximal development. Mind in society: The development of higher psychological processes 5291 (1987), 157 pages.

[29]

Blake Williford, Matthew Runyon, Josh Cherian, Wayne Li, Julie Linsey, and Tracy Hammond. 2019. A Framework for Motivating Sketching Practice with Sketch-based Gameplay. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play. ACM, ACM, USA, 533--544.

Digital Library

[30]

Blake Williford, Matthew Runyon, Adil Hamid Malla, Wayne Li, Julie Linsey, and Tracy Hammond. 2017. ZenSketch: A Sketch-based Game For Improving Line Work. In Extended Abstracts Publication of the Annual Symposium on Computer-Human Interaction in Play. ACM, ACM, USA, 591--598.

Digital Library

[31]

Blake Williford, Paul Taele, Trevor Nelligan, Wayne Li, Julie Linsey, and Tracy Hammond. 2016. PerSketchTivity: an intelligent pen-based educational application for design sketching instruction. In Revolutionizing Education with Digital Ink: The Impact of Pen and Touch Technology on Education (Human-Computer Interaction Series), A. Adler T. Hammond, S. Valentine (Ed.). Springer, Switzerland, 115--127. ISBN: 978-3-319-31193-7

[32]

Aaron Wolin, Brian Eoff, and Tracy Hammond. 2008. ShortStraw: A Simple and Effective Corner Finder for Polylines. In Proceedings of the Fifth Eurographics Conference on Sketch-Based Interfaces and Modeling (SBIM). Eurographics Association, Annecy, France, 33--40. ISBN: 978-3-905674-07-1.

[33]

Jun Xie, Aaron Hertzmann, Wilmot Li, and Holger Winnemöller. 2014. PortraitSketch: Face Sketching Assistance for Novices. In Proceedings of the 27^th Annual ACM Symposium on User Interface Software and Technology (UIST '14). ACM, New York, New York, USA, 407--417.

Digital Library

[34]

Jun Xing, Hsiang-Ting Chen, and Li-Yi Wei. 2014. Autocomplete painting repetitions. ACM Transactions on Graphics (TOG) 33, 6 (2014), 172.

Digital Library

[35]

Jun Xing, Li-Yi Wei, Takaaki Shiratori, and Koji Yatani. 2015. Autocomplete hand-drawn animations. ACM Transactions on Graphics (TOG) 34, 6 (2015), 169.

Digital Library

[36]

Ruhizan M. Yasin, Lilia Halim, and Azaman Ishar. 2012. Effects of Problem-Solving Strategies in the Teaching and Learning of Engineering Drawing Subject. Asian Social Science 8, 16 (2012), 65--79.

Cited By

Nghiem CBousseau ASypesteyn MHoftijzer JAgrawala MTsandilas T(2024)STIVi: Turning Perspective Sketching Videos into Interactive TutorialsProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670969(1-13)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670947.3670969
Gaspar AGil MPanach JRomero V(2024)Towards a general user model to develop intelligent user interfacesMultimedia Tools and Applications10.1007/s11042-024-18240-w83:26(67501-67534)Online publication date: 25-Jan-2024
https://doi.org/10.1007/s11042-024-18240-w
Qin JSun XXu W(2023)A State-of-Art Review on Intelligent Systems for Drawing AssistingHuman Interface and the Management of Information10.1007/978-3-031-35132-7_44(583-605)Online publication date: 9-Jul-2023
https://doi.org/10.1007/978-3-031-35132-7_44
Show More Cited By

Index Terms

Recognizing perspective accuracy: an intelligent user interface for assisting novices
1. Applied computing
  1. Education
    1. Computer-assisted instruction
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction techniques
    2. Interactive systems and tools

Recommendations

Exploring the Potential of an Intelligent Tutoring System for Sketching Fundamentals
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Sketching is a practical and useful skill that can benefit communication and problem solving. However, it remains a difficult skill to learn because of low confidence and motivation among students and limited availability for instruction and personalized ...
A Framework for Motivating Sketching Practice with Sketch-based Gameplay
CHI PLAY '19: Proceedings of the Annual Symposium on Computer-Human Interaction in Play

Sketching is a valuable skill to learn but requires extensive motivation and practice to improve. We present a framework for motivating practice with sketch-based gameplay that is rooted in a grounded theory study of the motivations of various ...
DrawMyPhoto: Assisting Novices in Drawing from Photographs
C&C '19: Proceedings of the 2019 Conference on Creativity and Cognition

We present DrawMyPhoto, an interactive system that can assist a drawing novice in producing a quality drawing by automatically parsing a photograph in to a step-by-step drawing tutorial. The system utilizes image processing to produce distinct line work ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '20: Proceedings of the 25th International Conference on Intelligent User Interfaces

March 2020

607 pages

ISBN:9781450371186

DOI:10.1145/3377325

General Chairs:
Fabio Paternò,
Nuria Oliver,
Program Chairs:
Cristina Conati,
Lucio Davide Spano,
Nava Tintarev

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 March 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

IUI '20

Sponsor:

IUI '20: 25th International Conference on Intelligent User Interfaces

March 17 - 20, 2020

Cagliari, Italy

Acceptance Rates

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
558
Total Downloads

Downloads (Last 12 months)92
Downloads (Last 6 weeks)14

Reflects downloads up to 28 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Nghiem CBousseau ASypesteyn MHoftijzer JAgrawala MTsandilas T(2024)STIVi: Turning Perspective Sketching Videos into Interactive TutorialsProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670969(1-13)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670947.3670969
Gaspar AGil MPanach JRomero V(2024)Towards a general user model to develop intelligent user interfacesMultimedia Tools and Applications10.1007/s11042-024-18240-w83:26(67501-67534)Online publication date: 25-Jan-2024
https://doi.org/10.1007/s11042-024-18240-w
Qin JSun XXu W(2023)A State-of-Art Review on Intelligent Systems for Drawing AssistingHuman Interface and the Management of Information10.1007/978-3-031-35132-7_44(583-605)Online publication date: 9-Jul-2023
https://doi.org/10.1007/978-3-031-35132-7_44
Lee SLee JBae S(2022)An Interactive Car Drawing System with Tick'n'Draw for Training Perceptual and Perspective Drawing SkillsExtended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491101.3519776(1-7)Online publication date: 27-Apr-2022
https://dl.acm.org/doi/10.1145/3491101.3519776
Prange ASonntag D(2021)Assessing Cognitive Test Performance Using Automatic Digital Pen Features AnalysisProceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization10.1145/3450613.3456812(33-43)Online publication date: 21-Jun-2021
https://dl.acm.org/doi/10.1145/3450613.3456812
Castro LRay SMerzdorf HDouglas KHammond T(2021)A Metalearning Approach to Personalized Automatic Assessment of Rectilinear Sketches2021 IEEE Frontiers in Education Conference (FIE)10.1109/FIE49875.2021.9637255(1-6)Online publication date: 13-Oct-2021
https://dl.acm.org/doi/10.1109/FIE49875.2021.9637255

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents