research-article

Open access

Sketchplore: Sketch and Explore with a Layout Optimiser

Authors:

Antti OulasvirtaAuthors Info & Claims

DIS '16: Proceedings of the 2016 ACM Conference on Designing Interactive Systems

Pages 543 - 555

https://doi.org/10.1145/2901790.2901817

Published: 04 June 2016 Publication History

Abstract

This paper studies a novel concept for integrating real-time design optimisation to a sketching tool. Although optimisation methods can attack very complex design problems, their insistence on precise objectives and a point optimum is a poor fit with sketching practices. Sketchplorer is a multitouch sketching tool that uses a real-time layout optimiser. It automatically infers the designer's task to search for both local improvements to the current design and global (radical) alternatives. Using predictive models of sensorimotor performance and perception, these suggestions steer the designer toward more usable and aesthetic layouts without overriding the designer or demanding extensive input.

Supplementary Material

suppl.mov (pn248.m4v)

Supplemental video

Download
18.45 MB

References

[1]

Seok-Hyung Bae, Ravin Balakrishnan, and Karan Singh. 2008. ILoveSketch: As-natural-as-possible Sketching System for Creating 3D Curve Models. In Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology (UIST '08). ACM, New York, NY, USA, 151--160.

Digital Library

[2]

Gilles Bailly, Antti Oulasvirta, Timo Kötzing, and Sabrina Hoppe. 2013. MenuOptimizer: Interactive Optimization of Menu Systems. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST '13). ACM, New York, NY, USA, 331--342.

Digital Library

[3]

Helen Y. Balinsky. 2006. Evaluating interface aesthetics: measure of symmetry. In Electronic Imaging 2006. International Society for Optics and Photonics, 607--608.

[4]

Helen Y. Balinsky, Anthony J. Wiley, and Matthew C. Roberts. 2009. Aesthetic Measure of Alignment and Regularity. In Proceedings of the 9th ACM Symposium on Document Engineering (DocEng '09). ACM, New York, NY, USA, 56--65.

Digital Library

[5]

Rainer E. Burkhard and J. Offerman. 1977. Entwurf von schreibmaschinentastaturen mittels quadratischer zuordnungsprobleme. Operations Res 21 (1977), B121--B132.

[6]

Bill Buxton. 2007. Sketching User Experiences: Getting the Design Right and the Right Design. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.

Digital Library

[7]

Daniel Cohen-Or, Olga Sorkine, Ran Gal, Tommer Leyvand, and Ying-Qing Xu. 2006. Color Harmonization. In ACM SIGGRAPH 2006 Papers (SIGGRAPH '06). ACM, New York, NY, USA, 624--630.

Digital Library

[8]

Nigel Cross. 2004. Expertise in design: an overview. Design studies 25, 5 (2004), 427--441.

[9]

Amine Drira, Henri Pierreval, and Sonia Hajri-Gabouj. 2007. Facility layout problems: A survey. Annual Reviews in Control 31, 2 (2007), 255--267.

[10]

Karim El Batran and Mark D. Dunlop. 2014. Enhancing KLM (Keystroke-level Model) to Fit Touch Screen Mobile Devices. In Proceedings of the 16th International Conference on Human-computer Interaction with Mobile Devices & Services (MobileHCI '14). ACM, New York, NY, USA, 283--286.

Digital Library

[11]

Steven K. Feiner. 1988. A Grid-based Approach to Automating Display Layout. In Proceedings on Graphics Interface '88. Canadian Information Processing Society, Toronto, Ont., Canada, Canada, 192--197. http://dl.acm.org/citation.cfm?id=102313.102339

Digital Library

[12]

Krzysztof Gajos and Daniel S. Weld. 2004. SUPPLE: Automatically Generating User Interfaces. In Proceedings of the 9th International Conference on Intelligent User Interfaces (IUI '04). ACM, New York, NY, USA, 93--100.

Digital Library

[13]

Mark D. Gross and Ellen Yi-Luen Do. 1996. Ambiguous Intentions: A Paper-like Interface for Creative Design. In Proceedings of the 9th Annual ACM Symposium on User Interface Software and Technology (UIST '96). ACM, New York, NY, USA, 183--192.

Digital Library

[14]

Pierre Hansen and Nenad Mladenović. 2001. Variable neighborhood search: Principles and applications. European journal of operational research 130, 3 (2001), 449--467.

[15]

Marc Hassenzahl. 2008. The Interplay of Beauty, Goodness, and Usability in Interactive Products. Hum.-Comput. Interact. 19, 4 (Dec. 2008), 319--349.

Digital Library

[16]

Eric Horvitz. 1999. Principles of Mixed-initiative User Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '99). ACM, New York, NY, USA, 159--166.

Digital Library

[17]

Gabe Johnson, Mark D. Gross, Jason Hong, and Ellen Yi-Luen Do. 2009. Computational Support for Sketching in Design: A Review. Found. Trends Hum.-Comput. Interact. 2, 1 (Jan. 2009), 1--93.

Digital Library

[18]

Levent Burak Kara, Chris M. D'Eramo, and Kenji Shimada. 2006. Pen-based Styling Design of 3D Geometry Using Concept Sketches and Template Models. In Proceedings of the 2006 ACM Symposium on Solid and Physical Modeling (SPM '06). ACM, New York, NY, USA, 149--160.

Digital Library

[19]

Rubaiat Habib Kazi, Fanny Chevalier, Tovi Grossman, and George Fitzmaurice. 2014. Kitty: Sketching Dynamic and Interactive Illustrations. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST '14). ACM, New York, NY, USA, 395--405.

Digital Library

[20]

Kenneth L. Kelly. 1965. Twenty-two colors of maximum contrast. Color Engineering 3, 26 (1965), 26--27.

[21]

David E. Kieras and Anthony J. Hornof. 2014. Towards Accurate and Practical Predictive Models of Active-vision-based Visual Search. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 3875--3884.

Digital Library

[22]

Ranjitha Kumar, Arvind Satyanarayan, Cesar Torres, Maxine Lim, Salman Ahmad, Scott R. Klemmer, and Jerry O. Talton. 2013. Webzeitgeist: Design Mining the Web. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 3083--3092.

Digital Library

[23]

James A. Landay and Brad A. Myers. 1995. Interactive Sketching for the Early Stages of User Interface Design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '95). ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 43--50.

Digital Library

[24]

Lissa Light and Peter Anderson. 1993. Designing better keyboards via simulated annealing. (1993).

[25]

James Lin, Mark W. Newman, Jason I. Hong, and James A. Landay. 2000. DENIM: Finding a Tighter Fit Between Tools and Practice for Web Site Design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '00). ACM, New York, NY, USA, 510--517.

Digital Library

[26]

Simon Lok and Steven Feiner. 2001. A survey of automated layout techniques for information presentations. Proceedings of SmartGraphics 2001 (2001).

[27]

Simon Lok, Steven Feiner, and Gary Ngai. 2004. Evaluation of Visual Balance for Automated Layout. In Proceedings of the 9th International Conference on Intelligent User Interfaces (IUI '04). ACM, New York, NY, USA, 101--108.

Digital Library

[28]

Scott I. MacKenzie. 1992. Fitts' law as a research and design tool in human-computer interaction. Human-Computer Interaction 7, 1 (1992), 91--139.

Digital Library

[29]

Barbara J. Meier, Anne Morgan Spalter, and David B. Karelitz. 2004. Interactive Color Palette Tools. IEEE Comput. Graph. Appl. 24, 3 (May 2004), 64--72.

Digital Library

[30]

Mark W. Newman and James A. Landay. 2000. Sitemaps, Storyboards, and Specifications: A Sketch of Web Site Design Practice. In Proceedings of the 3rd Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques (DIS '00). ACM, New York, NY, USA, 263--274.

Digital Library

[31]

Peter O'Donovan, Aseem Agarwala, and Aaron Hertzmann. 2014. Learning Layouts for Single-PageGraphic Designs. Visualization and Computer Graphics, IEEE Transactions on 20, 8 (2014), 1200--1213.

Digital Library

[32]

Peter O'Donovan, Aseem Agarwala, and Aaron Hertzmann. 2015. DesignScape: Design with Interactive Layout Suggestions. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 1221--1224.

Digital Library

[33]

Antti Oulasvirta, Anna Reichel, Wenbin Li, Yan Zhang, Myroslav Bachynskyi, Keith Vertanen, and Per Ola Kristensson. 2013. Improving Two-thumb Text Entry on Touchscreen Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 2765--2774.

Digital Library

[34]

Ruth Rosenholtz, Yuanzhen Li, and Lisa Nakano. 2007. Measuring visual clutter. Journal of vision 7, 2 (2007), 17.

[35]

Clayton Shepard, Ahmad Rahmati, Chad Tossell, Lin Zhong, and Phillip Kortum. 2011. LiveLab: Measuring Wireless Networks and Smartphone Users in the Field. SIGMETRICS Perform. Eval. Rev. 38, 3 (Jan. 2011), 15--20.

Digital Library

[36]

Surya P. Singh and Renduchintala RK Sharma. 2006. A review of different approaches to the facility layout problems. The International Journal of Advanced Manufacturing Technology 30, 5--6 (2006), 425--433.

[37]

Ivan E. Sutherland. 1963. Sketchpad: A Man-machine Graphical Communication System. In Proceedings of the May 21--23, 1963, Spring Joint Computer Conference (AFIPS '63 (Spring)). ACM, New York, NY, USA, 329--346.

Digital Library

[38]

Michael Terry, Elizabeth D. Mynatt, Kumiyo Nakakoji, and Yasuhiro Yamamoto. 2004. Variation in Element and Action: Supporting Simultaneous Development of Alternative Solutions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '04). ACM, New York, NY, USA, 711--718.

Digital Library

[39]

Marc Van Droogenbroeck and Sérard. 2011.ebastien Pié Object Descriptors Based on a List of Rectangles: Method and Algorithm. In Proceedings of the 10th International Conference on Mathematical Morphology and Its Applications to Image and Signal Processing (ISMM'11). Springer-Verlag, Berlin, Heidelberg, 155--165. http: //dl.acm.org/citation.cfm?id=2023043.2023061

Digital Library

[40]

L.G. Williams. 1966. A Study of Visual Search Using Eye Movement Recordings. Technical Report. DTIC Document.

[41]

Jacob O. Wobbrock, Andrew D. Wilson, and Yang Li. 2007. Gestures Without Libraries, Toolkits or Training: A $1 Recognizer for User Interface Prototypes. In Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology (UIST '07). ACM, New York, NY, USA, 159--168.

Digital Library

[42]

Yin Yin Wong. 1992. Rough and Ready Prototypes: Lessons from Graphic Design. In Posters and Short Talks of the 1992 SIGCHI Conference on Human Factors in Computing Systems (CHI '92). ACM, New York, NY, USA, 83--84.

Digital Library

[43]

Shengxiang Yang and Xin Yao. 2008. Population-based incremental learning with associative memory for dynamic environments. Evolutionary Computation, IEEE Transactions on 12, 5 (2008), 542--561.

Digital Library

[44]

Yeonsoo Yang and Scott R. Klemmer. 2009. Aesthetics Matter: Leveraging Design Heuristics to Synthesize Visually Satisfying Handheld Interfaces. In CHI '09 Extended Abstracts on Human Factors in Computing Systems (CHI EA '09). ACM, New York, NY, USA, 4183--4188.

Digital Library

[45]

Shumin Zhai, Michael Hunter, and Barton A. Smith. 2002. Performance Optimization of Virtual Keyboards. Human-Computer Interaction 17, 2--3 (2002), 229--269.

Cited By

Mo GDudley JChan LLiao YOulasvirta AKristensson P(2024)Cooperative Multi-Objective Bayesian Design OptimizationACM Transactions on Interactive Intelligent Systems10.1145/365764314:2(1-28)Online publication date: 17-Apr-2024
https://dl.acm.org/doi/10.1145/3657643
Takaffoli MLi SMäkelä V(2024)Generative AI in User Experience Design and Research: How Do UX Practitioners, Teams, and Companies Use GenAI in Industry?Proceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660720(1579-1593)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3660720
Wang ZShen LKuang EZhang SFan M(2024)Exploring the Impact of Artificial Intelligence-Generated Content (AIGC) Tools on Social Dynamics in UX CollaborationProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660703(1594-1606)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3660703
Show More Cited By

Index Terms

Sketchplore: Sketch and Explore with a Layout Optimiser
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Sketchplore: Sketch and Explore Layout Designs with an Optimiser
CHI EA '16: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems

Sketchploration is a novel concept of integrating real-time design optimisation to sketching tools, to enable interactive exploration of design alternatives. Although traditional optimisation methods can attack very complex design problems, their ...
DENIM: an informal tool for early stage web site design
CHI EA '01: CHI '01 Extended Abstracts on Human Factors in Computing Systems

DENIM is a system that helps web site designers in the early stages of design. DENIM supports sketching input; allows design at different refinement levels--site map, storyboard, and individual page; and unifies the levels through zooming. The design of ...
On the development of electronic design tools and associated guidelines for supporting the early stages of the design process
DESIRE '11: Procedings of the Second Conference on Creativity and Innovation in Design

According to the user-centred design (UCD) philosophy, users should be involved as much as possible in the different stages of the design process, to optimize the likelihood that design proposals will be appreciated by users. In this paper, first we ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

DIS '16: Proceedings of the 2016 ACM Conference on Designing Interactive Systems

June 2016

1374 pages

ISBN:9781450340311

DOI:10.1145/2901790

General Chair:
Marcus Foth
Queensland University of Technology
,
Program Chairs:
Wendy Ju
Stanford University
,
Ronald Schroeter
Queensland University of Technology
,
Stephen Viller
University of Queensland

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 June 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

European Research Council

Conference

DIS '16

Sponsor:

SIGCHI

DIS '16: Designing Interactive Systems Conference 2016

June 4 - 8, 2016

QLD, Brisbane, Australia

Acceptance Rates

DIS '16 Paper Acceptance Rate 107 of 418 submissions, 26%;

Overall Acceptance Rate 1,158 of 4,684 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

76
Total Citations
View Citations
1,669
Total Downloads

Downloads (Last 12 months)222
Downloads (Last 6 weeks)34

Reflects downloads up to 28 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mo GDudley JChan LLiao YOulasvirta AKristensson P(2024)Cooperative Multi-Objective Bayesian Design OptimizationACM Transactions on Interactive Intelligent Systems10.1145/365764314:2(1-28)Online publication date: 17-Apr-2024
https://dl.acm.org/doi/10.1145/3657643
Takaffoli MLi SMäkelä V(2024)Generative AI in User Experience Design and Research: How Do UX Practitioners, Teams, and Companies Use GenAI in Industry?Proceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660720(1579-1593)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3660720
Wang ZShen LKuang EZhang SFan M(2024)Exploring the Impact of Artificial Intelligence-Generated Content (AIGC) Tools on Social Dynamics in UX CollaborationProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660703(1594-1606)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3660703
Wu JTodi KChan JMyers BLafreniere B(2024)FrameKit: A Tool for Authoring Adaptive UIs Using KeyframesProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645176(660-674)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3640543.3645176
E JYen YPan ILin GLi MJin HChen MXia HDow S(2024)When to Give Feedback: Exploring Tradeoffs in the Timing of Design FeedbackProceedings of the 16th Conference on Creativity & Cognition10.1145/3635636.3656183(292-310)Online publication date: 23-Jun-2024
https://dl.acm.org/doi/10.1145/3635636.3656183
Lu FChen MHsu HDeshpande PWang CMacIntyre B(2024)Adaptive 3D UI Placement in Mixed Reality Using Deep Reinforcement LearningExtended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651059(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3651059
Duan PWarner JLi YHartmann B(2024)Generating Automatic Feedback on UI Mockups with Large Language ModelsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642782(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642782
Qian XWang TXu XJonker TTodi K(2024)Fast-Forward Reality: Authoring Error-Free Context-Aware Policies with Real-Time Unit Tests in Extended RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642158(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642158
Liu ZLiu FZhang M(2024)Intelligent Graphic Layout Generation: Current Status and Future Perspectives2024 27th International Conference on Computer Supported Cooperative Work in Design (CSCWD)10.1109/CSCWD61410.2024.10580676(2632-2637)Online publication date: 8-May-2024
https://doi.org/10.1109/CSCWD61410.2024.10580676
Shi YGao TJiao XCao N(2023)Understanding Design Collaboration Between Designers and Artificial Intelligence: A Systematic Literature ReviewProceedings of the ACM on Human-Computer Interaction10.1145/36102177:CSCW2(1-35)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610217
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents