Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3313831.3376628acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

VRSketchIn: Exploring the Design Space of Pen and Tablet Interaction for 3D Sketching in Virtual Reality

Published: 23 April 2020 Publication History

Abstract

Sketching in virtual reality (VR) enhances perception and understanding of 3D volumes, but is currently a challenging task, as spatial input devices (e.g., tracked controllers) do not provide any scaffolding or constraints for mid-air interaction. We present VRSketchIn, a VR sketching application using a 6DoF-tracked pen and a 6DoF-tracked tablet as input devices, combining unconstrained 3D mid-air with constrained 2D surface-based sketching. To explore what possibilities arise from this combination of 2D (pen on tablet) and 3D input (6DoF pen), we present a set of design dimensions and define the design space for 2D and 3D sketching interaction metaphors in VR. We categorize prior art inside our design space and implemented a subset of metaphors for pen and tablet sketching in our prototype. To gain a deeper understanding which specific sketching operations users perform with 2D and which with 3D metaphors, we present findings of usability walkthroughs with six participants.

Supplementary Material

SRT File (paper501pvc.srt)
Preview video captions
ZIP File (pn7038vfc.zip)
Video figure captions
MP4 File (paper501pv.mp4)
Preview video
MP4 File (pn7038vf.mp4)
Supplemental video
MP4 File (a501-drey-presentation.mp4)

References

[1]
Adobe Inc. 2019. Adobe Creative Cloud. (Aug 2019). https://www.adobe.com/creativecloud.html (Accessed on 08/17/2019).
[2]
Alphabet Inc. 2019. Tilt Brush by Google. (Aug 2019). https://www.tiltbrush.com/ (Accessed on 08/12/2019).
[3]
Rahul Arora, Rubaiat Habib Kazi, Tovi Grossman, George Fitzmaurice, and Karan Singh. 2018. SymbiosisSketch: Combining 2D & 3D Sketching for Designing Detailed 3D Objects in Situ. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, Article 185, 15 pages.
[4]
Rahul Arora, Rubaiat Habib Kazi, Fraser Anderson, Tovi Grossman, Karan Singh, and George Fitzmaurice. 2017. Experimental Evaluation of Sketching on Surfaces in VR. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, NY, NY, USA, 5643--5654.
[5]
Santiago Arroyave-Tobón, Gilberto Osorio-Gómez, and Juan F. Cardona-McCormick. 2015. AIR-MODELLING: A tool for gesture-based solid modelling in context during early design stages in AR environments. Computers in Industry 66 (2015), 73 -- 81.
[6]
Autodesk Corporation. 2019. 3ds Max | 3D Modeling, Animation & Rendering Software | Autodesk. (Aug 2019). https://www.autodesk.com/products/3ds-max/overview (Accessed on 08/17/2019).
[7]
Aaron Bangor, Philip Kortum, and James Miller. 2009. Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale. J. Usability Studies 4, 3 (May 2009), 114--123. http://dl.acm.org/citation.cfm?id=2835587.2835589
[8]
H. Benko, E. W. Ishak, and S. Feiner. 2004. Collaborative mixed reality visualization of an archaeological excavation. In Third IEEE and ACM International Symposium on Mixed and Augmented Reality. 132--140.
[9]
Mark Billinghurst, Sisinio Baldis, Lydia Matheson, and Mark Philips. 1997. 3D palette: a virtual reality content creation tool. In Proceedings of the ACM symposium on Virtual reality software and technology. ACM, 155--156.
[10]
D. A. Bowman, J. Wineman, L. F. Hodges, and D. Allison. 1998. Designing animal habitats within an immersive VE. IEEE Computer Graphics and Applications 18, 5 (Sep. 1998), 9--13.
[11]
D. A. Bowman and C. A. Wingrave. 2001. Design and evaluation of menu systems for immersive virtual environments. In Proceedings IEEE Virtual Reality 2001. 149--156.
[12]
John Brooke. 1996. SUS-A Quick and Dirty Usability Scale. Usability evaluation in industry 189, 194 (1996).
[13]
Bruno R. De Araùjo, Géry Casiez, and Joaquim A. Jorge. 2012. Mockup Builder: Direct 3D Modeling on and Above the Surface in a Continuous Interaction Space. In Proceedings of Graphics Interface 2012 (GI '12). Canadian Information Processing Society, Toronto, Ont., Canada, Canada, 173--180. http://dl.acm.org/citation.cfm?id=2305276.2305305
[14]
Michael F. Deering. 1995. HoloSketch: A Virtual Reality Sketching/Animation Tool. ACM Trans. Comput.-Hum. Interact. 2, 3 (Sept. 1995), 220--238.
[15]
Facebook Inc. 2019. Quill - VR illustration and animation tool built to empower artists and creators. (Sep 2019). https://quill.fb.com/ (Accessed on 09/14/2019).
[16]
J. D. Foley, V. L. Wallace, and P. Chan. 1984. The human factors of computer graphics interaction techniques. IEEE Computer Graphics and Applications 4, 11 (Nov 1984), 13--48.
[17]
Danilo Gasques, Janet G. Johnson, Tommy Sharkey, and Nadir Weibel. 2019a. PintAR: Sketching Spatial Experiences in Augmented Reality. In Companion Publication of the 2019 on Designing Interactive Systems Conference 2019 Companion (DIS '19 Companion). ACM, NY, NY, USA, 17--20.
[18]
Danilo Gasques, Janet G. Johnson, Tommy Sharkey, and Nadir Weibel. 2019b. What You Sketch Is What You Get: Quick and Easy Augmented Reality Prototyping with PintAR. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19). ACM, NY, NY, USA, Article LBW1416, 6 pages.
[19]
Gravity Sketch. 2019. Gravity Sketch. (Sep 2019). https://www.gravitysketch.com/ (Accessed on 09/14/2019).
[20]
hecomi. 2019. GitHub - hecomi/uWintab: Wintab API plugin for Unity. (Sep 2019). https://github.com/hecomi/uWintab (Accessed on 09/18/2019).
[21]
Rorik Henrikson, Bruno De Araujo, Fanny Chevalier, Karan Singh, and Ravin Balakrishnan. 2016. Storeoboard: Sketching Stereoscopic Storyboards. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, NY, NY, USA, 4587--4598.
[22]
Ken Hinckley, Michel Pahud, Hrvoje Benko, Pourang Irani, François Guimbretière, Marcel Gavriliu, Xiang 'Anthony' Chen, Fabrice Matulic, William Buxton, and Andrew Wilson. 2014. Sensing Techniques for Tablet+Stylus Interaction. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST '14). ACM, NY, NY, USA, 605--614.
[23]
Ken Hinckley and Daniel Wigdor. 2012. Input Technologies and Techniques (2012). CRC Press, Chapter 6, 95--132. https://www.microsoft.com/en-us/ research/publication/input-technologies-techniques/
[24]
Ken Hinckley, Koji Yatani, Michel Pahud, Nicole Coddington, Jenny Rodenhouse, Andy Wilson, Hrvoje Benko, and Bill Buxton. 2010. Pen + Touch = New Tools. In Proceedings of the 23Nd Annual ACM Symposium on User Interface Software and Technology (UIST '10). ACM, NY, NY, USA, 27--36.
[25]
Teresa Hirzle, Jan Gugenheimer, Florian Geiselhart, Andreas Bulling, and Enrico Rukzio. 2019. A Design Space for Gaze Interaction on Head-mounted Displays. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, NY, NY, USA, Article 625, 12 pages.
[26]
Icons8 LLC. 2019. Download free icons, music, stock photos, vectors. (Sep 2019). https://icons8.com/ (Accessed on 09/18/2019).
[27]
P. Issartel, F. Guéniat, and M. Ammi. 2014. Slicing techniques for handheld augmented reality. In 2014 IEEE Symposium on 3D User Interfaces (3DUI). 39--42.
[28]
D. Keefe, R. Zeleznik, and D. Laidlaw. 2007. Drawing on Air: Input Techniques for Controlled 3D Line Illustration. IEEE Transactions on Visualization and Computer Graphics 13, 5 (Sep. 2007), 1067--1081.
[29]
Daniel F. Keefe, Daniel Acevedo Feliz, Tomer Moscovich, David H. Laidlaw, and Joseph J. LaViola, Jr. 2001. CavePainting: A Fully Immersive 3D Artistic Medium and Interactive Experience. In Proceedings of the 2001 Symposium on Interactive 3D Graphics (I3D '01). ACM, NY, NY, USA, 85--93.
[30]
Yongkwan Kim, Sang-Gyun An, Joon Hyub Lee, and Seok-Hyung Bae. 2018. Agile 3D Sketching with Air Scaffolding. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, Article 238, 12 pages.
[31]
Kin Chung Kwan and Hongbo Fu. 2019. Mobi3DSketch: 3D Sketching in Mobile AR. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, NY, NY, USA, Article 176, 11 pages.
[32]
Paul Lapides, Ehud Sharlin, Mario Costa Sousa, and Lisa Streit. 2006. The 3D Tractus: A Three-Dimensional Drawing Board. In Proceedings of the First IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP '06). IEEE Computer Society, Washington, DC, USA, 169--176.
[33]
Robert W. Lindeman, John L. Sibert, and James K. Hahn. 1999. Towards Usable VR: An Empirical Study of User Interfaces for Immersive Virtual Environments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '99). ACM, NY, NY, USA, 64--71.
[34]
R. W. Lindeman, J. L. Sibert, and J. N. Templeman. 2001. The effect of 3D widget representation and simulated surface constraints on interaction in virtual environments. In Proceedings IEEE Virtual Reality 2001. 141--148.
[35]
Alfredo Liverani, Alessandro Ceruti, and Gianni Caligiana. 2013. Tablet-based 3D sketching and curve reverse modelling. International Journal of Computer Aided Engineering and Technology 8 5, 2--3 (2013), 188--215.
[36]
Logitech. 2019. Logitech R400 Wireless Presentation Remote with Laser Pointer & 15m Range. (Sep 2019). https://www.logitech.com/en-us/product/ wireless-presenter-r400?crid=11 (Accessed on 09/18/2019).
[37]
Mayra D. Barrera Machuca, Paul Asente, Wolfgang Stuerzlinger, Jingwan Lu, and Byungmoon Kim. 2018. Multiplanes: Assisted Freehand VR Sketching. In Proceedings of the Symposium on Spatial User Interaction (SUI '18). ACM, NY, NY, USA, 36--47.
[38]
Maxon Computer GmbH. 2019. Welcome - MAXON | 3D FOR THE REAL WORLD. (Aug 2019). https://www.maxon.net/en/ (Accessed on 08/17/2019).
[39]
A. Millette and M. J. McGuffin. 2016. DualCAD: Integrating Augmented Reality with a Desktop GUI and Smartphone Interaction. In 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct). 21--26.
[40]
Jörg Müller, Florian Alt, Daniel Michelis, and Albrecht Schmidt. 2010. Requirements and Design Space for Interactive Public Displays. In Proceedings of the 18th ACM International Conference on Multimedia (MM '10). ACM, NY, NY, USA, 1285--1294.
[41]
Andrew Nakas. 2019. GitHub andrewnakas/OpenBrushVR: Unity VR & AR Painting in 3d space for the Vive and Tango and ARKit. (Sep 2019). https://github.com/andrewnakas/OpenBrushVR (Accessed on 09/18/2019).
[42]
NaturalPoint, Inc. 2019. OptiTrack - Motion Capture Systems. (Sep 2019). https://optitrack.com/ (Accessed on 09/18/2019).
[43]
Daniel Paes, Eduardo Arantes, and Javier Irizarry. 2017. Immersive environment for improving the understanding of architectural 3D models: Comparing user spatial perception between immersive and traditional virtual reality systems. Automation in Construction 84 (2017), 292 -- 303.
[44]
Pixologic Inc. 2019. Pixologic : ZBrush - The all-in-one-digital sculpting solution. (Aug 2019). http://pixologic.com/ (Accessed on 08/17/2019).
[45]
E. Sachs, A. Roberts, and D. Stoops. 1991. 3-Draw: A Tool for Designing 3D Shapes. IEEE Computer Graphics and Applications 11, 06 (nov 1991), 18--26.
[46]
Steven Schkolne, Michael Pruett, and Peter Schröder. 2001. Surface Drawing: Creating Organic 3D Shapes with the Hand and Tangible Tools. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '01). ACM, NY, NY, USA, 261--268.
[47]
Dieter Schmalstieg, L Miguel Encarnação, and Zsolt Szalavári. 1999. Using transparent props for interaction with the virtual table. SI3D 99 (1999), 147--153.
[48]
Dieter Schmalstieg, Anton Fuhrmann, Gerd Hesina, Zsolt Szalavári, L. Miguel Encarnação, Michael Gervautz, and Werner Purgathofer. 2002. The Studierstube Augmented Reality Project. Presence: Teleoperators and Virtual Environments 11, 1 (2002), 33--54.
[49]
Martina Schütze, Pierre Sachse, and Anne Römer. 2003. Support value of sketching in the design process. Research in Engineering Design 14, 2 (2003), 89--97.
[50]
Richard Stoakley, Matthew J. Conway, and Randy Pausch. 1995. Virtual Reality on a WIM: Interactive Worlds in Miniature. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '95). ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 265--272.
[51]
Hemant Bhaskar Surale, Aakar Gupta, Mark Hancock, and Daniel Vogel. 2019. TabletInVR: Exploring the Design Space for Using a Multi-Touch Tablet in Virtual Reality. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, NY, NY, USA, Article 13, 13 pages.
[52]
Z. Szalavári, D. Schmalstieg, A. Fuhrmann, and M. Gervautz. 1998. "Studierstube": An environment for collaboration in augmented reality. Virtual Reality 3, 1 (01 Mar 1998), 37--48.
[53]
Zsolt Szalavári and Michael Gervautz. 1997. The Personal Interaction Panel -- a Two-Handed Interface for Augmented Reality. Computer Graphics Forum 16, 3 (1997), C335--C346.
[54]
Elkin Taborda, Senthil K Chandrasegaran, Lorraine Kisselburgh, Tahira Reid, and Karthik Ramani. 2013. Enhancing visual thinking in a toy design course using freehand sketching. In ASME 2012 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. American Society of Mechanical Engineers Digital Collection, 267--276.
[55]
Unity Technologies. 2019. Unity Real-Time Development Platform | 3D, 2D VR & AR Visualizations. (Sep 2019). https://unity.com/ (Accessed on 09/19/2019).
[56]
John Viega, Matthew J Conway, George Williams, and Randy Pausch. 1996. 3D magic lenses. In Proceedings of the 9th annual ACM symposium on User interface software and technology. Citeseer, 51--58.
[57]
Wacom. 2019. Intuos Draw | Wacom. (Sep 2019). https://www.wacom.com/en/products/pen-tablets/intuos-draw (Accessed on 09/18/2019).
[58]
Jia Wang and Robert Lindeman. 2014. Coordinated 3D Interaction in Tablet- and HMD-based Hybrid Virtual Environments. In Proceedings of the 2Nd ACM Symposium on Spatial User Interaction (SUI '14). ACM, NY, NY, USA, 70--79.
[59]
Kent Watsen, Rudolph Darken, and Michael Capps. 1999. A handheld computer as an interaction device to a virtual environment. In Proceedings of the third immersive projection technology workshop.
[60]
Gerold Wesche and Hans-Peter Seidel. 2001. FreeDrawer: A Free-form Sketching System on the Responsive Workbench. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology (VRST '01). ACM, NY, NY, USA, 167--174.
[61]
Jacob O. Wobbrock and Julie A. Kientz. 2016. Research Contributions in Human-computer Interaction. Interactions 23, 3 (April 2016), 38--44.
[62]
Min Xin, Ehud Sharlin, and Mario Costa Sousa. 2008. Napkin Sketch: Handheld Mixed Reality 3D Sketching. In Proceedings of the 2008 ACM Symposium on Virtual Reality Software and Technology (VRST '08). ACM, NY, NY, USA, 223--226.
[63]
Brandon Yee, Yuan Ning, and Hod Lipson. 2009. Augmented reality in-situ 3D sketching of physical objects. In Intelligent UI workshop on sketch recognition, Vol. 1. Citeseer.
[64]
F. Zwicky. 1967. The Morphological Approach to Discovery, Invention, Research and Construction. In New Methods of Thought and Procedure, Fritz Zwicky and Albert G. Wilson (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 273--297.

Cited By

View all
  • (2024)VRPaperCrafting: Exploring Child-Friendly Paper Crafting in Virtual Environments using Physical Proxies and Hand GesturesProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685379(1-13)Online publication date: 13-Oct-2024
  • (2024)PhoneCanvas: 3D Sketching System Using a Depth Camera-Equipped Smartphone as a CanvasProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682078(1-12)Online publication date: 7-Oct-2024
  • (2024)“I Try to Represent Myself as I Am”: Self-Presentation Preferences of People with Invisible Disabilities through Embodied Social VR AvatarsProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675620(1-15)Online publication date: 27-Oct-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
April 2020
10688 pages
ISBN:9781450367080
DOI:10.1145/3313831
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 April 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. design space
  2. interaction metaphors
  3. mid-air painting
  4. pen and tablet
  5. sketching
  6. virtual reality

Qualifiers

  • Research-article

Conference

CHI '20
Sponsor:

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI '25
CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)469
  • Downloads (Last 6 weeks)51
Reflects downloads up to 27 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2024)VRPaperCrafting: Exploring Child-Friendly Paper Crafting in Virtual Environments using Physical Proxies and Hand GesturesProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685379(1-13)Online publication date: 13-Oct-2024
  • (2024)PhoneCanvas: 3D Sketching System Using a Depth Camera-Equipped Smartphone as a CanvasProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682078(1-12)Online publication date: 7-Oct-2024
  • (2024)“I Try to Represent Myself as I Am”: Self-Presentation Preferences of People with Invisible Disabilities through Embodied Social VR AvatarsProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675620(1-15)Online publication date: 27-Oct-2024
  • (2024)3D-Layers: Bringing Layer-Based Color Editing to VR PaintingACM Transactions on Graphics10.1145/365818343:4(1-15)Online publication date: 19-Jul-2024
  • (2024)OptiBasePen: Mobile Base+Pen Input on Passive Surfaces by Sensing Relative Base Motion Plus Close-Range Pen PositionProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676467(1-9)Online publication date: 13-Oct-2024
  • (2024)Hands-on, Hands-off: Gaze-Assisted Bimanual 3D InteractionProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676331(1-12)Online publication date: 13-Oct-2024
  • (2024)Enhancing VR Sketching with a Dynamic Shape DisplayProceedings of the 30th ACM Symposium on Virtual Reality Software and Technology10.1145/3641825.3687714(1-11)Online publication date: 9-Oct-2024
  • (2024)RobotSketch: An Interactive Showcase of Superfast Design of Legged RobotsACM SIGGRAPH 2024 Emerging Technologies10.1145/3641517.3664382(1-2)Online publication date: 13-Jul-2024
  • (2024)Introducing AV-Sketch: An Immersive Participatory Design Tool for Automated Vehicle — Passenger InteractionProceedings of the 16th International Conference on Automotive User Interfaces and Interactive Vehicular Applications10.1145/3640792.3675705(83-93)Online publication date: 22-Sep-2024
  • (2024)ThermalPen: Investigating the Influence of Thermal Haptic Feedback for Creativity in 3D SketchingProceedings of the 16th Conference on Creativity & Cognition10.1145/3635636.3656191(661-673)Online publication date: 23-Jun-2024
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media