research-article

Multisensory texture exploration at the tip of the pen

Authors:

S. Delle Monache,

S. PapettiAuthors Info & Claims

International Journal of Human-Computer Studies, Volume 85, Issue C

Pages 47 - 56

https://doi.org/10.1016/j.ijhcs.2015.07.005

Published: 01 January 2016 Publication History

Abstract

A tool for the multisensory stylus-based exploration of virtual textures was used to investigate how different feedback modalities (static or dynamically deformed images, vibration, sound) affect exploratory gestures. To this end, we ran an experiment where participants had to steer a path with the stylus through a curved corridor on the surface of a graphic tablet/display, and we measured steering time, dispersion of trajectories, and applied force. Despite the variety of subjective impressions elicited by the different feedback conditions, we found that only nonvisual feedback induced significant variations in trajectories and an increase in movement time. In a post-experiment, using a paper-and-wood physical realization of the same texture, we recorded a variety of gestural behaviors markedly different from those found with the virtual texture. With the physical setup, movement time was shorter and texture-dependent lateral accelerations could be observed. This work highlights the limits of multisensory pseudo-haptic techniques in the exploration of surface textures. HighlightsSketch-a-Scratch is an abstract interactive object for multisensory texture exploration.Only nonvisual feedback induces significant behavioral variations in a steering task over a texture.Regardless of the richness of feedback, a virtual texture is experienced differently from the real one.

References

[1]

Accot, J., Zhai, S., 1997. Beyond Fitts' law: models for trajectory-based hci tasks. In: Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, 1997, pp. 295-302.

[2]

Accot, J., Zhai, S., 1999. Performance evaluation of input devices in trajectory-based tasks: an application of the steering law. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, 1999, pp. 466-472.

Digital Library

[3]

Ahlström, D., 2005. Modeling and improving selection in cascading pull-down menus using Fitts' law, the steering law and force fields. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, 2005, pp. 61-70.

[4]

T.H. Andersen, S. Zhai, Writing with music, ACM Trans. Appl. Percept., 7 (2008) 17:1-17:24.

Digital Library

[5]

F. Argelaguet, D.A.G. Jáuregui, M. Marchal, A. Lécuyer, Elastic images, ACM Trans. Appl. Percept., 10 (2013) 17:1-17:14.

Digital Library

[6]

F. Avanzini, S. Serafin, D. Rocchesso, Interactive simulation of rigid body interaction with friction-induced sound generation, IEEE Trans. Speech Audio Proc., 13 (2005) 1073-1081.

[7]

Bau, O., Poupyrev, I., Israr, A., Harrison, C., 2010. Teslatouch: electrovibration for touch surfaces. In: Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology, ACM, New York, NY, USA, 2010, pp. 283-292.

[8]

U. Castiello, B.L. Giordano, C. Begliomini, C. Ansuini, M. Grassi, When ears drive hands, PloS ONE, 5 (2010) e12240.

[9]

Conan, S., Thoret, E., Aramaki, M., Derrien, O., Gondre, C., Kronland-Martinet, R., Ystad, S., 2013. Navigating in a space of synthesized interaction-sounds: rubbing, scratching and rolling sounds. In: Proceedings of the 16th International Conference on Digital Audio Effects, Maynooth, Ireland, 2013.

[10]

S. Conan, E. Thoret, M. Aramaki, O. Derrien, C. Gondre, S. Ystad, R. Kronland-Martinet, An intuitive synthesizer of continuous-interaction sounds, Comput. Music J., 38 (2014) 24-37.

Digital Library

[11]

Delle Monache, S., Polotti, P., Rocchesso, D., 2010. A toolkit for explorations in sonic interaction design. In: Proceedings of the 5th Audio Mostly Conference: A Conference on Interaction with Sound, ACM, New York, NY, USA, 2010, pp. 1:1-1:7.

[12]

Delle Monache, S., Rocchesso, D., Papetti, S., 2014. Sketch a scratch. In: Eight International Conference on Tangible, Embedded and Embodied Interaction, Work in progress session, Munich, Germany.

[13]

Hachisu, T., Cirio, G., Marchal, M., Lecuyer, A., Kajimoto, H., 2011. Pseudo-haptic feedback augmented with visual and tactile vibrations. In: 2011 IEEE International Symposium on VR Innovation (ISVRI), 2011, pp. 327-328.

[14]

Harrison, C., Hudson, S., 2012. Using shear as a supplemental two-dimensional input channel for rich touchscreen interaction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, 2012, pp. 3149-3152.

[15]

R.L. Klatzky, S.J. Lederman, Multisensory texture perception, in: Multisensory Object Perception in the Primate Brain, Springer, New York, NY, 2010, pp. 211-230.

[16]

Kulikov, S., MacKenzie, I.S., Stuerzlinger, W., 2005. Measuring the effective parameters of steering motions. In: CHI '05 Extended Abstracts on Human Factors in Computing Systems, ACM, New York, NY, USA, 2005, pp. 1569-1572.

Digital Library

[17]

A. Lécuyer, Simulating haptic feedback using vision, Presence: Teleoper. Virtual Environ., 18 (2009) 39-53.

Digital Library

[18]

Lefebvre, A., Pusch, A., 2012. Object deformation illusion on a tactilely enhanced large tabletop device. In: 2012 IEEE VR Workshop on Perceptual Illusions in Virtual Environments (PIVE), 2012, pp. 27-28.

[19]

G. Lemaitre, O. Houix, P. Susini, Y. Visell, K. Franinovic, Feelings elicited by auditory feedback from a computationally augmented artifact, IEEE Trans. Affect. Comput., 3 (2012) 335-348.

Digital Library

[20]

G. Lemaitre, O. Houix, Y. Visell, K. Franinović, N. Misdariis, P. Susini, Toward the design and evaluation of continuous sound in tangible interfaces, Int. J. Hum.-Comput. Stud., 67 (2009) 976-993.

Digital Library

[21]

I.S. MacKenzie, Human-Computer Interaction: An Empirical Research Perspective, Morgan Kaufmann, Waltham, MA, 2013.

[22]

McDonald, C.G.; Kuchenbecker, K.J., "Dynamic simulation of tool-mediated texture interaction," World Haptics Conference (WHC), 2013, vol., no., pp.307,312, 14-17 April 2013. 10.1109/WHC.2013.6548426

[23]

K.V. Mensvoort, P. Vos, D.J. Hermes, R.V. Liere, Perception of mechanically and optically simulated bumps and holes, ACM Trans. Appl. Percept., 7 (2010) 10:1-10:24.

Digital Library

[24]

Merrill, D., Raffle, H., Aimi, R., 2008. The sound of touch: physical manipulation of digital sound. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, 2008, pp. 739-742.

[25]

S. O'Modhrain, G. Essl, Perceptual integration of audio and touch: a case study of PebbleBox, in: Sonic Interaction Design, MIT Press, Cambridge, MA, 2013, pp. 203-211.

[26]

S. Papetti, F. Avanzini, D. Rocchesso, Numerical methods for a nonlinear impact model, IEEE Trans. Audio, Speech Lang. Proc., 19 (2011) 2146-2158.

Digital Library

[27]

M. Rath, D. Rocchesso, Continuous sonic feedback from a rolling ball, IEEE Multimed., 12 (2005) 60-69.

Digital Library

[28]

G. Robles-De-La-Torre, The importance of the sense of touch in virtual and real environments, IEEE Multimed., 13 (2006) 24-30.

Digital Library

[29]

G. Robles-De-La-Torre, V. Hayward, Force can overcome object geometry in the perception of shape through active touch, Nature, 412 (2001) 445-448.

[30]

D. Rocchesso, P. Polotti, S. Delle Monache, Designing continuous sonic interaction, Int. J. Des., 3 (2009) 13-25.

[31]

Strobl, G., Eckel, G., Rocchesso, D., 2006. Sound texture modeling: a survey. In: Proceedings of the 2006 Sound and Music Computing (SMC) International Conference, 2006, Marseille, France.

[32]

M. Sun, X. Ren, X. Cao, Effects of multimodal error feedback on human performance in steering tasks, J. Inf. Process., 18 (2010) 284-292.

[33]

Sun, M., Ren, X., Zhai, S., Mukai, T., 2012. An investigation of the relationship between texture and human performance in steering tasks. In: Proceedings of the 10th Asia Pacific Conference on Computer Human Interaction, ACM, New York, 2012, NY, USA, pp. 1-6.

Digital Library

[34]

D. Svanæs, Interaction design for and with the lived body, ACM Trans. Comput.-Hum. Interact., 20 (2013) 8:1-8:30.

Digital Library

[35]

Tajadura-Jiménez, A., Liu, B., Bianchi-Berthouze, N., Bevilacqua, F., 2014. Using sound in multi-touch interfaces to change materiality and touch behavior. In: Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational, ACM, New York, NY, USA, 2014, pp. 199-202.

Digital Library

[36]

Yao, H.Y., Hayward, V., 2006. An experiment on length perception with a virtual rolling stone. In: Proceedings of Eurohaptics, 2006, pp. 325-330.

[37]

Zimmerman, J., Forlizzi, J., Evenson, S., 2007. Research through design as a method for interaction design research in hci. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, 2007, pp. 493-502.

Digital Library

Cited By

Bernard CMonnoyer JYstad SWiertlewski M(2022)Eyes-Off Your Fingers: Gradual Surface Haptic Feedback Improves Eyes-Free Touchscreen InteractionProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501872(1-10)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3501872
Cho YBianchi AMarquardt NBianchi-Berthouze NRekimoto JIgarashi TWobbrock JAvrahami D(2016)RealPenProceedings of the 29th Annual Symposium on User Interface Software and Technology10.1145/2984511.2984550(195-205)Online publication date: 16-Oct-2016
https://dl.acm.org/doi/10.1145/2984511.2984550

Multisensory texture exploration at the tip of the pen
1. Human-centered computing

Recommendations

Direct touch haptic display using immersive illusion with interactive virtual finger
EuroHaptics'12: Proceedings of the 2012 international conference on Haptics: perception, devices, mobility, and communication - Volume Part I

In attempt to improve haptic interaction and manipulation technique in surface computers, we propose an interactive haptic feedback display that utilizes ‘direct touch' on a surface computer without using a mediation device. The proposed haptic display ...
Enhancing Pen-based Interaction using Electrovibration and Vibration Haptic Feedback
CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

This paper presents the EV²-Pen which leverages electrovibration technology and vibration technology in pen interaction. Electrovibration technology can produce multisensory feedback when the pen is in motion (sliding/moving on the screen), and ...
Arc-Type and Tilt-Type: Pen-based Immersive Text Input for Room-Scale VR
SUI '20: Proceedings of the 2020 ACM Symposium on Spatial User Interaction

As immersive, room-scale virtual and augmented reality become more utilized in productive workflows, the need for fast, equally-immersive text input grows. Traditional keyboard interaction in these room-scale environments is limiting because of its ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image International Journal of Human-Computer Studies

International Journal of Human-Computer Studies Volume 85, Issue C

January 2016

78 pages

ISSN:1071-5819

Issue’s Table of Contents

Copyright © The Authors.

Publisher

Academic Press, Inc.

United States

Publication History

Published: 01 January 2016

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bernard CMonnoyer JYstad SWiertlewski M(2022)Eyes-Off Your Fingers: Gradual Surface Haptic Feedback Improves Eyes-Free Touchscreen InteractionProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501872(1-10)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3501872
Cho YBianchi AMarquardt NBianchi-Berthouze NRekimoto JIgarashi TWobbrock JAvrahami D(2016)RealPenProceedings of the 29th Annual Symposium on User Interface Software and Technology10.1145/2984511.2984550(195-205)Online publication date: 16-Oct-2016
https://dl.acm.org/doi/10.1145/2984511.2984550

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents