Crossmodal Audio and Tactile Interaction with Mobile Touchscreens
Author: 
Eve HogganAuthors Info & Claims
Pages 29 - 44
Abstract
This article asserts that using crossmodal auditory and tactile interaction can aid mobile touchscreen users in accessing data non-visually and, by providing a choice of modalities, can help to overcome problems that occur in different mobile situations where one modality may be less suitable than another Hoggan, 2010. By encoding data using the crossmodal parameters of audio and vibration, users can learn mappings and translate information between both modalities. In this regard, data may be presented to the most appropriate modality given the situation and surrounding environment.
References
[1]
Blattner, M. M., Sumikawa, D. A.,&Greenberg, R. M. 1989. Earcons and Icons: Their Structure and Common Design Principles. Human-Computer Interaction, 41, 11-44.
[2]
Brewster, S. A. 2002. Overcoming the Lack of Screen Space on Mobile Computers. Personal and Ubiquitous Computing, 63, 188-205.
[3]
Brown, L. M.,&Brewster, S. A. 2006. Multidimensional Tactons for Non-Visual Information Display in Mobile Devices. Paper presented at the MobileHCI '06.
[4]
Cockburn, A.,&Brewster, S. A. 2005. Multimodal feedback for the acquisition of small targets. Ergonomics, 489, 1129-1150. 16251152.
[5]
Enriquez, M.,&MacLean, K. 2008. The Role of Choice in Longitudinal Recall of Meaningful Tactile Signals. Paper presented at the IEEE Symposium on Haptic Interfaces for Virtual Environments and Teleoperator Systems.
[6]
Fastl, H. 1977. Roughness and Temporal Masking Patterns of Sinusoidally Amplitude Modulated Broadband Noise. Psychophysics and Physiology of Hearing, 403-414.
[7]
Fukumoto, M.,&Sugimura, T. 2001. Active Click: Tactile Feedback for Touch Panels. Paper presented at the CHI' 01.
[8]
Gaver, W. 1987. Auditory Icons: Using Sound in Computer Interfaces. ACM SIGCHI Bulletin, 191, 74.
[9]
Gibson, J. J. 1966. The Senses Considered as Perceptual Systems. Boston: Houghton Mifflin.
[10]
Hall, M., Hoggan, E.,&Brewster, S. A. 2008. T-Bars: Towards Tactile User Interfaces for Touchscreen Mobiles. Paper presented at the MobileHCI' 08.
[11]
Hoggan, E. 2010. Crossmodal Audio and Tactile Interaction with Mobile Touchscreens. Glasgow, UK: University of Glasgow.
[12]
Hoggan, E.,&Brewster, S. A. 2006a. Crossmodal Icons for Information Display. Paper presented at the CHI '06 Extended Abstracts.
[13]
Hoggan, E.,&Brewster, S. A. 2006b. Crossmodal Spatial Location: Initial Experiments. Paper presented at the NordiCHI '06, Norway.
[14]
Hoggan, E.,&Brewster, S. A. 2007a. Designing audio and tactile crossmodal icons for mobile devices. Paper presented at ICMI'07.
[15]
Hoggan, E.,&Brewster, S. A. 2007b. New Parameters for Tacton Design. Paper presented at the Vol II CHI 2007, San Jose, CA.
[16]
Hoggan, E.,&Brewster, S. A. 2010. CrossTrainer: Testing the Use of Multimodal Interfaces in Situ. Paper presented at the CHI'10, Atlanta, GA.
[17]
Hoggan, E., Brewster, S. A.,&Johnston, J. 2008. Investigating the Effectiveness of Tactile Feedback for Mobile Touchscreens. Paper presented at the CHI' 08, Florence, Italy.
[18]
Hoggan, E., Crossan, A., Brewster, S. A.,&Kaaresoja, T. 2009. Audio or Tactile Feedback: Which Modality When? Paper presented at the CHI'09, Boston.
[19]
Hsu, W. 2005. Using Timbre in a Computer-Based Improvisation System. Paper presented at the ICMC.
[20]
Jokiniemi, M., Raisamo, R., Lylykangas, J.,&Surakka, V. 2008. Crossmodal Rhythm Perception. Paper presented at the 3rd International Workshop on Haptic and Audio Interaction Design.
[21]
Kaaresoja, T., Brown, L. M.,&Linjama, J. 2006. Snap-Crackle-Pop: Tactile Feedback for Mobile Touch Screens. Paper presented at the Eurohaptics '06.
[22]
Lee, S.,&Zhai, S. 2009. The Performance of Touch Screen Soft Buttons. Paper presented at the CHI'09, Boston.
[23]
Lenay, C., Canu, S.,&Villon, P. 1997. Technology and Perception: The Contribution of Sensory Substitution Systems. Paper presented at the ICCT.
[24]
Lewkowicz, D. J. 2000. The Development of Intersensory Temporal Perception: An Epigenetic Systems/Limitations View. Psychological Bulletin, 126, 281-308. 10748644.
[25]
Marks, L. E. 1978. The Unity of Senses. Interrelations Among The Modalities. New York: Academic Press.
[26]
McGookin, D.,&Brewster, S. A. 2004. Space, The Final Frontearcon: The Identification of Concurrently Presented Earcons in a Synthetic Spatialised Auditory Environment. Paper presented at the ICAD 2004.
[27]
Mendelson, M. J. 1979. Acoustic-Optical Correspondences and Auditory-Visual Coordination in Infancy. Canadian Journal of Experimental Psychology, 33, 334-346. 546498.
[28]
Mereu, S.,&Kazman, R. 1996. Audio Enhanced 3D Interfaces for Visually Impaired Users. Paper presented at the CHI 1996.
[29]
Mortimer, B., Zets, G.,&Cholewiak, R. W. 2007. Vibrotactile Transduction and Transducers. The Journal of the Acoustical Society of America, 121, 2970-2977. 17550194.
[30]
Mursell, J. L. 1937. The Psychology of Music. New York: W.W. Norton and Company, Inc.
[31]
Parncutt, R. 1989. Harmony: A Psychoacoustical Approach. Springer Series in Information Sciences, 19.
[32]
Poupyrev, I.,&Maruyama, S. 2003. Tactile Interfaces for Small Touch Screens. Paper presented at UIST, Canada.
[33]
Reeves, L. M., KLai, J., Larson, J. A., Oviatt, S., Balaji, T. S., Buisine, S., et al. 2004. Guidelines for Multimodal User Interface Design. Communications of the ACM, 471, 57-59.
[34]
Spence, C. 2002. Multisensory attention and tactile information-processing. Behavioural Brain Research, 1351, 57-64. 12356434.
[35]
Terhardt, E. 1974. On the Perception of Periodic Sound Fluctuations Roughness. Acoustica, 30, 201-213.
- Crossmodal Audio and Tactile Interaction with Mobile Touchscreens
Recommendations
Investigating the effectiveness of tactile feedback for mobile touchscreens
CHI '08: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsThis paper presents a study of finger-based text entry for mobile devices with touchscreens. Many devices are now coming to market that have no physical keyboards (the Apple iPhone being a very popular example). Touchscreen keyboards lack any tactile ...
Designing audio and tactile crossmodal icons for mobile devices
ICMI '07: Proceedings of the 9th international conference on Multimodal interfacesThis paper reports an experiment into the design of crossmodal icons which can provide an alternative form of output for mobile devices using audio and tactile modalities to communicate information. A complete set of crossmodal icons was created by ...
Crossmodal congruence: the look, feel and sound of touchscreen widgets
ICMI '08: Proceedings of the 10th international conference on Multimodal interfacesOur research considers the following question: how can visual, audio and tactile feedback be combined in a congruent manner for use with touchscreen graphical widgets? For example, if a touchscreen display presents different styles of visual buttons, ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Publisher
IGI Global
United States
Publication History
Published: 01 October 2010
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 26 Dec 2024