Abstract
Past research has demonstrated that speech communication on the skin is entirely achievable. However, there is still no definitive conclusion on the best training method that minimizes the time it takes for users to reach a prescribed performance level with a speech communication device. The present study reports the design and testing of two learning approaches with a system that translates English phonemes to haptic stimulation patterns (haptic symbols). With the phoneme-based learning approach, users learned the haptic symbols associated with the phonemes before attempting to acquire words made up of the phonemes. With the word-based approach, users learned words on day one. Two experiments were conducted with the two learning approaches, each employing twelve participants who spent 100 min each learning 100 English words made up of 39 phonemes. Results in terms of the total number of words learned show that performance levels vary greatly among the individuals tested (with the best learners in both methods achieving word-recognition scores > 90%-correct on a 100-word vocabulary), both approaches are feasible for successful acquisition of word through the skin, and the phoneme-based approach provides a more consistent path for learning across users in a shorter period of time.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Debriefing with participants after the experiments revealed that even though 8 of the 12 participants with the word-based approach noticed on Day 1 the repeating haptic symbols associated with the phonemes making up the words, only 2 of them were able to “decode” the phonemes successfully. These two top performers then focused on the learning of new phonemes on subsequent learning days and contrasted them with the old phonemes learned on previous days. The other participants appeared to be less efficient at learning the haptic symbols for phonemes. As more phonemes and words were added to the task, learning became even more challenging since confusions with similar haptic symbols remained unresolved.
References
Reed, C.M., Rabinowitz, W.M., Durlach, N.I., Braida, L.D., Conway-Fithian, S., Schultz, M.C.: Research on the Tadoma method of speech communication. J. Acoust. Soc. Am. 77(1), 247–257 (1985)
Reed, C.M., Durlach, N.I., Braida, L.D.: Research on tactile communication of speech: a review. American Speech-Language-Hearing Association Monographs, no. 20 (1982)
Reed, C.M., Durlach, N.I., Delhorne, L.A., Rabinowitz, W.M.: Research on tactual communication of speech: ideas, issues, and findings. Volta Rev. 91, 65–78 (1989)
Tan, H.Z., Pentland, A.: Tactual displays for sensory substitution and wearable computers. In: Barfield, W., Caudell, T. (eds.) Fundamentals of Wearable Computers and Augmented Reality, pp. 579–598. Lawrence Erlbaum Associates, Mahwah (2001)
Plant, G.: The selection and training of tactile aid users. In: Summers, I.R. (ed.) Tactile Aids for the Hearing Impaired, pp. 146–166. Whurr Publishers, London (1992)
Novich, S.D., Eagleman, D.M.: Using space and time to encode vibrotactile information: toward an estimate of the skin’s achievable throughput. Exp. Brain Res. 233(10), 2777–2788 (2015)
Novich, S.D.: Sound-to-Touch Sensory Substitution and Beyond. Doctoral Dissertation, Department of Electrical and Computer Engineering, Rice University (2015)
Turcott, R., Chen, J., Castillo, P., Knott, B., Setiawan, W., Briggs, F., Klumb, K., Abnousi, F., Chakka, P.: Efficient evaluation of coding strategies for transcutaneous language communication. In: Proceedings of EuroHaptics 2018, Pisa, Italy, 13–16 June 2018 (2018, to appear)
Luzhnica, G., Veas, E., Pammer, V.: Skin reading: encoding text in a 6-channel haptic display. In: Proceedings of the 2016 ACM International Symposium on Wearable Computers (ISWC 2016), pp. 148–155 (2016)
Reed, C.M., Durlach, N.I.: Note on information transfer rates in human communication. Presence Teleoperators Virtual Environ. 7(5), 509–518 (1998)
Reed, C.M., Tan, H.Z., Perez, Z.D., Wilson, E.C., Severgnini, F.M., Jung, J., Martinez, J.S., Jiao, Y., Israr, A., Lau, F., Klumb, K., Abnousi, F.: A phonemic-based tactual display for speech communication. IEEE Trans. Haptics (submitted)
Jung, J., Jiao, Y., Severgnini, F.M., Tan, H.Z., Reed, C.M., Israr, A., Lau, F., Abnousi, F.: Speech communication through the skin: design of learning protocols and initial findings. In: Proceedings of HCI International, 15–20 July 2018 (2018, to appear)
Hall, A.D., Newman, S.E.: Braille learning: relative importance of seven variables. Appl. Cognit. Psychol. 1(2), 133–141 (1987)
Herzog, M.H., Fahle, M.: The role of feedback in learning a vernier discrimination task. Vis. Res. 37(15), 2133–2141 (1997)
Henshaw, H., Ferguson, M.A.: Efficacy of individual computer-based auditory training for people with hearing loss: a systematic review of the evidence. PLoS ONE 8(5), e62836 (2013)
Nitchie, E.: Lipreading, an art. Volta Rev. 15, 276–278 (1913)
Walden, B.E., Erdman, S.A., Montgomery, A.A., Schwartz, D.M., Prosek, R.A.: Some effects of training on speech recognition by hearing-impaired adults. J. Speech Hear. Res. 24(2), 207–216 (1981)
Rubinstein, A., Boothroyd, A.: Effect of two approaches to auditory training on speech recognition by hearing-impaired adults. J. Speech Lang. Hear. Res. 30(2), 153–160 (1987)
Ecroyd, D.H.: Voice and Articulation: A Handbook, pp. 63–87. Scott Foresman & Co Publisher, Glenview (1966)
Jones, L.A., Tan, H.Z.: Application of psychophysical techniques to haptic research. IEEE Trans. Haptics 6(3), 268–284 (2013)
Dudai, Y., Karni, A., Born, J.: The consolidation and transformation of memory. Neuron 88(1), 20–32 (2015)
Zhao, S., Israr, A., Lau, F., Abnousi, F.: Coding tactile symbols for phonemic communication. In: Proceedings of CHI 2018, 21–26 April 2018 (2018, to appear)
Acknowledgments
This work was partially supported by a research grant from Facebook Inc. The authors thank Emily Fredette for her assistance with data collection.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Jiao, Y. et al. (2018). A Comparative Study of Phoneme- and Word-Based Learning of English Words Presented to the Skin. In: Prattichizzo, D., Shinoda, H., Tan, H., Ruffaldi, E., Frisoli, A. (eds) Haptics: Science, Technology, and Applications. EuroHaptics 2018. Lecture Notes in Computer Science(), vol 10894. Springer, Cham. https://doi.org/10.1007/978-3-319-93399-3_53
Download citation
DOI: https://doi.org/10.1007/978-3-319-93399-3_53
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-93398-6
Online ISBN: 978-3-319-93399-3
eBook Packages: Computer ScienceComputer Science (R0)