Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

BendableSound

Published: 01 November 2017 Publication History

Abstract

BendableSound is an elastic multisensory surface to assist music therapy sessions.BendableSound has the potential to provide a more natural and intuitive multisensory experience.Elastic multisensory surfaces promote the discovery of new tactile experiences.BendableSound helps children with autism to maintain their attention during therapy.BendableSound could have therapeutic benefits regarding motor development. Neurological Music Therapy uses live music to improve the sensorimotor regulation of children with severe autism. However, they often lack musical training and their impairments limit their interactions with musical instruments. In this paper, we present our co-design work that led to the BendableSound prototype: an elastic multisensory surface encouraging users to practice coordination movements when touching a fabric to play sounds. We present the results of a formative study conducted with 18 teachers showing BendableSound was perceived as usable and attractive. Then, we present a deployment study with 24 children with severe autism showing BendableSound is easy to use and may potentially have therapeutic benefits regarding attention and motor development. We propose a set of design insights that could guide the design of natural user interfaces, particularly elastic multisensory surfaces. We close with a discussion and directions for future work.

References

[1]
American Psychiatric Association, 2013. Diagnostic and Statistical Manual of Mental Disorders (DSM-5®). American Psychiatric Pub
[2]
Bangor, A., Kortum, P.T., Miller, J.T., 2008. An empirical evaluation of the system usability scale. Int. J. Hum. Comput. Interact. 24, 574-594.
[3]
Bellan, V., Wallwork, S.B., Hons, B.P., Gallace, A., Spence, C., Moseley, G.L., 2017. Integrating self-localization, proprioception, pain, and performance. J. Danc. Med. Sci. 21, 24-35.
[4]
Berger, D.S., 2002. Music Theapy, Sensory Integration and Autistic Child. Jessica Kingsley Publishers, London and New York
[5]
Boulay, M., Benveniste, S., Boespflug, S., Jouvelot, P., Rigaud, A.S., 2011. A pilot usability study of MINWii, a music therapy game for demented patients. Technol. Heal. Care 19, 233-246.
[6]
Brakke, K., Fragaszy, D.M., Simpson, K., Hoy, E., Cummins-Sebree, S., 2007. The production of bimanual percussion in 12- to 24-month-old children. Infant Behav. Dev. 30, 2-15.
[7]
Branje, C., Fels, D.I., 2014. Playing vibrotactile music: a comparison between the Vibrochord and a piano keyboard. Int. J. Hum. Comput. Stud. 72, 431-439.
[8]
Brooke, J., 2013. SUS : a retrospective. J. Usability Stud. 8, 29-40.
[9]
Büitefisch, C., Hummelsheim, H., Denzler, P., Mauritz, K.H., 1995. Repetitive training of isolated movements improves the outcome of motor rehabilitation of the centrally paretic hand. J. Neurol. Sci. 130, 59-68.
[10]
Burland, K., Magee, W., 2012. Developing identities using music technology in therapeutic settings. Psychol. Music 42, 177-189.
[11]
Cassinelli, A., Ishikawa, M., 2005. Khronos projector. ACM SIGGRAPH 2005 Emerg. Technol. - SIGGRAPH '05 10
[12]
Cesarini, D., Calvaresi, D., Farnesi, C., Taddei, D., Frediani, S., Ungerechts, B.E., Hermann, T., 2016. MEDIATION: an eMbEddeD system for auditory feedback of handwater InterAcTION while swimming. Procedia Eng. 147, 324-329.
[13]
Charmaz, K., 2006. Constructing Grounded Theory: A Practical Guide to Through Qualitative Analysis. Sage Publications Ltd.
[14]
Cibrian, F.L., Pena, O., Vazquez, V., Cardenas, C., Tentori, M., 2016a. Designing a deformable musical surface for children with autism. In: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing Adjunct - UbiComp '16, pp. 977-982.
[15]
Cibrian, F.L., Weibel, N., Tentori, M., 2016b. Collective use of a fabric-based interactive surface to support early development in toddler classrooms. In: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing - UbiComp '16, pp. 328-339.
[16]
Corbin, J., Strauss, A.L., 1998. Basics of qualitative research: Techniques and procedures for developing grounded theory. SAGE. ed.
[17]
Cuvo, A.J., May, M.E., Post, T.M., 2001. Effects of living room, Snoezelen room, and outdoor activities on stereotypic behavior and engagement by adults with profound mental retardation. Res. Dev. Disabil. 22, 183-204.
[18]
Darrow, A.A., 2009. Adapting for students with autism. Gen. Music Today 22, 24-26.
[19]
Fagard, J., Wolff, P.H., 1991. The Development of Timing Control and Temporal Organization in Coordinated Action: Invariant Relative Timing, Rhythms and Coordination. Elsevier
[20]
Follmer, S., Leithinger, D., Olwal, A., Cheng, N., Ishii, H., 2012. Jamming user interfaces: programmable particle stiffness and sensing for malleable and shape-changing devices. In: Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology - UIST '12, pp. 519-528.
[21]
Fournier, K.A., Hass, C.J., Naik, S.K., Lodha, N., Cauraugh, J.H., 2010. Motor coordination in autism spectrum disorders: a synthesis and meta-analysis. J. Autism Dev. Disord. 40, 1227-1240.
[22]
Giordano, M., Wanderley, M.M., 2013. Perceptual and technological issues in the design of vibrotactile-augmented interfaces for music technology and media. In: Haptic and Audio Interaction Design, pp. 89-98.
[23]
Gorman, M., Lahav, A., Saltzman, E., Betke, M., 2007. A camera-based music-making tool for physical rehabilitation. Comput. Music J. 31, 39-53.
[24]
Green, D., Charman, T., Pickles, A., Chandler, S., Loucas, T., Simonoff, E., Baird, G., 2009. Impairment in movement skills of children with autistic spectrum disorders. Dev. Med. Child Neurol. 51, 311-316.
[25]
Grierson, M., Kiefer, C., 2013. NoiseBear: a malleable wireless controller designed in participation with disabled children. In: Proc. Int. Conf. New Interfaces Music. Expr., pp. 413-416.
[26]
Grosshauser, T., Bläsing, B., Spieth, C., Hermann, T., 2012. Wearable sensor-based real-time sonification of motion and foot pressure in dance teaching and training. J. Audio Eng. Soc. 60, 580-589.
[27]
Grunder, T., Kammer, D., Brade, M., Groh, R., 2013. Towards a design space for elastic displays. In: CHI '13 Extended Abstracts on Human Factors in Computing Systems on - CHI EA '13, p. 4.
[28]
Hardy, M.W., Lagasse, aB., 2013. Rhythm, movement, and autism: using rhythmic rehabilitation research as a model for autism. Front. Integr. Neurosci. 7, 1-9.
[29]
Hayes, G.R., Hirano, S., Marcu, G., Monibi, M., Nguyen, D.H., Yeganyan, M., 2010. Interactive visual supports for children with autism. Pers. Ubiquitous Comput. 14, 663-680.
[30]
Hobbs, D., Worthington-Eyre, B., 2008. The efficacy of combining augmented reality and music therapy with traditional teaching - preliminary results. i-CREATe 2008 - Int. Conv. Rehabil. Eng. Assist. Technol. 2008, 241-244.
[31]
Hopkins, I.M., Gower, M.W., Perez, T.A., Smith, D.S., Amthor, F.R., Casey Wimsatt, F., Biasini, F.J., 2011. Avatar assistant: Improving social skills in students with an ASD through a computer-based intervention. J. Autism Dev. Disord. 41, 1543-1555.
[32]
Hourcade, J.P., Bullock-Rest, N.E., Hansen, T.E., 2012. Multitouch tablet applications and activities to enhance the social skills of children with autism spectrum disorders. Pers. Ubiquitous Comput. 16, 157-168.
[33]
Ichino, J., Pon, A., Sharlin, E., Eagle, D., Carpendale, S., 2014. Vuzik: the effect of large gesture interaction on children's creative musical expression. In: Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: The Future of Design. ACM, pp. 240-249.
[34]
Jordà, S., Geiger, G., Alonso, M., Kaltenbrunner, M., 2007. The reacTable: exploring the synergy between live music performance and tabletop tangible interfaces. In: Proceedings of the 1st International Conference on Tangible and Embedded Interaction. New York, NY, USA. ACM, pp. 15-17.
[35]
Keele, S.W., Ivry, R.I., Pokorny, R.a, 1987. Force control and its relation to timing. J. Mot. Behav. 19, 96-114.
[36]
Kientz, J.A., Goodwin, M.S., Hayes, G.R., Abowd, G.D., 2013. Interactive Technologies for Autism, Synthesis Lectures on Assistive, Rehabilitative, and Health-Preserving Technologies. Morgan & Claypool Publishers.
[37]
LaGasse, A.B., Hardy, M.W., 2013. Considering rhythm for sensorimotor regulation in children with autism spectrum disorders. Music Ther. Perspect. 31, 67-77.
[38]
Laugwitz, B., Held, T., Schrepp, M., 2008. Construction and evaluation of a user experience questionnaire. In: HCI and Usability for Education and Work, pp. 63-76.
[39]
Longo, M.R., Haggard, P., 2009. Sense of agency primes manual motor responses. Perception 38, 69-78.
[40]
Lundy-Ekman, L., Ivry, R., Keele, S., Woollacott, M., 1991. Timing and force control deficits in clumsy children. J. Cogn. Neurosci. 3, 367-376.
[41]
Magee, W.L., 2006. Electronic technologies in clinical music therapy : a survey of practice and attitudes. Technol. Disabil. 18, 139-146.
[42]
Malizia, A., Bellucci, A., 2012. The artificiality of natural user interfaces. Commun. ACM 55, 36.
[43]
Meldrum, D., Glennon, A., Herdman, S., Murray, D., McConn-Walsh, R., 2012. Virtual reality rehabilitation of balance: assessment of the usability of the Nintendo Wii(®) Fit Plus. Disabil. Rehabil. Assist. Technol. 7, 205-210.
[44]
Mertel, K., 2014. Therapeutical instrumental music performance (TIMP). In: Handbook of Neurologic Music Therapy. Oxford University Press, pp. 116-139.
[45]
Mintzberg, H., 1970. Structured observation as a method to study managerial work. J. Manag. Stud. 7, 87-104.
[46]
Moore, J.W., Fletcher, P.C., 2012. Sense of agency in health and disease: a review of cue integration approaches. Conscious. Cogn. 21, 59-68.
[47]
Müller, M., Gründer, T., Groh, R., 2015. Data exploration on elastic displays using physical metaphors. In: xCoAx 2015: Proceedings of the Third Conferenc on Computation, Communication, Aesthetics and X, pp. 111-124.
[48]
Nawaz, A., Helbostad, J.L., Chiari, L., Chesani, F., Cattelani, L., 2015. User Experience (UX) of the fall risk assessment tool (FRAT-up). In: 2015 IEEE 28th Int. Symp. Comput. Med. Syst., pp. 19-22.
[49]
Oliveros, P., Miller, L., Heyen, J., Siddall, G., Hazard, S., 2011. A musical improvisation interface for people with severe physical disabilities. Music Med 3, 172-181.
[50]
Ortega, D., Cibrian, F.L., Tentori, M., 2015. BendableSound: a fabric-based interactive surface to promote free play in children with autism. In: In Proceedings of the 16th International ACM SIGACCESS Conference on Computers & Accessibility. ACM, pp. 315-316.
[51]
Ospina, M.B., Seida, J.K., Clark, B., Karkhaneh, M., Hartling, L., Tjosvold, L., Vandermeer, B., Smith, V., 2008. Behavioural and developmental interventions for autism spectrum disorder: a clinical systematic review. PLoS One 3
[52]
Parés, N., Soler, M., Sanjurjo, À., Carreras, A., Durany, J., Ferrer, J., Freixa, P., Gómez, D., Kruglanski, O., Parés, R., Ribas, J.I., 2005. Promotion of creative activity in children with severe autism through visuals in an interactive multisensory environment. In: Proceeding 2005 Conf. Interact. Des. Child. - IDC '05, pp. 110-116.
[53]
Piochon, C., Kloth, A.D., Grasselli, G., Titley, H.K., Nakayama, H., Hashimoto, K., Wan, V., Simmons, D.H., Eissa, T., Nakatani, J., Cherskov, A., Miyazaki, T., Watanabe, M., Takumi, T., Kano, M., Wang, S.S.-H., Hansel, C., 2014. Cerebellar plasticity and motor learning deficits in a copy-number variation mouse model of autism. Nat. Commun. 5, 5586.
[54]
Putnam, C., Chong, L., 2008. Software and technologies designed for people with autism: what do users want? In: Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility. ACM, pp. 3-10.
[55]
Rauschenberger, M., Schrepp, M., Perez-Cota, M., Olschner, S., Thomaschewski, J., 2013. Efficient measurement of the user experience of interactive products. How to use the user experience questionnaire (UEQ). Example: spanish language version. Int. J. Interact. Multimed. Artif. Intell. 2, 39-45.
[56]
Repp, B.H., Su, Y.-H., 2013. Sensorimotor synchronization: a review of recent research (2006-2012). Psychon. Bull. Rev. 20, 403-452.
[57]
Ringland, K.E., Zalapa, R., Neal, M., Escobedo, L., Tentori, M., Hayes, G.R., 2014. SensoryPaint: a multimodal sensory intervention for children with neurodevelopmental disorders. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 873-884.
[58]
Rogers, K., Amrei, R., Weing, M., Gugenheimer, J., Bastian, K., Klepsch, M., Schaub, F., Rukzio, E., Seufert, T., Weber, M., 2014. P.I.A.N.O : faster piano learning with interactive projection. In: Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces. ACM, pp. 149-158.
[59]
Rosati, G., Roda, A., Avanzini, F., Masiero, S., 2013. On the role of auditory feedback in robotic-assisted movement training after stroke. Comput. Intell. Neurosci. 2013.
[60]
Sahoo, D.R., Hornbæk, K., Subramanian, S., 2016. TableHop: an actuated fabric display using transparent electrodes. In: Proc. 2016 CHI Conf. Hum. Factors Comput. Syst., pp. 3767-3780.
[61]
Salgado-Montejo, A., Marmolejo-Ramos, F., Alvarado, J.A., Arboleda, J.C., Suarez, D.R., Spence, C., 2016. Drawing sounds: representing tones and chords spatially. Exp. Brain Res 1-14.
[62]
Santoso, H.B., Isal, R.Y.K., Basaruddin, T., Sadira, L., Schrepp, M., 2014. Research-in-progress: User experience evaluation of student centered e-learning environment for computer science program. In: 2014 3rd Int. Conf. User Sci. Eng., pp. 52-55.
[63]
Schaaf, R.C., Benevides, T., Mailloux, Z., Faller, P., Hunt, J., van Hooydonk, E., Freeman, R., Leiby, B., Sendecki, J., Kelly, D., 2014. An intervention for sensory difficulties in children with autism: a randomized trial. J. Autism Dev. Disord. 44, 1493-1506.
[64]
Schaffert, N., Mattes, K., 2015. Interactive sonification in rowing: acoustic feedback for on-water training. IEEE Multimed 22, 58-67.
[65]
Schmidt, R.A., Lee, T., 1988. Motor Control and Learning: A behavioral Emphasis, second ed. Human kinetics, Champaign, IL.
[66]
Schmidt, R.A., Wrisberg, C.A., 2008. Motor Learning and Performance: A Situation-Based Learning Approach, fourth ed. ed. Human kinetics.
[67]
Schonauer, C., Pintaric, T., Kaufmann, H., Jansen-Kosterink, S., Vollenbroek-Hutten, M., 2011. Chronic pain rehabilitation with a serious game using multimodal input. 2011 Int. Conf. Virtual Rehabil. ICVR 2011
[68]
Shams, L., Seitz, A.R., 2008. Benefits of multisensory learning. Trends Cogn. Sci. 12, 411-417.
[69]
Sigrist, R., Rauter, G., Marchal-Crespo, L., Riener, R., Wolf, P., 2014. Sonification and haptic feedback in addition to visual feedback enhances complex motor task learning. Exp. Brain Res. 233, 909-925.
[70]
Singh, A., Piana, S., Pollarolo, D., Volpe, G., Varni, G., Tajadura-Jiménez, A., Williams, A.C., Camurri, A., Bianchi-Berthouze, N., 2015. Go-with-the-Flow : tracking, analysis and sonification of movement and breathing to build confidence in activity despite chronic pain. Human Comput. Interact 24, 1-49.
[71]
Sitdhisanguan, K., Chotikakamthorn, N., Dechaboon, A., Out, P., 2012. Using tangible user interfaces in computer-based training systems for low-functioning autistic children. Pers. Ubiquitous Comput. 16, 143-155.
[72]
Spence, C., 2011. Crossmodal correspondences: a tutorial review. Attention, Perception, Psychophys. 73, 971-995.
[73]
Srinivasan, S.M., Bhat, A.N., 2013. A review of "music and movement" therapies for children with autism: embodied interventions for multisystem development. Front. Integr. Neurosci. 7, 1-15.
[74]
Staples, K.L., Reid, G., 2010. Fundamental movement skills and autism spectrum disorders. J. Autism Dev. Disord. 40, 209-217.
[75]
Sterr, a, Freivogel, S, Voss, a, 2002. Exploring a repetitive training regime for upper limb hemiparesis in an in-patient setting: a report on three case studies. Brain Inj. 16, 1093-1107.
[76]
Tang, S.T., McCorkle, R., 2002. Use of family proxies in quality of life research for cancer patients at the end of life: a literature review. Cancer Invest 20, 1086-1104.
[77]
Tartaro, A., Cassell, J., 2008. Playing with virtual peers: bootstrapping contingent discourse in children with autism. In: Proceedings of the 8th International Conference on International Conference for the Learning Sciences - Volume 2, pp. 382-389.
[78]
Thaut, M.H., Hoemberg, V., 2014. Handbook of Neurologic Music Therapy. Oxford University Press.
[79]
Torres, E.B., Donnellan, A.M., 2015. Autism: the movement perspective. Front. Integr. Neurosci.
[80]
Troiano, G.M., Pedersen, E.W., Hornbæk, K., 2014. User-defined gestures for elastic, deformable displays. In: Proc. 2014 Int. Work. Conf. Adv. Vis. Interfaces - AVI '14, pp. 1-8.
[81]
Troiano, G.M., Pedersen, E.W., Hornbæk, K., Copenhagen, D., 2015. Deformable interfaces for performing music. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. New York, NY, USA. ACM, pp. 377-386.
[82]
Vega-Barbas, M., Pau, I., Ferreira, J., Lebis, E., Seoane, F., 2015. Utilizing smart textiles-enabled sensorized toy and playful interactions for assessment of psychomotor development on children. J. Sensors 2015, 1-9.
[83]
Villafuerte, L., Markova, M., Jorda, S., 2012. Acquisition of social abilities through musical tangible user interface : Children with autism spectrum condition and the Reactable. In: CHI '12 Extended Abstracts on Human Factors in Computing Systems. New York, NY, USA. ACM, pp. 745-760.
[84]
Visi, F., Schramm, R., Miranda, E., 2014. Gesture in performance with traditional musical instruments and electronics - use of embodied music cognition and multimodal motion capture to design gestural mapping strategies. In: Proc. Int. Work. Mov. Comput., pp. 100-105.
[85]
Wendell, J.B., Shelley, W., Karen, H., 2004. Rapid Contextual Design a How-To Guide to Key Techniques for User-Centered Design. Elsevier
[86]
Wigram, Tony, Pedersen Inge Nygaard, B.L.O., 2011. A Comprehensive Guide to Music Therapy - Theory. Clinical Practice, Research and Training.
[87]
Xiao, X., Puentes, P., Ackermann, E., Ishii, H., 2016. Andantino : teaching children piano with projected animated characters. In: ACM SIGCHI Conference on Interaction Design and Children 2016, pp. 37-45.
[88]
Yun, K., Song, J., Youn, K., Cho, S., Bang, H., 2013. ElaScreenn: Exploring multi-dimensional data using elastic screen. In: CHI '13 Ext. Abstr. Hum. Factors Comput. Syst. - CHI EA '13, p. 1311.

Cited By

View all
  • (2024)CoMo.education: Collective and embodied storytelling in kindergarten using a gesture-sound interactive systemProceedings of the 9th International Conference on Movement and Computing10.1145/3658852.3659070(1-11)Online publication date: 30-May-2024
  • (2024)Unmasking the Power of Play Through TUI DesignsACM Transactions on Computer-Human Interaction10.1145/364861931:4(1-43)Online publication date: 19-Sep-2024
  • (2024)Let's Play: Early Explorations of Child-Caregiver Embodied InteractionsProceedings of the 23rd Annual ACM Interaction Design and Children Conference10.1145/3628516.3659417(913-918)Online publication date: 17-Jun-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image International Journal of Human-Computer Studies
International Journal of Human-Computer Studies  Volume 107, Issue C
November 2017
70 pages

Publisher

Academic Press, Inc.

United States

Publication History

Published: 01 November 2017

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)CoMo.education: Collective and embodied storytelling in kindergarten using a gesture-sound interactive systemProceedings of the 9th International Conference on Movement and Computing10.1145/3658852.3659070(1-11)Online publication date: 30-May-2024
  • (2024)Unmasking the Power of Play Through TUI DesignsACM Transactions on Computer-Human Interaction10.1145/364861931:4(1-43)Online publication date: 19-Sep-2024
  • (2024)Let's Play: Early Explorations of Child-Caregiver Embodied InteractionsProceedings of the 23rd Annual ACM Interaction Design and Children Conference10.1145/3628516.3659417(913-918)Online publication date: 17-Jun-2024
  • (2024)MusicTraces: A collaborative music and paint activity for autistic peopleExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651101(1-7)Online publication date: 11-May-2024
  • (2024)Co-designing the Collaborative Digital Musical Instruments for Group Music TherapyProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642649(1-18)Online publication date: 11-May-2024
  • (2023)Ethics without IRB, is that possible? The case study of participatory sessions with ASD Children in MexicoAdjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2023 ACM International Symposium on Wearable Computing10.1145/3594739.3610729(407-410)Online publication date: 8-Oct-2023
  • (2023)Furthering Development of Smart Fabrics to Improve the Accessibility of Music TherapyAdjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2023 ACM International Symposium on Wearable Computing10.1145/3594739.3610727(400-403)Online publication date: 8-Oct-2023
  • (2023)Me, My Health, and My Watch: How Children with ADHD Understand Smartwatch Health DataACM Transactions on Computer-Human Interaction10.1145/357700830:4(1-25)Online publication date: 12-Sep-2023
  • (2023)Understanding the musical interaction of children with autism spectrum disorder using elastic displayPersonal and Ubiquitous Computing10.1007/s00779-022-01703-y27:5(1843-1860)Online publication date: 3-Feb-2023
  • (2022)Material Matters: Exploring Materiality in Digital Musical Instruments DesignProceedings of the 2022 ACM Designing Interactive Systems Conference10.1145/3532106.3533523(976-986)Online publication date: 13-Jun-2022
  • Show More Cited By

View Options

View options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media