research-article

Examining Narrative Sonification: Using First-Person Retrospection Methods to Translate Radio Production to Interaction Design

Authors:

Jordan Wirfs-Brock,

Laura Devendorf,

Brian KeeganAuthors Info & Claims

ACM Transactions on Computer-Human Interaction (TOCHI), Volume 28, Issue 6

Article No.: 41, Pages 1 - 34

https://doi.org/10.1145/3461762

Published: 15 November 2021 Publication History

Abstract

We present a first-person, retrospective exploration of two radio sonification pieces that employ narrative scaffolding to teach audiences how to listen to data. To decelerate and articulate design processes that occurred at the rapid pace of radio production, the sound designer and producer wrote retrospective design accounts. We then revisited the radio pieces through principles drawn from guidance design, data storytelling, visualization literacy, and sound studies. Finally, we speculated how these principles might be applied through interactive, voice-based technologies. First-person methods enabled us to access the implicit knowledge embedded in radio production and translate it to technologies of interest to the human–computer-interaction community, such as voice user interfaces that rely on auditory display. Traditionally, sonification practitioners have focused more on generating sounds than on teaching people how to listen; our process, however, treated sound and narrative as a holistic, sonic-narrative experience. Our first-person retrospection illuminated the role of narrative in designing to support people as they learn to listen to data.

References

[1]

Marketplace. 2011. Marketplace FAQs: When You “do the numbers” Each Day, What Are the Songs You Play? Retrieved on June 6, 2021 fromhttps://www.marketplace.org/faqs/.

[2]

Thomas Hermann, Andy Hunt, and John G. Neuhoff. 2011. The Sonification Handbook. Logos Publishing House.

[3]

NETFLIX. 2020. Lin-Manuel Miranda - Wait for It. Retrieved on September 22, 2020 from https://www.netflix.com/watch/81025975.

[4]

Fereshteh Amini, Nathalie Henry Riche, Bongshin Lee, Christophe Hurter, and Pourang Irani. 2015. Understanding data videos: Looking at narrative visualization through the cinematography lens. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 1459–1468.

Digital Library

[5]

Benjamin Bach, Nathalie Henry Riche, Sheelagh Carpendale, and Hanspeter Pfister. 2017. The emerging genre of data comics. IEEE Computer Graphics and Applications 37, 3 (2017), 6–13.

Digital Library

[6]

Anna Bagnoli. 2009. Beyond the standard interview: The use of graphic elicitation and arts-based methods. Qualitative Research 9, 5 (Nov. 2009), 547–570. DOI:https://doi.org/10.1177/1468794109343625

[7]

J. Barbour. 2004. Analytic listening: A case study of radio production. In Proceedings of the International Conference on Auditory Display. Georgia Institute of Technology. Retrieved from https://smartech.gatech.edu/handle/1853/50759.

[8]

Stephen Barrass. 2015. An annotated portfolio of research through design in acoustic sonification. Leonardo 49, 1 (July 2015), 72–73. DOI:https://doi.org/10.1162/LEON_a_01116

[9]

H. Russell Bernard, Peter Killworth, David Kronenfeld, and Lee Sailer. 1984. The problem of informant accuracy: The validity of retrospective data. Annual Review of Anthropology 13, 1 (1984), 495–517. DOI:https://doi.org/10.1146/annurev.an.13.100184.002431

[10]

L. R. Berney and D. B. Blane. 1997. Collecting retrospective data: Accuracy of recall after 50 years judged against historical records. Social Science & Medicine 45, 10 (Nov. 1997), 1519–1525. DOI:https://doi.org/10.1016/S0277-9536(97)00088-9

[11]

Karin Bijsterveld. 2019. Sonic Skills: Listening for Knowledge in Science, Medicine and Engineering (1920s-Present). Springer Nature. DOI:https://doi.org/10.1057/978-1-137-59829-5

[12]

Jennifer K. Bizley and Yale E. Cohen. 2013. The what, where and how of auditory-object perception. Nature Reviews Neuroscience 14, 10 (Oct. 2013), 693–707. DOI:https://doi.org/10.1038/nrn3565

[13]

Katy Börner, Andreas Bueckle, and Michael Ginda. 2019. Data visualization literacy: Definitions, conceptual frameworks, exercises, and assessments. Proceedings of the National Academy of Sciences 116, 6 (2019), 1857–1864.

[14]

E. Eric Boschmann and Emily Cubbon. 2014. Sketch maps and qualitative GIS: Using cartographies of individual spatial narratives in geographic research. The Professional Geographer 66, 2 (April 2014), 236–248. DOI:https://doi.org/10.1080/00330124.2013.781490

[15]

John Bowers. 2012. The logic of annotated portfolios: Communicating the value of ‘research through design’. In Proceedings of the Designing Interactive Systems Conference. ACM.

Digital Library

[16]

Geoffrey C. Bowker and Susan Leigh Star. 2000. Sorting Things Out: Classification and Its Consequences. MIT Press.

Digital Library

[17]

Michel Callon. 1984. Some elements of a sociology of translation: Domestication of the scallops and the fishermen of St Brieuc Bay. The Sociological Review 32, 1_suppl (1984), 196–233.

[18]

Davide Ceneda, Theresia Gschwandtner, Thorsten May, Silvia Miksch, Hans-Jörg Schulz, Marc Streit, and Christian Tominski. 2016. Characterizing guidance in visual analytics. IEEE Transactions on Visualization and Computer Graphics 23, 1 (2016), 111–120.

Digital Library

[19]

Davide Ceneda, Theresia Gschwandtner, and Silvia Miksch. 2019. A review of guidance approaches in visual data analysis: A multifocal perspective. Computer Graphics Forum 38, 3 (2019), 861–879. DOI:https://doi.org/10.1111/cgf.13730

[20]

Alan Chamberlain, Andy Crabtree, Tom Rodden, Matt Jones, and Yvonne Rogers. 2012. Research in the wild: Understanding ‘in the wild’ approaches to design and development. In Proceedings of the Designing Interactive Systems Conference. ACM, New York, NY, 795–796. DOI:https://doi.org/10.1145/2317956.2318078

Digital Library

[21]

Roy Peter Clark. 2020. Explanatory Journalism is Entering a Golden Age in the Middle of the Coronavirus Pandemic. Retrieved on June 6, 2021 from https://www.poynter.org/reporting-editing/2020/explanatory-journalism-is-entering-a-golden-age-in-the-middle-of-the-coronavirus-pandemic/.

[22]

Fred Collopy. 1996. Biases in retrospective self-reports of time use: An empirical study of computer users. Management Science 42, 5 (May 1996), 758–767. DOI:https://doi.org/10.1287/mnsc.42.5.758

Digital Library

[23]

Martin Davies. 2011. Concept mapping, mind mapping and argument mapping: What are the differences and do they matter?Higher Education 62, 3 (Sept. 2011), 279–301. DOI:https://doi.org/10.1007/s10734-010-9387-6

[24]

William J. Davies, Mags D. Adams, Neil S. Bruce, Rebecca Cain, Angus Carlyle, Peter Cusack, Deborah A. Hall, Ken I. Hume, Amy Irwin, Paul Jennings, Melissa Marselle, Christopher J. Plack, and John Poxon. 2013. Perception of soundscapes: An interdisciplinary approach. Applied Acoustics 74, 2 (2013), 224–231.

[25]

Alberto de Campo. 2007. Toward a data sonification design space map. In Proceedings of the International Conference on Auditory Display. Georgia Institute of Technology, 342–347.

[26]

Tilman Dingler, Jeffrey Lindsay, and Bruce N. Walker. 2008. Learnabiltiy of sound cues for environmental features: Auditory icons, earcons, spearcons, and speech. In Proceedings of the 2008 International Conference on Auditory Display. Georgia Institute of Technology. Retrieved from https://smartech.gatech.edu/handle/1853/49940.

[27]

Graham Dove and Sara Jones. 2012. Narrative visualization: Sharing insights into complex data. In Interfaces and Human Computer Interaction (IHCI’12), 21-23 Jul 2012, Lisbon, Portugal. https://openaccess.city.ac.uk/id/eprint/1134

[28]

Milena Droumeva and Iain McGregor. 2019. Sound stories: A context-based study of everyday listening to augmented soundscapes. Interacting with Computers 31, 3 (May 2019), 336–347. DOI:https://doi.org/10.1093/iwc/iwz024.

[29]

Elizabeth Dubois and Heather Ford. 2015. Trace interviews: An actor-centered approach. International Journal of Communication 9, 0 (June 2015), 25. Retrieved from https://ijoc.org/index.php/ijoc/article/view/3378.

[30]

Thomas Dykes, Mark Blythe, Jayne Wallace, James Thomas, and Tim Regan. 2016. RtD comics: A medium for representing research through design. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems. ACM, New York, NY, 971–982. DOI:https://doi.org/10.1145/2901790.2901821

Digital Library

[31]

Frederick Erickson. 2012. Definition and analysis of data from videotape: Some research procedures and their rationales. In Handbook of Complementary Methods in Education Research. J. L. Green, G. Camilli, P. B. Elmore, A. Skukauskaiti, and E. Grace (Eds.), Routledge.

[32]

Kathy Roberts Forde. 2007. Discovering the explanatory report in American newspapers. Journalism Practice 1, 2 (June 2007), 227–244. DOI:https://doi.org/10.1080/17512780701275531

[33]

Deborah Freedman, Arland Thornton, Donald Camburn, Duane Alwin, and Linda Young-DeMarco. 1988. The life history calendar: A technique for collecting retrospective data. Sociological Methodology 18 (1988), 37–68. DOI:https://doi.org/10.2307/271044

[34]

Mikhaila Friske, Jordan Wirfs-Brock, and Laura Devendorf. 2020. Entangling the roles of maker and interpreter in interpersonal data narratives: Explorations in yarn and sound. In Proceedings of the 2020 ACM Designing Interactive Systems Conference. ACM, 297–310. DOI:https://doi.org/10.1145/3357236.3395442

Digital Library

[35]

William W. Gaver. 1986. Auditory icons: Using sound in computer interfaces. Human–Computer Interaction 2, 2 (June 1986), 167–177. DOI:https://doi.org/10.1207/s15327051hci0202_3

Digital Library

[36]

William W. Gaver. 1993. How do we hear in the world? Explorations in ecological acoustics. Ecological Psychology 5, 4 (1993), 285–313. DOI:https://doi.org/10.1207/s15326969eco0504_2

[37]

William W. Gaver and Donald A. Norman. 1988. Everyday Listening and Auditory Icons. Ph.D. Dissertation. University of California, San Diego, Department of Cognitive Science and Psychology.

Digital Library

[38]

Bruno L. Giordano, John McDonnell, and Stephen McAdams. 2010. Hearing living symbols and nonliving icons: Category specificities in the cognitive processing of environmental sounds. Brain and Cognition 73, 1 (June 2010), 7–19. DOI:https://doi.org/10.1016/j.bandc.2010.01.005

[39]

Jacob Goldstein and David Kestenbaum. 2011. U.S. Home Prices, Sung As Opera. Retrieved June 6, 2021 from https://www.npr.org/sections/money/2011/04/27/135737940/the-case-shiller-index-sung-as-opera.

[40]

Rebecca Gulotta, William Odom, Jodi Forlizzi, and Haakon Faste. 2013. Digital artifacts as legacy: Exploring the lifespan and value of digital data. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 1813–1822. DOI:https://doi.org/10.1145/2470654.2466240

Digital Library

[41]

Rebecca Gulotta, Alex Sciuto, Aisling Kelliher, and Jodi Forlizzi. 2015. Curatorial agents: How systems shape our understanding of personal and familial digital information. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 3453–3462.

Digital Library

[42]

Gunnar Harboe and Elaine M. Huang. 2015. Real-world affinity diagramming practices: Bridging the paper-digital gap. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, 95–104. DOI:https://doi.org/10.1145/2702123.2702561

Digital Library

[43]

Sabrina Hauser, Doenja Oogjes, Ron Wakkary, and Peter-Paul Verbeek. 2018. An annotated portfolio on doing postphenomenology through research products. In Proceedings of the 2018 Designing Interactive Systems Conference. ACM, 459–471. DOI:https://doi.org/10.1145/3196709.3196745

Digital Library

[44]

Kieran Healy. 2018. Data Visualization: A Practical Introduction. Princeton University Press.

[45]

John Hudak. Shining Light on Explanatory Journalism’s Impact on Media, Democracy, and Society. Retrieved February 24, 2016 from https://www.brookings.edu/blog/fixgov/2016/02/24/shining-light-on-explanatory-journalisms-impact-on-media-democracy-and-society/.

[46]

Kelly Jakubowski, Sebastian Finkel, Lauren Stewart, and Daniel Müllensiefen. 2017. Dissecting an earworm: Melodic features and song popularity predict involuntary musical imagery. Psychology of Aesthetics, Creativity, and the Arts 11, 2 (May 2017), 122–135. DOI:https://doi.org/10.1037/aca0000090

[47]

Jane Johnston and Susan Forde. 2017. Churnalism. Digital Journalism 5, 8 (Sept. 2017), 943–946. DOI:https://doi.org/10.1080/21670811.2017.1355026

[48]

Brigitte Jordan and Austin Henderson. 1995. Interaction analysis: Foundations and practice. The Journal of the Learning Sciences 4, 1 (1995), 39–103.

[49]

Julia Barton and Roman Mars. 2013. 99 Percent Invisible: The Broadcast Clock. Retrieved September 22, 2020 from https://99percentinvisible.org/episode/the-broadcast-clock/.

[50]

Sara Lenzi and Paolo Ciuccarelli. 2020. Intentionality and design in the data sonification of social issues. Big Data & Society 7, 2 (July 2020), 2053951720944603. DOI:https://doi.org/10.1177/2053951720944603

[51]

Grace Li and Bruce N. Walker. 2019. Mixed speech and non-speech auditory displays: Impacts of design, learning, and individual differences in musical engagement. In 25th International Conference on Auditory Display (ICAD’19) 23-27 June 2019, Northumbria University, Newcastle upon Tyne, UK. DOI:https://doi.org/10.21785/icad2019.019

[52]

Otto Hans-Martin Lutz, Jacob Leon Kroger, Manuel Schneiderbauer, and Manfred Hauswirth. 2019. Surfing in sound: Sonification of hidden web tracking. In Proceedings of the 2019 International Conference on Auditory Display. Georgia Institute of Technology. DOI:https://doi.org/10.21785/icad2019.071

[53]

William J. T. Mitchell. 2005. There are no visual media. Journal of Visual Culture 4, 2 (2005), 257–266.

[54]

Tamara Munzner. 2014. Visualization Analysis and Design. CRC Press.

[55]

Michael A. Nees. 2018. Auditory graphs are not the “killer app” of sonification, but they work. Ergonomics in Design 26, 4 (2018), 25–28.

[56]

John G. Neuhoff. 2019. Is sonification doomed to fail? In Proceedings of the International Conference on Auditory Display. Georgia Institute of Technology. DOI:https://doi.org/10.21785/icad2019.069

[57]

Darren Newtson and Gretchen Engquist. 1976. The perceptual organization of ongoing behavior. Journal of Experimental Social Psychology 12, 5 (1976), 436–450.

[58]

Elinor Ochs. 1979. Transcription as theory. Developmental Pragmatics 10, 1 (1979), 43–72.

[59]

Tor Ole B. Odden and Rosemary S. Russ. 2019. Defining sensemaking: Bringing clarity to a fragmented theoretical construct. Science Education 103, 1 (2019), 187–205. DOI:https://doi.org/10.1002/sce.21452

[60]

Doenja Oogjes, Ron Wakkary, Henry Lin, and Omid Alemi. 2020. Fragile! Handle with care: The morse things. In Proceedings of the 2020 ACM Designing Interactive Systems Conference. ACM, New York, NY, 2149–2162. DOI:https://doi.org/10.1145/3357236.3395584

Digital Library

[61]

Julian Edgerton Orr. 1987. Talking about machines: Social aspects of expertise. Xerox Palo Alto Research Center Intelligent Systems Laboratory.

[62]

Trevor Pinch and Karin Bijsterveld. 2012. New keys to the world of sound. In The Oxford Handbook of Sound Studies. T. Pinch and K. Bijsterveld (Eds.), Oxford University Press, Oxford.

[63]

Søren Rasmussen, Jeanette Falk Olesen, and Kim Halskov. 2019. Co-notate: Exploring real-time annotations to capture situational design knowledge. In Proceedings of the 2019 on Designing Interactive Systems Conference. ACM, 161–172. DOI:https://doi.org/10.1145/3322276.3322310

Digital Library

[64]

Nathalie Henry Riche, Christophe Hurter, Nicholas Diakopoulos, and Sheelagh Carpendale. 2018. Data-Driven Storytelling. CRC Press.

Digital Library

[65]

Stephen Roddy and Brian Bridges. 2016. Sounding human with data: The role of embodied conceptual metaphors and aesthetics in representing and exploring data sets. In Proceedings of the Music Technology Workshop 2016: Establishing a Partnership between Music Technology, Business Analytics and Industry in Ireland. University College Dublin, 64–76.

[66]

Stephen Roddy and Brian Bridges. 2020. Mapping for meaning: The embodied sonification listening model and its implications for the mapping problem in sonic information design. Journal on Multimodal User Interfaces 14, 2 (2020), 143–151.

[67]

Sophia Roosth. 2009. Screaming yeast: Sonocytology, cytoplasmic milieus, and cellular subjectivities. Critical Inquiry 35, 2 (2009), 332–350.

[68]

Daniel M. Russell and Ed H. Chi. 2014. Looking back: Retrospective study methods for HCI. In Ways of Knowing in HCI. Olson J. and Kellogg W. (Eds.), Springer, 373–393.

[69]

Kai Ryssdal and Alli Fam. 2020. Here’s What the Crescendo of Unemployment Sounds Like. Retrieved October 2, 2021 from https://www.marketplace.org/2020/06/24/heres-what-the-crescendo-of-unemployment-sounds-like/.

[70]

Kai Ryssdal and Alli Fam. 2020. What Sound does a Volatile Stock Market Make? Retrieved October 2, 2021 from https://www.marketplace.org/2020/03/31/the-sounds-of-a-volatile-stock-market/.

[71]

Christine Schmidt. 2018. “Journalism practice may feel like a product on a conveyor belt”: Researchers on the Future of Automated News Production and Consumption. Retrieved September 22, 2020 from https://www.niemanlab.org/2018/05/journalism-practice-may-feel-like-a-product-on-a-conveyor-belt-researchers-on-the-future-of-automated-news-production-and-consumption/.

[72]

Sarita Schoenebeck, Nicole B. Ellison, Lindsay Blackwell, Joseph B. Bayer, and Emily B. Falk. 2016. Playful backstalking and serious impression management: How young adults reflect on their past identities on Facebook. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing. ACM, New York, NY, 1475–1487. DOI:https://doi.org/10.1145/2818048.2819923

Digital Library

[73]

Donald A. Schon. 1983. The Reflective Practitioner: How Professionals Think in Action (1st. ed.). Basic Books, New York, NY.

[74]

Stephan Schwan and Bärbel Garsoffky. 2004. The cognitive representation of filmic event summaries. Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition 18, 1 (2004), 37–55.

[75]

Edward Segel and Jeffrey Heer. 2010. Narrative visualization: Telling stories with data. IEEE Transactions on Visualization and Computer Graphics 16, 6 (Nov. 2010), 1139–1148. DOI:https://doi.org/10.1109/TVCG.2010.179

Digital Library

[76]

Taliesin L. Smith and Emily B. Moore. 2020. Storytelling to sensemaking: A systematic framework for designing auditory description display for interactives. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 1–12. DOI:https://doi.org/10.1145/3313831.3376460

Digital Library

[77]

Pieter Jan Stappers and Elisa Giaccardi. 2012. Research through design. In The Encyclopedia of Human-Computer Interaction, Claude Ghaoui (Ed.). Interaction Design Foundation. Retrieved from https://www.interaction-design.org/literature/book/the-encyclopedia-of-human-computer-interaction-2nd-ed/research-through-design.

[78]

Susan Leigh Star and James R. Griesemer. 1989. Institutional ecology, ‘translations’ and boundary objects: Amateurs and professionals in Berkeley’s museum of vertebrate zoology, 1907–39. Social Studies of Science 19, 3 (1989), 387–420. Retrieved from https://www.jstor.org/stable/285080.

[79]

Dean Starkman. 2010. The Hamster Wheel. Retrieved September 22, 2020 from https://www.cjr.org/cover_story/the_hamster_wheel.php.

[80]

Alexandra Supper. 2011. The search for the “killer application”: Drawing the boundaries around the sonification of scientific data. In The Oxford Handbook of Sound Studies. T. Pinch and K. Bijsterveld (Eds.). Oxford University Press, New York. DOI:https://doi.org/10.1093/oxfordhb/9780195388947.013.0064

[81]

Alexandra Supper and Karin Bijsterveld. 2015. Sounds convincing: Modes of listening and sonic skills in knowledge making. Interdisciplinary Science Reviews 40, 2 (June 2015), 124–144. DOI:https://doi.org/10.1179/0308018815Z.000000000109

[82]

Alice Thudt, Jagoda Walny, Theresia Gschwandtner, Jason Dykes, and John Stasko. 2018. Exploration and explanation in data-driven storytelling. In Data-Driven Storytelling, Nathalie Henry Riche, Christophe Hurter, Nicholas Diakopoulos, and Sheelagh Carpendale (Eds.). AK Peters/CRC Press, 59–83.

[83]

Barbara Tversky. 2018. Storytelling in the wild: Implications for data storytelling. In Data-Driven Storytelling, Nathalie Henry Riche, Christophe Hurter, Nicholas Diakopoulos, and Sheelagh Carpendale (Eds.). AK Peters/CRC Press, 17–57.

[84]

Shawn VanCour. 2018. Making Radio: Early Radio Production and the Rise of Modern Sound Culture. Oxford University Press.

[85]

Bruce N. Walker and Michael A. Nees. 2005. Brief training for performance of a point estimation sonification task. In Proceedings of the 11th Meeting of the International Conference on Auditory Display. Georgia Institute of Technology. DOI:https://doi.org/1853/50101

[86]

Jordan Wirfs-Brock. 2016. Listen to U.S. Coal Production Fall Off A Cliff. Retrieved June 6, 2021 from http://insideenergy.org/2016/05/03/listen-to-u-s-coal-production-fall-off-a-cliff/.

[87]

David Worrall. 2009. Sonification and Information: Concepts, Instruments and Techniques. Ph.D. Dissertation. University of Canberra.

[88]

Roxanna Xia, Swetha Kannan, and Terry Castleman. 2019. The Ocean Game: The sea is rising. Can you save your town? - Los Angeles Times. Retrieved June 6, 2021 from https://www.latimes.com/projects/la-me-climate-change-ocean-game/.

[89]

Xiao Zhang and Ron Wakkary. 2014. Understanding the role of designers’ personal experiences in interaction design practice. In Proceedings of the 2014 Conference on Designing Interactive Systems. ACM, 895–904. DOI:https://doi.org/10.1145/2598510.2598556

Digital Library

[90]

John Zimmerman and Jodi Forlizzi. 2014. Research through design in HCI. In Ways of Knowing in HCI. J. Olson and W. Kellogg (Eds.), Springer, 167–189.

Cited By

Lenzi STerenghi GMeacci DMoreno Fernandez-de-Leceta ACiuccarelli P(2024)The design of Datascapes: toward a design framework for sonification for anomaly detection in AI-supported networked environmentsFrontiers in Computer Science10.3389/fcomp.2023.12546785Online publication date: 11-Jan-2024
https://doi.org/10.3389/fcomp.2023.1254678
Bang TAlaoui STyse GSchwartz EBevilacqua F(2024)A Retrospective Autoethnography Documenting Dance Learning Through Data PhysicalisationsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661607(2357-2373)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661607
Wirfs-Brock JPerera JGeere D(2024)Open Sonifications: A Manifesto for Many Ecologies of Data and SoundProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660757(2660-2674)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3660757
Show More Cited By

Index Terms

Examining Narrative Sonification: Using First-Person Retrospection Methods to Translate Radio Production to Interaction Design
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction techniques
      1. Auditory feedback
  2. Interaction design
    1. Interaction design process and methods

Recommendations

Evaluation of psychoacoustic sound parameters for sonification
ICMI '17: Proceedings of the 19th ACM International Conference on Multimodal Interaction

Sonification designers have little theory or experimental evidence to guide the design of data-to-sound mappings. Many mappings use acoustic representations of data values which do not correspond with the listener's perception of how that data value ...
Aesthetic strategies in sonification

Sound can be listened to in various ways and with different intentions. Multiple factors influence how and what we perceive when listening to sound. Sonification, the acoustic representation of data, is in essence just sound. It functions as ...
Learning to Listen to Data: Voice Interaction, Sonification, and Narrative
DIS '21 Companion: Companion Publication of the 2021 ACM Designing Interactive Systems Conference

My dissertation explores how to help people learn to listen to data through new voice-first technological capabilities, by combining narrative storytelling and data sonification. For example, what if a voice assistant could help you craft personalized ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Computer-Human Interaction

ACM Transactions on Computer-Human Interaction Volume 28, Issue 6

December 2021

354 pages

ISSN:1073-0516

EISSN:1557-7325

DOI:10.1145/3492441

Editor:
Kristina Höök
KTH Royal Institute of Technology, Sweden

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 November 2021

Accepted: 01 April 2021

Revised: 01 February 2021

Received: 01 October 2020

Published in TOCHI Volume 28, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
747
Total Downloads

Downloads (Last 12 months)191
Downloads (Last 6 weeks)23

Reflects downloads up to 13 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lenzi STerenghi GMeacci DMoreno Fernandez-de-Leceta ACiuccarelli P(2024)The design of Datascapes: toward a design framework for sonification for anomaly detection in AI-supported networked environmentsFrontiers in Computer Science10.3389/fcomp.2023.12546785Online publication date: 11-Jan-2024
https://doi.org/10.3389/fcomp.2023.1254678
Bang TAlaoui STyse GSchwartz EBevilacqua F(2024)A Retrospective Autoethnography Documenting Dance Learning Through Data PhysicalisationsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661607(2357-2373)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661607
Wirfs-Brock JPerera JGeere D(2024)Open Sonifications: A Manifesto for Many Ecologies of Data and SoundProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660757(2660-2674)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3660757
Meiklejohn EDevendorf LPosch I(2024)Design Bookkeeping: Making Practice Intelligible through a Managerial LensProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660754(35-49)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3660754
Rodríguez Mateos DTapia López A(2023)Sonificación y periodismo: la representación de datos mediante sonidosRevista de Comunicación10.26441/RC22.1-2023-302222:1Online publication date: 15-Jan-2023
https://doi.org/10.26441/RC22.1-2023-3022
Reed CMcPherson A(2023)The Body as Sound: Unpacking Vocal Embodiment through Auditory BiofeedbackProceedings of the Seventeenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3569009.3572738(1-15)Online publication date: 26-Feb-2023
https://dl.acm.org/doi/10.1145/3569009.3572738
Reed CStrohmeier PMcPherson A(2023)Negotiating Experience and Communicating Information Through Abstract MetaphorProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580700(1-16)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580700

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents