short-paper

MirrorFugue iii: conjuring the recorded pianist

Authors:

Xiao Xiao,

Paula Aguilera,

Jonathan Williams,

Hiroshi IshiiAuthors Info & Claims

CHI EA '13: CHI '13 Extended Abstracts on Human Factors in Computing Systems

Pages 2891 - 2892

https://doi.org/10.1145/2468356.2479564

Published: 27 April 2013 Publication History

Get Access

Abstract

The body channels rich layers of information when playing music, from intricate manipulations of the instrument to vivid personifications of expression. But when music is captured and replayed across distance and time, the performer's body is rarely present.

MirrorFugue conjures the recorded performer at the piano by combining the moving keys of a player piano with life-sized projection of the pianist's hands and upper body. Inspired by reflections on a lacquered grand piano, MirrorFugue evokes the sense that the virtual pianist is playing the physically moving keys.

This video tells two stories of interactions across space and time mediated by MirrorFugue. One presents a great concert broadcasted across the world, where a young pianist learns by playing along. The other depicts a woman who plays a duet with her childhood self.

Supplementary Material

suppl.mov (vid0117-file3.m4v)

Supplemental video

Download
88.06 MB

Cited By

View all

Echtler FMaierhöfer VHansen NWimmer R(2023)SurfaceCast: Ubiquitous, Cross-Device Surface SharingProceedings of the ACM on Human-Computer Interaction10.1145/36264757:ISS(286-308)Online publication date: 31-Oct-2023
https://dl.acm.org/doi/10.1145/3626475
Deja JMayer SČopič Pucihar KKljun M(2022)A Survey of Augmented Piano Prototypes: Has Augmentation Improved Learning Experiences?Proceedings of the ACM on Human-Computer Interaction10.1145/35677196:ISS(226-253)Online publication date: 14-Nov-2022
https://dl.acm.org/doi/10.1145/3567719
Suzuki RKarim AXia THedayati HMarquardt N(2022)Augmented Reality and Robotics: A Survey and Taxonomy for AR-enhanced Human-Robot Interaction and Robotic InterfacesProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517719(1-33)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517719
Show More Cited By

Index Terms

MirrorFugue iii: conjuring the recorded pianist
1. Information systems
  1. Information systems applications
    1. Multimedia information systems

Recommendations

MirrorFugue for the Composer, Performer and Improviser
CHI EA '16: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems

This video illustrates the usage of an integrated workbench for learning and composing music on the piano. Built on a Yamaha Disklavier augmented with projection, the system suports music represented either as symbolic notation or as MirrorFugue, where ...
Andantino: Teaching Children Piano with Projected Animated Characters
IDC '16: Proceedings of the The 15th International Conference on Interaction Design and Children

This paper explores how multi-modal body-syntonic interactive systems may be used to teach children to play the piano beyond the typical focus on reading musical scores and "surface correctness". Our work draws from Dalcroze Eurhythmics, a method of ...
Interactive piano training using augmented reality and the Internet of Things
Abstract
The technology of augmented reality in education is a significant attempt to reform the training mode and develop education in general. Piano training usually includes using well-known computer means of teaching music on-screen and hardware. In ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

CHI EA '13: CHI '13 Extended Abstracts on Human Factors in Computing Systems

April 2013

3360 pages

ISBN:9781450319522

DOI:10.1145/2468356

General Chair:
Wendy E. Mackay
INRIA
,
Program Chairs:
Stephen Brewster
Glasgow University
,
Susanne Bødker
University of Aarhus

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 April 2013

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

CHI '13

Sponsor:

SIGCHI

CHI '13: CHI Conference on Human Factors in Computing Systems

April 27 - May 2, 2013

Paris, France

Acceptance Rates

CHI EA '13 Paper Acceptance Rate 630 of 1,963 submissions, 32%;

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
225
Total Downloads

Downloads (Last 12 months)13
Downloads (Last 6 weeks)3

Reflects downloads up to 16 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Echtler FMaierhöfer VHansen NWimmer R(2023)SurfaceCast: Ubiquitous, Cross-Device Surface SharingProceedings of the ACM on Human-Computer Interaction10.1145/36264757:ISS(286-308)Online publication date: 31-Oct-2023
https://dl.acm.org/doi/10.1145/3626475
Deja JMayer SČopič Pucihar KKljun M(2022)A Survey of Augmented Piano Prototypes: Has Augmentation Improved Learning Experiences?Proceedings of the ACM on Human-Computer Interaction10.1145/35677196:ISS(226-253)Online publication date: 14-Nov-2022
https://dl.acm.org/doi/10.1145/3567719
Suzuki RKarim AXia THedayati HMarquardt N(2022)Augmented Reality and Robotics: A Survey and Taxonomy for AR-enhanced Human-Robot Interaction and Robotic InterfacesProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517719(1-33)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517719
Khulusi RKusnick JMeinecke CGillmann CFocht JJänicke S(2020)A Survey on Visualizations for Musical DataComputer Graphics Forum10.1111/cgf.1390539:6(82-110)Online publication date: 5-Mar-2020
https://doi.org/10.1111/cgf.13905
Sandnes FEika E(2019)Enhanced Learning of Jazz Chords with a Projector Based Piano Keyboard AugmentationInnovative Technologies and Learning10.1007/978-3-030-35343-8_21(194-203)Online publication date: 2-Dec-2019
https://dl.acm.org/doi/10.1007/978-3-030-35343-8_21
Suzuki GMurase TFujii YNichols JMahmud JO'Donovan JConati CZancanaro M(2016)Projecting Recorded Expert Hands at Real Size, at Real Speed, and onto Real Objects for Manual WorkCompanion Publication of the 21st International Conference on Intelligent User Interfaces10.1145/2876456.2879476(13-17)Online publication date: 7-Mar-2016
https://dl.acm.org/doi/10.1145/2876456.2879476
Xiao XIshii HKaye JDruin ALampe CMorris DHourcade J(2016)Inspect, Embody, InventProceedings of the 2016 CHI Conference on Human Factors in Computing Systems10.1145/2858036.2858577(5397-5408)Online publication date: 7-May-2016
https://dl.acm.org/doi/10.1145/2858036.2858577
Xiao XHantrakul LIshii HKaye JDruin ALampe CMorris DHourcade J(2016)MirrorFugue for the Composer, Performer and ImproviserProceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems10.1145/2851581.2889435(12-12)Online publication date: 7-May-2016
https://dl.acm.org/doi/10.1145/2851581.2889435
Zhang YLiu STao LYu CShi YXu YDuh HChen YWang HHsieh GBi XJi YWang R(2015)ChinARProceedings of the Third International Symposium of Chinese CHI10.1145/2739999.2740003(23-31)Online publication date: 18-Apr-2015
https://dl.acm.org/doi/10.1145/2739999.2740003
Rogers KRöhlig AWeing MGugenheimer JKönings BKlepsch MSchaub FRukzio ESeufert TWeber MDachselt RGraham NHornbæk KNacenta M(2014)P.I.A.N.O.Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces10.1145/2669485.2669514(149-158)Online publication date: 16-Nov-2014
https://dl.acm.org/doi/10.1145/2669485.2669514
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

Supplementary Material

Cited By

Index Terms

Recommendations

MirrorFugue for the Composer, Performer and Improviser

Andantino: Teaching Children Piano with Projected Animated Characters

Interactive piano training using augmented reality and the Internet of Things

Comments

Information

Published In

Sponsors

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations