Books by Federico Fontana
We suggest that studies on active touch psychophysics are needed to inform the design of haptic m... more We suggest that studies on active touch psychophysics are needed to inform the design of haptic musical interfaces and better understand the relevance of haptic cues in musical performance. Following a review of the previous literature on vibrotactile perception in musical performance, two recent experiments are reported. The first experiment investigated how active finger-pressing forces affect vibration perception, finding significant effects of vibration type and force level on perceptual thresholds. Moreover, the measured thresholds were considerably lower than those reported in the literature, possibly due to the concurrent effect of large (unconstrained) finger contact areas, active pressing forces, and long-duration stimuli. The second experiment assessed the validity of these findings in a real musical context by studying the detection of vibrotactile cues at the keyboard of a grand and an upright piano. Sensitivity to key vibrations in fact not only was highest at the lower octaves and gradually decreased toward higher pitches; it was also significant for stimuli having spectral peaks of acceleration similar to those of the first experiment, i.e., below the standard sensitivity thresholds measured for sinusoidal vibrations under passive touch conditions.
Bookmarks Related papers MentionsView impact
While a standard approach is more or less established for rendering basic vibratory cues in consu... more While a standard approach is more or less established for rendering basic vibratory cues in consumer electronics, the implementation of advanced vibrotac-tile feedback still requires designers and engineers to solve a number of technical issues. Several off-the-shelf vibration actuators are currently available, having different characteristics and limitations that should be considered in the design process. We suggest an iterative approach to design in which vibrotactile interfaces are validated by testing their accuracy in rendering vibratory cues and in measuring input gestures. Several examples of prototype interfaces yielding audio-haptic feedback are described, ranging from open-ended devices to musical interfaces, addressing their design and the characterization of their vibratory output.
Bookmarks Related papers MentionsView impact
The first time I went to a conference of computer music, in 1995, I was amazed by the many panels... more The first time I went to a conference of computer music, in 1995, I was amazed by the many panels presenting papers on acoustics, psychoacoustics, audio signal processing, applied computer science and audio engineering, which addressed musical performance, the art of composition, the specification of ideal listening and the ways to achieve that ideal. This musical emphasis was demonstrated by the conference's social event, normally including an artistic performance such as a concert of contemporary music: I later realized that this was a must in most of those conferences.
In the science of sound there is a clear link between research and art that is less common than other disciplines.
The study of the everyday listening experience and its systematic - and not necessarily artistic, as it has been happening since long time ago: think, for instance, of Luigi Russolo's Intonarumori - inclusion as a branch of research in sound is quite recent. Perhaps this new branch has not yet been totally accepted by some research communities in the field. Obviously, during most of their lifetime humans use the hearing system to acquire information from the external world, but they spend little time appreciating the acoustical subtleties of state of the art technology such as high-end sound synthesis and reproduction equipment. Likewise, most of the researchers working in other disciplines focus on solving problems that, directly or not, concern practical matters; rarely do they deal with the artistic derivations of their research.
Do we choose to do study in the everyday listening spectrum only because our methods of investigation cannot deal with the artistic level of the hearing experience? At least two arguments can be given to nullify that suspicion. First, most everyday sounds are so familiar to the listener that they can be hardly reproduced by a model accurately enough to cheat the hearing system, if not the subject. Second, art must not be used to mystify fair research results, as it happens sometimes in this field.
To put it plainly, we would say that models that simulate everyday listening, including those reproducing the spatial location of a sound source, those synthesizing ecological sounds, and more generally those adding some sense of presence to a virtual scenario, are prone to more criticism and severe evaluation compared, for instance, to models for the synthesis of hybrid sounds or for the automatic generation of scores.
On the other hand most of those actually doing research in sound are (or were) sound practitioners or expert music listeners, if not musicians, composers or professionals in the musical field. Those people ought to consider a different research point of view, where quality is evaluated by means of psychophysical tests, and musical sounds are substituted by ecological events such as bounces, footsteps, rolling, crushing and breaking sounds.
This is not an easy perspective to adopt, especially for those having a musical background. It is opinion of the author that accepting this new perspective will be a key step in helping the science of sound to achieve a prominent position among the most respected scientific disciplines.
Pordenone, February 2nd, 2003.
Bookmarks Related papers MentionsView impact
During the last couple of decades we have been involved in a revolution: computer-based virtual, ... more During the last couple of decades we have been involved in a revolution: computer-based virtual, distributed, and augmented realities increasingly affect our perception of the world. Despite the abundant technological facilities, we are now faced with lack of knowledge on how to generate sounds which please a perceptual and cognitive system whose evolution developed slowly, over many thousands of years. The authors of this book are striving to find and develop principles that may open the way to get appropriate informative sounds out of future artifacts. They present novel research in perception, physics, numeral analysis, computer science, and human-machine interaction.
Bookmarks Related papers MentionsView impact
As we walk along concrete city sidewalks, over gravel paths, or across tiled building lobbies, we... more As we walk along concrete city sidewalks, over gravel paths, or across tiled building lobbies, we are continuously exposed through our footsteps to highly structured information about the ground, through the feelings we experience and the sounds we hear. The present volume documents recent research that has aimed at reinforcing our understanding of how our feet interact with surfaces on which we walk, and at characterizing those sensations we have when walking that help us to interpret space in intuitive ways and that can be replicated via new technologies toward building realistic virtual environments.
The chapters it contains notably review advances that were achieved within the multidisciplinary European project Natural Interactive Walking. Through the development of new technologies for enhancing spaces, floors, and footwear in ways that allow them to provide simulated experiences of attributes of everyday walking surfaces, the research covered here attempts to enable the designers of new technological systems to engender a real sense of ``being there'', particularly through the use of data coming from the haptic (touch) and auditory (sound) perceptual channels, or in tandem with them. It illustrates how knowledge about the ways that users experience their surroundings during walking can be used to create perceptually rich and plausible experiences of walking in diverse natural and man-made environments.
This work may lead to radically new approaches to interaction with digital information, for example in airports, railway stations, public urban spaces, or in virtual environments used for immersive training purposes. It could, for example, be applied to the creation of intuitive navigation aids, such as landmarking, guidance to locations of interest, ``eyes-free'' signaling, and warning about obstacles and restricted areas. Such research may also open the door to the creation of better assistive tools for visually-impaired and other special-needs users.
Bookmarks Related papers MentionsView impact
Papers by Federico Fontana
Arts
The physics-based design and realization of a digital musical interface asks for the modeling and... more The physics-based design and realization of a digital musical interface asks for the modeling and implementation of the contact-point interaction with the performer. Musical instruments always include a resonator that converts the input energy into sound, meanwhile feeding part of it back to the performer through the same point. Specifically during plucking or bowing interactions, musicians receive a handful of information from the force feedback and vibrations coming from the contact points. This paper focuses on the design and realization of digital music interfaces realizing two physical interactions along with a musically unconventional one, rubbing, rarely encountered in assimilable forms across the centuries on a few instruments. Therefore, it aims to highlight the significance of haptic rendering in improving quality during a musical experience as opposed to interfaces provided with a passive contact point. Current challenges are posed by the specific requirements of the hapt...
Bookmarks Related papers MentionsView impact
IEEE Transactions on Haptics
Bookmarks Related papers MentionsView impact
DMT Lab, Birmingham City University, 2019
Bookmarks Related papers MentionsView impact
Results from three experiments are presented, showing that the perceived acoustic and vibrotactil... more Results from three experiments are presented, showing that the perceived acoustic and vibrotactile quality of a reproduced piano does not require models simulating every aspect of the original instrument with great accuracy. It was found that high-quality loudspeaker array passive listening at the pianist's position admits distortions of the sound field. Furthermore, pianists during playing seem to compensate for errors in the auditory scene description. Finally, they are particularly sensitive to the existence of vibrotactile feedback on their fingers meanwhile tolerant about the precision with which this feedback is reproduced. Based on these results we are currently working on a lightweight portable physics-based digital piano design, that should improve upon the experience a pianist with no keyboards at hand makes when interacting with a touch-screen piano software running on smartphones and laptops
Bookmarks Related papers MentionsView impact
Presentiamo una tastiera virtuale che mira a sostituire quella del pianoforte con dei tasti in re... more Presentiamo una tastiera virtuale che mira a sostituire quella del pianoforte con dei tasti in realt\ue0 aumentata. Lo sforzo maggiore \ue8 stato compiuto nel rilevamento accurato nonch\ue9 rapido dei movimenti delle mani. Per raggiungere questo obiettivo sono stati confrontati un dispositivo di rilevamento a infrarossi ed uno a ultrasuoni, valutandone pro e contro nell\u2019impiego specifico. Abbiamo poi realizzato un sistema di feedback multimodale riprendendo le azioni delle mani dell\u2019esecutore mediante la fotocamera posteriore di un comune dispositivo mobile, e presentandole all\u2019utente unite alla visualizzazione di una tastiera di pianoforte animata. Connettendo il dispositivo Leap Motion, responsabile del rilevamento a infrarossi, e altri dispositivi hardware per la generazione di stimoli audio e vibrotattili, il prototipo promette potenziali applicazioni come interfaccia musicale mobile a basso costo, trasformando un comune tavolo in uno scenario di realt\ue0 aumenta...
Bookmarks Related papers MentionsView impact
Proceedings of the SMC Conferences, Jul 4, 2018
Bookmarks Related papers MentionsView impact
This paper reviews recent developments in physics-based synthesis of piano. The paper considers t... more This paper reviews recent developments in physics-based synthesis of piano. The paper considers the main components of the instrument, that is, the hammer, the string, and the soundboard. Modeling techniques are discussed for each of these elements, together with implementation strategies. Attention is focused on numerical issues, and each implementation technique is described inlightofitsefficiency and accuracy properties. As the structured audio coding approach is gaining popularity, the authors argue that the physical modeling approach will have relevant applications in the field of multimedia communication.
Bookmarks Related papers MentionsView impact
Journal on Multimodal User Interfaces, 2016
Bookmarks Related papers MentionsView impact
Bookmarks Related papers MentionsView impact
Questo lavoro presenta i risultati di due esperimenti percettivi compiuti su un prototipo di pian... more Questo lavoro presenta i risultati di due esperimenti percettivi compiuti su un prototipo di pianoforte digitale aumentato. L'obiettivo di entrambi gli esperimenti era studiare la percezione da parte del pianista di feedback vibrotattile sulla tastiera. Nel primo esperimento i soggetti coinvolti dovevano suonare liberamente al variare del feedback, e fornire un giudizio sulla qualità percepita dello strumento in una griglia di cinque attributi: controllo dinamico, ricchezza, coinvolgimento, naturalezza, e preferenza complessiva. Nel secondo esperimento si è misurata l'accuratezza (in termini di tempo e di controllo dinamico) nell'esecuzione di una scala, al variare del feedback. I risultati mostrano una preferenza per le condizioni in cui è presente feedback vibrotattile, tuttavia per quanto riguarda l'accuratezza dell'esecuzione non sono state osservate differenze significative tra le diverse condizioni.
Bookmarks Related papers MentionsView impact
Besides vision and audition, everyday materials can be passively explored also using touch if the... more Besides vision and audition, everyday materials can be passively explored also using touch if they provide tactile feedback to users, for instance in consequence of an external force exciting their natural resonances. If such resonances are known to provide informative auditory cues of material, on the other hand their role when a recognition is made through touch is debatable. Even more questionable is a material recognition from their reproductions: if happening, then they could be used to enrich existing touch-screen interactions with ecological auditory and haptic feedback furthermore requiring inexpensive actuation. With this goal in mind, two experiments are proposed evaluating user's ability to classify wooden, plastic, and metallic surfaces respectively using auditory and haptic cues. Although the literature reports successful auditory classification of everyday material simulations, especially the passive recognition of such material reproductions by holding a finger on...
Bookmarks Related papers MentionsView impact
Proceedings of the SMC Conferences, Jul 6, 2011
Bookmarks Related papers MentionsView impact
Proceedings of the SMC Conferences, Aug 31, 2016
Bookmarks Related papers MentionsView impact
Bookmarks Related papers MentionsView impact
A shoe-based interface is presented, which enables users to play percussive virtual instruments b... more A shoe-based interface is presented, which enables users to play percussive virtual instruments by tapping their feet. The wearable interface consists of a pair of sandals equipped with four force sensors and four actuators affording audiotactile feedback. The sensors provide data via wireless transmission to a host computer, where they are processed and mapped to a physics-based sound synthesis engine. Since the system provides OSC and MIDI compatibility, alternative electronic instruments can be used as well. The audio signals are then sent back wirelessly to audio-tactile exciters embedded in the sandals’ sole, and optionally to headphones and external loudspeakers. The round-trip wireless communication only introduces very small latency, thus guaranteeing coherence and unity in the multimodal percept and allowing tight timing while playing.
Bookmarks Related papers MentionsView impact
Uploads
Books by Federico Fontana
In the science of sound there is a clear link between research and art that is less common than other disciplines.
The study of the everyday listening experience and its systematic - and not necessarily artistic, as it has been happening since long time ago: think, for instance, of Luigi Russolo's Intonarumori - inclusion as a branch of research in sound is quite recent. Perhaps this new branch has not yet been totally accepted by some research communities in the field. Obviously, during most of their lifetime humans use the hearing system to acquire information from the external world, but they spend little time appreciating the acoustical subtleties of state of the art technology such as high-end sound synthesis and reproduction equipment. Likewise, most of the researchers working in other disciplines focus on solving problems that, directly or not, concern practical matters; rarely do they deal with the artistic derivations of their research.
Do we choose to do study in the everyday listening spectrum only because our methods of investigation cannot deal with the artistic level of the hearing experience? At least two arguments can be given to nullify that suspicion. First, most everyday sounds are so familiar to the listener that they can be hardly reproduced by a model accurately enough to cheat the hearing system, if not the subject. Second, art must not be used to mystify fair research results, as it happens sometimes in this field.
To put it plainly, we would say that models that simulate everyday listening, including those reproducing the spatial location of a sound source, those synthesizing ecological sounds, and more generally those adding some sense of presence to a virtual scenario, are prone to more criticism and severe evaluation compared, for instance, to models for the synthesis of hybrid sounds or for the automatic generation of scores.
On the other hand most of those actually doing research in sound are (or were) sound practitioners or expert music listeners, if not musicians, composers or professionals in the musical field. Those people ought to consider a different research point of view, where quality is evaluated by means of psychophysical tests, and musical sounds are substituted by ecological events such as bounces, footsteps, rolling, crushing and breaking sounds.
This is not an easy perspective to adopt, especially for those having a musical background. It is opinion of the author that accepting this new perspective will be a key step in helping the science of sound to achieve a prominent position among the most respected scientific disciplines.
Pordenone, February 2nd, 2003.
The chapters it contains notably review advances that were achieved within the multidisciplinary European project Natural Interactive Walking. Through the development of new technologies for enhancing spaces, floors, and footwear in ways that allow them to provide simulated experiences of attributes of everyday walking surfaces, the research covered here attempts to enable the designers of new technological systems to engender a real sense of ``being there'', particularly through the use of data coming from the haptic (touch) and auditory (sound) perceptual channels, or in tandem with them. It illustrates how knowledge about the ways that users experience their surroundings during walking can be used to create perceptually rich and plausible experiences of walking in diverse natural and man-made environments.
This work may lead to radically new approaches to interaction with digital information, for example in airports, railway stations, public urban spaces, or in virtual environments used for immersive training purposes. It could, for example, be applied to the creation of intuitive navigation aids, such as landmarking, guidance to locations of interest, ``eyes-free'' signaling, and warning about obstacles and restricted areas. Such research may also open the door to the creation of better assistive tools for visually-impaired and other special-needs users.
Papers by Federico Fontana
In the science of sound there is a clear link between research and art that is less common than other disciplines.
The study of the everyday listening experience and its systematic - and not necessarily artistic, as it has been happening since long time ago: think, for instance, of Luigi Russolo's Intonarumori - inclusion as a branch of research in sound is quite recent. Perhaps this new branch has not yet been totally accepted by some research communities in the field. Obviously, during most of their lifetime humans use the hearing system to acquire information from the external world, but they spend little time appreciating the acoustical subtleties of state of the art technology such as high-end sound synthesis and reproduction equipment. Likewise, most of the researchers working in other disciplines focus on solving problems that, directly or not, concern practical matters; rarely do they deal with the artistic derivations of their research.
Do we choose to do study in the everyday listening spectrum only because our methods of investigation cannot deal with the artistic level of the hearing experience? At least two arguments can be given to nullify that suspicion. First, most everyday sounds are so familiar to the listener that they can be hardly reproduced by a model accurately enough to cheat the hearing system, if not the subject. Second, art must not be used to mystify fair research results, as it happens sometimes in this field.
To put it plainly, we would say that models that simulate everyday listening, including those reproducing the spatial location of a sound source, those synthesizing ecological sounds, and more generally those adding some sense of presence to a virtual scenario, are prone to more criticism and severe evaluation compared, for instance, to models for the synthesis of hybrid sounds or for the automatic generation of scores.
On the other hand most of those actually doing research in sound are (or were) sound practitioners or expert music listeners, if not musicians, composers or professionals in the musical field. Those people ought to consider a different research point of view, where quality is evaluated by means of psychophysical tests, and musical sounds are substituted by ecological events such as bounces, footsteps, rolling, crushing and breaking sounds.
This is not an easy perspective to adopt, especially for those having a musical background. It is opinion of the author that accepting this new perspective will be a key step in helping the science of sound to achieve a prominent position among the most respected scientific disciplines.
Pordenone, February 2nd, 2003.
The chapters it contains notably review advances that were achieved within the multidisciplinary European project Natural Interactive Walking. Through the development of new technologies for enhancing spaces, floors, and footwear in ways that allow them to provide simulated experiences of attributes of everyday walking surfaces, the research covered here attempts to enable the designers of new technological systems to engender a real sense of ``being there'', particularly through the use of data coming from the haptic (touch) and auditory (sound) perceptual channels, or in tandem with them. It illustrates how knowledge about the ways that users experience their surroundings during walking can be used to create perceptually rich and plausible experiences of walking in diverse natural and man-made environments.
This work may lead to radically new approaches to interaction with digital information, for example in airports, railway stations, public urban spaces, or in virtual environments used for immersive training purposes. It could, for example, be applied to the creation of intuitive navigation aids, such as landmarking, guidance to locations of interest, ``eyes-free'' signaling, and warning about obstacles and restricted areas. Such research may also open the door to the creation of better assistive tools for visually-impaired and other special-needs users.