Chapter 4 Perception of Vibrotactile Cues in Musical Performance Federico Fontana, Stefano Papetti, Hanna Järveläinen, Federico Avanzini and Bruno L. Giordano Abstract We suggest that studies on active touch psychophysics are needed to inform the design of haptic musical interfaces and better understand the relevance of haptic cues in musical performance. Following a review of the previous literature on vibrotactile perception in musical performance, two recent experiments are reported. The first experiment investigated how active finger-pressing forces affect vibration perception, finding significant effects of vibration type and force level on perceptual thresholds. Moreover, the measured thresholds were considerably lower than those reported in the literature, possibly due to the concurrent effect of large (unconstrained) finger contact areas, active pressing forces, and long-duration stimuli. The second experiment assessed the validity of these findings in a real musical context by studying the detection of vibrotactile cues at the keyboard of a grand and an upright piano. Sensitivity to key vibrations in fact not only was highest at the lower octaves and gradually decreased toward higher pitches; it was also significant for stimuli having spectral peaks of acceleration similar to those of the first experiment, i.e., below the standard sensitivity thresholds measured for sinusoidal vibrations under passive touch conditions. F. Fontana (B) Dipartimento di Scienze Matematiche, Informatiche e Fisiche, Università di Udine, via delle Scienze 206, 33100 Udine, Italy e-mail: federico.fontana@uniud.it S. Papetti · H. Järveläinen ICST—Institute for Computer Music and Sound Technology, Zürcher Hochschule der Künste, Pfingsweidstrasse 96, 8005 Zurich, Switzerland e-mail: stefano.papetti@zhdk.ch H. Järveläinen e-mail: hanna.jarvelainen@zhdk.ch F. Avanzini Dipartimento di Informatica, Università di Milano, via Comelico 39, 20135 Milano, Italy e-mail: federico.avanzini@di.unimi.it B. L. Giordano Institut de Neurosciences de la Timone UMR 7289, Aix-Marseille Université-Centre National de la Recherche Scientifique, 13005 Marseille, France e-mail: brungio@gmail.com © The Author(s) 2018 S. Papetti and C. Saitis (eds.), Musical Haptics, Springer Series on Touch and Haptic Systems, https://doi.org/10.1007/978-3-319-58316-7_4 49 50 F. Fontana et al. 4.1 Introduction For what we have seen in Chap. 3, the somatosensory system relies on input from receptors that operate within deformable human tissues. One solution for measuring their activity precisely is to keep those tissues free from any kinematic perturbation. Such experiments—in which subjects were typically stimulated with vibrations at selected areas of their skin while remaining still—have set the roots of the psychophysics of passive touch. However, as Gibson observed in 1962, “passive touch involves only the excitation of receptors in the skin and its underlying tissue,” while “active touch involves the concomitant excitation of receptors in the joints and tendons along with new and changing patterns in the skin” [24]. This observation suggests that the psychophysics of active touch may exhibit relevant differences from the passive case. Furthermore, a systematic investigation of active touch psychophysics presents additional practical difficulties in experimental settings due to interactivity, which seems to motivate the current lack of results in the field. Even if we assume a small and well-defined vibrating contact at the fingertip, any change in this contact—as typically found in finger actions such as sliding or pressing—gives rise to new normal and longitudinal forces acting on the skin and to different contact areas. Such side-effects are indeed known to alter the tactile percept [9, 10, 28, 34, 36, 54]. The surrounding skin regions, which contribute to tactile sensations, are also dynamically affected by such changes and by the patterns of vibrations propagating across them [49]. The perception of vibrations generated by musical instruments during playing does not make an exception to the above mechanisms. In fact, the respective experimental scenario is conceptually even more complicated and technically challenging. While in general tactile stimuli may be controlled reasonably well in active touch psychophysics experiments, when considering instrumental performance one has to take into account that vibrations are elicited by the subjects themselves while playing and that concurrent auditory feedback may affect tactile perception [30, 46, 50, 59]. As explained in Chap. 2, a tight closed loop is established between musicians and their instruments during performance. Experimentation on active touch in the context of musical performance hypothesizes that tactile feedback affects such interaction in a number of ways and eventually has a role in the production of musical sounds. 4.1.1 Open-Loop Experimentation The study of haptic properties of musical instruments outside of the musician– instrument interaction (i.e., in open loop) conceptually simplifies the experimental design, while effectively preparing the ground for further studies in closed loop. The violin, due to its intimate contact with the player, represents one of the most fascinating instruments for researchers in musical haptics. A rich literature has grown 4 Perception of Vibrotactile Cues in Musical Performance 51 to explain the physical mechanisms at the base of its range of expressive features [60]. However, the mechanical coupling of the violin with the performer is strong, so that its vibratory response measured in free-suspension conditions cannot fully represent the vibrotactile cues generated by the instrument when in use [38]. The vibratory response of the piano is relatively easier to assess, as the instrument’s interface with the musician is limited to the keyboard and pedals. Furthermore, the mass of the piano is such that the mechanical coupling with the performer’s limbs cannot affect its vibrations significantly. However, pianos couple with the floor; hence, vibrations can reach the pianist’s body through it and the seat. Piano vibrations have been carefully studied by researchers in musical acoustics, who measured them mainly at the strings or soundboard [51]. In contrast, keyboard vibrations as conveyed to the player have been less researched. In the early 1990s, Askenfelt and Jansson performed extensive measurements on several stringed instruments, including the double bass, violin, guitar, and piano [4]. Overall, vibration amplitude was measured above the standard sensitivity thresholds for passive touch [54], suggesting a role for tactile feedback at least in conveying a feeling of a resonating and responding object. This conclusion, though, was mitigated for the piano keyboard, whose vibration amplitude was mostly found below such thresholds and hence supposedly perceptually negligible. More recently, Keane and Dodd reported significant differences between upright and grand piano keyboard vibrations, while hypothesizing a perceptual role of vibrotactile feedback during piano playing [32]. Other classes of instruments, such as aerophones, likely offer measurable vibrotactile cues to the performer, but to our knowledge a systematic assessment of the perceivable effects of such vibratory feedback has not been yet conducted. Percussion instruments, on the other hand, respond with a strong kinesthetic feedback that is necessary for performers to rearm their limbs instantaneously, and for executing rebounds and rolls without strain. In this regard, Dahl suggested that the interaction of a drumstick or a hand with the percussion point happens so rapidly, that it does not seem possible for a performer to adjust a single hit simultaneously with the tactile feedback coming from it [11]. The percussive action, in other words, appears to be purely feed-forward as far as multiple hit sequences are not considered (see also Sect. 2.2 in this regard). Finally, electroacoustic and electronic instruments do not seem able to generate relevant vibrotactile feedback, unless a loudspeaker system is mounted directly aboard them. 4.1.2 Experiments with Musicians Once an instrument has been identified as a source of relevant tactile cues, their potential impact on musical performance and produced musical sound may be tested with musicians. The inclusion of human participants, however, introduces several 52 F. Fontana et al. issues. To start with, as mentioned above, interactive contexts such as the musical one prevent the implementation of experiments with full control over contact areas and forces, or the generation of vibratory stimuli. Also, acoustical emissions from musical instruments engage musicians in a multisensory process where the tactile and auditory channels are entangled at different levels, ranging from the peripheral and central nervous system, to cross-modal perceptual and cognitive processes. Tactile and auditory cues start to interfere with each other in the middle ear. Vibrations in fact propagate from the skin to the cochlear system through bone and tendon conduction, via several pathways [12]. Especially if an instrument is played close to the ear (e.g., a violin) or enters into contact with large areas of the body (e.g., a cello or double bass), such vibrations can reach the cochlea with sufficient energy to produce auditory cues. Cochlear by-products of tactile feedback may be masked by overloading the hearing system with sufficiently loud sound that does not correlate with tactile feedback: Masking noise provided through headphones is often necessary in tactile perception tests [6, 58]. The use of bone-conduction headphones may improve experimental control, as bone-conducted cues could be jammed on their way to the cochlea by vibratory noise transferred to the skull [47]. Even when considering only airborne auditory feedback, earmuffs or earplugs may not provide sufficient cutoff, and uncorrelated masking noise may be needed. The question, then, is how to analyze answers from musicians who had to perform while listening to loud noise. The literature on audio-tactile sensory integration is particularly rich and can help explain possible perceptual synergies or cancellations occurring during this integration [46, 50, 57, 58]. Any tactile interaction experiment that involves musicians should take the aforementioned issues into account. In a groundbreaking study from 2003, Galembo and Askenfelt showed that grand pianos are mainly recognized—and possibly even rated—based on the tactile and kinesthetic feedback offered by their keyboards, more than based on the produced sound [20]. Similarly, in a later study on percussive musical gestures, Giordano et al. showed that haptic feedback has a bigger influence on performance than on auditory cues [25]. Focusing on tactile cues alone, Keane and Dodd reported significant preference of pianists for an upright instrument whose keybed had been modified to decrease vibrations intensity at the keyboard, thus making them comparable to those produced by a grand piano [31, 32]. In parallel, some authors of the present chapter augmented a digital piano with synthesized vibrotactile feedback, showing that it significantly modified the performer’s preference [16, 18]. In the same period, one of the world’s top manufacturers equipped its flagship digital pianos with vibration transducers making the instruments’ body vibrate while playing [27], thus testifying concrete interest from the industry at least for the aesthetic value of tactile cues. More recently, Wollman et al. showed that salient perceptual features of violin playing are influenced by vibrations at the violin’s neck [59], and Altinsoy et al. found similar results using reproduced vibratory cues [3]. Saitis et al. discussed the influence of vibrations on quality perception and evaluation as manifested in the way 4 Perception of Vibrotactile Cues in Musical Performance 53 that musicians conceptualize violin quality [48]. Further details on the influence of haptic cues on the perceived quality of instruments are given in Chap. 5. 4.1.3 Premises to the Present Experiments Compared to other interfaces of stringed instruments, the piano keyboard is easier to control experimentally, as the performer is only supposed to hit and then release one or more keys with one or more fingers. Other body contacts can be prevented by excluding the use of the pedals. Also, non-airborne auditory feedback—a by-product of the tactile response—can be masked by employing the techniques mentioned above. Furthermore, the sound and string vibrations produced by a key press are in good correspondence with the velocity with which the hammer hits a string [33]. If a keyboard is equipped with sensors complying with the MIDI protocol, then such map is encoded for each key and made available as digital messages. Together, these properties allow the experimenter to (i) record the vibratory response of the keyboard to measurable key actions; (ii) create a database of reproducible action– response relationships; (iii) make use of those data in experiments where pianists perform simple tasks on the keyboard, such as hitting one or few keys. Our interest in the piano keyboard is not only motivated by its relatively easy experimental control: As mentioned above, its tactile feedback measured in open loop was found hardly above the standard vibrotactile sensitivity thresholds [4]. Did this evidence set an end point to the perception of piano keyboard vibrations? This chapter discusses and compares the results of two previously reported experiments on vibrotactile perception in active tasks: The first one conducted in a controlled setting and the other in an ecological, musical setting. The goal was twofold: (i) to assess how finger pressing (similar to a key-press task) affected vibrotactile detection thresholds and (ii) to investigate whether pianists perceive keyboard vibrations while playing. Somewhat surprisingly, in Experiment 1 we found sensitivity thresholds much lower than those previously reported for passive tasks. Experiment 2 demonstrated that pianists do perceive keyboard vibration, with detection rates highest at the lower octaves and gradually decreasing toward higher pitches. Importantly, vibrations at the piano keyboard were also measured with an accelerometer for the conditions used in the experiment: While their intensity was generally lower than the standard thresholds for passive touch, conversely a comparison with the thresholds obtained in Experiment 1 provided a solid explanation to how pianists detected vibrations across the keyboard. These findings suggest that studies on active touch psychophysics are required to better understand the relevance of haptic cues in musical performance and, consequently, to inform the development of future haptic musical interfaces. 54 F. Fontana et al. 4.2 Experiment 1: Vibrotactile Sensitivity Thresholds Under Active Touch Conditions In this experiment, vibrotactile perceptual thresholds at the finger were measured for several levels of pressing force actively exerted against a flat rigid surface [43]. Vibration of either sinusoidal or broadband nature and of varying intensity was provided in return. The act of pressing a finger is indeed a gesture found while performing on many musical instruments (e.g., keyboard, reed, and string instruments) and therefore represents a case study of wide interest for musical haptics. Based on the results reported by several previous studies [9, 10, 28, 34, 36], we expected perceptual thresholds to be influenced by the strength of the pressing force. 4.2.1 Setup A self-designed tabletop device called the Touch-Box was utilized to measure the applied normal force and area of contact of a finger pressing its top surface and to provide vibrotactile stimuli in return. Technical details on the device are given in Sect. 13.3.1. The Touch-Box was placed on a thick layer of stiff rubber, and sound emissions were masked by noise played back through headphones. To minimize variability of hand posture, an arm rest was used. The experiment made use of two vibrotactile stimuli, implementing two different conditions: Band-passed white noise with 48 dB/octave cutoffs at 50 and 500 Hz and a sine wave at 250 Hz. Both stimuli focus around the range of maximal vibrotactile sensitivity (200–300 Hz [55]). During the experiment, stimulus amplitude was varied in fixed steps according to a staircase procedure (see Sect. 4.2.2). Stimulus level was calculated as the RMS value of the acceleration signal, accounting for the power of vibration acceleration averaged across the stimulation time. Pressing force was a within-subject condition with three target levels, covering a range from light touch to hard press, while still being comfortable for participants [13], as well as compatible with forces found in instrumental practice [4]. In what follows, the three force levels are referred to as Low, Mid, and High, which correspond, respectively, to 1.9, 8, and 15 N, with a tolerance of ±1.5 N. 4.2.2 Procedure Twenty-seven subjects participated in the sinusoidal condition, and seventeen in the noise condition. They were 19–39-year old (mean = 26, SD = 4.5), and half of them were music students. The experiment lasted between 35 and 60 min, depending on the participants’ performance, and a 1-minute break was allowed every 5 min to prevent fatigue. 4 Perception of Vibrotactile Cues in Musical Performance 55 85 vibration Acceleration [dB RMS] noise sine 80 75 70 low 5 mid 10 high Pressing force [N] Fig. 4.1 Thresholds measured at three pressing force levels, for sinusoidal and noise vibrations. Error bars represent the standard error of the mean. Figure reprinted from [43] Perceptual thresholds were measured using a one-up-two-down staircase algorithm with fixed step size (2 dB1 ) and eight reversals, and a two-alternative forced choice (2AFC) procedure. The method targets the stimulus level corresponding to a correct detection rate of 70.7% [35], estimated as the mean of the last six reversals of the up-down algorithm. Three staircases were implemented, each corresponding to a target force level, which were presented in interleaved and randomized fashion. Participants were instructed to use their dominant index finger throughout the experiment. A trial consisted of two subsequent finger presses, with vibration randomly assigned to only one of them. The participants’ task was to identify which press contained the vibration stimulus. Before the observation interval began, a LCD screen turning green signaled the stable reaching of the requested force level. 4.2.3 Results As shown in Fig. 4.1, at each pressing force level thresholds for sinusoidal vibration were lower than for noise. For both vibration conditions, higher thresholds (i.e., worse detection performance) were obtained at the Low force condition, while at the other two force levels the thresholds were generally lower. The lowest mean threshold (68.5 dB RMS acceleration) was measured at the High force condition with sinusoidal vibration, and the highest at the Low force condition with noise vibration (83.1 dB)— thus thresholds varied over a wide range across conditions. Individual differences were also large: The lowest and highest individual thresholds differ typically by about 20 dB in each condition. the remainder of this chapter, vibration acceleration values expressed in dB use 10−6 m/s2 as a reference. 1 In 56 F. Fontana et al. Perceptual thresholds were analyzed by means of a mixed ANOVA. A significant main effect was found for type of vibration (F1,41 = 14.64, p < 0.001, generalized η2 = 0.23) and force level (F2,82 = 137.5, p < 0.0001, η2 = 0.35), while the main effect of musical experience was not significant. Post hoc pairwise comparisons with Bonferroni correction (sphericity assumption was not violated in the within-subject force level factor) indicated that the Low force condition differed from both the Mid and High force conditions, for both vibration types (t (82) > 8.85, p < 0.0001 for all comparisons). For noise vibration, the difference between Mid and High force conditions was significant (t (82) = −3.17, p = 0.02), but the respective contrast for sinusoidal vibration was not (t (82) = 1.64, p > 0.05). The difference between sinusoidal and noise vibrations was significant for the Low (t (57.44) = 4.37, p < 0.001) and High (t (57.44) = 4.29, p < 0.001) force conditions, but not for the Mid force (t (57.44) = 1.85, p > 0.05). 4.2.4 Discussion Vibrotactile perceptual thresholds were found in the range 68.5–83.1 dB RMS acceleration—values that are considerably lower than what generally reported in the literature. Maeda and Griffin [36] compared acceleration thresholds from various studies addressing passive touch, finding that most of them are in the range 105–115 dB for sinusoidal stimuli ranging from 100 to 250 Hz. The lowest reported acceleration thresholds are 97–98.5 dB, for contact areas (probe size) ranging from 53 to 176.7 mm2 [1, 2, 15]. It is worth noticing that the widely accepted results by Verrillo [55] report lowest displacement thresholds of approximately −20 dB (re 10−6 m) at 250 Hz, equivalent to about 105 dB RMS acceleration.2 The main result of the present experiment is that vibrotactile sensitivity depends on the applied pressing force. Thresholds were highest at the Low force condition and decreased significantly at both Mid and High force levels. In good accordance with what reported in a preliminary study [44], for noise vibration the lowest threshold was obtained at the Mid force condition, while at the Low and High conditions thresholds were higher, resulting in a U-shaped threshold contour with respect to the applied force. However, as shown in Sect. 13.3.1.4, the spectral centroid of the noise vibration generally shifted toward 300 Hz and higher frequencies for the Mid and High force conditions. Therefore, we suggest that the U-shape of the threshold-force curve might be partially due to the response of the Pacinian channel, which shows a U-shaped contour over the frequency range 40–800 Hz with maximum sensitivity in the 200–300 Hz range [8]. Conversely, for sinusoidal vibrations at 250 Hz, mean dB thresholds decreased roughly logarithmically for increasing pressing forces (see Fig. 4.1). This simpler trend may be due to the more consistent behavior of our system 2 For a sinusoidal vibration signal s, it is straightforward to convert between acceleration and displacement: sacc = sdispl · (2π f)2 , where f is the frequency. Also, RMS values can be obtained √ directly from peak values: sRMS = speak / 2. 4 Perception of Vibrotactile Cues in Musical Performance 57 when reproducing simpler sinusoidal vibrations (see Sect. 13.3.1.4). An improved version of the Touch-Box would be needed to test whether a similar trend can be found when noise stimuli are reproduced more linearly for varying pressing forces. Further studies are needed to precisely assess how vibratory thresholds might be affected by passive forces of strength equivalent to the active forces used in the present study. However, since the Low condition in our experiment was already satisfied by applying light pressing force (the measured mean is about 1.49 N), it may be compared to studies addressing passive static forces. Craig and Sherrick [10] found that increasing static force on the contactor produces an increase in vibrotactile magnitude. They considered vibration bursts at 20, 80, and 250 Hz lasting 1240 ms, contact areas up to 66.3 mm2 , and static forces of about 0.12 and 1.2 N. Harada and Griffin [28] used a contact area of 38.5 mm2 and found that forces in the range 1–3 N led to significant lowering of thresholds by 2–6 dB RMS at 125, 250, and 500 Hz. The lowest thresholds reported are however around 100 dB RMS acceleration. On the other hand, Brisben et al. [9] reported that passive static contact forces from 0.05 to 1.0 N did not have an effect on thresholds. However, with only four participants, the statistics of those results are not robust. Nevertheless, the authors suggested that extending these investigations to higher forces, as found in everyday life, would be important. They also hypothesized that increasing the force beyond 1–2 N could lower thresholds by better coupling of vibrating surfaces to bones and tendons, which could result in more effective vibration transmission to distant Pacinian corpuscles. That might also contribute to explain the generally lower thresholds that we found for higher forces. In our study, force level was found strongly correlated to contact area, resulting in larger areas for higher forces, which clearly contributed to further lowering perceptual thresholds [43]. Only a few related studies are found in the literature dealing with non-sinusoidal stimuli. Gescheider et al. [22] studied difference limens for the detection of changes in vibration amplitude, with either sinusoidal stimuli at 25 or 250 Hz or narrowband noise with spectrum centered at 175 Hz and 24 dB/octave falloff at 150 and 200 Hz (contact area 2.9 cm2 ). They found that the nature of the stimuli had no effect on difference limens. Wyse et al. [61] conducted a study with hearing-impaired participants and found that, for complex stimuli and whole hand contact (area of about 50–80 cm2 ), the threshold at 250 Hz was 80 dB RMS acceleration, i.e., comparable with our results, especially in the Low force condition. In that study, it is hypothesized that the temporal dynamics of spectrally complex vibration might play a key role in detecting vibrotactile stimulation. In our case, however, the stimuli had no temporal dynamics. Sinusoidal stimuli resulted in lower RMS acceleration thresholds as compared to noise vibration. This may be explained intuitively by considering that equivalent RMS acceleration values for sinusoidal and noise stimuli actually result in a similar amount of vibration power being concentrated at 250 Hz (a frequency characterized by peak tactile sensitivity [55]), or spread across the 50–500 Hz band, respectively. This explanation is supported by the findings by Young et al. [64], who reported lower thresholds produced by sinusoidal stimuli than spectrally more complex signals (square and ramp waves). 58 F. Fontana et al. The Pacinian channel, targeted by this study, is capable of spatial summation. Previous studies [21, 55] showed that for contact areas between 2 and 510 mm 2 at the thenar eminence of the hand, and for frequencies in the 40–800 Hz range, displacement thresholds decrease by approximately 3 dB with every doubling of the area. Intuitively, a reason for that is that the number of stimulated skin receptors increases with larger contact areas. In the present experiment, the interactive nature of the task resulted in high variability of the contact area [43]. The mean contact areas measured in the experiment were in the range 103–175 mm2 , contributing to explaining the reported enhanced sensitivity. The Pacinian channel is also sensitive to temporal summation, which lowers sensitivity thresholds and enhances sensation magnitude [21]. Verrillo [53] found that thresholds decrease for stimuli at 250 Hz for increasing duration up to about 1 s, when delivered through a 2.9 cm2 contactor to the thenar eminence of the hand. Gescheider and Joelson [23] examined temporal summation with stimulus intensities ranging from the threshold to 40 dB above it: For 80 and 200 Hz stimuli, peak displacement thresholds were lowered by up to about 8 dB for duration increasing from 30 to 1000 ms. The present study made use of stimuli lasting 1.5 s, which likely contributed to enhancing vibrotactile sensitivity. Large inter-individual differences in sensitivity were found in our experiment, which we could not fully explain by contact area or age. However, this observation is in accordance with other studies [1, 29, 36, 41]. Sources for large variations in sensitivity may be many. While exposure to vibration is a known occupational health issue and can cause acute impairment of tactile sensitivity [28], experience in conditions similar to the present experiment seemed a possible advantage. Therefore, we further analyzed the performance of musician participants, who are often exposed to vibrations when performing on their instruments: Indeed, musicians’ mean threshold in the Low force condition was about 3 dB lower than non-musicians’, but there was no significant difference at the other force levels. Overall, enhanced sensitivity in musicians—previously observed by other authors [14, 45, 65]—could not be confirmed. By considering actively applied forces and unconstrained contact of the finger pad, the present study adopted a somewhat more ecological approach [24] as compared to the studies mentioned above. An analogous approach was adopted by Brisben et al. [9], who studied vibrotactile thresholds in an active task that required participants to grab a vibrating cylinder. While the exerted forces were not measured, in accordance with our results much lower thresholds were reported than in the most previous literature: At 150 and 200 Hz, the average displacement threshold was 0.03 µm peak (down to 0.01 µm in some subjects), which is equivalent to RMS acceleration values of 85.5 dB at 150 Hz, and 90.5 dB at 200 Hz. The authors suggested that such low figures could be due to the multiple stimulation areas on the hand involved in grabbing the vibrating cylinder, the longitudinal direction of vibration, and the force exerted by the participants. A few studies report that active movement results in lower sensitivity thresholds [63] or better percept possibly due to the involvement of planning and additional cognitive load as compared to the passive case [52]. 4 Perception of Vibrotactile Cues in Musical Performance 59 Despite its partially ecological setting, this experiment kept control over the generation of sinusoidal and noise vibrations, with focus on the region of maximal human vibrotactile sensitivity (200–300 Hz). Vibratory cues at the piano keyboard, however similar in form to the respective tones, are more complex than either of the conditions in Experiment 1 and are likely to be perceived differently depending on the type of touch and the number of depressed keys. The following experiment tested first vibration detection in a piano-playing task, and second whether active touch sensitivity threshold curves of Experiment 1 could predict the measured results. 4.3 Experiment 2: Vibration Detection at the Piano Keyboard During Performance A second experiment investigated vibrotactile sensitivity in a musical setting [19]. Specifically, the goal was to measure the ability of pianists to detect vibration at the keyboard while playing. Vibration detection was measured for single and multiple tones of varying pitch, duration, and dynamics. 4.3.1 Setup The experiment was performed at two separate laboratories using similar setups, centered around two Yamaha Disklavier pianos: A grand model DC3 M4 and an upright model DU1A with control unit DKC-850. The Disklaviers are MIDI-compliant acoustic pianos equipped with sensors for recording performances and electromechanical motors for playback. They can be switched from normal operation to a “silent mode.” In the latter modality, the hammers do not hit the strings and therefore the instruments neither resound nor vibrate, while their MIDI features and other mechanical operations are left unaltered. The two setups are shown in Fig. 4.2. During the experiment, the normal and silent modes were switched back and forth across trials, letting participants receive respectively either natural or no vibrations from the keys. In both configurations, participants were exposed to the same auditory feedback produced by a physical modeling piano synthesizer (Modartt Pianoteq), set to simulate either a grand or an upright piano, and driven in real time by MIDI data sent by the Disklaviers. The synthesized sound was reproduced through Sennheiser HDA200 isolating reference headphones (grand piano) or Shure SE425 earphones (upright piano). In the latter case, 3M Peltor X5A earmuffs were worn on top of the earphones for additional isolation. Preliminary testing confirmed that through these setups the Disklaviers’ operating modes (normal or silent) were indistinguishable while listening to the piano synthesizer from the performer’s seat position, meaning that any acoustic sound coming from the pianos in normal mode was fully masked. 60 F. Fontana et al. Fig. 4.2 The two Disklavier setups used in the experiment. Left: Yamaha DC3 M4 grand piano. Right: Yamaha DU1A upright piano. Figure adapted from [19] The loudness and dynamic response of the piano synthesizer were preliminary calibrated to match those of the corresponding Disklavier model in use (details are given in Sect. 13.3.2). Participants could sense the instrument’s vibration only through their fingers on the keyboard. Other sources of vibration were excluded: The pedals were made inaccessible, while the stool, the player’s feet, and the piano were isolated from the floor by various means [17]. Vibration measurements confirmed that, as a result of the mechanical insulation, playing the piano did not cause vibrations at the player’s seat exceeding the noise floor in the room. The experiment was conducted under human control, with the help of software developed in the Pure Data environment, which was used to: (i) read computergenerated playlists describing the experimental trials; (ii) set the Disklavier’s playing mode accordingly; (iii) check if the requested tasks were executed correctly; (iv) record the participants’ answers. 4.3.2 Procedure Sensitivity was measured at six A tones of different pitch ranging from A0 to A5, chosen after a pilot study [17], reporting a significant drop in detection above A5. Tone duration was either “long” (8 metronome beats at 120 BPM) or “short” (2 beats), and dynamics either “loud” (mf to ff, corresponding to MIDI key velocities in the range 72–108) or “soft” (p to mp, key velocities 36–54). In addition to single tones, participants were requested to play three-tone clusters around D4 and D5. The experiment consisted of two parts: In part A, participants played long and loud single tones; in part B, tone dynamics and duration were modified so as to make the detection task potentially harder in the low range, where vibrations should be most easily perceived [17]. Additionally, by extending the contact area, the note clusters 4 Perception of Vibrotactile Cues in Musical Performance 61 Table 4.1 Factors and conditions in the piano experiment Part A Part B # of keys 1 1 Pitch A0/A1/A3/D4/A4/D5/A5 A0/A1 3 CDE4/CDE5 Playing style Long and loud Short and loud/ Long and soft Long and loud were expected to facilitate detection in the high range, where sensitivity should be low [17]. The conditions are summarized in Table 4.1. The experiment followed a 2AFC (yes/no) procedure, which required participants to report whether they had detected vibrations during a trial or not. Each condition was repeated eight times in normal mode and eight times in silent mode, in randomized order. However, part A was performed before part B. Participants were instructed to use their index fingers for single keys or fingers 2-3-4 for chords and to play pitches lower than the middle C with their left hand and the rest with the right hand. Fourteen piano students participated in the upright piano condition, and fourteen in the grand piano condition. Their average age was 27 years and they had in average 15 years of training, mainly on the acoustic piano. 4.3.3 Results Sensitivity index d ′ , as defined in signal detection theory [26], was computed for each subject and condition as follows: d ′ = Z (hits) − Z (false alarms), where Z ( p) is the inverse of the Gaussian cumulative distribution function, hits is the proportion of “yes” responses with vibrations present, and false alarms is the proportion of “yes” responses with vibrations absent. Thus, a proportion of correct responses p(c) = 0.69 corresponds to d ′ = 1 and chance performance p(c) = 0.50 to d ′ = 0. Perfect proportions 1 and 0 would result in infinite d ′ and were therefore corrected by (1 − 1/16) and (1/16), respectively [26]. Results of part A are presented at the top of Fig. 4.3: Sensitivity was highest in the lower range and decreased toward higher pitches. At A4 (440 Hz), vibrations were still detected with mean d ′ = 0.84, while at D5 (587 Hz) and A5, performance dropped to chance level. A mixed ANOVA indicated a significant main effect of pitch (F(6, 156) = 26.98, p < 0.001). The results for the upright and the grand piano did not differ significantly, nor was there a significant interaction of pitch and piano type. The Mauchly test showed that sphericity had not been violated. 62 F. Fontana et al. Grand Upright d' 2 1 0 A0 A1 A3 D4 A4 D5 A5 Pitch 3 Grand Upright d' 2 1 A5 D5 A4 hord C D4 A3 A1soft A1short A1 A0soft A0short A0 0 Pitch Fig. 4.3 Sensitivity d ′ in part A (top) or parts A and B (bottom). Error bars represent the standard error of the mean [40]. Chance performance (d ′ = 0) is represented by the dashed line. Figures reprinted from [19] 4 Perception of Vibrotactile Cues in Musical Performance 63 The results were collapsed over upright and grand pianos, and a trend analysis was conducted. A linear trend was significant (t (156) = −12.3, p < 0.0001), indicating that as pitch increases, sensitivity to vibrations decreases. Results from parts A and B are presented together at the bottom of Fig. 4.3, showing small differences in mean sensitivity between normal, soft, and short conditions. However, none of the contrasts between long and short duration or loud and soft dynamics at A0 or A1 was significant. The difference was more notable between clusters and single notes: For the cluster CDE4, sensitivity was significantly higher than for the isolated note D4 (t (294) = 5.96, p < 0.0001), whereas the much smaller difference between D5 and the cluster CDE5 was not significant. Even considering the possible effect of learning between part A and B (average sensitivity at pitches A0 and A1 was 0.23 higher in part B), the result suggests that at D4, playing a cluster of notes facilitates vibration detection. 4.3.4 Vibration Characterization In order to gain further insight into the results, vibration signals at the keyboard were measured on both the grand and upright Disklaviers. An in-depth description of the measurements and related issues is given in Sect. 13.3.2.2. For convenience, only essential details are reported here. Vibration signals were acquired for different MIDI velocities at each of the 88 keys of the Disklavier pianos via a measurement accelerometer and recorded as audio signals. A digital audio sequencer software was used to record vibration signals, while reproducing MIDI tracks that played back each single key of the Disklaviers. Additional MIDI tracks were used to play CDE4 and CDE5 clusters, while vibration was recorded with the accelerometer attached to the respective C, D, and E keys in sequence. The MIDI velocities were chosen to cover the entire dynamics reproducible by the Disklaviers’ motors. Acceleration signals had a large onset in the attack, corresponding to the initial fly of the keys followed by their impact against the keybed. Figure 4.4 shows a typical attack, recorded from the grand Disklavier playing the A2 note at MIDI velocity 12. These onsets, appearing in the first 200–250 ms, are not related to the vibratory response of the keys and were therefore manually removed from the samples. Acceleration values in m/s2 were computed from the acquired signals by making use of the nominal sensitivity parameters of the audio interface and the accelerometer. Similarly to what was done by Askenfelt and Jansson [4], the spectra of the resulting acceleration signals were compared to Verrillo’s reference vibrotactile sensitivity curve [55]. Note that this curve reports sensitivity as the smallest, frequency-dependent displacement A( f ) (in meters) of a sinusoidal stimulus s(t) = A( f ) sin(2π f t) that is detected at the fingertip. Therefore, a corresponding acceleration curve was computed from the original displacement curve in order to compare with our acceleration signals. Thanks to the sinusoidal nature of the stimuli employed by Verrillo, the corresponding acceleration signal could be found 64 F. Fontana et al. Fig. 4.4 Attack of the acceleration signal recorded for note A2, MIDI velocity 12, grand Disklavier. Figure reprinted from [19] 160 140 Acceleration magnitude [dB] Fig. 4.5 Vibration spectrum of A0 played with ff dynamics (MIDI velocity 111) on the upright Disklavier, represented as magnitude acceleration in dB. The vertical dotted line shows the nominal fundamental frequency f0 = 27.5 Hz. The dashed curve represents vibrotactile acceleration thresholds at the fingertip adapted from Verrillo [55]. Figure reprinted from [19] 120 100 80 60 40 20 0 10 2 3 10 Frequency [Hz] analytically as s̈(t) = −A( f ) · (2π f )2 sin(2π f t). Consequently, the acceleration threshold curve A( f ) · (2π f )2 was used for comparison to our signals. Confirming the results by Askenfelt and Jansson [4], no spectral peaks were found to exceed the acceleration threshold curve, even for notes played with high dynamics. To exemplify this, Fig. 4.5 shows the spectrum of the highest dynamics of the note that participants detected with the highest sensitivity (part A), i.e., A0 played at MIDI velocity 111, along with the threshold curve. Since Verrillo’s thresholds cannot explain the results of Experiment 2, RMS acceleration values were computed in place of spectral peak amplitudes, in analogy with Experiment 1. Vibration signals were first processed with a specifically designed low-pass filter to shape stimuli according to human vibrotactile band [19]. RMS 4 Perception of Vibrotactile Cues in Musical Performance 65 Fig. 4.6 RMS acceleration values of keys played as in part A (top) or parts A and B (bottom). The horizontal lines represent (min/max) vibrotactile thresholds as measured in Experiment 1 for noise and sinusoidal stimuli over a range of active pressing forces. Figure adapted from [19] values in dB were then extracted from the filtered signals over time windows equal to the lengths of the stimuli, that is 1 s for short and 4 s for long trials. Figure 4.6 shows the resulting RMS values for parts A and B, respectively, together with the RMS thresholds of vibration reported in Experiment 1. A comparison of the RMS acceleration values and perceptual thresholds for noise shown in these figures against the sensitivity curves of Fig. 4.3 suggests that RMS values of broadband stimuli have more potential to explain the results of Experiment 2. 66 F. Fontana et al. 4.3.5 Discussion The results presented in the previous section show that sensitivity to key vibrations is highest in the lowest range and decreases toward higher pitches. Vibrations are clearly detected in many cases where the vibration acceleration signals hardly reached typical thresholds found in the literature for sinusoidal stimuli. The literature on the detection of complex stimuli provides support to our results, although it does not explain them completely. As already discussed in Sect. 4.2.4, Wyse et al. [61] report RMS acceleration threshold values at 250 Hz corresponding to 80 dB, a value compatible with our results. However, the characteristics of those stimuli may have occasionally produced significant energy at lower frequencies, causing the thresholds to lower once they were presented to the whole hand. The pianist receives the initial transient when the hammer hits the string; then, the vibration energy promptly decreases and its partials fade each with its own decay curve. The initial peak may produce an enhancement effect similar to those measured by Verrillo and Gescheider limited to sinusoids [56] and hence contribute to sensitivity. As discussed earlier, the P-channel is sensitive to the signal energy, while is not able to recognize complex waveforms. Loudness summation instead occurs when vibration stimulates both the Pacinian and non-Pacinian (NP) channels, lowering the thresholds accordingly [7, 37, 56]. In our experiment, summation effects were likely to occur when the A0 key and, possibly, the A1 key were pressed. From A3 on, only the P-channel became responsible for vibration perception. Figure 4.3 seems to confirm these conclusions, since they show a pronounced drop in sensitivity between A1 and A3 in both parts of Experiment 2. As Fig. 4.6 demonstrates, this drop is only partially motivated by a proportional attenuation of the vibration energy in the grand piano, while it is not motivated at all in the upright piano. Hence, it is reasonable to conclude that the NP-channel played a perceptual role until A3. Beyond that pitch, loudness summation effects ceased. In analogy with Experiment 1, the results of this experiment also suggest the occurrence of spatial summation effects [10] when a cluster of notes, whose fundamentals overlap with the tactile band, is played instead of single notes. As Fig. 4.3 (bottom panel) shows, playing the cluster in the fourth octave boosted the detection in that octave, whereas the same effect did not occur in the fifth octave. Unlike Experiment 1, this summation originates from multifinger interaction rather than varying contact areas in single-finger interaction. This evidence opens an interesting question about the interaction of complex vibrations reaching the fingers simultaneously. Measurements of cutaneous vibration propagation patterns in the hand resulting from finger tapping show, however, an increase in both intensity and propagation distance with the number of fingers involved [49], which may partially explain the increased sensitivity we observed. 4 Perception of Vibrotactile Cues in Musical Performance 67 Unlike Experiment 1, where uni-modal tactile stimuli were used, here we employed bimodal audio-tactile stimuli. Therefore, the possibility of cross-modal amplification effects needs to be shortly discussed, even though Experiment 2 did not investigate this aspect. As discussed earlier, previous studies on cross-modal integration effects [46, 58] support the concrete possibility that an audible piano tone, whose vibratory components are a subset of the auditory components, helps detect a tactile signal near threshold. Although in our case the sound came from a synthesizer, both the auditory and tactile signals shared the same fundamental frequency of the piano tone, and furthermore the first partials were close to each other, respecting the hypothesis of proximity in frequency investigated by Wilson [58]. We did not test a condition in which subjects played the piano in normal mode in the absence of auditory feedback, or using sound uncorrelated with vibration (e.g., white noise). Although that may provide significant data about the effective contribution of auditory cues to vibration detection on the piano, a different experimental setup should be devised. Other cross-modal effects that may have instead contributed to impair the detection [62] should be considered as minor with respect to the spectral compatibility and temporal synchronization of the audio-tactile stimulus occurring when a piano key was pressed. Yet another relevant difference with Experiment 1 is that in this case the pressing forces exerted by pianists were unknown and most likely not constant throughout a single trial. The maximum and minimum sensitivity thresholds lines in Fig. 4.6, which report the results of Sect. 4.2.3, correspond to constant pressing forces of 1.9 and 8 N for noise vibration, and 1.9 and 15 N for 250 Hz sinusoidal vibration. These force values occur when piano keys are hit at dynamics between pp and f, with negligible difference between struck and pressed touch style [20, 33]. Conversely, ff dynamics require stronger forces up to 50 N [4]. In Experiment 2, it seems reasonable to assume that pianists initially pressed the key according to the dynamics required by the trial and then, once the key had reached the keybed, accommodated the finger force on a comfortable value while attending the detection process. If our participants adapted finger forces toward the range mentioned above, then their performance in this experiment would fall in between the results for sinusoidal and noise stimuli in Experiment 1. Experiment 1 additionally found that, when using low finger force, musicians on average exhibit slightly better tactile acuity than non-musicians. Even if this difference was not significant, our participants could have reduced the finger force only after starting a trial that required loud dynamics, while leaving the force substantially unvaried during the entire task in the other cases. This behavior seems indeed quite natural. The hypothesis that vibrotactile sensitivity to RMS acceleration falls in between the thresholds for 250 Hz sine wave and filtered noise is coherent with the temporal and spectral characteristics of the stimuli: Right after its initial transient, a piano tone closely resembles a decaying noisy sinusoid. For instance, it can be simulated by employing several hundreds of damped oscillators whose outputs are subsequently filtered using a high-order transfer characteristic [5]. A remaining question is whether 68 F. Fontana et al. the RMS acceleration values of filtered noise plotted in Fig. 4.6 explain our thresholds sufficiently, or if there is a need to discuss them further. Other elements in favor of further discussion are the mentioned potential existence of a cross-modal amplification and evidences of superior tactile acuity in musicians [65]. 4.4 Conclusions We have given an introduction to the role of active touch in musical haptic research. A closed loop between musicians and their instrument during performance poses a major challenge to experimental setups: While playing, musicians generate themselves the vibrotactile feedback and are at the same time influenced by the produced sound. To discuss the possible links between music performance tasks and basic active touch psychophysics, we presented two experiments, one in a controlled and one in an ecological setting, showing evidence that pianists perceive keyboard vibrations with sensitivity values resembling those obtained under controlled active touch conditions. Overall, the results presented here suggest that research on active touch in musical performance may prove precious to understand the role, mechanisms, and prospective applications of active touch perception also outside the musical context. An example application that seems at immediate reach of current tactile interfaces is to create illusory effects of loudness change by varying the intensity of vibratory feedback [39, 42]. Although interesting and necessary, our results represent only a premise for further research activities aimed at precisely understanding the role of tactile feedback during piano playing. Exploratory experiments have already been performed in an attempt to understand whether changes in the “timbre” of tactile feedback may determine equivalent auditory sensations. Some results in this regard are presented in Sect. 5.3.2.2. If confirmed, after excluding the influence of non-airborne sonic cues on auditory perception, such results would imply the ability of the tactile and auditory systems to interact so as to form a wider, multimodal notion of musical timbre, for which some partial evidence has been found in musicians [59] and non-musicians [47]. Several questions related to the role of tactile feedback in musical performance remain open. For instance, feedback from percussion instruments is likely to define strong patterns of skin vibration extending far beyond the interaction point. The propagation of vibration across the skin has been recent object of research having potentially interesting haptic applications outside the musical context [49]. It cannot be excluded that percussionists control their playing by testing specific wide-area tactile patterns they learned, and then retained in the somatosensory memory after years of practice with their instrument: Some sense of unnatural interaction with the instrument otherwise 4 Perception of Vibrotactile Cues in Musical Performance 69 should not be experienced by drummers and percussionists when they play rubber pads and other digital interfaces. Furthermore, while it is not precisely known how wind instrument players make use of the vibrations transmitted by the mouthpiece, digital wind controllers like the Yamaha WX series never achieved wide popularity, possibly also due to their unnatural haptic feedback. Acknowledgements The authors wish to thank Francesco and Valerio Zanini for recording piano vibrations and contributing to perform the piano experiment. This research was pursued as part of project AHMI (Audio-Haptic modalities in Musical Interfaces, 2014–2016), funded by the Swiss National Science Foundation. References 1. Aaserud, O., Juntunen, J., Matikainen, E.: Vibration sensitivity thresholds: methodological considerations. Acta Neurologica Scandinavica 82, 277–283 (1990) 2. Aatola, S., Färkkilä, M., Pyykkö, I., Korhonen, O.: Measuring method of vibration perception threshold of fingers and its application to vibration exposed workers. Int. Arch. Occup. Environ. Health 62, 239–242 (1990) 3. Altinsoy, M.E., Merchel, S., Tilsch, S.: Perceptual evaluation of violin vibrations and audiotactile interaction. Proc. Meet. Acoust. 19(1), 15–26 (2013) 4. Askenfelt, A., Jansson, E.V.: On vibration sensation and finger touch in stringed instrument playing. Music Percept. 9(3), 311–349 (1992) 5. Bank, B., Zambon, S., Fontana, F.: A modal-based real-time piano synthesizer. IEEE Trans. Audio Speech Lang. Process. 18(4), 809–821 (2010) (Special Issue on Virtual Analog Audio Effects and Musical Instruments) 6. Bensmaïa, S., Hollins, M., Yau, J.: Vibrotactile intensity and frequency information in the pacinian system: a psychophysical model. Percept. Psychophys. 67(5), 828–841 (2005) 7. Bensmaïa, S.J., Hollins, M.: Complex tactile waveform discrimination. J. Acoust. Soc. Am. 108(3), 1236–1245 (2000) 8. Bolanowski, S.J., Gescheider, G.A., Verrillo, R.T., Checkosky, C.M.: Four channels mediate the mechanical aspects of touch. J. Acoust. Soc. Am. 84(5), 1680–1694 (1988) 9. Brisben, A.J., Hsiao, S.S., Johnson, K.O.: Detection of vibration transmitted through an object grasped in the hand. J. Neurophysiol. 81(4), 1548–1558 (1999) 10. Craig, J.C., Sherrick, C.E.: The role of skin coupling in the determination of vibrotactile spatial summation. Percept. Psychophys. 6(2), 97–101 (1969) 11. Dahl, S.: Striking movements: a survey of motion analysis of percussionists. Acoust. Sci. Technol. 32(5), 168–173 (2011) 12. Dauman, R.: Bone conduction: an explanation for this phenomenon comprising complex mechanisms. Eur. Ann. Otorhinolaryngol. Head Neck Dis. 130(4), 209–213 (2013) 13. DiDomenico Astin, A.: Finger force capability: measurement and prediction using anthropometric and myoelectric measures. Master’s thesis, Virginia Polytechnic Institute and State University, Blacksburg, VA, USA (1999) 14. Dinse, H.R., Kalisch, T., Ragert, P., Pleger, B., Schwenkreis, P., Tegenthoff, M.: Improving human haptic performance in normal and impaired human populations through unattended activation-based learning. ACM Trans. Appl. Percept. 2(2), 71–88 (2005) 15. Ekenvall, L., Gemne, G., Tegner, R.: Correspondence between neurological symptoms and outcome of quantitative sensory testing in the hand-arm vibration syndrome. Br. J. Ind. Med, 46, 570–574 (1989) 16. Fontana, F., Avanzini, F., Järveläinen, H., Papetti, S., Klauer, G., Malavolta, L.: Rendering and subjective evaluation of real vs. synthetic vibrotactile cues on a digital piano keyboard. In: 70 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. F. Fontana et al. Proceedings of the Sound and Music Computing conference (SMC), Maynooth, Ireland, pp. 161–167 (2015) Fontana, F., Avanzini, F., Järveläinen, H., Papetti, S., Zanini, F., Zanini, V.: Perception of interactive vibrotactile cues on the acoustic grand and upright piano. In: Proceedings of the Sound and Music Computing Conference (SMC), Athens, Greece (2014) Fontana, F., Papetti, S., Civolani, M., del Bello, V., Bank, B.: An exploration on the influence of vibrotactile cues during digital piano playing. In: Proceedings of the Sound and Music Computing conference (SMC), Padua, Italy (2011) Fontana, F., Papetti, S., Järveläinen, H., Avanzini, F.: Detection of keyboard vibrations and effects on perceived piano quality. J. Acoust. Soc. Am. 142(5), 2953–2967 (2017) Galembo, A., Askenfelt, A.: Quality assessment of musical instruments - Effects of multimodality. In: Proceedings of the 5th Triennial Conference of the European Society for the Cognitive Sciences of Music (ESCOM5), Hannover, Germany (2003) Gescheider, G., Bolanowski, S., Verrillo, R.: Some characteristics of tactile channels. Behav. Brain Res. 148(1–2), 35–40 (2004) Gescheider, G.A., Bolanowski, S.J., Verrillo, R.T., Arpajian, D.J., Ryan, T.F.: Vibrotactile intensity discrimination measured by three methods. J. Acoust. Soc. Am. 87(1), 330 (1990) Gescheider, G.A., Joelson, J.M.: Vibrotactile temporal summation for threshold and suprathreshold levels of stimulation. Percept. Psychophys. 33(2), 156–162 (1983) Gibson, J.J.: Observations on active touch. Psychol. Rev. 69, 477–491 (1962) Giordano, B.L., Avanzini, F., Wanderley, M.M., McAdams, S.: Multisensory integration in percussion performance. In: S.F. d’Acoustique SFA (ed.) 10ème Congrès Français d’Acoustique, Lyon, France (2010) Green, D., Swets, J.: Signal Detection Theory and Psychophysics. Wiley, New York (1966) Guizzo, E.: Keyboard maestro. IEEE Spectr. 47(2), 32–33 (2010) Harada, N., Griffin, M.J.: Factors influencing vibration sense thresholds used to assess occupational exposures to hand transmitted vibration. Br. J. Ind. Med. 48(48), 185–192 (1991) Harazin, B., Kuprowski, J., Stolorz, G.: Repeatability of vibrotactile perception thresholds obtained with two different measuring systems. Int. J. Occup. Med. Environ. Health 16(4), 311–319 (2003) Kayser, C., Petkov, C.I., Augath, M., Logothetis, N.K.: Integration of touch and sound in auditory cortex. Neuron 48(2), 373–384 (2005) Keane, M.: Separation of piano keyboard vibrations into tonal and broadband components. Appl. Acoust. 68(10), 1104–1117 (2007) Keane, M., Dodd, G.: Subjective assessment of upright piano key vibrations. Acta Acustunited Ac. 97(4), 708–713 (2011) Kinoshita, H., Furuya, S., Aoki, T., Altenmüller, E.: Loudness control in pianists as exemplified in keystroke force measurements on different touches. J. Acoust. Soc. Am. 121(5), 2959–2969 (2007) Lamoré, P.J., Keemink, C.J.: Evidence for different types of mechanoreceptors from measurements of the psychophysical threshold for vibrations under different stimulation conditions. J. Acoust. Soc. Am. 83(6), 2339–2351 (1988) Levitt, H.: Transformed up-down methods in psychoacoustics. J. Acoust. Soc. Am. 49(2), 467–477 (1971) Maeda, S., Griffin, M.J.: A comparison of vibrotactile thresholds on the finger obtained with different equipment. Ergonomics 37(8), 1391–1406 (1994) Makous, J., Friedman, R., Vierck, C.: A critical band filter in touch. J. Neurosci. 15(4), 2808– 2818 (1995) Marshall, K., Genter, B.: The musician and the vibrational behavior of a violin. J. Catgut Acoust. Soc. 45, 28–33 (1986) Merchel, S., Leppin, A., Altinsoy, E.: Hearing with your body: the influence of whole-body vibrations on loudness perception. In: Proceedings of the 16th International Congress on Sound and Vibration (ICSV), Kraków, Poland (2009) 4 Perception of Vibrotactile Cues in Musical Performance 71 40. Morey, R.D.: Confidence intervals from normalized data: a correction to cousineau (2005). Tutor. Quant. Methods Psychol. 4(2), 61–64 (2008) 41. Morioka, M., Griffin, M.J.: Dependence of vibrotactile thresholds on the psychophysical measurement method. Int. Arch. Occup. Environ. Health 75(1–2), 78–84 (2002) 42. Okazaki, R., Kajimoto, H., Hayward, V.: Vibrotactile stimulation can affect auditory loudness: a pilot study. In: Proceedings of the Eurohaptics Conference, Tampere, Finland, pp. 103–108 (2012) 43. Papetti, S., Järveläinen, H., Giordano, B.L., Schiesser, S., Fröhlich, M.: Vibrotactile sensitivity in active touch: effect of pressing force. IEEE Trans. Haptics 10(1), 113–122 (2017) 44. Papetti, S., Järveläinen, H., Schmid, G.M.: Vibrotactile sensitivity in active finger pressing. In: Proceedings of the IEEE World Haptics, Evanston, Illinois, USA (2015) 45. Ragert, P., Schmidt, A., Altenmüller, E., Dinse, H.R.: Superior tactile performance and learning in professional pianists: evidence for meta-plasticity in musicians. Eur. J. Neurosci. 19(2), 473– 478 (2004) 46. Ro, T., Hsu, J., Yasar, N.E., Elmore, L.C., Beauchamp, M.S.: Sound enhances touch perception. Exp. Brain Res. 195(1), 135–143 (2009) 47. Russo, F., Ammirante, P., Fels, D.: Vibrotactile discrimination of musical timbre. J. Exp. Psychol. Hum. Percept. Perform. 38(4), 822–826 (2012) 48. Saitis, C., Fritz, C., Scavone, G.P., Guastavino, C., Dubois, D.: Perceptual evaluation of violins: A psycholinguistic analysis of preference verbal descriptions by experienced musicians. J. Acoust. Soc. Am. 141(4), 2746–2757 (2017) 49. Shao, Y., Hayward, V., Visell, Y.: Spatial patterns of cutaneous vibration during whole-hand haptic interactions. Proc. Natl. Acad. Sci. U.S.A 113(15), 4188–4193 (2016) 50. Soto-Faraco, S., Deco, G.: Multisensory contributions to the perception of vibrotactile events. Behav. Brain Res. 196(2), 145–154 (2009) 51. Suzuki, H.: Vibration and sound radiation of a piano soundboard. J. Acoust. Soc. Am. 80(6), 1573–1582 (1986) 52. Van Doorn, G.H., Dubaj, V., Wuillemin, D.B., Richardson, B.L., Symmons, M.A.: Cognitive load can explain differences in active and passive Touch. In: Isokoski, P., Springare, J. (eds.) Haptics Perception, Devices, Mobility, Commun., Lecture Notes in Computer Science, vol. 7282, pp. 91–102. Springer, Berlin, Heidelberg (2012) 53. Verrillo, R.T.: Temporal summation in vibrotactile sensitivity. J. Acoust. Soc. Am. 37, 843–846 (1965) 54. Verrillo, R.T.: Psychophysics of vibrotactile stimulation. J. Acoust. Soc. Am. 77(1), 225–232 (1985) 55. Verrillo, R.T.: Vibration sensation in humans. Music Percept. 9(3), 281–302 (1992) 56. Verrillo, R.T., Gescheider, G.A.: Enhancement and summation in the perception of two successive vibrotactile stimuli. Percept. Psychophys. 18(2), 128–136 (1975) 57. Wilson, E.C., Braida, L.D., Reed, C.M.: Perceptual interactions in the loudness of combined auditory and vibrotactile stimuli. J. Acoust. Soc. Am. 127(5), 3038–3043 (2010) 58. Wilson, E.C., Reed, C.M., Braida, L.D.: Integration of auditory and vibrotactile stimuli: effects of phase and stimulus-onset asynchrony. J. Acoust. Soc. Am. 126(4), 1960–1974 (2009) 59. Wollman, I., Fritz, C., Poitevineau, J.: Influence of vibrotactile feedback on some perceptual features of violins. J. Acoust. Soc. Am. 136(2), 910–921 (2014) 60. Woodhouse, J.: The acoustics of the violin: a review. Rep. Prog. Phys. 77(11), 115901 (2014) 61. Wyse, L., Nanayakkara, S., Seekings, P., Ong, S.H., Taylor, E.A.: Palm-area sensitivity to vibrotactile stimuli above 1 kHz. In: Proceedings of the Conference on New Interfaces for Musical Expression (NIME), Ann Arbor, MI, USA (2012) 62. Yau, J.M., Olenczak, J.B., Dammann, J.F., Bensmaïa, S.J.: Temporal frequency channels are linked across audition and touch. Curr. Biol. 19(7), 561–566 (2009) 63. Yildiz, M.Z., Toker, I., Özkan, F.B., Güçlü, B.: Effects of passive and active movement on vibrotactile detection thresholds of the Pacinian channel and forward masking. Somatosens. Mot. Res. 32(4), 262–272 (2015) 72 F. Fontana et al. 64. Young, G.W., Murphy, D., Weeter, J.: Auditory discrimination of pure and complex waveforms combined with vibrotactile feedback. In: Proceedings of the Conference on New Interfaces for Musical Expression (NIME), Baton Rouge, LA, USA (2015) 65. Zamorano, A.M., Riquelme, I., Kleber, B., Altenmüller, E., Hatem, S.M., Montoya, P.: Pain sensitivity and tactile spatial acuity are altered in healthy musicians as in chronic pain patients. Front. Hum. Neurosci. 8, 1016 (2014) Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made. The images or other third party material in this chapter are included in the chapter’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.