IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, VOL. 25, NO. 3, MARCH 2017 215 An Empirical Evaluation of Force Feedback in Body-Powered Prostheses Jeremy D. Brown, Member, IEEE , Timothy S. Kunz, Duane Gardner, Mackenzie K. Shelley, Alicia J. Davis, and R. Brent Gillespie, Member, IEEE Abstract — Myoelectric prostheses have many advantages over body-powered prostheses, yet the absence of sensory feedback in myoelectric devices is one reason body-powered devices are often preferred by amputees. While considerable progress has been made in the mechanical design and control of myoelectric prostheses, research on haptic feedback has not had a similar impact. In this study, we seek to develop a fundamental understanding of the utility of force feedback and vision in the functional operation of a body-powered upper-limb prosthesis. Using a custom body-powered prosthesis in which force feedback can be conditionally removed, we asked N = 10 nonamputee participants to identify objects based on stiffness in four separate conditions with and without visual and/or force feedback. Results indicate that the combination of visual and force feedback allows for the best accuracy, followed by force feedback only, then visual feedback only. In addition, combining force feedback with visual feedback does not significantly affect identification timing compared to visual feedback alone. These findings suggest that consideration should be given to the development of force feedback displays for myoelectric prostheses that function like a Bowden cable, coupling the amputee’s control input to the resulting feedback. Index Terms — Body-powered, force feedback, prosthetics, visual feedback. Manuscript received January 13, 2015; revised July 21, 2015, and October 6, 2015; accepted November 24, 2015. Date of publication April 14, 2016; date of current version March 20, 2017. This work was supported in part by the National Science Foundation Grant IIS-1065027, and by a NSF Graduate Research Fellowship held by the first author. J. D. Brown was with the Department of Mechanical Engineering, University of Michigan, Ann Arbor, MI 48109 USA. He is now with the Department of Mechanical Engineering and Applied Mechanics, University of Pennsylvania, Philadelphia, PA 19104 USA (e-mail: brownjer@seas.upenn.edu). T. S. Kunz was with the Department of Physical Medicine and Rehabilitation Orthotics and Prosthetics Center, University of Michigan, Ann Arbor, MI 48109 USA. He is now with Kootenai Prosthetics and Orthotics Services, Post Falls, ID 83854 USA (e-mail: timothy-kunz@ouhsc.edu). D. Gardner was with the Department of Mechanical Engineering, University of Michigan, Ann Arbor, MI 48109 USA. He is now with the Boeing Corporation, St. Louis, MO 63134 USA (e-mail: duane.gardner@boeing.com). M. K. Shelley is with the Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI 48109 USA (e-mail: mshelley@umich.edu). A. J. Davis is with the Department of Physical Medicine and Rehabilitation Orthotics and Prosthetics Center, University of Michigan, Ann Arbor, MI 48109 USA (e-mail: aliciad@med.umich.edu). R. B. Gillespie is with the Department of Mechanical Engineering, University of Michigan, Ann Arbor, MI, 48109 USA (e-mail: brentg@umich.edu). Digital Object Identifier 10.1109/TNSRE.2016.2554061 I. I NTRODUCTION OR an upper-limb amputee, the choice between a bodypowered or a myoelectric prosthesis involves a tradeoff. The body-powered and myoelectric devices each offer unique advantages in terms of weight, available degrees of freedom, ease of control, and availability of sensory feedback that are—for the most part—mutually exclusive. One advantage of body-powered devices is their support for force feedback, generated naturally by the mechanical linkage between the terminal device and shoulder harness used for control. Using this haptic feedback, amputees are able to develop closed-loop, force-control schemes. This helps explain why body-powered devices are still widely used today, despite remaining relatively unchanged in design since their development over 60 years ago [1]. In contrast, myoelectric1 devices do not provide haptic feedback, thus forcing amputees to rely more heavily on vision and incidental auditory cues. Amputees who wear myoelectric or body-powered prostheses list gripping, steadying, and manipulating as most important among the functional roles of their prosthesis, and they rank function and comfort as top design priorities for future device development [2]. Most notably, amputees who currently wear a myoelectric device as their primary prosthesis identify the lack of adequate sensory feedback as one area of dissatisfaction with their device [2]. While amputees have been desiring improved haptic feedback from their myoelectric prostheses for over 25 years [3], the greatest advancements have come in the form of limbs that have more degrees of freedom. Examples include the Deka Arm [4], the APL arm [5], and the Touch Bionics iLimb Ultra prosthetic hand [6]. To give amputees adequate control over these improved devices, researchers have focused on developing advanced control schemes based on myoelectric pattern recognition algorithms [7] and targeted muscle reinnervation [8]. To provide haptic sensory feedback to supplement vision, researchers have been developing haptic display mechanisms since the 1960s [1]. To date, numerous technologies have been studied that use haptic display to relay signals sensed electronically from the terminal device, including electrocutaneous stimulation [9], [10], vibrotactile stimulation [11]–[13], skin stretch stimulation [14], mechanotactile stimulation F 1 While the term myoelectric devices generally refers to a class of externally powered prostheses that can receive their control input from many sources, we restrict our definition here to only include those devices that rely on transduction of muscle activity through surface EMG electrodes. 1534-4320 © 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. Authorized licensed use limited to: Johns Hopkins University. Downloaded on July 28,2020 at 13:01:31 UTC from IEEE Xplore. Restrictions apply. 216 IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, VOL. 25, NO. 3, MARCH 2017 (displaying forces normal to the skin [15]) [16]–[18], and devices that combine multiple types of stimulation [19], [20]. These technologies, however, have not made their way into clinical practice or commercial availability while the advances in mechanical design and improved control have made considerable impact. The lack of impact by advancements in haptic display for myoelectric devices might be attributed to an inability of the haptic display to provide the rich information intrinsically available through the Bowden cable in bodypowered devices. By allowing a direct mapping between body action and terminal device aperture, the Bowden cable provides amputees with a proprioceptive sense of the terminal device. This mapping is consonant with the principle of extended physiological proprioception first introduced by Simpson in the 1970s [21] and explored in a limited scaled by others [22]–[24]. At the same time, the Bowden cable allows for all control input (action) to and force feedback (re-action) from the terminal device to originate and terminate, respectively, at the same point on the body. In this manner, the dynamics of the body are coupled to the dynamics of the world encountered in the grasp of the terminal device. These coupled dynamics have been shown to improve performance over the case where body and world are decoupled [25]. Development of a haptic display for myoelectric prostheses that features haptic feedback akin to that available in bodypowered prostheses presents a unique challenge: myoelectric prostheses lack a Bowden cable. It still may be possible, however, if the utility provided by the Bowden cable can be replicated by other means. To date, there are no empirical evaluations of Bowden cable utility, even in body-powered devices. To inform the design of a sensory feedback display for myoelectric devices, it would first be worthwhile to quantify the benefits of force feedback in body-powered devices. Any direct comparison of myoelectric and body-powered devices, however, would be confounded with both a change in control (EMG versus body) and sensory feedback (visual versus visual+force) that prevents easy interpretation. Conditionally removing force feedback from a body-powered prosthesis would produce valuable comparative results. The literature on teleoperation might prove useful for quantifying the value of force feedback. Indeed, a prosthesis can be considered a teleoperator in the sense that it connects the amputee’s residual limb to the world experienced at the distal end of the prosthesis. The obvious difference here is that for traditional teleoperators, the hand is used to interact with the master side of the device; for prosthetics, the absence of a physical hand on the residual limb necessitates referral to another body part, such as the shoulder. Nonetheless, body-powered prostheses are very similar to the mechanical teleoperators first developed by Goertz to handle radioactive material [26]: both feature a mechanical linkage between the master and slave (the shoulder harness and terminal device for body-powered devices) that provides inherent force feedback. When Goertz went on to develop electromechanical teleoperators featuring both a motorized master and slave [27], [28], he opened the possibility for directly evaluating the utility of force feedback. The empirical evaluations that have followed show that providing force feedback in teleoperation improves task completion times [29] and task accuracy [30], [31]. If an electromechanical body-powered prosthesis existed that featured a motorized master and slave, similar evaluations could be undertaken to assess the utility of force feedback in body-powered devices. With this aim in mind, we have developed a custom bodypowered prosthesis that can be worn by non-amputees and that features force feedback that can be turned on and off. Our prosthesis is built on the concept of an electromechanical teleoperator with the master (a cable-driven exoskeleton) worn on the left arm connected to the slave (a cable-driven voluntary-closing terminal device) worn on the right arm through a series of linear actuators and an interrupted Bowden cable. The angular position of the elbow serves as the control input to the terminal device, and forces measured during grasping of the terminal device can be conditionally displayed through the exoskeleton. In this study, we use our custom prosthesis as an experimental control to evaluate the utility of force feedback and vision in the functional performance of a body-powered prosthesis. We ask participants to identify a set of objects based on stiffness using the prosthesis in the four conditions that result by selectively gating vision and force feedback. We expect results similar to teleoperation— improved performance with force feedback over vision alone. II. M ETHODS A. Participants We tested N = 10 non-amputee participants (seven male, three female; mean age = 24.3 ± 2.9 years). Prior to starting the study, each participant was consented according to a protocol approved by the University of Michigan Institutional Review Board (IRB) and provided with an overview of the study. B. Experimental Apparatus Our experimental apparatus consisted of two linear actuator drives, a custom mock prosthesis, and a custom cable-driven elbow brace. A Dell Precision T3500 Desktop computer with a Sensoray 626 PCI data acquisition card was used for data acquisition and control. Both linear actuator drives featured a Maxon RE65 rotary motor and a linear ballscrew with a 20 mm lead [see Fig. 1(a)]. Each motor was equipped with a rotary optical encoder (US Digital EM1-1-1024) and driven with a current sourcing amplifier (Advanced Motion Control 12A8). A 10-kg rated beam load cell (Transducer Techniques LSP-10) was attached to the ball nut of each ball screw through a 3D printed ABS carriage. The carriage was attached to linear slides so that it could move freely with the ball nut. Two limit switches (not pictured) were placed along the length of the ballscrew to limit the actuator’s range of motion. Each linear drive provided pulling actuation for a Bowden cable. The cable housing was secured at the end of the ballscrew to a mount, and the cable was attached to the loadcell through a cable anchor. The cable-driven exoskeleton consisted of two 3D printed ABS halves (upper and lower) that were connected about two coaxial rotational bearings on either side of the participant’s Authorized licensed use limited to: Johns Hopkins University. Downloaded on July 28,2020 at 13:01:31 UTC from IEEE Xplore. Restrictions apply. BROWN et al.: EMPIRICAL EVALUATION OF FORCE FEEDBACK IN BODY-POWERED PROSTHESES 217 Fig. 1. Experimental apparatus components. A: Linear actuator drive featuring rotational motor, encoder, ballscrew, beam loadcell, loadcell carriage, Bowden cable anchor and housing mount, and linear slide. In operation, the motor, ballscrew, and carriage tracked a desired position to either generate no force on the cable or to track a setpoint pulling force on the cable. B: Cable-driven exoskeleton featuring an encoder and thermoplastic cuffs with velcro straps, and anchor and mount for Bowden cable and housing. In operation, pulling on the cable would produce and extension moment about the elbow. The exoskeleton is shown here on the right arm for illustrative purposes. C: Mock prosthesis featuring a thermoplastic shell with accommodations for the fist of a non-amputee participant, a voluntary-closing terminal device, and anchor and mount points for a Bowden cable and housing. To ensure a snug fit, participants were required to wear prosthetic socks. In operation, pulling on the cable closed the terminal device. elbow [see Fig. 1(b)]. One of the axes was fixed to the lower half of the exoskeleton, and a rotary optical encoder (US Digital EM1-1-1250) was mounted to the axis to measure angular position. Custom thermoplastic cuffs (in one of three available sizes) were attached to each half and featured Velcro straps and foam padding. The housing of the Bowden cable attached to a housing mount on the upper half, and the cable itself attached to the lower half through a cable anchor that swiveled. The exoskeleton was fit to the left arm of a nonamputee participant with the axis of rotation of the elbow joint aligned coaxially with the exoskeleton axis of rotation. Velcro straps were used to secure the exoskeleton to the upper and lower portion of the participant’s arm. In operation, pulling on the cable caused the exoskeleton to produce an extension moment about the elbow. This extension moment served as force feedback. The mock prosthesis [see Fig. 1(c)] consisted of a thermoplastic shell mated with a Hosmer Quick Disconnect Wrist (USMC style) and voluntary-closing terminal device (TRS Grip 2S), which was nominally held open by an internal torsional spring. The prosthesis was designed to mate to the right arm of a non-amputee participant. The thermoplastic shell was fabricated by casting the forearm and hand of a non-amputee individual with their hand in a closed (fist) position. The cast was digitized using an Ohio WillowWood Omega Tracer Scanner where the overall diameter of the model was increased by 15 mm to accommodate larger sized forearms. The digitized model was fabricated on a Milltronics 4-axis mill using an ISO technologies 4.0 density foam blank as the carving medium. A 3/16 in AIN Plastic co-poly thermoplastic sheet was then vacuum-formed over the foam model to create the thermoplastic shell. Royal Knit prosthetic socks were worn over the participant’s arm to create a tight and comfortable fit of the thermoplastic shell. The Bowden cable housing was attached to the housing mount, and the cable was attached to the terminal device via the cable anchor. In operation, pulling on the cable closed the terminal device. Together, the individual elements of the experimental apparatus created a body-powered prosthesis that could be used by non-amputee participants. As shown schematically in Fig. 2(a), one actuator is mechanically linked through a Bowden cable to the cable-driven prosthesis (prosthesis actuator), and the other is mechanically linked via a separate cable to the cabledriven exoskeleton (exoskeleton actuator). Angle θ E A is the exoskeleton actuator angular position, θ P A is the prosthesis actuator angular position, TE is the tension in the Bowden cable that generates an extension moment about the exoskeleton axis of rotation, and TG is the tension in the Bowden cable that generates terminal device prehension (TG is related to the grip force through the mechanical advantage and torsional spring of the terminal device). This body-powered prosthesis differs from a traditional body-powered prosthesis in that the participant’s left elbow (as opposed to a shoulder through a shoulder harness) is used for control. In addition, haptic (force) feedback can be removed, which is possible because the user is connected to the prosthesis through the two actuators and a computerized controller. Authorized licensed use limited to: Johns Hopkins University. Downloaded on July 28,2020 at 13:01:31 UTC from IEEE Xplore. Restrictions apply. 218 IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, VOL. 25, NO. 3, MARCH 2017 Fig. 2. Custom body-powered prosthesis. A: Schematic of custom body-powered prosthesis for non-amputee participants. Exoskeleton is worn on the left arm, and the mock prosthesis is worn on the right arm. θE is the exoskeleton angular position. θEA is the exoskeleton actuator angular position. TE is the tension in the Bowden cable that generates an extension moment about the exoskeleton axis of rotation. TG is the tension in the Bowden cable that generates terminal device prehension. B: Block diagrams of prosthesis actuator, and exoskeleton actuator with force feedback “off” and “on.” Both linear actuators were controlled through a proportional-derivative controller [see Fig. 2(b)]. The control law for the exoskeleton actuator u E A with force feedback “off” (H = 0) and force feedback “on” (H = 1) is u E A = (K p + K d s)[K E A θ E − θ E A ] + H [K F TG ] (II.1) where K p is the proportional gain, K d is the derivative gain, K F is the feedback gain, and K E A is the exoskeleton actuator gain. The values of K p , K d , and K F were manually tuned during pilot tests to provide an acceptable level of position tracking and discernible force feedback from the device. In operation, K p = 2.0, K d = 0.06, and K F = 2.0. TG was measured by the loadcell on the prosthesis actuator. The exoskeleton actuator gain K E A scales the exoskeleton position to that of the exoskeleton actuator and is determined through a calibration routine described below. The control law for the prosthesis actuator u P A is u P A = (K p + K d s)[K P A θ E − θ P A ] (II.2) where K P A , the prosthesis actuator gain, scales the exoskeleton position to that of the prosthesis actuator and is determined through a calibration routine described below. When force feedback was turned “off,” the body-powered device operated like a non force-reflecting position controlled teleoperator. The position of the exoskeleton was mapped to the position of the terminal device via the prosthesis actuator. Any tension generated by the prosthesis actuator closing the terminal device TG , however, was not displayed to the participant. The exoskeleton actuator tracked the position of the exoskeleton to minimize the device impedance. When force feedback was turned “on,” the body-powered device operated like a force-reflecting position-force teleoperator. The angular position of the exoskeleton was mapped to the position of the terminal device via the prosthesis actuator and the tension TG generated by the prosthesis actuator was displayed to the user through the actuated exoskeleton as an extension moment. The following calibration routine was used to scale the range of motion of each actuator to the range of motion of the participant’s arm. For the prosthesis actuator, the gain K P A was tuned such that when the participant’s left arm Fig. 3. Sample stiffness (force/displacement) profiles generated by the (A) Prosthesis actuator and (B) exoskeleton actuator with force feedback “off”, and by the (C) Prosthesis actuator and (D) exoskeleton actuator with force feedback “on”. Blue dotted traces refer to the “hard” block. Purple solid traces refer to the “medium” block. Brown dashed traces refer to the “soft” block. was fully extended, the terminal device was open and when the arm was fully flexed, the terminal device was closed. For the exoskeleton actuator, the gain K E A was tuned so that the loadcell carriage (and cable) moved in-sync with the participant’s left arm. C. Stimuli Our stimuli consisted of foam blocks (Temper Foam R-Lite Foam Blocks) in three different stiffnesses: extra-soft, soft, and medium. In the experiment they were referred to as “soft,” “medium,” and “hard,” respectively. The blocks were covered with black athletic socks to hide their unique colors. D. Sample Apparatus Performance With feedback “off,” the prosthesis actuator loadcell and encoder measured the force/displacement relationship of each block. As demonstrated by the sample traces in Fig. 3(a), three Authorized licensed use limited to: Johns Hopkins University. Downloaded on July 28,2020 at 13:01:31 UTC from IEEE Xplore. Restrictions apply. BROWN et al.: EMPIRICAL EVALUATION OF FORCE FEEDBACK IN BODY-POWERED PROSTHESES Fig. 4. Sample stiffness (force/displacement) profiles generated by the exoskeleton actuator with force feedback “on” when the block was (A) squeezed and (B) released. Blue dotted traces refer to the “hard” block. Purple solid traces refer to the “medium” block. Brown dashed traces refer to the “soft” block. distinct stiffnesses were discernible by the prosthesis actuator’s encoder and loadcell. These stiffnesses, however, were not displayed to the exoskeleton [see Fig. 3(b)]. With feedback “on,” both the prosthesis actuator loadcell and encoder as well as the exoskeleton actuator loadcell and encoder capture the force/displacement relationship of the blocks. The sample traces in Fig. 3(c) are somewhat similar to the traces for the prosthesis actuator with no feedback with the exception that the medium and hard blocks appear to excite some resonant dynamics in the closed-loop system with our PD controller that make the stiffness traces not as smooth. For the exoskeleton actuator [see Fig. 3(d)], the traces fall into three distinct groupings based on the stiffness of each block. This is particularly true when the blocks are squeezed [see Fig. 4(a)] as opposed to when the blocks are released [see Fig. 4(b)]. The traces presented here highlight the hysteresis present in the stiffness relationship of the foam blocks used in the experiment. Still, the overall force/displacement profiles for each of the three foam blocks are distinguishable visually with feedback “off.” This holds true as well, albeit to a lesser degree, with feedback “on,” where the force/displacement profiles appears to be heavily influenced by the resonant dynamics of our closed-loop system. Therefore, being able to differentiate the force/displacement profiles visually would suggest that they may also be distinguishable by feel when presented through our device. E. Setup and Training Participants sat on a stool facing the table where the experiment would take place. The appropriate size cuffs were attached to the exoskeleton, and the exoskeleton was mated to each participant’s left arm with the exoskeleton’s axis of rotation in line with the participant’s elbow axis of rotation. Additional padding was used to ensure the velcro straps did not pinch the participant’s skin. The participants’ right arms were donned with Royal Knit prosthetic socks, and their arms were placed inside the custom prosthesis. Participants were then instructed to make a loose fist with their right hand to ensure the prosthesis stayed in place. The experimental apparatus was then calibrated as described in Section II-B. Participants were made aware of the fact that if they moved their arm too fast, the motor would not be able to keep up, and they would feel 219 the actuator’s impedance through the exoskeleton. Participants were also told that the blocks were made of memory foam. Prior to testing, participants were given an opportunity to experience the four experimental conditions: visual+force feedback, visual feedback alone, force feedback alone, and no feedback. In each condition, they were allowed to feel a sample block. This sample block was a foam block similar to the stimuli blocks, except it had a higher stiffness than all three stimuli blocks. This sample block was also covered in a black athletic sock for consistency. F. Protocol Participants were not given an opportunity to feel the three test blocks with their hands or through the device prior to beginning the test. Participants were told that the goal of the experiment was to accurately identify the blocks in the shortest time possible. The test consisted of 20 trials (five trials for each of the four conditions). The trial order was randomized into five sets of four, with each set containing a randomized order of the four experimental conditions. In each trial, the experimenter randomly selected eight blocks from a group of 12 (four blocks of each stiffness). The blocks were then presented one at a time to participants. For each block presentation, participants started from a rest position (terminal device resting on the edge of the table). When the experimenter announced “begin,” participants were instructed to explore the block through the prosthesis and sort it in a corresponding bin (“soft,” “medium,” or “hard”). The participant was instructed to approach the block from the front, as opposed to the top, to avoid any stiffness cues associated with displacing the block with respect to the table. Participants were allowed to squeeze any portion of the block through the prosthesis as many times as desired while the block rested on the table in a specified location and could request that the experimenter rotate the block. After picking the block up, participants were not allowed to place it back down on the table to squeeze it again. Participants were also not allowed to re-sort the blocks once they were placed in the bins. The no vision trials were treated differently. The prosthesis was held in a specified location, and the terminal device was shielded from the participant’s view by a posterboard curtain. When the experimenter announced “begin,” participants were allowed to squeeze the foam block while it was held by the experimenter, and then verbalize their choice (hard/medium/soft) as to which block it was. The experimenter would then place the object in the corresponding bin on their behalf. After verbalizing their bin choice, participants were not allowed to change. After the participants sorted all eight blocks, the experimenter would verbally indicate how many blocks of each stiffness (hard/medium/soft) occupied each bin as a means of correct answer feedback. This was also recorded by the experimenter. Short breaks (∼2 min) were taken between trials, and participants were made aware of the condition before starting the trial. Noise-canceling headphones were not used so that verbal instructions could be understood clearly. Authorized licensed use limited to: Johns Hopkins University. Downloaded on July 28,2020 at 13:01:31 UTC from IEEE Xplore. Restrictions apply. 220 IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, VOL. 25, NO. 3, MARCH 2017 The auditory cues provided by the actuators were thought to be consistent with the auditory cues available in current prostheses, especially myoelectric prostheses. G. Metrics and Data Analysis The kinematic, kinetic, and performance data were recorded to disk with a 1 kHz sampling rate. Our performance metrics were the object identification accuracy (%), object identification completion time (s), and the number of times the object was probed. Object identification accuracy was computed as an overall accuracy for each group of eight blocks (i.e., the percentage of blocks in the correct bin). Object identification completion time was measured as the time in milliseconds from the time the tester announced “begin” to the time the participant placed the block in a bin (vision trials) or verbalized their bin choice (no vision trials). Unlike the object identification accuaracy metric, the completion time was recorded for each block presentation in the group of eight. To measure the number of times the object was probed (# of Probes), two threshold values were set at 80 mm and 120 mm on the gripper actuator, whose entire range was ∼200 mm. These threshold values served to capture whether the participant passed the halfway point in either direction when probing the block. Passing both thresholds in one direction as the terminal device closed accumulated a 0.5 (half) probe. Subsequently, passing both thresholds in the opposite direction as the terminal device opened accumulated another 0.5 probe. Passing only one threshold in either direction accumulated 0.25 probe. As with object identification completion time, the number-of-probes was recorded for each block presentation. 1) Post-Test Survey: Our post-test survey represents a quantitative and qualitative self-assessment of each participant’s subjective assessment of the four conditions, as well as strategies employed for each. The survey contained a mix of 12 Likert, ranking, and short-answer questions. Only the Likert and ranking questions will be discussed further. Questions 2–9 consisted of two questions for each of the four feedback conditions. The first question asked participants to rank on a 3-point scale how easy/difficult the given condition was, and the second question asked participants what strategy (if any) they employed in the given feedback condition. Question 10 asked participants to rank each of the four conditions in order of how distinguishable (First-“most distinguishable” and Fourth- “least distinguishable”) the foam blocks were with that particular condition. Question 11 asked participants to rank the four conditions in order of preference. We acknowledge that there will likely be a certain level of correlation between participants’ perceived accuracy (based on the correct answer feedback) in a certain condition and their corresponding preference for that condition. H. Statistical Analyses All statistical analyses were performed using SPSS (v.21). Linear models were used to assess the effect of condition. For identification accuracy, a univariate general linear model (GLM) was used with condition as a fixed effect, and the order in which each trial was presented as a covariate to account for potential learning effects. Linear Mixed models (LMM) were used for the object identification completion time and the number-of-probes because of the strong amount of within-subject correlation. Within the model, participants were a random effect, condition was a fixed effect, and the order in which each block was presented as a covariate to account for any learning effects. The first block presentation for each condition was considered a baseline and is, therefore, not included in the completion time or number-of-probes model. In addition, completion time results were not compared between the two conditions with vision, and the two conditions without vision due to differences in the experimental protocol. Bonferroni adjustments were applied to the estimated means to control for Type I errors in both the GLM and LMM. A significance level of 0.05 was used as the threshold for significance for all analyses, and all reported p-values have been adjusted according to the Bonferroni correction. III. R ESULTS We found one participant to be an outlier from the remaining nine participants. Overall, this participant appeared to struggle with the operation of the device and kept expressing confusion as to what cues were important. The participant mentioned feeling “drowsy” on several occasions and did not seem to be able to concentrate, but this participant never requested to withdraw from the study. Since the experiment posed no medical risk to the participant, this participant was not asked to withdraw by the study coordinator. When plotting this participant’s results, we found this participant had a higher object identification accuracy with the no feedback condition than the force feedback condition and a higher identification accuracy with the visual feedback condition than the visual+force feedback condition. These trends were inconsistent with the other nine participants. In addition, after testing this participant informed the experimenter that they had not slept the night before and had consumed a significant amount of caffeine. Our analysis will, therefore, focus on the results of our remaining nine participants. For the purposes of a sensitivity assessment, we will also present our results with the outlier participant included. A. Foam Block Identification Accuracy Participants were able to identify the foam blocks more accurately in the trials featuring force feedback. A bar plot of the model based means of identification accuracy for the nine non-outlier participants is shown in Fig. 5(a). Results of our univariate general linear model for the nine non-outlier participants indicated significant main effects of intercept (F(1, 175) = 335.54, MSE = 9.29, p < 0.001) and feedback condition (F(3, 175) = 41.59, MSE = 1.15, p < .001). The order in which each trial was presented (the covariate in the model) did not produce a significant effect (F(1, 175) = 1.45, MSE = 0.04, p = 0.230), indicating no linear impact of trial order on accuracy, Authorized licensed use limited to: Johns Hopkins University. Downloaded on July 28,2020 at 13:01:31 UTC from IEEE Xplore. Restrictions apply. BROWN et al.: EMPIRICAL EVALUATION OF FORCE FEEDBACK IN BODY-POWERED PROSTHESES 221 intercept (F(1, 195) = 334.18, MSE = 10.4, p < 0.001), and feedback condition (F(3, 195) = 32.87, MSE = 1.02, p < 0.001). The covariate, trial order, did not produce a significant fixed effect (F(1, 195) = 0.237, MSE = 0.01, p = 0.627). The significant pairwise comparisons include a higher identification accuracy for the visual+force feedback condition (M = 65.0%, SE = 2.5%) than the visual feedback condition (M = 44.2%, SE = 2.5%) (β = 20.8%, SE = 3.5%, p < 0.001), the force feedback condition (M = 53.5%, SE = 2.5%) (β = 11.5%, SE = 3.5%, p = 0.008), or the no feedback condition (M = 31.2%, SE = 2.5%) (β = 33.8%, SE = 3.5%, p < 0.001). There was a higher identification accuracy for the force feedback condition than the no feedback condition (β = 22.3%, SE = 2.5%, p < 0.001). There was also a higher identification accuracy for the visual feedback condition than the no feedback condition (β = 13.0%, SE = 3.5%, p = 0.002). All other comparisons were not significant. B. Foam Block Identification Completion Time Fig. 5. Experimental Results. A: Model based means (adjusted for trial order) of identification accuracy for all nine non-outlier participants in all five trials of the four conditions. B: Model based means (adjusted for block presentation order) of identification completion time for all nine non-outlier participants in all five trials of the visual feedback and visual+force feedback conditions. Note that the first object exploration for each condition is excluded. C: Model based means (adjusted for block presentation order) of identification completion time for all nine nonoutlier participants in all five trials of the no feedback and force feedback conditions. Note that the first object exploration for each condition is excluded. D: Model based means (adjusted for block presentation order) of number of probes for all nine non-outlier participants in all five trials of the four conditions. Note that the first object exploration for each condition is excluded. Error bars represent 1 standard error. thereby suggesting a weak learning effect. The significant pairwise comparisons included a higher identification accuracy for the visual+force feedback condition (M = 68.1%, SE = 2.5%) than the visual feedback condition (M = 44.4%, SE = 2.5%) (β = 23.7%, SE = 3.5%, p < 0.001), the force feedback condition (M = 56.4%, SE = 2.5%) (β = 11.6%, SE = 3.5%, p = 0.007), or the no feedback condition (M = 30.8%, SE = 2.5%) (β = 37.3%, SE = 3.5%, p < 0.001). There was a higher identification accuracy for the force feedback condition than the visual feedback condition (β = 12.0%, SE = 3.5%, p = 0.005), or the no feedback condition (β = 25.6%, SE = 3.5%, p < 0.001). There was also a higher identification accuracy for the visual feedback condition than the no feedback condition (β = 13.6%, SE = 3.5%, p = 0.001). All other comparisons were not significant. Results of our univariate general linear model including our outlier participant indicated significant main effects of Overall, the conditions had little to no effect on the duration of time needed to identify the blocks. A bar plot of the model based means of identification completion time for the nine non-outlier participants is shown in Fig. 5(B) and (C). Results of the linear mixed model for the nine non-outlier participants in the two conditions featuring vision indicated significant fixed effects of intercept (F(1, 8.75) = 140.64, p < 0.001) and foam block presentation order (F(1, 666.11) = 102.97, p < 0.001), indicating a linear impact of foam block presentation order on completion time, thereby suggesting a learning effect was present throughout the experiment. In particular, trial duration decreased with increasing object presentation. There was no significant effect of feedback condition (F(1, 666.04) = 1.03, p = 0.311). Results of the linear mixed model for the nine non-outlier participants in the two conditions without vision indicated significant fixed effects of intercept (F(1, 8.49) = 61.28, p < 0.001), foam block presentation order (F(1, 680.06) = 106.9, p < 0.001), and feedback condition (F(1, 680.02) = 7.34, p = 0.007). In particular, the force feedback condition (M = 6.05 s, SE = 0.95 s) resulted in a significantly longer completion time than the no feedback condition (M = 5.57 s, SE = 0.95 s) (β = 0.48 s, SE = 0.18 s, p = 0.007). All other comparisons were not significant. Results of the linear mixed model for the vision trials including the outlier participant indicated significant fixed effects of intercept (F(1, 9.96) = 169.01, p < 0.001) and foam block presentation order (F(1, 740.15) = 95.86, p < 0.001). There was no significant fixed effect of feedback condition (F(1, 740.05) = 1.55, p = 0.214). Results of the linear mixed model for the no vision trials including the outlier participant indicated significant fixed effects of intercept (F(1, 9.66) = 69.97, p < 0.001) and foam block presentation order (F(1, 755.1) = 70.72, p < 0.001). There were no significant fixed effects of feedback condition (F(1, 755.03) = 2.58, p = 0.108). Authorized licensed use limited to: Johns Hopkins University. Downloaded on July 28,2020 at 13:01:31 UTC from IEEE Xplore. Restrictions apply. 222 IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, VOL. 25, NO. 3, MARCH 2017 TABLE I P OST-T EST S URVEY R ESULTS (# R ESPONSES ) C. # of Probes The manner in which participants explored the foam blocks depended heavily on the availability of visual and force feedback. In particular, participants probed the foam blocks more in the trials featuring only visual feedback. A bar plot of the model based means of the number of probes for the nonoutlier participants is shown in Fig. 5(d). Results of the linear mixed model for the nine non-outlier participants indicated significant fixed effects of intercept (F(1, 8.21) = 39.77, p < 0.001), foam block presentation order (F(1, 1355.04) = 12.63, p < 0.001), and feedback condition (F(3, 1355.02) = 5.56, p = 0.001). The significant linear impact of foam block presentation order on the number of probes suggests a learning effect was present. In particular, the number of probes decreases with increasing foam block presentation. The significant pairwise comparisons included a higher number of probes for the visual feedback condition (M = 1.55, SE = 0.25) than the force feedback condition (M = 1.41, SE = 0.25) (β = 0.14, SE = 0.4, p = 0.005) and the no feedback condition (M = 1.41, SE = 0.25) (β = 0.15, SE = 0.4, p = 0.003). All other comparisons were not significant. Results of the linear mixed model including the outlier participant indicated significant fixed effects of intercept (F(1, 9.26) = 45.14, p < 0.001), foam block presentation order (F(1, 1505.05) = 5.96, p = 0.015), and feedback condition (F(3, 1505.02) = 5.61, p = 0.001). The significant pairwise comparisons included a higher number of probes for the visual feedback condition (M = 1.53, SE = 0.22) than the force feedback condition (M = 1.37, SE = 0.22) (β = 0.16, SE = 0.4, p < 0.001), and the no feedback condition (M = 1.42, SE = 0.22) (β = 0.12, SE = 0.4, p = 0.024). All other comparisons were not significant. 1) Survey Results: From a qualitative perspective, participants preferred the conditions with force feedback (F and V+F) over the conditions lacking force feedback (V and N) (see Table I). All participants thought the no feedback condition was difficult (Question 2). The majority of participants felt neutral about the force feedback condition (Question 4). The majority of participants thought the visual feedback condition was difficult (Question 6). A slight majority of participants thought the visual+force feedback condition was easy; the remaining minority felt neutral (Question 8). In terms of distinguishing the foam blocks in each condition, participants thought the blocks were most distinguishable with visual+force feedback, followed by force feedback, then by visual feedback, and least distinguishable with no feedback (Question 10). Overall, the majority of our participants ranked in order of preference the visual+force feedback condition first, the force feedback condition second, the visual feedback condition third, and the no feedback condition fourth. IV. D ISCUSSION In this study, we have quantified the value of force feedback in a body-powered prosthesis and found that, even without vision, it has the potential to provide greater utility to an amputee than visual feedback alone. We arrived at this conclusion through an experiment involving a custom body-powered prosthesis that is capable of being worn by non-amputee participants and features force feedback that can be conditionally removed. Our custom prosthesis, which was modeled after an electromechanical teleoperator, acts to provide its own experimental control, in which the benefits of force feedback can be parsed without the confounds present in any comparison between body-powered and myoelectric prostheses. While the findings presented here are applicable only to the task of differentiating object stiffness, the principles underlying the ability to recognize an object’s unique force/displacement relationship carry over to many other manual tasks and suggest potential utility in many areas of prosthetic use. In four conditions, we evaluated our participants’ ability to discriminate objects (foam blocks) based on their stiffness (with visual+force feedback, force feedback, visual feedback, and no feedback). Vision was controlled with a simple curtain. Force feedback, however, was controlled through the device itself. Rather than using a shoulder harness like a traditional trans-radial body-powered prostheses, our device Authorized licensed use limited to: Johns Hopkins University. Downloaded on July 28,2020 at 13:01:31 UTC from IEEE Xplore. Restrictions apply. BROWN et al.: EMPIRICAL EVALUATION OF FORCE FEEDBACK IN BODY-POWERED PROSTHESES coupled the terminal device of the prosthesis to a cabledriven exoskeleton worn about the elbow. These exoskeletons only allow uniaxial loading about the joint and have been a topic of research interest in our group for quite some time [32]–[34]. In addition, these exoskeletons allow for easy position tracking about the joint, which was essential given our particular control paradigm (see θ E A in (II.1)). The manner in which our device operates when feedback is “on” is still consonant with traditional body-powered devices in that our device electromechanically links the action at the elbow joint to the action of the terminal device through a Bowden cable. Considering our nine, non-outlier participants, we found the accuracy with which they identified the foam blocks was best when both visual and force feedback were available, followed by force feedback only, then by visual feedback only, and finally no feedback. That participants were least accurate in the no feedback condition is not surprising. Without visual or haptic cues to inform their assessment of foam block stiffness, participants were forced to guess, which is confirmed by an identification accuracy close to that expected for guessing. In addition, compared to the force feedback condition, participants spent less time exploring the foam blocks and making a decision. This suggests that they were quite aware that increased time spent in the no feedback condition yielded no increase in identification accuracy. It is also worth noting that there were no significant differences in the number of probes between the no feedback condition and the force feedback condition. Apparently, participants simply probed at a faster rate. For the two conditions with vision, the accuracy/time trade-off was approached differently. In particular, there were no significant differences in identification time or the number of probes between the visual+force feedback condition and the visual feedback condition. Therefore, it appears that participants gave the same amount of effort in each condition, and the differences in identification accuracy are due to the availability of force feedback. Although our experimental protocol precluded a comparison of identification time between the conditions featuring visual feedback and those that do not, we did find that participants probed more in the visual feedback condition than either the force feedback condition or the no feedback condition. This suggests that the visual estimate of the block stiffness required more effort. By drawing inspiration for our custom prosthesis from electromechanical teleoperators, we were able to evaluate the utility of force feedback in a body-powered prosthesis in a manner consistent with the teleoperator literature. Our visual+force feedback condition can be likened to an electromechanical force-reflecting teleoperator, and our visual feedback condition can be likened to a non force-reflecting teleoperator. As with studies on the utility of force-reflection in teleoperation, we found that force feedback in body-powered prostheses improves performance over vision alone in terms of reduced errors (incorrect object identification) [30], [31]. Unlike the teleoperator literature [29], we did not find any differences in completion time when force feedback and visual feedback were available compared to visual feedback only. It appears then that visual feedback contributes mostly to 223 identification time when it is available. A comparison to the force feedback only condition would verify whether this is true. Unfortunately, our protocol does not support that comparison. That participants used more probes in the visual feedback condition compared to the force feedback condition suggests that there might in fact have existed differences in completion time, if they were measured. The decline in identification accuracy with vision alone is consistent with findings on the visual estimation of stiffness [35]. Even though visual estimation can be performed, the measurement is highly variable, and depends heavily upon the ability to discriminate object deformations under common forcing conditions. This could perhaps explain why the blocks were probed more often in the visual feedback condition than the force feedback condition. Note here that there were a lot of single-probe occurrences for each condition, and the differences observed in the number of probes represent differences in the number of multi-probe occurrences for a given condition. Adding force feedback alongside visual feedback appears to reduce the need for multiple probes, which is why the visual+force condition did not differ from any of the other conditions in terms of times the object was probed. Still, in even the most accurate condition, the visual+force condition, participants were only able to consistently discriminate two of the three blocks at an average completion time of 7 s. Overall, these low accuracies and long completion times point to a few areas in our study that could be improved upon. First, our blocks were made of memory foam and featured hysteresis in their stiffness relationship that potentially made stiffness estimation more difficult. In addition, the blocks had unique color identifiers that had to be masked with black socks. These socks also had the unfortunate consequence of potentially masking some of the visual and haptic cues needed for stiffness estimation. Second, our device featured dynamic behavior that potentially affected the stiffness relationship felt through the force feedback display. While the choice of device components and controller allowed participants to perform the experiment, they were not completely optimized, thus resulting in the added unwanted dynamics in the closed-loop system (see Fig. 3). Improving upon the limitations of our apparatus, in addition to increasing exposure and practice by our participants, would most likely result in improved performance. This improved performance would inevitably be realized in each of the conditions, except perhaps the no feedback condition. What would remain unchanged, however, is the trend of improved accuracy with force feedback over vision alone, and the potential for less reliance on vision in a prosthetics application. Although many factors such as cost, the amputation cause, and the aesthetics of the device affect the decision as to which prosthesis technology to choose for a given amputee, device function still ranks as a top priority [2]; and many amputees desire increased feedback and less reliance on vision [3], [36]. It is worth noting that the current findings are based on the use of a voluntary-closing terminal device. While Haverkate et al. found that participants performed better with a voluntary-opening terminal device than a voluntary-closing Authorized licensed use limited to: Johns Hopkins University. Downloaded on July 28,2020 at 13:01:31 UTC from IEEE Xplore. Restrictions apply. 224 IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, VOL. 25, NO. 3, MARCH 2017 terminal device in a box and blocks test and nine-hole peg test [37], the voluntary-closing device has specific benefits in terms of stiffness perception. In particular, this type of end-effector gives participants direct control over their prehension force. With a voluntary-opening device, participants are still able to modulate the prehension force, but it requires a good estimation of the preload generated by the rubber bands or springs producing the prehension. Another benefit of the voluntary-closing device is that it allowed for good visual field of view of the object being grasped. Our unique experimental control and the use of nonamputee participants also allows for interpretation of the results in the context of trends underlying past and present observations on upper-limb prosthetics. By design, the visual+force feedback condition resembles the operation of a standard body-powered prosthesis. The visual feedback condition, on the other hand, resembles the operation of a standard myoelectric prosthesis in that vision serves as the primary sensory feedback modality. The significantly improved identification accuracy with force feedback over vision gives credence to the widespread use of body-powered devices today that remain relatively unchanged in design since their development over 60 years ago [1]. Amputees want satisfactory function out of their prostheses, and the lack of sensory feedback is one area of dissatisfaction with myoelectric devices [2]. The differences in performance presented here help quantify the functional underpinnings of this dissatisfaction. The qualitative results tell a similar story with the majority of participants finding the visual feedback condition “Difficult” and “Third” both in terms of preference and ability to distinguish the blocks; the visual+force feedback condition was ranked “Easy” and “First,” respectively. When visual feedback is removed, as was the case for the force feedback and no feedback conditions, our findings allude to the operation of body-powered and myoelectric devices in the absence of vision. For an amputee, operation of the prosthesis in the absence of vision can be a common occurrence. This happens for instance when operating the prosthesis in a dimly lit or dark room, or in an environment where vision is obscured altogether, such as a jacket pocket. The results, however, are similar to the case where vision is available: identification accuracy is significantly improved when force feedback is present. Although identification completion time is smaller with no feedback, the shorter duration can be attributed to a strategy based largely on guessing. This is supported by the qualitative findings that all participants found the no feedback condition “difficult” and “Fourth” both in terms of preference and ability to distinguish the blocks; the force feedback condition was ranked “Neutral” and “Second,” respectively. Together, these findings provide evidence supporting the preference of many amputees to use a body-powered prostheses, and provide a possible explanation as to why many amputees are dissatisfied with the lack of haptic feedback in their myoelectric prostheses. Still, the future of upperlimb prosthetics seems to favor myoelectric over bodypowered devices. Indeed, much of the current work in the field is focused on improved mechanical design [4]–[6], advanced control [7], [8] and haptic display for myoelectric devices [11]–[17]. Unfortunately, these efforts fail to capture the simple but effective essence of the Bowden cable in bodypowered prostheses. By allowing a proprioceptive sense of the terminal device, as well as exteroceptive feedback that is displayed to the same body site used to generate control for the terminal device, the Bowden cable couples the dynamics of the amputee’s body to the dynamics of the world encountered in the grasp of the terminal device. It may prove beneficial to shift research focus to myoelectric controllers that give amputees direct control over terminal device aperture and to haptic displays that feature force feedback in a manner that supports coupled dynamics between body and world. This concept was first introduced by Simpson as extended physiological proprioception (EPP) over 30 years ago [21] and was briefly investigated by others [22]–[24]. We have also begun to explore these concepts [25], [32]–[34] and feel that a myoelectric prosthesis that features the principles of EPP would undoubtedly move closer to externally powered prostheses that are capable of being embodied by an amputee. Our hope is that the findings presented in this study will lead to a revival of focus on the benefits of EPP as it pertains to externally powered prostheses. In addition to the previously mentioned limitations of our current study, there are a few limitations that should be addressed in future investigations. First, we only tested nonamputee participants, and we did so with a small sample size that was not determined a priori through a power analysis. While we are encouraged by the fact that we still observed significant differences in performance between the conditions, the sensitivity analysis demonstrated how susceptible our findings are to the amount of rest and alertness in one of our participants. At the same time, our findings need to be verified in an amputee population before their real clinical impact can be realized. Second, while our device functioned along the same principles as a body-powered prosthesis, the use of the contralateral arm limits its use outside of the simple stiffness identification task presented here. Therefore, alternative control schemes, such as bi-scapular abduction, need to be considered when attempting more complicated bimanual tasks such as activities of daily living. Third, because each group of eight objects presented for a given feedback condition represented a random sample from the 12 available blocks (four from each stiffness), our protocol does not allow for a systematic analysis of which blocks were more easily discriminated. This random presentation of eight blocks also precluded an explicit investigation of the learning effects in each condition. Finally, our protocol did not allow for a comparison between the two conditions featuring visual feedback, and the two conditions without visual feedback in terms of completion time. At the same time, studying the effects of haptic and visual feedback on both accuracy and time may have presented conflicting goals to our participants (although it seems that our participants on average chose to focus on accuracy). Despite all of these limitations, however, we feel that in this small exploratory study we have empirically validated the utility of force feedback in an upper-limb prosthesis, a result that is useful to the larger prosthetics research community. Authorized licensed use limited to: Johns Hopkins University. Downloaded on July 28,2020 at 13:01:31 UTC from IEEE Xplore. Restrictions apply. BROWN et al.: EMPIRICAL EVALUATION OF FORCE FEEDBACK IN BODY-POWERED PROSTHESES V. C ONCLUSION AND F UTURE W ORK In this study, we have effectively made non-amputee participants trans-radial amputees using a custom body-powered prosthesis. Since our participants had no exposure to a bodypowered prosthesis, we were able to see the effect of both vision and force feedback in a prosthesis without the influence of prior experience. Our findings help explain on a functional level why body-powered prostheses are still in widespread use today. At the same time, our findings are supported by similar observations on teleoperation. It would, therefore, seem appropriate that myoelectric prostheses would benefit from the addition of haptic display mechanisms that feature force feedback. While the non-trivial nature of this task cannot be ignored (myoelectric prostheses rely on control from the muscle’s electrical activity, not mechanical limb movement), the impact of these findings and their consistency with those in teleoperation suggest this research aim is a worthwhile undertaking. While the present study has provided an empirical foundation of the benefits of force feedback in body-powered devices, it has also presented a unique experimental platform that can be used for future investigations. In addition to more thoroughly investigating the learning effects in each of the four conditions presented here, we envision many follow-on studies that investigate the utility of other types of haptic feedback (such as vibrotactile feedback), other types of terminal devices (such as voluntary-opening), other types of tasks, and other control schemes (such as myoelectric control). We also envision other studies comparing performance with our mock prosthesis with improved dynamic response to performance with the intact hand, comparing performance differences between amputee and non-amputee participants, as well as comparing our visual+force and visual feedback conditions to body-powered and myoelectric prostheses, respectively. These latter comparison will further assess the clinical relevance of our device as an experimental platform. All of these studies should be guided by the outcome of a power analysis to ensure a sufficient sample size, and the potential confound of the time versus accuracy tradeoff should be avoided where possible. R EFERENCES [1] D. S. Childress, “Closed-loop control in prosthetic systems: Historical perspective,” Ann. Biomed. Eng., vol. 8, no. 4–6, pp. 293–303, Jan. 1980. [2] E. Biddiss, D. Beaton, and T. Chau, “Consumer design priorities for upper limb prosthetics,” Disabil. Rehabil. Assist. Technol., vol. 2, no. 6, pp. 346–357, Jan. 2007. [3] E. a. Biddiss and T. T. Chau, “Upper limb prosthesis use and abandonment: A survey of the last 25 years,” Prosthet. Orthot. Int., vol. 31, no. 3, pp. 236–57, Sep. 2007. [4] Deka Arm, [Online]. Available: http://www.dekaresearch.com/deka_arm. shtml [5] Johns Hopkins Applied Physics Laboratory Modular Prosthetic Limb, [Online]. Available: http://www.jhuapl.edu/prosthetics/scientists/mpl.asp [6] Touch Bionics I-Limb Ultra, [Online]. Available: http://www. touchbionics.com/products/active-prostheses/i-limb-ultra [7] A. B. Ajiboye and R. F. ff Weir, “A heuristic fuzzy logic approach to EMG pattern recognition for multifunctional prosthesis control,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 13, no. 3, pp. 280–91, Sep. 2005. [8] T. A. Kuiken et al., “Targeted muscle reinnervation for real-time myoelectric control of multifunction artificial arms,” J. Am. Med. Assoc., vol. 301, no. 6, pp. 619–28, Feb. 2009. [9] G. F. Shannon, “A myoelectrically-controlled prosthesis with sensory feedback,” Med. Biol. Eng., vol. 17, no. 1, pp. 73–80, Jan. 1979. 225 [10] G. W. G. Wang, X. Z. X. Zhang, J. Z. J. Zhang, and W. Gruver, “Gripping force sensory feedback for a myoelectrically controlled forearm prosthesis,” in Proc. IEEE Int. Conf. Syst., Man Cybern., 1995, vol. 1, pp. 501–504. [11] R. Christiansen, J. L. Contreras-Vidal, R. B. Gillespie, P. A. Shewokis, and M. K. O’Malley, “Vibrotactile feedback of pose error enhances myoelectric control of a prosthetic hand,” in IEEE World Haptics Conf., 2013, pp. 531–536. [12] H. Witteveen, F. Luft, J. Rietman, and P. Veltink, “Stiffness feedback for myoelectric forearm prostheses using vibrotactile stimulation,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 22, no. 1, pp. 53–61, Jun. 2014. [13] E. Rombokas, C. E. Stepp, C. Chang, M. Malhotra, and Y. Matsuoka, “Vibrotactile sensory substitution for electromyographic control of object manipulation,” IEEE Trans. Biomed. Eng., vol. 60, no. 8, pp. 2226–32, Aug. 2013. [14] J. Wheeler, K. Bark, J. Savall, and M. Cutkosky, “Investigation of rotational skin stretch for proprioceptive feedback with application to myoelectric systems,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 18, no. 1, pp. 58–66, Feb. 2010. [15] C. Antfolk et al., “Transfer of tactile input from an artificial hand to the forearm: Experiments in amputees and able-bodied volunteers,” Disabil. Rehabil., Assist. Technol., vol. 8, no. 3, pp. 249–54, May 2013. [16] S. G. Meek, S. C. Jacobsen, and P. P. Goulding, “Extended physiologic taction: Design and evaluation of a proportional force feedback system,” J. Rehabil. Res. Develop., vol. 26, no. 3, pp. 53–62, Jan. 1989. [17] C. Antfolk et al., “Artificial redirection of sensation from prosthetic fingers to the phantom hand map on transradial amputees: Vibrotactile versus mechanotactile sensory feedback,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 21, no. 1, pp. 112–120, Jan. 2013. [18] P. D. Marasco, K. Kim, J. E. Colgate, M. a. Peshkin, and T. A. Kuiken, “Robotic touch shifts perception of embodiment to a prosthesis in targeted reinnervation amputees,” Brain, J. Neurol., vol. 134, pp. 747–58, Mar. 2011. [19] K. Kim and J. E. Colgate, “Haptic feedback enhances grip force control of sEMG-controlled prosthetic hands in targeted reinnervation amputees,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 20, no. 6, pp. 798–805, Nov. 2012. [20] K. Kim, J. E. Colgate, J. J. Santos-Munne, A. Makhlin, and M. A. Peshkin, “On the design of miniature haptic devices for upper extremity prosthetics,” IEEE/ASME Trans. Mechatron., vol. 15, no. 1, pp. 27–39, Feb. 2010. [21] D. Simpson, “The choice of control system for the multimovement prosthesis: Extended physiological proprioception (EPP),” in The Control of Upper-Extremity Prostheses and Orthoses. New York: Thomas, 1974, ch. 15, pp. 146–150. [22] J. A. Doubler and D. S. Childress, “Design and evaluation of a prosthesis control system based on the concept of extended physiological proprioception,” J. Rehabil. Res. Develop., vol. 21, no. 1, pp. 19–31, May 1984. [23] J. A. Doubler and D. S. Childress, “An analysis of extended physiological proprioception as a prosthesis-control technique,” J. Rehabil. Res. Develop., vol. 21, no. 1, pp. 5–18, 1984. [24] R. F. Weir, C. W. Heckathorne, and D. S. Childress, “Cineplasty as a control input for externally powered prosthetic components,” J. Rehabil. Res. Develop., vol. 38, no. 4, pp. 357–63, 2001. [25] J. D. Brown, R. B. Gillespie, D. Gardner, and E. a. Gansallo, “Co-location of force and action improves identification of forcedisplacement features,” in Proc. IEEE Haptics Symp., Mar. 2012, pp. 187–193. [26] R. C. Goertz, “Mechanical master-slave manipulator,” Nucleonics, vol. 12, no. 11, pp. 45–46, 1954. [27] R. C. Goertz, “A force-reflecting positional servomechanism,” Nucleonics, vol. 10, no. 11, pp. 43–45, 1952. [28] R. C. Goertz and W. M. Thompson, “Electronically controlled manipulator,” Nucleonics, vol. 12, no. 11, pp. 46–47, 1954. [29] J. G. Wildenbeest, D. A. Abbink, C. J. Heemskerk, F. C. van der Helm, and H. Boessenkool, “The impact of haptic feedback quality on the performance of teleoperated assembly tasks,” IEEE Trans. Haptics, vol. 6, no. 2, pp. 242–252, Apr. 2013. [30] J. Draper, W. Moore, J. Herndon, and B. Weil, “Effects of force reflection on servomanipulator task performance,” in Proc. Int. Topical Meet. Remote Syst. Robot. Hostile Environ., 1986. [31] B. Hannaford, L. Wood, D. A. Mcaffee, and H. Zak, “Performance evaluation of a six-axis generalized force-reflecting teleoperator,” IEEE Trans. Syst., Man, Cybern., vol. 21, no. 3, pp. 620–633, May/Jun. 1991. Authorized licensed use limited to: Johns Hopkins University. Downloaded on July 28,2020 at 13:01:31 UTC from IEEE Xplore. Restrictions apply. 226 IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, VOL. 25, NO. 3, MARCH 2017 [32] R. B. Gillespie, D. Kim, J. M. Suchoski, B. Yu, and J. D. Brown, “Series elasticity for free free-space motion for free,” in Proc. IEEE Haptics Symp., Feb. 2014, pp. 609–615. [33] R. B. Gillespie et al., “Toward improved sensorimotor integration and learning using upper-limb prosthetic devices,” in Proc. IEEE Eng. Med. Biol. Soc., Sep. 2010, pp. 5077–5080. [34] J. D. Brown and R. B. Gillespie, “The effect of force/motion coupling on motor and cognitive performance,” in Proc. IEEE World Haptics Conf., Jun. 2011, pp. 197–202. [35] K. Drewing, A. Ramisch, and F. Bayer, “Haptic, visual and visuo-haptic softness judgments for objects with deformable surfaces,” in Proc. IEEE World Haptics Conf., 2009, pp. 640–645. [36] D. Atkins, D. C. Heard, and W. H. Donovan, “Epidemiologic overview of individuals with upper-limb loss and their reported research priorities,” J. Prosthet. Orthot., vol. 8, no. 1, pp. 1–11, 1996. [37] L. Haverkate, G. Smit, and D. H. Plettenburg, “Assessment of bodypowered upper limb prostheses by able-bodied subjects, using the box and blocks test and the nine-hole peg test,” Prosthet. Orthot. Int., p. , Oct. 2014. Jeremy D. Brown (M’11) received the Ph.D. degree in mechanical engineering from the University of Michigan, Ann Arbor, MI, USA, in 2014. He is now a Postdoctoral Research Fellow in the Department of Mechanical Engineering and Applied Mechanics and the Haptics Group in the GRASP Lab at the University of Pennsylvania, Philadelphia, PA, USA. His research focuses on the interface between humans and robots with a specific focus on medical applications and haptic feedback. Dr. Brown was honored to receive several awards including the National Science Foundation (NSF) Graduate Research Fellowship, the Best Student Paper award from the IEEE Haptics Symposium in 2012, and the Penn Postdoctoral Fellowship for Academic Diversity. Mackenzie K. Shelley will receive the B.S. degree in computer science from the University of Michigan, Ann Arbor, MI, USA, in 2016. Her undergraduate research experience has been centered around medical applications ranging from haptic feedback in prosthetic devices to automated organ segmentation in CT scans. She has previously worked for Google, Square, and Airbnb. Upon graduation, she will return to Square as a Software Engineer on risk systems. Alicia J. Davis has been a clinical prosthetist and orthotist since 1991 and is currently the Residency Program Director at the University of Michigan Orthotic and Prosthetic Center. She taught at Eastern Michigan University’s Master of Prosthetics and Orthotics Program in upper extremity prosthetics. Her research interests are focused on upper extremity prosthetics. Dr. Davis serves on the Board of Directors of the American Academy of Orthotists and Prosthetists. Timothy S. Kunz completed his prosthetic residency training at the University of Michigan Health System in 2014 and orthotic residency training from the University of Oklahoma Health Sciences Center in 2015. He is now a prosthetic and orthotic clinician at Kootenai Prosthetics and Orthotics Services in Post Falls, ID, USA. Duane Gardner received the B.S. degree in mechanical engineering from the University of Michigan, Ann Arbor, MI, USA, in 2014. As an undergraduate, he served as a research assistant focusing on the incorporation of feedback into prosthetic devices. He is currently an engineer at the Boeing Company. R. Brent Gillespie (M’97) received the B.S. degree in mechanical engineering from the University of California, Davis, CA, USA, in 1986, the M.S. degree in piano performance from the San Francisco Conservatory of Music, San Francisco, CA, USA, in 1989, and the M.S. and Ph.D. degrees in mechanical engineering from Stanford University, Stanford, CA, USA, in 1992 and 1996, respectively. He is currently with the Department of Mechanical Engineering, University of Michigan, Ann Arbor, USA. His current research interests include haptic interface and teleoperator control, human motor control, and robot-assisted rehabilitation after neurological injury. Authorized licensed use limited to: Johns Hopkins University. Downloaded on July 28,2020 at 13:01:31 UTC from IEEE Xplore. Restrictions apply.