Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/1774674.1774693acmotherconferencesArticle/Chapter ViewAbstractPublication PagespermisConference Proceedingsconference-collections
research-article

Survey of domain-specific performance measures in assistive robotic technology

Published: 19 August 2008 Publication History

Abstract

Assistive robotics have been developed for several domains, including autism, eldercare, intelligent wheelchairs, assistive robotic arms, external limb prostheses, and stroke rehabilitation. Work in assistive robotics can be divided into two larger research areas: technology development, where new devices, software, and interfaces are created; and clinical application, where assistive technology is applied to a given end-user population. Moving from technology development towards clinical applications is a significant challenge. Developing performance metrics for assistive robots can unveil a larger set of challenges. For example, what well established performance measures should be used for evaluation to lend credence to a particular assistive robotic technology from a clinician's perspective? In this paper, we survey several areas of assistive robotic technology in order to demonstrate domain-specific means for evaluating the performance of an assistive robot system.

References

[1]
N. K. Aaronson, C. Acquadro, J. Alonso, G. Apolone, D. Bucquet, M. Bullinger, K. Bungay, S. Fukuhara, B. Gandek, S. Keller, D. Razavi, R. Sanson-Fisher, M. Sullivan, S. Wood-Dauphinee, A. Wagner, and J. E. Ware Jr. Intl. Quality of Life Assessment (IQOLA) Project. Quality of Life Research, 1(5):349--351, 2004.
[2]
AMPS.com. Assess of Motor and Process Skills. In http://www.ampsintl.com/tasks.htm, 2008.
[3]
S. Au. Powered Ankle-Foot Prosthesis for the Improvement of Amputee Walking Economy. PhD thesis, MIT, 2007.
[4]
R. Bohannon and M. Smith. Interrater reliability of a modified Ashworth scale of muscle spasticity. Physical Therapy, 67(2):206--7, 1987.
[5]
B. R. Brewer, S. Pradhan, G. Carvell, P. Sparto, D. Josbeno, and A. Delitto. Application of Machine Learning to the Development of a Quantitative Clinical Biomarker for the Progression of Parkinson's Disease. In Rehab. Eng. Society of North America Conf., 2008.
[6]
J. Brody. Prospects for an ageing population. Nature, 315(6019):463--466, 1985.
[7]
H. Burger, F. Franchignoni, A. Heinemann, S. Kotnik, and A. Giordano. Validation of the orthotics and prosthetics user survey upper extremity functional status module in people with unilateral upper limb amputation. J. of Rehab. Medicine, 40(5):393--399, 2008.
[8]
CALL Centre. Smart wheelchair. In http://callcentre.education.ed.ac.uk/Smart_WheelCh/smart_wheelch.html, 2008.
[9]
T. Carlson and Y. Demiris. Human-wheelchair collaboration through prediction of intention and adaptive assistance. In IEEE Intl. Conf. on Robotics and Automation, 2008.
[10]
O. Celik, M. K. O'Malley, C. Boake, H. Levin, S. Fischer, and T. Reistetter. Comparison of robotic and clinical motor function improvement measures for sub-acute stroke patients. In IEEE Intl. Conf. on Robotics and Automation, 2008.
[11]
E. Chaves, A. Koontz, S. Garber, R. Cooper, and A. Williams. Clinical evaluation of a wheelchair mounted robotic arm. Technical report, Univ. of Pittsburgh, 2003.
[12]
J. A. Cohen. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20:37--46, 1960.
[13]
R. Cooper, M. Boninger, R. Cooper, A. Dobson, J. Kessler, M. Schmeler, and S. Fitzgerald. Use of the Independence 3000 IBOT Transporter at home and in the community. J. of Spinal Cord Medicine, 26(1):79--85, 2003.
[14]
H. Day and J. Jutai. Piads in the world. In http://www.piads.ca/worldmapshtm/worldmap.asp, 2008.
[15]
H. Day, J. Jutai, and K. Campbell. Development of a scale to measure the psychosocial impact of assistive devices: lessons learned and the road ahead. Disability and Rehab., 24(1--3):31--37, 2002.
[16]
DEKA Research and Development Corporation. DEKA Evolved Thinking. In http://www.dekaresearch.com, 2008.
[17]
W. DeWeerdt and M. Harrison. Measuring recovery of arm-hand function in stroke patients: a comparison of the Brunnstrom-Fugl-Meyer test and the Action Research Arm Test. Physiother Can, 37(2):65--70, 1985.
[18]
M. D. Ellis, T. Sukal, T. DeMott, and J. P. A. Dewald. ACT3D exercise targets gravity-induced discoordination and improves reaching work area in individuals with stroke. In IEEE Intl. Conf. on Rehab. Robotics, 2007.
[19]
S. Fahn, R. Elton, et al. Unified Parkinson's Disease Rating Scale. Recent developments in Parkinson's disease, 2:153--163, 1987.
[20]
D. J. Feil-Seifer and M. J. Matarić. Robot-assisted therapy for children with autism spectrum disorders. In Conf. on Interaction Design for Children: Children with Special Needs, 2008.
[21]
D. J. Feil-Seifer and M. J. Matarić. Toward socially assistive robotics for augmenting interventions for children with autism spectrum disorders. In Intl. Symposium on Experimental Robotics, 2008.
[22]
M. Ferraro, J. Demaio, J. Krol, C. Trudell, K. Rannekleiv, L. Edelstein, P. Christos, M. Aisen, J. England, and S. Fasoli. Assessing the Motor Status Score: A Scale for the Evaluation of Upper Limb Motor Outcomes in Patients after Stroke. Neurorehabilitation and Neural Repair, 16(3):283, 2002.
[23]
A. Fisher. AMPS: Assessment of Motor and Process Skills Volume 1: Development, Standardisation, and Administration Manual. Ft Collins, CO: Three Star Press Inc, 2003.
[24]
B. Freeman. Guidelines for Evaluating Intervention Programs for Children with Autism. J. of Autism and Developmental Disorders, 27(6):641--651, 1997.
[25]
A. Fugl-Meyer, L. Jaasko, I. Leyman, S. Olsson, and S. Steglind. The post-stroke hemiplegic patient. 1. a method for evaluation of physical performance. Scandinavian J. of Rehab. Medicine, 7(1):13--31, 1975.
[26]
G. Fulk, M. Frick, A. Behal, and M. Ludwig. A wheelchair mounted robotic arm for individuals with multiple sclerosis. Technical report, Clarkson Univ., 2005.
[27]
D. Gladstone, C. Danells, and S. Black. The Fugl-Meyer Assessment of Motor Recovery after Stroke: A Critical Review of Its Measurement Properties. Neurorehabilitation and Neural Repair, 16(3):232, 2002.
[28]
D. Greenwood, W. Whyte, and I. Harkavy. Participatory Action Research as a Process and as a Goal. Human Relations, 46(2):175--192, 1993.
[29]
J. Guralnik, E. Simonsick, L. Ferrucci, R. Glynn, L. Berkman, D. Blazer, P. Scherr, and R. Wallace. A short physical performance battery assessing lower extremity function: association with self-reported disability and prediction of mortality and nursing home admission. J. of Gerontology, 49(2): M85--94, 1994.
[30]
K. Haigh and H. A. Yanco. Automation as caregiver: A survey of issues and technologies. In AAAI-2002 Workshop on Automation as Caregiver: The Role of Intelligent Technology in Elder Care, 2002.
[31]
A. W. Heinemann, R. K. Bode, and C. O'Reilly. Development and measurement properties of the orthotics and prosthetics users' survey (opus): a comprehensive set of clinical outcome instruments. Prosthetics and Orthotics Intl., 27(3):191--206, 2003.
[32]
M. Hillman. Rehabilitation robotics from past to present - a historical perspective. In IEEE 8th Intl. Conf. on Rehab. Robotics, 2003.
[33]
S. J. Housman, V. Le, T. Rahman, R. J. Sanchez, and D. J. Reinkensmeyer. Arm-training with t-wrex after chronic stroke: Preliminary results of a randomized controlled trial. In IEEE Intl. Conf. on Rehab. Robotics, 2007.
[34]
X. L. Hu, K. Y. Tong, R. Song, X. j. Zheng, I. F. Lo, and K. H. Lui. Myoelectrically controlled robotic systems that provide voluntary mechanical help for persons after stroke. In IEEE Intl. Conf. on Rehab. Robotics, 2007.
[35]
W. Jenkins and M. Merzenich. Reorganization of neocortical representations after brain injury: a neurophysiological model of the bases of recovery from stroke. Progress in Brain Research, 71:249--66, 1987.
[36]
E. S. Kim, E. Newland, R. Paul, and B. Scassellati. A Robotic Therapist For Positive, Affective Prosody in High-Functioning Autistic Children. In Poster Pres. at the Intl. Meeting for Autism Research, 2008.
[37]
H. Kozima and C. Nakagawa. Longitudinal child-robot interaction at preschool. In AAAI Spring Symposium on Multidisciplinary Collaboration for Socially Assistive Robotics, pages 27--32, 2007.
[38]
W. Lee, M. Zhang, P. Chan, and D. Boone. Gait Analysis of Low-Cost Flexible-Shank Trans-Tibial Prostheses. IEEE Trans. on Neural Systems and Rehab. Eng., 14(3):370--377, 2006.
[39]
R. Likert. A technique for the measurement of attitudes. Archives of Psychology, 140(5):1--55, 1932.
[40]
C. Lord, S. Risi, L. Lambrecht, E. H. C. Jr., B. L. Leventhal, P. C. DiLavore, A. Pickles, and M. Rutter. The autism diagnostic observation schedule-generic: A standard measure of social and communication deficits associated with the spectrum of autism. J. of Autism and Developmental Disorders, 30(3):205--223, 2000.
[41]
D. M. McNair, M. Lorr, and L. F. Droppleman. Profile of mood states. In Educational and Industrial Testing Service, 1992.
[42]
MedFriendly.com. Functional independence measure. In http://www.medfriendly.com/functionalindependencemeasure.html, 2007.
[43]
F. Michaud, T. Salter, A. Duquette, H. Mercier, M. Lauria, H. Larouche, and F. Larose. Assistive technologies and child-robot interaction. In AAAI Spring Symposium on Multidisciplinary Collaboration for Socially Assistive Robotics, 2007.
[44]
L. Miller, K. Stubblefield, R. Lipschutz, B. Lock, and T. Kuiken. Improved Myoelectric Prosthesis Control Using Targeted Reinnervation Surgery: A Case Series. IEEE Trans. on Neural Systems and Rehab. Eng., 16(1):46--50, 2008.
[45]
MobileRobots Inc. Independence-enhaning wheelchair. In http://www.activrobots.com/RESEARCH/wheelchair.html, 2008.
[46]
MobileRobots Inc. Robotic chariot. In http://activrobots.com/robots/robochariot.html, 2008.
[47]
D. Molloy and T. Standish. A Guide to the Standardized Mini-Mental State Examination. Intl. Psychogeriatrics, 9(S1):87--94, 2005.
[48]
B. Robins, K. Dautenhahn, and J. Dubowsky. Robots as Isolators or Mediators for Children with Autism? A Cautionary Tale. In AISB05: Social Intelligence and Interaction in Animals, Robots and Agents, 2005.
[49]
B. Robins, K. Dautenhahn, R. te Boekhorst, and A. Billard. Robots as assistive technology -- does appearance matter? In IEEE Int. Workshop on Robot and Human Interactive Communication, 2004.
[50]
B. Robins, K. Dautenhahn, R. te Boekhorst, and C. Nehaniv. Behaviour Delay and Robot Expressiveness in Child-Robot Interactions: A User Study on Interaction Kinesics. In Intl. Conf. on Human-Robot Interaction, 2008.
[51]
R. Robins, C. Fraley, and R. Krueger. Handbook of Research Methods in Personality Psychology. Guilford Press, 2007.
[52]
G. Römer and H. Stuyt. Compiling a Medical Device File and a Proposal for an Intl. Standard for Rehabilitation Robots. IEEE Intl. Conf. on Rehab. Robotics, pages 489--496, 2007.
[53]
T. Salter, F. Michaud, D. Létourneau, D. Lee, and I. Werry. Using proprioceptive sensors for categorizing interactions. In Human-Robot Interaction, 2007.
[54]
E. Sanderson and R. Scott. UNB test of prosthetic function: a test for unilateral amputees {test manual}. Fredericton, New Brunswick, Canada, Univ. of New Brunswick, 1985.
[55]
B. Scassellati. How Social Robots Will Help Us to Diagnose, Treat, and Understand Autism. Robotics Research: Results of the 12th Intl. Symposium ISRR, 28:552--563, 2007.
[56]
F. Shic, B. Scassellati, D. Lin, and K. Chawarska. Measuring context: The gaze patterns of children with autism evaluated from the bottom-up. Intl. Conf. on Development and Learning, pages 70--75, 2007.
[57]
R. Simpson. Smart wheelchairs: A literature review. J. of Rehab. Research Development, 42(4):423--36, 2005.
[58]
R. C. Simpson. Improved automatic adaption through the combination of multiple information sources. PhD thesis, Univ. of Michigan, Ann Arbor, 1997.
[59]
H. Sintonen. The 15-d measure of health related quality of life: Reliability, validity and sensitivity of its health state descriptive system. Working Paper 41, Center for Health Program Evaluation, 1994.
[60]
C. Stanton, P. Kahn, R. Severson, J. Ruckert, and B. Gill. Robotic animals might aid in the social development of children with autism. In Intl. Conf. on Human Robot Interaction, pages 271--278, 2008.
[61]
C. Stranger, C. Anglin, W. S. Harwin, and D. Romilly. Devices for assisting manipulation: A summary of user task priorities. IEEE Trans. on Rehab. Eng., 4(2):256--265, 1994.
[62]
T. Taha, J. V. Miró, and G. Dissanayake. Pompdp-based long-term user intention prediction for wheelchair navigation. In IEEE Intl. Conf. on Robotics and Automation, 2008.
[63]
A. Tapus, J. Fasola, and M. J. Matarić. Socially assistive robots for individuals suffering from dementia. In Human-Robot Interaction Intl. Conf., Workshop on Robotic Helpers: User Interaction, Interfaces and Companions in Assistive and Therapy Robotics, 2008.
[64]
C. Tardif, M. Plumet, J. Beaudichon, D. Waller, M. Bouvard, and M. Leboyer. Micro-analysis of social interactions between autistic children and normal adults in semi-structured play situations. Intl. J. of Behavioral Development, 18(4):727--747, 1995.
[65]
H. Tijsma, F. Liefhebber, and J. Herder. Evaluation of new user interface features for the manus robot arm. In IEEE Intl. Conf. on Rehab. Robotics, pages 258--263, 2005.
[66]
K. Tsui, H. Yanco, D. Kontak, and L. Beliveau. Development and evaluation of a flexible interface for a wheelchair mounted robotic arm. In Intl. Conf. on Human Robot Interaction, 2008.
[67]
Univ. of Illinois Chicago. Power mobility skills checklist. In http://internet.dscc.uic.edu/forms/0534.pdf, 2008.
[68]
US Food and Drug Administration. Guidance for industry, E6 good clinical practice: consolidated guidance. Federal Register, 10:691--709, 1997.
[69]
G. Uswatte, E. Taub, D. Morris, K. Light, and P. Thompson. The Motor Activity Log-28: Assessing daily use of the hemiparetic arm after stroke. Neurology, 67(7):1189, 2006.
[70]
K. Wada, T. Shibata, T. Saito, and K. Tanie. Effects of robot-assisted activity for elderly people and nurses at a day service center. IEEE, 92(11):1780--1788, 2004.
[71]
J. Wagner and H. Van der Loos. Training strategies for the user interface of vocational assistive robots. Intl. Conf. on Eng. in Medicine and Biology Society, EMBC, 2, 2004.
[72]
S. Wang, J. Keller, K. Burks, M. Skubic, and H. Tyrer. Assessing Physical Performance of Elders Using Fuzzy Logic. IEEE Intl. Conf. on Fuzzy Systems, pages 2998--3003, 2006.
[73]
J. A. Ward, S. Balasubramanian, T. Sugar, and J. He. Robotic gait trainer reliability and stroke patient case study. In IEEE Intl. Conf. on Rehab. Robotics, 2007.
[74]
D. Wilson, L. Baker, and J. Craddock. Functional test for the hemiparetic upper extremity. American J. of Occupational Therapy, 38(3):159--64, 1984.
[75]
S. Wolf, P. Thompson, D. Morris, D. Rose, C. Winstein, E. Taub, C. Giuliani, and S. Pearson. The EXCITE Trial: Attributes of the Wolf Motor Function Test in Patients with Subacute Stroke. Neurorehabilitation and Neural Repair, 19(3):194, 2005.
[76]
S. Wood-Dauphinee. Assessing quality of life in clinical research: From where have we come and where are we going? J. of Clinical Epidemiology, 52(4):355--363, 1999.
[77]
H. Yanco. Evaluating the performance of assistive robotic systems. Performance Metrics for Intelligent Systems (PerMIS) Workshop, pages 21--25, 2002.
[78]
F. Zijlstra. Efficiency in work behaviour: A design approach for modern tools. PhD thesis, Delft Univ., 1993.

Cited By

View all
  • (2022)Learning from Carers to inform the Design of Safe Physically Assistive Robots - Insights from a Focus Group StudyProceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction10.5555/3523760.3523859(703-707)Online publication date: 7-Mar-2022
  • (2022)Learning from Carers to inform the Design of Safe Physically Assistive Robots - Insights from a Focus Group Study2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI)10.1109/HRI53351.2022.9889658(703-707)Online publication date: 7-Mar-2022
  • (2020)Towards the acceptance of care robots by senior usersEighth International Conference on Technological Ecosystems for Enhancing Multiculturality10.1145/3434780.3436545(422-429)Online publication date: 21-Oct-2020
  • Show More Cited By

Index Terms

  1. Survey of domain-specific performance measures in assistive robotic technology

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    PerMIS '08: Proceedings of the 8th Workshop on Performance Metrics for Intelligent Systems
    August 2008
    333 pages
    ISBN:9781605582931
    DOI:10.1145/1774674
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    • NIST: National Institute of Standards & Technology

    In-Cooperation

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 19 August 2008

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. assistive technology
    2. end-user evaluation
    3. human-robot interaction
    4. performance measures
    5. robotics

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    PerMIS '08
    Sponsor:
    • NIST
    PerMIS '08: Performance Metrics for Intelligent Systems
    August 19 - 21, 2008
    Maryland, Gaithersburg

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)27
    • Downloads (Last 6 weeks)3
    Reflects downloads up to 21 Nov 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2022)Learning from Carers to inform the Design of Safe Physically Assistive Robots - Insights from a Focus Group StudyProceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction10.5555/3523760.3523859(703-707)Online publication date: 7-Mar-2022
    • (2022)Learning from Carers to inform the Design of Safe Physically Assistive Robots - Insights from a Focus Group Study2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI)10.1109/HRI53351.2022.9889658(703-707)Online publication date: 7-Mar-2022
    • (2020)Towards the acceptance of care robots by senior usersEighth International Conference on Technological Ecosystems for Enhancing Multiculturality10.1145/3434780.3436545(422-429)Online publication date: 21-Oct-2020
    • (2017)Soft brain-machine interfaces for assistive robotics: A novel control approach2017 International Conference on Rehabilitation Robotics (ICORR)10.1109/ICORR.2017.8009357(863-869)Online publication date: Jul-2017
    • (2016)A survey of reconfigurable service robots2016 International Conference on Research Advances in Integrated Navigation Systems (RAINS)10.1109/RAINS.2016.7764426(1-4)Online publication date: May-2016
    • (2016)Robot Enhanced Therapy for Children with Autism Disorders: Measuring Ethical AcceptabilityIEEE Technology and Society Magazine10.1109/MTS.2016.255470135:2(54-66)Online publication date: Jun-2016
    • (2015)Evaluating the use of robots to enlarge AAL services1Journal of Ambient Intelligence and Smart Environments10.3233/AIS-1503157:3(301-313)Online publication date: 8-Jun-2015
    • (2015)Adaptive motion control for a differentially driven semi-autonomous wheelchair platform2015 International Conference on Advanced Robotics (ICAR)10.1109/ICAR.2015.7251470(288-294)Online publication date: Jul-2015
    • (2014)Multi-modal control framework for a semi-autonomous wheelchair using modular sensor designsIntelligent Service Robotics10.1007/s11370-014-0149-77:3(145-155)Online publication date: 1-Jul-2014
    • (2013)Modular Robot Arm Design for Physical Human-Robot InteractionProceedings of the 2013 IEEE International Conference on Systems, Man, and Cybernetics10.1109/SMC.2013.762(4482-4487)Online publication date: 13-Oct-2013
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media