3D localization of objects of daily living activities for rehabilitation applications

Assisting paralysed people for conducting activities of daily living (ADL) is a major challenge in the area of collaborative robotics. The first predicament of wheelchair mounted robotic arms and/or exoskeletons is to identify and localize the objects for ADL in a 3-dimensional (3D) space so that they can be manipulated according to the user’s need. This requires robust integration and adaptation of state-of-the-art algorithms in a seamless framework for faster, accurate, and reliable manner ensuring the user’s safety. To this end, the current work focuses on the adaptation of the popular YOLO algorithm for image based object identification for detecting objects for ADL. In order to facilitate 3D localization of these objects we integrated the YOLO with a stereo vision based algorithm for depth perception. Thus the integrated system can provide both object identification and 3D localization in real-time so that a wheelchair mounted robotic arm and/or exoskeleton can be semi-autonomously guided to perform ADL for the disabled people. The proposed solution is resource efficient as it can be implemented using low-cost hardware so that the overall product can be affordable, environmentally efficient (as it may consume less power due to low computational complexity). This will lead to enhanced environment perception for the robots which can drastically improve the human robot interaction models for optimal assistance.