Search Results (44)

Ensuring the accurate tracking of hand and fingers movements is an ongoing challenge for upper limb rehabilitation assessment, as the high number of degrees of freedom and segments in the limited volume of the hand makes this a difficult task. The objective of this study is to evaluate the performance of two markerless approaches (the Leap Motion Controller and the Google MediaPipe API) in comparison to a marker-based one, and to improve the precision of the markerless methods by introducing additional data processing algorithms fusing multiple recording devices. Fifteen healthy participants were instructed to perform five distinct hand movements while being recorded by the three motion capture methods simultaneously. The captured movement data from each device was analyzed using a skeletal model of the hand through the inverse kinematics method of the OpenSim software. Finally, the root mean square errors of the angles formed by each finger segment were calculated for the markerless and marker-based motion capture methods to compare their accuracy. Our results indicate that the MediaPipe-based setup is more accurate than the Leap Motion Controller-based one (average root mean square error of 10.9° versus 14.7°), showing promising results for the use of markerless-based methods in clinical applications. Full article

This paper presents the development of a hardware-in-the-loop ground testbed featuring active gravity compensation via software-in-the-loop integration, specially designed to support research in autonomous robotic removal of space debris. The testbed is designed to replicate six degrees of freedom (6DOF) motion maneuvering to accurately simulate the dynamic behaviors of free-floating robotic manipulators and free-tumbling space debris under microgravity conditions. The testbed incorporates two industrial 6DOF robotic manipulators, a three-finger robotic gripper, and a suite of sensors, including cameras, force/torque sensors, and tactile tensors. Such a setup provides a robust platform for testing and validating technologies related to autonomous tracking, capture, and post-capture stabilization within the context of active space debris removal missions. Preliminary experimental results have demonstrated advancements in motion control, computer vision, and sensor fusion. This facility is positioned to become an essential resource for the development and validation of robotic manipulators in space, offering substantial improvements to the effectiveness and reliability of autonomous capture operations in space missions. Full article

Brain–computer interfaces (BCIs) are pivotal in translating neural activities into control commands for external assistive devices. Non-invasive techniques like electroencephalography (EEG) offer a balance of sensitivity and spatial-temporal resolution for capturing brain signals associated with motor activities. This work introduces MOVING, a Multi-Modal dataset of EEG signals and Virtual Glove Hand Tracking. This dataset comprises neural EEG signals and kinematic data associated with three hand movements—open/close, finger tapping, and wrist rotation—along with a rest period. The dataset, obtained from 11 subjects using a 32-channel dry wireless EEG system, also includes synchronized kinematic data captured by a Virtual Glove (VG) system equipped with two orthogonal Leap Motion Controllers. The use of these two devices allows for fast assembly (∼1 min), although introducing more noise than the gold standard devices for data acquisition. The study investigates which frequency bands in EEG signals are the most informative for motor task classification and the impact of baseline reduction on gesture recognition. Deep learning techniques, particularly EEGnetV4, are applied to analyze and classify movements based on the EEG data. This dataset aims to facilitate advances in BCI research and in the development of assistive devices for people with impaired hand mobility. This study contributes to the repository of EEG datasets, which is continuously increasing with data from other subjects, which is hoped to serve as benchmarks for new BCI approaches and applications. Full article

Upper limb functionality is essential to perform activities of daily living. It is critical to investigate neurorehabilitation therapies in order to improve upper limb functionality in post-stroke patients. This paper presents a soft-robotic bilateral system to provide rehabilitation therapy for hand and wrist joints. A sensorized glove that tracks finger and wrist joint movements is worn on the healthy limb, which guides the movement of the paretic limb. The input of sensors from the healthy limb is provided to the soft robotic exoskeleton attached to the paretic limb to mimic the motion. A proportional derivative flow-based control algorithm is used to perform bilateral therapy. To test the feasibility of the developed system, two different applications are performed experimentally: (1) Wrist exercise with a dumbbell, and (2) Object pick-and-place task. The initial tests of the developed system verified its capability to perform bilateral therapy. Full article

Non-linear and dynamic systems analysis of human movement has recently become increasingly widespread with the intention of better reflecting how complexity affects the adaptability of motor systems, especially after a stroke. The main objective of this scoping review was to summarize the non-linear measures used in the analysis of kinetic, kinematic, and EMG data of human movement after stroke. PRISMA-ScR guidelines were followed, establishing the eligibility criteria, the population, the concept, and the contextual framework. The examined studies were published between 1 January 2013 and 12 April 2023, in English or Portuguese, and were indexed in the databases selected for this research: PubMed^®, Web of Science^®, Institute of Electrical and Electronics Engineers^®, Science Direct^® and Google Scholar^®. In total, 14 of the 763 articles met the inclusion criteria. The non-linear measures identified included entropy (n = 11), fractal analysis (n = 1), the short-term local divergence exponent (n = 1), the maximum Floquet multiplier (n = 1), and the Lyapunov exponent (n = 1). These studies focused on different motor tasks: reaching to grasp (n = 2), reaching to point (n = 1), arm tracking (n = 2), elbow flexion (n = 5), elbow extension (n = 1), wrist and finger extension upward (lifting) (n = 1), knee extension (n = 1), and walking (n = 4). When studying the complexity of human movement in chronic post-stroke adults, entropy measures, particularly sample entropy, were preferred. Kinematic assessment was mainly performed using motion capture systems, with a focus on joint angles of the upper limbs. Full article

As human–robot interaction and teleoperation technologies advance, anthropomorphic control of humanoid arms has garnered increasing attention. However, accurately translating sensor-detected arm motions to the multi-degree freedom of a humanoid robotic arm is challenging, primarily due to occlusion issues with single-sensor setups, which reduce recognition accuracy. To overcome this problem, we propose a human-like arm control strategy based on multi-sensor fusion. We defined the finger bending angle to represent finger posture and employed a depth camera to capture arm movement. Consequently, we developed an arm movement tracking system and achieved anthropomorphic control of the imitation human arm. Finally, we verified our proposed method’s effectiveness through a series of experiments, evaluating the system’s robustness and real-time performance. The experimental results show that this control strategy can control the motion of the humanoid arm stably, and maintain a high recognition accuracy in the face of complex situations such as occlusion. Full article

The main goal of this research is to develop a highly advanced anthropomorphic control system utilizing multiple sensor technologies to achieve precise control of a robotic arm. Combining Kinect and IMU sensors, together with a data glove, we aim to create a multimodal sensor system for capturing rich information of human upper body movements. Specifically, the four angles of upper limb joints are collected using the Kinect sensor and IMU sensor. In order to improve the accuracy and stability of motion tracking, we use the Kalman filter method to fuse the Kinect and IMU data. In addition, we introduce data glove technology to collect the angle information of the wrist and fingers in seven different directions. The integration and fusion of multiple sensors provides us with full control over the robotic arm, giving it flexibility with 11 degrees of freedom. We successfully achieved a variety of anthropomorphic movements, including shoulder flexion, abduction, rotation, elbow flexion, and fine movements of the wrist and fingers. Most importantly, our experimental results demonstrate that the anthropomorphic control system we developed is highly accurate, real-time, and operable. In summary, the contribution of this study lies in the creation of a multimodal sensor system capable of capturing and precisely controlling human upper limb movements, which provides a solid foundation for the future development of anthropomorphic control technologies. This technology has a wide range of application prospects and can be used for rehabilitation in the medical field, robot collaboration in industrial automation, and immersive experience in virtual reality environments. Full article

This study introduces a new wearable fiber-optic sensor glove. The glove utilizes a flexible material, polydimethylsiloxane (PDMS), and a silicone tube to encapsulate fiber Bragg gratings (FBGs). It is employed to enable the self-perception of hand posture, gesture recognition, and the prediction of grasping objects. The investigation employs the Support Vector Machine (SVM) approach for predicting grasping objects. The proposed fiber-optic sensor glove can concurrently monitor the motion of 14 hand joints comprising 5 metacarpophalangeal joints (MCP), 5 proximal interphalangeal joints (PIP), and 4 distal interphalangeal joints (DIP). To expand the measurement range of the sensors, a sinusoidal layout incorporates the FBG array into the glove. The experimental results indicate that the wearable sensing glove can track finger flexion within a range of 0° to 100°, with a modest minimum measurement error (Error) of 0.176° and a minimum standard deviation (SD) of 0.685°. Notably, the glove accurately detects hand gestures in real-time and even forecasts grasping actions. The fiber-optic smart glove technology proposed herein holds promising potential for industrial applications, including object grasping, 3D displays via virtual reality, and human–computer interaction. Full article

Soft robotic gloves have attracted significant interest in hand rehabilitation in the past decade. However, current solutions are still heavy and lack finger-state monitoring and versatile treatment options. To address this, we present a lightweight soft robotic glove actuated by twisted string actuators (TSA) that provides whole-hand finger motion tracking. We have developed a virtual reality environment for hand rehabilitation training, allowing users to interact with various virtual objects. Fifteen small inertial measurement units are placed on the glove to predict finger joint angles and track whole-hand finger motion. We performed TSA experiments to identify design and control rules, by understanding how their response varies with input load and voltages. Grasping experiments were conducted to determine the grasping force and range of motion. Finally, we showcase an application of the rehabilitation glove in a Unity-based VR interface, which can actuate the operator’s fingers to grasp different virtual objects. Full article

Research studies show that serious games can increase patient motivation regardless of age or illness and be an affordable and promising solution with respect to conventional physiotherapy. In this paper, we present the latest evolution of our system for shoulder rehabilitation based on hand-finger tracking and projected augmented reality. This version integrates metrics to assess patient performance, monitors the game progress, and allows the selection of the game visualization mode (standard on-screen or projected augmented reality). Additionally, the new software tracks the velocity, acceleration, and normalized jerk of the arm-hand movements of the user. Specifically, sixteen healthy volunteers (eight technical and eight rehabilitation experts) tested our current prototype. The results showed that the serious game is engaging, its design is ergonomically sound, and the overall system could be a useful tool in shoulder rehabilitation. However, clinical validation is needed to assess that the serious game has the same effects as the selected therapy. This is the preliminary step toward laying the foundation for future studies that investigate abnormalities in shoulder movements by using hand-finger tracking. Full article

This paper presents a quasi-static model-based control algorithm for controlling the motion of a soft robotic exo-digit with three independent actuation joints physically interacting with the human finger. A quasi-static analytical model of physical interaction between the soft exo-digit and a human finger model was developed. Then, the model was presented as a nonlinear discrete-time multiple-input multiple-output (MIMO) state-space representation for the control system design. Input–output feedback linearization was utilized and a control input was designed to linearize the input–output, where the input is the actuation pressure of an individual soft actuator, and the output is the pose of the human fingertip. The asymptotic stability of the nonlinear discrete-time system for trajectory tracking control is discussed. A soft robotic exoskeleton digit (exo-digit) and a 3D-printed human-finger model integrated with IMU sensors were used for the experimental test setup. An Arduino-based electro-pneumatic control hardware was developed to control the actuation pressure of the soft exo-digit. The effectiveness of the controller was examined through simulation studies and experimental testing for following different pose trajectories corresponding to the human finger pose during the activities of daily living. The model-based controller was able to follow the desired trajectories with a very low average root-mean-square error of 2.27 mm in the x-direction, 2.75 mm in the y-direction, and 3.90 degrees in the orientation of the human finger distal link about the z-axis. Full article

One of the most difficult components of stroke therapy is regaining hand mobility. This research describes a preliminary approach to robot-assisted hand motion therapy. Our objectives were twofold: First, we used machine learning approaches to determine and describe hand motion patterns in healthy people. Surface electrodes were used to collect electromyographic (EMG) data from the forearm’s flexion and extension muscles. The time and frequency characteristics were used as parameters in machine learning algorithms to recognize seven hand gestures and track rehabilitation progress. Eight EMG sensors were used to capture each contraction of the arm muscles during one of the seven actions. Feature selection was performed using the Pareto front. Our system was able to reconstruct the kinematics of hand/finger movement and simulate the behaviour of every motion pattern. Analysis has revealed that gesture categories substantially overlap in the feature space. The correlation of the computed joint trajectories based on EMG and the monitored hand movement was 0.96 on average. Moreover, statistical research conducted on various machine learning setups revealed a 92% accuracy in measuring the precision of finger motion patterns. Full article

Electronic skin (e-skin) has the potential to detect large-scale strain, which is typically achieved by integrating multiple strain sensors into an array. However, the latency and limited resolution of sensing have hindered its large-scale sensing applications. Here, we have developed a high-resolution detection sensing system capable of detecting static and dynamic strain with a simple fabrication process by combining capacitive and mechanoluminescent (ML) sensor units. An elastic polydimethylsiloxane (PDMS) composite film doped with ZnS:Cu and BaTiO₃(BT) particles are fabricated as the functional film of the capacitive sensor. In contrast, the transparent electrode was fabricated on the surface of the as-prepared film. By incorporating BT nanoparticles into the elastic substrate, the ML intensity of the ZnS:Cu was improved up to 2.89 times that without BT addition, and the sensitivity of the capacitive sensor was increased as well. The capacitive part of the sensor presented a GF of 0.9 and good stability, while the ML part exhibited excellent performance, making it suitable for both static and dynamic sensing. Furthermore, the strain sensor integrated by 10 × 10 sensing units is demonstrated to detect large-scale strain with high resolution. Moreover, finger joint strain distribution tracking is achieved by attaching the strain sensor unit to the finger joint. With these characteristics, the e-skin may have great potential for bio-motion monitoring and human-computer interaction applications. Full article

Many diseases, such as stroke, arthritis, and spinal cord injury, can cause severe hand impairment. Treatment options for these patients are limited by expensive hand rehabilitation devices and dull treatment procedures. In this study, we present an inexpensive soft robotic glove for hand rehabilitation in virtual reality (VR). Fifteen inertial measurement units are placed on the glove for finger motion tracking, and a motor—tendon actuation system is mounted onto the arm and exerts forces on fingertips via finger-anchoring points, providing force feedback to fingers so that the users can feel the force of a virtual object. A static threshold correction and complementary filter are used to calculate the finger attitude angles, hence computing the postures of five fingers simultaneously. Both static and dynamic tests are performed to validate the accuracy of the finger-motion-tracking algorithm. A field-oriented-control-based angular closed-loop torque control algorithm is adopted to control the force applied to the fingers. It is found that each motor can provide a maximum force of 3.14 N within the tested current limit. Finally, we present an application of the haptic glove in a Unity-based VR interface to provide the operator with haptic feedback while squeezing a soft virtual ball. Full article

Human motion analysis using inertial measurement units (IMUs) has recently been shown to provide accuracy similar to the gold standard, optical motion capture, but at lower costs and while being less restrictive and time-consuming. However, IMU-based motion analysis requires precise knowledge of the orientations in which the sensors are attached to the body segments. This knowledge is commonly obtained via time-consuming and error-prone anatomical calibration based on precisely defined poses or motions. In the present work, we propose a self-calibrating approach for magnetometer-free joint angle tracking that is suitable for joints with two degrees of freedom (DoF), such as the elbow, ankle, and metacarpophalangeal finger joints. The proposed methods exploit kinematic constraints in the angular rates and the relative orientations to simultaneously identify the joint axes and the heading offset. The experimental evaluation shows that the proposed methods are able to estimate plausible and consistent joint axes from just ten seconds of arbitrary elbow joint motion. Comparison with optical motion capture shows that the proposed methods yield joint angles with similar accuracy as a conventional IMU-based method while being much less restrictive. Therefore, the proposed methods improve the practical usability of IMU-based motion tracking in many clinical and biomedical applications. Full article