Open AccessArticle

A Convolutional Neural Network-Based Broad Incremental Learning Filter for Attenuating Physiological Tremors in Telerobot Systems

Guanyu Lai

¹,

Weizhen Liu

¹,

Weijun Yang

^2,*

and

Yun Zhang

School of Automation, Guangdong University of Technology, Guangzhou 510006, China

School of Mechanical and Electrical Engineering, Guangzhou City Polytechnic, Guangzhou 510405, China

Author to whom correspondence should be addressed.

Appl. Sci. 2023, 13(2), 890; https://doi.org/10.3390/app13020890

Submission received: 5 December 2022 / Revised: 31 December 2022 / Accepted: 5 January 2023 / Published: 9 January 2023

(This article belongs to the Special Issue Intelligent Control and Applications for Robotics)

Download

Browse Figures

Versions Notes

Abstract

While master-slave teleoperated robotic systems have extensive applications in practice, the physiological tremors can easily affect the control accuracy and even destroy the stability of the closed-loop control systems during operation. Hence, the development of some effective approaches for counteracting physiological tremors is of both theoretical and practical importance. In this paper, a broad learning network-based filter integrating a deep learning network and modified incremental learning algorithms is proposed to reconstruct and compensate for tremor signals. To strengthen the recognition of correlations between different moments, the lateral connectivity structure is adopted to obtain multi-scale feature maps. Each feature window is obtained from multi-scale feature maps generated by the convolutional neural network, which has an advantage that makes the feature nodes fuse the feature information of long time series and short time series by the lateral connection. The broad learning network is a unique construction, which only needs to obtain the input and the output to conveniently calculate the connection weights by the pseudo-inverse without involving backpropagation. It is known that the relation between the data X and the label Y can be represented as

X W = Y

, and the solution W can be obtained by the pseudo-inverse

W = X^{+} Y

. In addition, to guarantee the ill-posed problem, a ridge regression algorithm is used for the pseudo-inverse calculation. The effectiveness of our raised network architecture is illustrated by comparative simulation and experiment results.

Keywords:

broad learning system; convolutional neural network; incremental learning; teleoperation robot systems

1. Introduction

It is well known that teleoperation systems have gained widespread applications in many fields, such as medicine, space exploration, etc.; see [1,2,3] for examples. For such applications, it is generally required that the telerobot or its end-effector moves along the same trajectory as the operator hand to accomplish some specific tasks [4,5,6]. According to each human’s mechanism, slight tremors always exist in every human hand and are not pathological but natural [7,8]. During teleoperation, these physiological tremors can cause movement deflections of the telerobot from the objective. The physiological tremors have characteristics, i.e., a small amplitude range between 50 μm and 100 μm and a domain frequency range between 8 Hz and 12 Hz [9,10]. Hence, in some occasions requiring an accuracy of about 10 μm, such as microsurgery [11,12], the physiological tremor attenuation is a significant yet challenging problem.

To overcome the problem, numerous studies have been performed, with particular emphasis on cancelling out the physiological tremors, see [13,14], just to list a few. Generally, linear low-pass filters (LPFs) are employed for canceling tremors, but digital filters have a disadvantage that introduces amplitude and phase distortions [15]. Related studies have been proposed to overcome the limits of digital filters [16,17,18]. An adaptive band-limited multiple Fourier linear combiner with Kalman filter (BMFLC-KF) has been proposed to provide accurate time-frequency decomposition components in the band-limited signal for filtering and compensation [19]. Moreover, an autoregressive model with Kalman filter (AR-KF) has been proposed to estimate a real-time tremor based on the previous outputs [20]. These methods can reduce most phase distortion but still have a problem with a phase delay of 5 to 10 ms. Later, the multistep prediction-based algorithms were proposed to address the existence of phase delay in sensors and filter for accurate canceling of real-time tremors [21]. Because of the fusion of various algorithms, the computation costs are increased, which can be alleviated by the reduced-order Kalman-based enhanced band-limited multiple Fourier linear combiner (RKE-BMFLC) [22].

Although the algorithms mentioned above have a good performance in predicting tremors, they have some restrictions that need to be removed: the AR-KF method is a linear prediction model, which characterizes tremor signals as linear Gaussian distribution. It needs to consider the characteristic of tremors; when applying the BMFLC-KF method, a careful choice of proper parameters is required, since even a small frequency gap can make the method ineffective [19]. To overcome the problems, some small-scale sample machine learning algorithms are raised to compensate tremors; see typically the support vector machine (SVM)-based approaches [23,24], which exhibit a good generalization capability and are computationally attractive. Note that SVM may not be assigned exactly in the learning process and is not sensitive to small amplitude signals, and this could cause substantial losses in accuracy [25]. Such an issue can be well addressed by the deep-learning methodology. Moreover, the deep learning network has a high accuracy in predicting small signals due to its strong network depth [26]. In addition, to improve the accuracy of tremor prediction, a multi-step long-short-term memory (MS-LSTM) method has been used in [27].

While the deep learning network is a powerful tool, due to the need for many hyperparameters, and the use of complicated structures, a time-consuming training process is generally required. In addition, the deep structures are the black box that is difficult to analyze. Recently, a novel network structure broad learning system (BLS) has been proposed in [28]. Unlike the deep network, BLS has a simple structure and trains without hyper-parameters. It updates the weights of network by the pseudo-inverse calculation. In [29], it shows that BLS has the generalization capability in function approximation and proposes variant structures of original BLS. However, combined with our recent study [30], we find that the original BLS has some limitations in extracting the features’ process, as the original BLS cannot extract the potential information of time-frequency features for non-stationary signals. Thus, we point out that the original BLS must be redesigned as a novel variant to accommodate tremor signals.

Motivated by the above, in this study, we propose a novel convolutional neural network-based broad learning system filter (CNN-BLSF) combined with broad expansion: incremental learning to predict and compensate for tremor signals. In summary, the work of this study has the following novelties and contributions:

For the original BLS, the task of feature extraction is generally achieved by the sparse autoencoder (SAE), which could not reflect the underlying relationship of the adjacent time sequence. To overcome the problem, a convolutional neural network is introduced to extract features, based on which a novel network structure, called the convolutional neural network-based BLS (CNN-BLS), is established.
With our raised convolutional neural network, various feature maps, including feature information in long and short time series regions, can be well obtained. A lateral connection structure, such as the residual network, is employed in the network so that the feature information of the time sequence can be extracted. Our purpose of design is to construct multi-scale feature maps and fuse them.
With the unique construction of BLS, an incremental learning algorithm, which adapts to BLS and can improve the performance of systems without retraining, is raised to remodel the network. Combining such an algorithm with our CNN-BLS, a convolutional neural network-based broad learning system with incremental learning (CNN-BLS-IL) is developed. It is worth mentioning that the raised incremental learning algorithm is better than the original incremental learning algorithm and thus could be more effective, as illustrated in Section 4.1.3.

The rest of this paper is organized as follows. The problem to be investigated is presented in Section 2. Some control strategies are given in Section 3. The proposed CNN-BLS-based approach is described in Section 4. The simulation and experiment results are shown in Section 5 to demonstrates the effectiveness of our approach. The paper is concluded in Section 6.

2. Problem Description

A teleoperated robotic system is composed of three parts: (master, channel, and slave). The effects of tremors are concentrated in the master part (haptic device), which leads the slave part (slave robot) to generate noisy signals.

In Figure 1, the haptic device (Touch X) has 6 degrees of freedom (DOFs). Its homogeneous transformation matrix between the i-th joint and the (i-1)-th joint generates deviation variables when the operator’s hand with tremor touches the 6-th joint of Touch X. Firstly, the original joint angle

θ_{o}

and the original link offset

d_{o}

of each joint change as

θ^{*}

and

d^{*}

, which can be described as

\begin{matrix} θ_{i}^{*} = θ_{o i} + θ_{m i} + Δ θ_{i}, i = 1, 2, \dots, 6 \\ d_{i}^{*} = d_{o i} + d_{m i} + Δ d_{i}, i = 1, 2, \dots, 6, \end{matrix}

(1)

where

θ_{m i}

and

d_{m i}

are the joint angle and the link offset generated by the operator, and

Δ θ_{i}

and

Δ d_{i}

are the deviation joint angle and the deviation link offset caused by tremors. i represents i-th joint of the haptic device. With these disturbance variables, the homogeneous transformation matrix between adjacent joints

_{i - 1}^{i} T

, according to the modified D-H notation, can be rewritten as

\begin{matrix} _{i - 1}^{i} T^{*} = [\begin{matrix} c θ_{i}^{*} & - s θ_{i}^{*} & 0 & a_{i - 1} \\ s θ_{i}^{*} c α_{i - 1} & c θ_{i}^{*} c α_{i - 1} & - s α_{i - 1} & - s α_{i - 1} d_{i}^{*} \\ s θ_{i}^{*} s α_{i - 1} & c θ_{i}^{*} s α_{i - 1} & c α_{i - 1} & c α_{i - 1} d_{i}^{*} \\ 0 & 0 & 0 & 1 \end{matrix}], \\ s = s i n, c = c o s, i = 1, 2, \dots, 6, \end{matrix}

(2)

where

α

is the kinematic link twist. The six matrixes

_{i - 1}^{i} T^{*}

(

i = 1, 2, \dots, 6

) can be multiplied together as the pose matrix of the end effector

_{0}^{6} T^{*}

_{6}^{0} T^{*} = [\begin{matrix} n_{x 1}^{*} & n_{x 2}^{*} & n_{x 3}^{*} & p_{x}^{*} \\ n_{y 1}^{*} & n_{y 2}^{*} & n_{y 3}^{*} & p_{y}^{*} \\ n_{z 1}^{*} & n_{z 2}^{*} & n_{z 3}^{*} & p_{z}^{*} \\ 0 & 0 & 0 & 1 \end{matrix}],

(3)

where

n_{x i}^{*}

n_{y i}^{*}

n_{z i}^{*}

(

i = 1, 2, 3

) are the rotational elements and

p_{x}^{*}

p_{y}^{*}

p_{z}^{*}

are the position vector.

Influenced by tremors, the pose of the end effector change, which makes the slave robot deviates from its original track. The trajectory deviation over 100 μm results in failure on the occasion of requiring precise manual positioning [31,32,33]. The tremor filter needs to be designed in the teleoperation system for the tremor compensation to eliminate tremors.

3. Control Strategies

Considering the stability and precision of the teleoperation system, we integrate some control strategies into the system’s control mode. As shown in Figure 2, the operator gives a command to the haptic device to obtain an actual trajectory

S_{m 1}

, and

S_{m}

is a filtered trajectory. After receiving the operation signal, the robot follows the haptic device and generates a trajectory

S_{s}

S_{e}

is the position error of

S_{m}

and

S_{s}

J_{m}^{+}

and

J_{s}^{+}

are the pseudo-inverse of the Jacobian of the master device and the slave device, respectively.

{\dot{q}}_{m d}

q_{m}

are the master desired joint velocity, the master joint angle.

{\dot{q}}_{s d}

q_{s}

are the slave desired joint velocity and the slave joint angle.

τ_{m}

τ_{s}

and

τ_{o}

are the master torque and the slave torque generated by the PD controller, and the torque generated by the operator.

3.1. Haptic Force Feedback

Force feedback is integrated into the haptic device to improve tactile perceptual interaction between an operator and a slave robot, which can be written as [34,35,36].

\begin{matrix} F_{m} = K_{m} (S_{m} - S_{s}) = K_{m} S_{e} \\ F_{s} = K_{s} (S_{m} - S_{s}) = K_{s} S_{e} . \end{matrix}

(4)

Its control object is the tracking error

S_{e}

, i.e.,

S_{e} = S_{m} - S_{s}

. The value of feedback force decreases with the tracking error

S_{e}

reducing, and finally the whole control system converges.

Depending on the adjustment of the feedback force,

S_{m}

and

S_{s}

can be modified to the desired trajectories

S_{m d}

and

S_{s d}

. The pose information

S_{m d}

and

S_{s d}

are translated into the joint velocity information

{\dot{q}}_{m d}

and

{\dot{q}}_{s d}

based on the Jacobian matrix.

\begin{matrix} {\dot{S}}_{m d} = J_{m} {\dot{q}}_{m d} \to {\dot{q}}_{m d} = J_{m}^{+} {\dot{S}}_{m d} \\ {\dot{S}}_{s d} = J_{s} {\dot{q}}_{s d} \to {\dot{q}}_{s d} = J_{s}^{+} {\dot{S}}_{s d} . \end{matrix}

(5)

3.2. PD Controller

The joint information is obtained as the control variables from Equation (5). The PD controller is designed for the master part and the slave part, which can be described as follows:

\begin{matrix} τ_{m} = P_{m} (q_{m} - q_{m d}) + D_{m} ({\dot{q}}_{m} - {\dot{q}}_{m d}) = P_{m} Δ q_{m} + D_{m} Δ {\dot{q}}_{m} \\ τ_{s} = P_{s} (q_{s} - q_{s d}) + D_{s} ({\dot{q}}_{s} - {\dot{q}}_{s d}) = P_{s} Δ q_{s} + D_{s} Δ {\dot{q}}_{s}, \end{matrix}

(6)

where

τ_{m}

and

τ_{s}

are the master torque and the slave torque.

P_{m}

P_{s}

are proportion gain of the controller and

D_{m}

D_{s}

are differential gain of the controller. The control objects of the controller are the joint angle

Δ q_{m}

Δ q_{s}

and the joint velocity

Δ {\dot{q}}_{m}

Δ {\dot{q}}_{s}

To describe the controller module clearly, a block diagram of the designed controller is given in Figure 3, where P is the proportion control unit, and D is the differential control unit.

δ

denotes m (master device) or s (slave device). The master device (Touch X) and the slave device (Telerobot) are both 6 DOFs, where each joint is driven by a motor and is controlled by a power steering gear. The micro-controller unit (MCU) controls the power steering gear to move devices by torque.

3.3. Model Structure of CNN-BLSF

3.3.1. Physical Model Structure of CNN-BLSF

In order to address the effects caused by physiological tremors, a novel CNN-BLS-based filter is proposed in this work. The physical model of CNN-BLSF contains: (i) sampling unit; (ii) tremor filtering unit; (iii) control unit. In Figure 4, each unit of the whole CNN-BLSF system is shown.

As the central part of the sampling module, the internal measurement unit (IMU) is used to sample the operator’s real-time hand movements. It can obtain a three-axis position acceleration and a three-axis joint angular velocity from a human.
In the tremor filtering unit module, the CNN-BLS network algorithm is integrated to forecast the tremor signals. The compensation tremor signals $x_{p}$ , $y_{p}$ , $z_{p}$ and $θ_{x p}$ , $θ_{y p}$ , $θ_{z p}$ have the same magnitude but opposite phase compared with tremor signals, which can neutralize the tremor signals in the actual signals x, y, and z.
The control module integrates the calculation of inverse kinematics, the driver of a single joint, and the motion feedback of deflection sensors, which converts inverse kinematics into motion control variables for the robot manipulator.

Figure 4. Block diagram of the tremor filter.

3.3.2. Mathematical Model Structure of CNN-BLSF

To better analyze the relationship between the input and output of the tremor filter, the mathematical model is shown in Figure 5, where

d (k)

n (k)

and

n_{p} (k)

are a desired signal, a faint signal caused by tremors and a predicted tremor signal, respectively. An actual signal

S_{1} (k)

with the tremor can be expressed as

S_{1} (k) = d (k) + n (k) .

(7)

The input data of the CNN-BLSF is (

S_{1} (k)

n_{p} (k)

), and the output of the CNN-BLSF

S (k)

S (k) = S_{1} (k) - n_{p} (k) = d (k) + n (k) - n_{p} (k) = d (k) + Δ n .

(8)

According to the Equation (8), when

n (k) = n_{p} (k)

, the end item

Δ n = n (k) - n_{p} (k)

is zero. We expect the tremor attenuation filter to be designed so that the predicted error

Δ n

is closer to zero.

Figure 5. Mathematical model of the tremor filter.

4. Design of CNN-BLS Tremor Filter

To guarantee the predicted error

Δ n

is closer to zero, a novel variant of the broad learning network is proposed in this paper, which integrates the advantage of the broad learning network (fast) and convolutional neural network (accurate). In addition, an incremental learning algorithm can be applied to specific networks, such as the broad learning network. To better complete the build of the tremor filter, the increment learning is combined with our designed network.

4.1. Algorithm of CNN-BLS

4.1.1. Broad Learning Network

Professor Chen and Liu proposed a novel and simple network frame in [28]. Moreover, Chen and Liu further demonstrated the approximation capability of the BLS for generic functions and proposed several variants of the original BLS in [29]. Firstly, as shown in Figure 6,

G = [x, y, z]

is denoted as the input data, where X, Y, and Z are the three-axis input signals, respectively. The input data

G = [x, y, z]

are randomly mapped to n groups feature nodes, which can be expressed as the equation of the form

Z_{i} = ϕ (G W_{f_{i}} + β_{f_{i}}), i = 1, \dots, n,

(9)

where

W_{f_{i}}

and

β_{f_{i}}

are random weights and random biases.

ϕ (\cdot)

represents a random mapping function.

Remark 1.

For the original BLS, the feature layer is obtained from a random mapping function, which regularizes the broad network. In addition, SAE is used to fine-tune the weights of feature extraction (

W_{f i}

β_{f_{i}}

), and it has a good performance in classification. However, the original BLS feature extraction method cannot obtain enough information for time-series features that adjacent sequences have connections. Thus, a new network structure for feature extraction is proposed to replace the SAE of the original BLS.

All generated feature nodes can be written as

Z^{n} = [Z_{1}, Z_{2}, \dots, Z_{n}]

, and then, are activated to be m enhancement nodes by a activation function

ξ (\cdot)

. The j-th enhancement node is represented as

H_{j} = ξ (Z^{n} W_{e_{j}} + β_{e_{j}}), j = 1, \dots, m,

(10)

where

W_{e_{j}}

and

β_{e_{j}}

are also random weights and random biases. Denote the input layer as

I_{n}^{m} = [Z_{i} | H_{j}], i = 1, \dots, n, j = 1, \dots, m .

(11)

Finally, the output of BLS can be represented as

Y = [Z_{1}, \dots, Z_{n} ∣ H_{1}, \dots, H_{m}] W^{*} = I_{n}^{m} W^{*} .

(12)

According to the pseudo-inverse calculation equation, the Equation (12) can be rewritten as the following form

W^{*} = {(I_{n}^{m})}^{+} Y = {({(I_{n}^{m})}^{T} I_{n}^{m})}^{- 1} {(I_{n}^{m})}^{T} Y .

(13)

A ridge regression algorithm is used to avoid the inexistence of pseudo-inverse of

I_{n}^{m}

[37], and Equation (13) is rewritten as

W^{*} = {(λ E + {(I_{n}^{m})}^{T} I_{n}^{m})}^{- 1} {(I_{n}^{m})}^{T} Y .

(14)

4.1.2. Structure of Our CNN-BLS Network

In Figure 7, the internal algorithm of the designed tremor filter is shown and the algorithm of our designed network is given in Algorithm 1, where

γ

represents the number of feature maps,

δ

represents the number of enhancement nodes and

f^{(γ)} (\cdot)

represents the output of the

γ

-th pooling layer. We point out that the primary network framework is the BLS. Since the original BLS does not extract time feature sequences well, the network fuses a simple CNN with a lateral connection approach [38].

Algorithm 1: Convolutional Broad Learning Filter Algorithm: Increment of Feature Mapping Nodes and Enchancement Nodes.

The input data

G = [x, y, z]

are sent to the convolutional network layer, which contains

γ

convolution and pooling process and can yield some feature sequences concerning the input. In Figure 8, the convolution and pooling operation process is given. Firstly, the input time series can form the feature maps. The process uses auto-padding to generate a

3 \times 3

feature map of the same size as the convolution kernel. Then, the feature map can be mapped as feature series based on the max pooling. Finally, through the process of convolution and pooling, the i-th feature sequence

\vec{z_{i}}

is represented as

\vec{z_{i}} = f^{(i)} (G W_{c o n v_{i}}), i = 1, \dots, γ,

(15)

where

W_{c o n v_{i}}

is the kernel weight of the i-th convolution layer and

f^{(i)} (\cdot)

represents the output of the i-th pooling layer. Denote

γ

feature sequences as network feature nodes, and they are written as

Z^{γ} = [\vec{z_{1}}, \dots, \vec{z_{γ}}] = [Z_{1}, \dots, Z_{γ}] .

(16)

Similar to the Equations (10)–(14), the

δ

enhancement nodes can be generated by

Z^{γ}

and they can be written as

H_{j} = ξ (Z^{γ} W_{e_{j}} + β_{e_{j}}), j = 1, \dots, δ .

(17)

Our designed network input layer can be written as

I I = [Z_{i} | H_{j}], i = 1, \dots, γ, j = 1, \dots, δ .

(18)

We have

W = {(λ E + {(I I)}^{T} I I)}^{- 1} {(I I)}^{T} Y .

(19)

Figure 8. Process of single convolution.

Remark 2.

We point out that the BLS filter shows the capacity to cancel tremor is unpowerful because the feature extraction part may not effectively obtain corresponding feature maps. Hence, the above work shows our employed CNN to replace the SAE of BLS.

To enrich the feature extraction process of the network, we propose an approach concerning multi-scale extraction. In other words, it can be defined as a feature fusion method. By obtaining these multi-scale feature maps, we solve the problem of BLS and obtain better performance.

4.1.3. Broad Expansion: Incremental Learning

For the specific broad network construction, the incremental learning algorithm can be applied to improve the performance of the network, which contains three incremental methods: (i) increment of additional enhancement nodes; (ii) increment of additional feature mapping nodes; (iii) increment of input. To improve the regression capacity of the proposed algorithm, methods (i) and (ii) are applied, which have a fast remodeling to form robust feature extraction and non-linear capability of the model.

Increment of feature nodes: Suppose the tracking performance of the model is terrible. In that case, the reason may be that the feature layer cannot obtain underlying input data information, leading to the corresponding enhancement nodes not being enough to learn. Here, we show the detail of method (i), as follows.

Assume the initial model structure comprises:

γ

convolution layers and pooling layers, and

δ

enhancement nodes. The additional feature nodes are denoted as

{\vec{z}}_{γ + 1} = f^{(γ + 1)} (G W_{c o n v_{γ + 1}}),

(20)

The corresponding yeild enhancement nodes can be respresented as

Δ H_{γ + 1} = [ξ ({\vec{z}}_{γ + 1} Δ W_{1} + Δ β_{1}), \dots, ξ ({\vec{z}}_{γ + 1} Δ W_{m} + Δ β_{δ})],

(21)

where

Δ W_{i}

and

Δ β_{i}

are incremental random parameters corresponding to the

(n + 1)

-th feature node. Then, denote the new input layer as

I I_{γ + 1} = [I I | {\vec{z}}_{γ + 1} | Δ H_{γ + 1}],

(22)

and its pseudo-inverse can be calculated by the following equations:

{(I I_{γ + 1})}^{+} = [\begin{matrix} {(I I)}^{+} (E - D B^{T}) \\ B^{T} \end{matrix}],

(23)

where

D = {(I I)}^{+} [{\vec{z}}_{γ + 1} | Δ H_{γ + 1}]

C = I I_{γ + 1} - I I \cdot D

and

B^{T} = \{\begin{matrix} C^{+} & if C \neq 0 \\ {(\vec{1} + D^{T} D)}^{- 1} D^{T} {(I I)}^{+} & if C = 0 \end{matrix} .

(24)

Thus, the new weights can be obtained as

W_{γ + 1} = [\begin{matrix} W - D B^{T} Y \\ B^{T} Y \end{matrix}] .

(25)

Remark 3.

It is noted that the incremental learning algorithm is a practical and fast remodeling approach. It can reduce the computational complexity of numerous training data. Since the feature extraction part of the original BLS is replaced by our method, the incremental learning with added feature nodes should be different from the original incremental learning. According to the above contents, the novel incremental learning is combined with multi-scale feature maps, where the added feature nodes are given by other scale feature maps.

Increment of enhancement nodes: If the non-linear degrees of the fitting curves are not enough to fit the target curves, the reason may be that the number of enhancement nodes is insufficient and can not achieve a good performance. Similar to method (i), the network connection weight of method (ii) can be calculated and obtained. Denote the new input layer as

I I^{δ + 1} = [I I | ξ (Z^{γ} W_{e_{δ + 1}} + β_{e_{δ + 1}})] = [I I | H_{δ + 1}],

(26)

where

W_{e_{δ + 1}}

and

β_{e_{δ + 1}}

are randomly generated, and we have

W^{δ + 1} = [\begin{matrix} W - D B^{T} Y \\ B^{T} Y \end{matrix}],

(27)

where

D = {(I I)}^{+} [H_{δ + 1}]

C = I I^{δ + 1} - I I \cdot D

and

B^{T} = \{\begin{matrix} C^{+} & if C \neq 0 \\ {(\vec{1} + D^{T} D)}^{- 1} D^{T} {(I I)}^{+} & if C = 0 \end{matrix} .

(28)

5. Experiments

To explain the performance of different algorithms, sum square error (SSE), root mean square error (RMSE) and R-square (

R^{2}

) are applied, which can be written as

S S E = \sum_{t = 1}^{T} {(n (t) - n_{p} (t))}^{2}

(29)

R M S E = \sqrt{\frac{\sum_{t = 1}^{T} {(n (t) - n_{p} (t))}^{2}}{N}}

(30)

R^{2} = 1 - \frac{\sum_{t = 1}^{T} n (t) - n_{p} (t)}{\sum_{t = 1}^{T} n (t) - \bar{n} (t)}

(31)

where T and N are the period and the number of samples.

n (t)

n_{p} (t)

and

\bar{n} (t)

are the real values, the predicted values and the average of the real values, respectively. The comparative simulation is processed in the MATLAB robotics toolbox and Libsvm toolbox. Figure 9 shows a simulated robot manipulator and the desired and actual motion and pose.

5.1. Parameters Setting

According to the standard D-H table construction rules, we set four parameters (d, a, alpha, offset) to build the simulation robot, respectively. See Table 1 for specific parameter information. For the different filters, the parameters of filters were shown in Table 2.

Here, the sample time is 50 s, and the interval is set as 1 s. To achieve the construction of the SVM filter, we used the Libsvm toolbox, where p and n represent the loss parameter of epsilon-SVR, and RBF is the type of kernel function. For other filters, C is the regularization parameter for sparse regularization, and s is the shrinkage parameter for enhancement nodes.

N 11

N 2

, and

N 33

are the feature nodes of each window, the number of windows of feature nodes, and the number of enhancement nodes, respectively.

m 1

m 2

, and

m 3

are the number of feature nodes of each increment step, the number of enhancement nodes related to the incremental feature nodes of each increment step, and the number of enhancement nodes in each incremental learning, respectively. For the CNN, we used SGDM to update weights, where the initial learning rate is 0.001. The number of convolution layers and pooling layers is 3. As shown in Figure 10a,b, the track performance varies as the number of feature nodes increases. To guarantee the performance of track and track error, we proposed that the number of feature nodes is set as 30.

5.2. Simulated Tremor Signal

Considering the characteristics of tremors, we define the tremor signal as having a small amplitude and high-low frequency, which can be written as

\begin{matrix} n (t) = 0.04 s i n (0.055 π t) + 0.03 s i n (0.078 π t) \\ + 0.05 s i n (0.083 π t) + 0.007 s i n (1000 π t) + 0.01 c o s (500 π t) \end{matrix}

(32)

where the front part is the low-frequency signal and the back part is the high-frequency signal. Figure 11a shows the trajectory of our simulated tremor signal.

5.3. Tremor Forecast Results

Figure 11b shows the desired operation trajectory and the actual operation trajectory with tremor. Our actual operation trajectory deviates from the desired trajectory within 100 μm. The bias 100 μm leads to the failure of high-precision tasks. Figure 12a,b represent the track performance and the track errors between the different algorithms, respectively. From Figure 12a,b, we can observe that our designed algorithm has a good performance in regression and loss. The loss of CNN-BLS-IL converges to zero throughout, indicating that it almost overlaps with the actual tremor signal. In Figure 13b, it shows the recovery trajectory by the different filters. The specific evaluation metrics are given in Table 3. In summary, the results verify the effectiveness of our proposed approach (CNN-BLS-IL) in the tremor attenuation.

6. Conclusions

In this paper, we propose a novel variant network CNN-BLSF for forecasting and compensating physiological tremors. At first, to simplify the calculation of model parameters, the structure of the original BLS is used as our rebuild basis. Then, the SAE of the original BLS is replaced by the CNN to extract the time series feature efficiently. Moreover, to better obtain in-depth feature information from the multi-feature layers, the structure of lateral connection is adopted. Moreover, the incremental learning algorithm is adopted to update the network in real-time, which can improve the accuracy by adding new nodes. Finally, compared with the different existing methods, our proposed filter is more efficient in forecasting tremors, and we believe our filter can be applied to the medical area requiring higher accuracy.

The work improves the correlation of series by using the lateral connection, which seems to be somewhat complex. In view of this, and the superiorities of long short term memory (LSTM) in solving sequence correlation problems, in future study, we will try to achieve the combination of LSTM with BLS.

Author Contributions

Conceptualization, G.L.; methodology, W.Y.; software, W.L.; validation, G.L. and Y.Z.; formal analysis, Y.Z.; writing—original draft preparation, W.Y.; writing—review and editing, W.L. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Tertiary Education Scientific research project of Guangzhou Municipal Education Bureau [No. 202235364], and the Science and Technology Program of Guangzhou, China [No. 201804010098].

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Acknowledgments

This work was supported by the Tertiary Education Scientific research project of Guangzhou Municipal Education Bureau (No. 202235364), the Special projects in key fields of colleges and universities in Guangdong Province (No. 2021ZDZX1109), the Science and Technology Program of Guangzhou, China (No. 201804010098), the Natural Science Foundation of Guangdong Province (No. 2019A1515012109), and the National Natural Science Foundation of China (No. 61803090).

Conflicts of Interest

The authors declare no conflict of interest.

References

Mortensen Ernits, R.; Reiß, M.; Bauer, M.; Becker, A.; Freitag, M. Individualisation of Inflight Catering Meals—An Automation Concept for Integrating Pre-Ordered Meals during the Flight for All Passengers. Aerospace 2022, 9, 736. [Google Scholar]
Xu, W.; Peng, J.; Liang, B.; Mu, Z. Hybrid modeling and analysis method for dynamic coupling of space robots. IEEE Trans. Aerosp. Electron. Syst. 2016, 52, 85–98. [Google Scholar] [CrossRef]
Okamura, A.M.; Mataric, M.J.; Christensen, H.I. Medical and Health-Care Robotics. Robot. Autom. Mag. IEEE 2010, 17, 26–37. [Google Scholar] [CrossRef]
Cogan, B.; Tandetzki, J.; Milius, B. Passenger Acceptability of Teleoperation in Railways. Future Transp. 2022, 2, 956–969. [Google Scholar] [CrossRef]
Mehrdad, S.; Liu, F.; Pham, M.T.; Lelevé, A.; Atashzar, S.F. Review of Advanced Medical Telerobots. Appl. Sci. 2021, 11, 209. [Google Scholar] [CrossRef]
Ehrampoosh, A.; Shirinzadeh, B.; Pinskier, J.; Smith, J.; Moshinsky, R.; Zhong, Y. A Force-Feedback Methodology for Teleoperated Suturing Task in Robotic-Assisted Minimally Invasive Surgery. Sensors 2022, 22, 7829. [Google Scholar] [CrossRef]
Zhi, L.; Wu, Q.; Yun, Z.; Wang, Y.; Chen, C. Adaptive fuzzy wavelet neural network filter for hand tremor canceling in microsurgery. Appl. Soft Comput. 2011, 11, 5315–5329. [Google Scholar]
Liu, Z.; Mao, C.; Luo, J.; Zhang, Y.; Chen, C.P. A three-domain fuzzy wavelet network filter using fuzzy PSO for robotic assisted minimally invasive surgery. Knowl. Based Syst. 2014, 66, 13–27. [Google Scholar] [CrossRef]
Tatinati, S.; Veluvolu, K.C.; Ang, W.T. Multistep Prediction of Physiological Tremor Based on Machine Learning for Robotics Assisted Microsurgery. IEEE Trans. Cybern. 2015, 45, 328–339. [Google Scholar] [CrossRef]
Latt, W.T.; Veluvolu, K.C.; Ang, W.T. Drift-Free Position Estimation of Periodic or Quasi-Periodic Motion Using Inertial Sensors. Sensors 2011, 11, 5931–5951. [Google Scholar] [CrossRef]
Li, X.; Guo, S.; Shi, P.; Jin, X.; Kawanishi, M.; Suzuki, K. A Bimodal Detection-Based Tremor Suppression System for Vascular Interventional Surgery Robots. IEEE Trans. Instrum. Meas. 2022, 71, 1–12. [Google Scholar] [CrossRef]
Sun, Z.; Guo, J.; Guo, S.; Song, Y. Study on A Novel Strategy for Eliminating Tremor in Vascular Interventional Robot. In Proceedings of the 2022 IEEE International Conference on Mechatronics and Automation (ICMA), Guilin, China, 7–10 August 2022; pp. 1263–1268. [Google Scholar] [CrossRef]
Lin, J.; Liu, Z.; Chen, C.P.; Zhang, Y. Quaternion broad learning system: A novel multi-dimensional filter for estimation and elimination tremor in teleoperation. Neurocomputing 2020, 380, 78–86. [Google Scholar] [CrossRef]
Yang, Q.; Liang, K.; Su, T.; Geng, K.; Pan, M. Broad learning extreme learning machine for forecasting and eliminating tremors in teleoperation. Appl. Soft Comput. 2021, 112, 107863. [Google Scholar] [CrossRef]
Mellone, S.; Palmerini, L.; Cappello, A.; Chiari, L. Hilbert–Huang-Based Tremor Removal to Assess Postural Properties From Accelerometers. IEEE Trans. Biomed. Eng. 2011, 58, 1752–1761. [Google Scholar] [CrossRef]
Tatinati, S.; Veluvolu, K.C.; Ang, W.T. Autoregressive model with Kalman filter for estimation of physiological tremor in surgical robotic applications. In Proceedings of the 2011 11th International Conference on Control, Automation and Systems, Gyeonggi-do, Republic of Korea, 26–29 October 2011; pp. 454–459. [Google Scholar]
Gonzalez, J.; Heredia, E.; Rahman, T.; Barner, K.; Arce, G. Optimal digital filtering for tremor suppression. IEEE Trans. Biomed. Eng. 2000, 47, 664–673. [Google Scholar] [CrossRef] [PubMed]
Riviere, C.; Thakor, N. Modeling and canceling tremor in human-machine interfaces. IEEE Eng. Med. Biol. Mag. 1996, 15, 29–36. [Google Scholar] [CrossRef]
Wang, Y.; Veluvolu, K.C. Time-frequency decomposition of band-limited signals with BMFLC and Kalman filter. In Proceedings of the 2012 7th IEEE Conference on Industrial Electronics and Applications (ICIEA), Singapore, 18–20 July 2012; pp. 582–587. [Google Scholar]
Tatinati, S.; Veluvolu, K.C.; Hong, S.M.; Latt, W.T.; Ang, W.T. Physiological Tremor Estimation With Autoregressive (AR) Model and Kalman Filter for Robotics Applications. IEEE Sens. J. 2013, 13, 4977–4985. [Google Scholar] [CrossRef]
Veluvolu, K.; Tatinati, S.; Hong, S.M.; Ang, W. Multistep Prediction of Physiological Tremor for Surgical Robotics Applications. IEEE Trans. Biomed. Eng. 2013, 60, 3074–3082. [Google Scholar] [CrossRef]
Ghassab, V.K.; Mohammadi, A.; Atashzar, S.F.; Patel, R.V. Dynamic estimation strategy for E-BMFLC filters in analyzing pathological hand tremors. In Proceedings of the 2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP), Montreal, QC, Canada, 14–16 November 2017; pp. 442–446. [Google Scholar]
Yang, C.; Luo, J.; Pan, Y.; Liu, Z.; Su, C.Y. Personalized Variable Gain Control with Tremor Attenuation for Robot Teleoperation. IEEE Trans. Syst. Man, Cybern. Syst. 2018, 48, 1759–1770. [Google Scholar] [CrossRef]
Tatinati, S.; Wang, Y.; Shafiq, G.; Veluvolu, K.C. Online LS-SVM based multi-step prediction of physiological tremor for surgical robotics. In Proceedings of the 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Osaka, Japan, 3–7 July 2013; pp. 6043–6046. [Google Scholar]
Liu, Z.; Luo, J.; Wang, L.; Zhang, Y.; Chen, C.L.P.; Chen, X. A time-sequence-based fuzzy support vector machine adaptive filter for tremor cancelling for microsurgery. Int. J. Syst. Sci. 2015, 46, 1131–1146. [Google Scholar] [CrossRef]
Ibrahim, A.; Zhou, Y.; Jenkins, M.E.; Trejos, A.L.; Naish, M.D. Real-Time Voluntary Motion Prediction and Parkinson’s Tremor Reduction Using Deep Neural Networks. IEEE Trans. Neural Syst. Rehabil. Eng. 2021, 29, 1413–1423. [Google Scholar] [CrossRef] [PubMed]
Zhang, L.; Guo, S.; Yang, C. Prediction of Physiological Tremor Based on Deep Learning for Vascular Interventional Surgery Robot. In Proceedings of the 2021 IEEE International Conference on Mechatronics and Automation (ICMA), Takamatsu, Japan, 8–11 August 2021; pp. 1339–1344. [Google Scholar]
Chen, C.; Liu, Z. Broad Learning System: An Effective and Efficient Incremental Learning System without the Need for Deep Architecture. IEEE Trans. Neural Netw. Learn. Syst. 2018, 29, 10–24. [Google Scholar] [CrossRef] [PubMed]
Chen, C.L.P.; Liu, Z.; Feng, S. Universal Approximation Capability of Broad Learning System and Its Structural Variations. IEEE Trans. Neural Netw. Learn. Syst. 2019, 30, 1191–1204. [Google Scholar] [CrossRef] [PubMed]
Liu, W.; Lai, G.; Liu, A. Tremor Attenuation For Robot Teleoperation By A Broad Learning System-Based Approach. In Proceedings of the 2021 China Automation Congress (CAC), Beijing, China, 22–24 October 2021; pp. 7627–7632. [Google Scholar]
Veluvolu, K.C.; Tan, U.X.; Latt, W.T.; Shee, C.Y.; Ang, W.T. Adaptive filtering of physiological tremor for real-time compensation. In Proceedings of the 2008 IEEE International Conference on Robotics and Biomimetics, Bangkok, Thailand, 22–25 February 2009; pp. 524–529. [Google Scholar] [CrossRef]
Adhikari, K.; Tatinati, S.; Ang, W.T.; Veluvolu, K.C.; Nazarpour, K. A Quaternion Weighted Fourier Linear Combiner for Modeling Physiological Tremor. IEEE Trans. Biomed. Eng. 2016, 63, 2336–2346. [Google Scholar] [CrossRef] [Green Version]
Adhikari, K.; Tatinati, S.; Veluvolu, K.C.; Nazarpour, K. Modeling 3D tremor signals with a quaternion weighted Fourier Linear Combiner. In Proceedings of the 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER), Montpellier, France, 22–24 April 2015; pp. 799–802. [Google Scholar] [CrossRef] [Green Version]
Yang, C.; Wang, X.; Li, Z.; Li, Y.; Su, C.Y. Teleoperation Control Based on Combination of Wave Variable and Neural Networks. IEEE Trans. Syst. Man Cybern. Syst. 2017, 47, 2125–2136. [Google Scholar] [CrossRef] [Green Version]
Al-Mouhamed, M.A.; Nazeeruddin, M.; Merah, N. Design and Instrumentation of Force Feedback in Telerobotics. IEEE Trans. Instrum. Meas. 2009, 58, 1949–1957. [Google Scholar] [CrossRef]
Ju, Z.; Yang, C.; Li, Z.; Cheng, L.; Ma, H. Teleoperation of humanoid baxter robot using haptic feedback. In Proceedings of the International Conference on Multisensor Fusion & Information Integration for Intelligent Systems, Beijing, China, 28–29 September 2014; pp. 1–6. [Google Scholar]
Hoerl, A.E.; Kennard, R.W. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics J. Stats Phys. Chem. Eng. Ences 2000, 42, 80–86. [Google Scholar] [CrossRef]
Lin, T.Y.; Goyal, P.; Girshick, R.; He, K.; Dollár, P. Focal Loss for Dense Object Detection. IEEE Trans. Pattern Anal. Mach. Intell. 2020, 42, 318–327. [Google Scholar] [CrossRef]

Figure 1. Physical structure model of Touch X. (a) Touch X. (b) Structure of Touch X.

Figure 2. Control mode of system.

Figure 3. Control structure of the controller.

Figure 6. Network structure of BLS.

Figure 7. Network structure of CNN-BLS.

Figure 9. Simulated robot manipulator, motion and pose.

Figure 10. The simulated result metrics of different feature nodes. (a) Tremor prediction by different feature nodes in the case of tremors with small amplitude and high−low frequency. (b) Estimation error of different feature nodes in the case of tremors with small amplitude and high−low frequency.

Figure 11. The trajectory with tremor and the effect of tremor. (a) The signal with small amplitude and high−low frequency tremors. (b) The desired operation trajectory and the actual operation trajectory with small amplitude and high−low frequency tremors.

Figure 12. The simulated result metrics of different algorithms. (a) Tremor prediction by different algorithms in the case of tremors with small amplitude and high−low frequency. (b) Estimation error of different algorithms in the case of tremors with small amplitude and high−low frequency.

Figure 13. The trajectory with tremor and the effect of tremor attenuation. (a) The signal with small amplitude and high−low frequency tremors. (b) Tremor attenuation performance of different algorithms in the case of tremors with small amplitude and high−low frequency.

Table 1. D-H table of the simulated robot manipulator.

i	Theta	d/mm	a/mm	Alpha/rad	Offset/rad
1	q1	105.03	0	1.571	0
2	q2	0	−174.42	0	−1.571
3	q3	0	−174.42	0	0
4	q4	75.66	0	1.571	−1.571
5	q5	80.09	0	−1.571	0
6	q6	44.36	0	0	0

Table 2. Parameters and initial conditions of various methods.

Various Methods	Parameters and Initial Conditions
SVM	$p = 0.4$ ; $n = 0.1$ ; Radial Basis Function (RBF)
BLS	$C = 2^{- 30}$ ; $s = 0.8$ ; $N 11 = 8$ ; $N 2 = 10$ ; $N 33 = 80$
BLS-IL	$C = 2^{- 30}$ ; $s = 0.8$ ; $N 11 = 8$ ; $N 2 = 10$ ; $N 33 = 10$ ; $m 1 = 10$ ; $m 2 = 10$ ; $m 3 = 50$
CNN-BLS	$s = 0.8$ ; $N 11 = 3$ ; $N 2 = 10$ ; $N 33 = 8$ ; $I n i t i a l L e a r n R a t e = 1 e - 3$ ; Stochastic Gradient Descent with Momentum (SGDM); $γ = 3$
CNN-BLS-IL	$s = 0.8$ ; $N 11 = 3$ ; $N 2 = 10$ ; $N 33 = 8$ ; $I n i t i a l L e a r n R a t e = 1 e - 3$ ; SGDM; $m 1 = 3$ ; $m 2 = 3$ ; $m 3 = 5$ ; $γ = 3$ ;

Table 3. Compare among evaluation results with different methods: SSE; RMSE; R2; Time.

Methods and Evaluations	Train		Test		Total Time	R2
Methods and Evaluations	SSE	RMSE	SSE	RMSE	Total Time	R2
BLS	0.0184	0.0027	0.0184	0.0192	0.05	84.6%
BLS-IL	0.0161	0.0025	0.0191	0.0196	0.054	83.9%
CNN-BLS	0.0016	0.0008	0.0012	0.005	4.02	98.9%
CNN-BLS-IL(Our)	0.00073	0.00054	0.0005	0.0031	4.03	99.6%
SVM	0.0302	0.0246	0.0302	0.0246	0.1	76.1%

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

Share and Cite

MDPI and ACS Style

Lai, G.; Liu, W.; Yang, W.; Zhang, Y. A Convolutional Neural Network-Based Broad Incremental Learning Filter for Attenuating Physiological Tremors in Telerobot Systems. Appl. Sci. 2023, 13, 890. https://doi.org/10.3390/app13020890

AMA Style

Lai G, Liu W, Yang W, Zhang Y. A Convolutional Neural Network-Based Broad Incremental Learning Filter for Attenuating Physiological Tremors in Telerobot Systems. Applied Sciences. 2023; 13(2):890. https://doi.org/10.3390/app13020890

Chicago/Turabian Style

Lai, Guanyu, Weizhen Liu, Weijun Yang, and Yun Zhang. 2023. "A Convolutional Neural Network-Based Broad Incremental Learning Filter for Attenuating Physiological Tremors in Telerobot Systems" Applied Sciences 13, no. 2: 890. https://doi.org/10.3390/app13020890

APA Style

Lai, G., Liu, W., Yang, W., & Zhang, Y. (2023). A Convolutional Neural Network-Based Broad Incremental Learning Filter for Attenuating Physiological Tremors in Telerobot Systems. Applied Sciences, 13(2), 890. https://doi.org/10.3390/app13020890

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu

A Convolutional Neural Network-Based Broad Incremental Learning Filter for Attenuating Physiological Tremors in Telerobot Systems

Abstract

1. Introduction

2. Problem Description

3. Control Strategies

3.1. Haptic Force Feedback

3.2. PD Controller

3.3. Model Structure of CNN-BLSF

3.3.1. Physical Model Structure of CNN-BLSF

3.3.2. Mathematical Model Structure of CNN-BLSF

4. Design of CNN-BLS Tremor Filter

4.1. Algorithm of CNN-BLS

4.1.1. Broad Learning Network

4.1.2. Structure of Our CNN-BLS Network

4.1.3. Broad Expansion: Incremental Learning

5. Experiments

5.1. Parameters Setting

5.2. Simulated Tremor Signal

5.3. Tremor Forecast Results

6. Conclusions

Author Contributions

Funding

Institutional Review Board Statement

Informed Consent Statement

Data Availability Statement

Acknowledgments

Conflicts of Interest

References

Share and Cite

Article Metrics

Article Access Statistics

Further Information

Guidelines

MDPI Initiatives

Follow MDPI