Optimal Tracking A Moving Target For Integrated Mobile Robot-Pan Tilt-Stereo Camera
Optimal Tracking A Moving Target For Integrated Mobile Robot-Pan Tilt-Stereo Camera
Optimal Tracking A Moving Target For Integrated Mobile Robot-Pan Tilt-Stereo Camera
Abstract— This paper proposes a dynamic model and control In this paper, the pan-tilt robot is controlled so the target’s
method for the integrated system, including mobile robot, image holds on the origin of the camera coordinate. Two
pan/tilt and stereo camera system. It used to track a moving wheels of mobile robot moved by the suitable torque to reach
object. A pair of cameras holding by pan/tilt robot placed on to the target. The controller is constructed to optimize about
the mobile robot. Mobile robot is Pioneer 3DX moved by two position error and energy. The paper is organized as follows.
wheels and a steering wheel. Pan/tilt platform holding camera In section 2, solve the kinematic control problem for their
is a two degree-of-freedom robot, controlled camera rotation system. In section 3, a dynamic control algorithm with
following pan and tilt directions. The proposed control optimal control and the asymptotic stability of the overall
algorithm is highly optimal response qualities for minimizing system is proved by Lyapunov stability method. Section 4
error of velocity and energy of the system. shows simulation results on Matlab-Simulink to demonstrate
I. INTRODUCTION the performance of the proposed control algorithm. Finally,
some conclusions are summarized in the section 5.
The integrated system of mobile robot – pan/tilt – camera
is a system including a mobile robot, pan tilt robot and II. CONSTRUCT KINEMATIC MODEL AND KINEMATIC
stereo camera. This is a complex system with many force CONTROLLER FOR MOBILE ROBOT – PAN/TILT – STEREO
links, mutual influence between the moving of mobile robot CAMERA TO TRACK MOVING TARGET
and pan/tilt platforms [1]. With the simple system, only The image collected from each camera used to find the
mobile robot [4] or pan/tilt pedestal, we can only perform center of the target’s image. However, image-processing
tasks such as move to reach a fixed target and anticipate or algorithm is not mentioned here. The objective of this paper
tracking target [6]. To track and reach moving target is finding the controller to control the mobile robot-pan/tilt
requires a system that combined all the above elements. camera system, tracking moving targets, so that the tracking
Some papers showed that the construction of dynamics is a images and distant to target error converges on zero.
separate part of mobile robot and pan/tilt platforms [2, 5, 9].
The information obtained from cameras used to calculate out A. Determination of Image Jacobian matrix
the desire coordinates for mobile robot [10]. Some author Parameters such as distance, coordinates of target in 3D
integrated the dynamic of the system, including mobile space can be determined by using two cameras. In Fig. 1, the
robot, pan/tilt platform and mono camera [1], but the coordinates are assigned to mobile robot, pan/tilt robot and
condition to apply the system is limited on moving space of cameras. Figure 2 shows the relationship between the camera
the target. To track or reach to the target, the authors use coordinate system and its image.
many types of control algorithms from kinematic [9, 11] to Assumption 1: The intrinsic parameters of camera as focal
dynamic control method [1, 2]. Recently, some authors have length f, number of pixels, etc. are the same, placing at the
used advance methods like as neural network [3,6], sliding same height and the optical axis of two cameras is parallel.
mode [2], optimal control [5, 4, 7] or time-energy optimal The target moves in the height limited of system view space.
control [8] for having better results following a target Notations: Left camera coordinates is OLXLYLZL with the
function. origin located at the focal point of left camera, right camera
In the tracking target problem, the target moves with coordinates is ORXRYRZR, with the origin located at the focal
variable velocity. Then, the velocity of pan/tilt’s joint and point of right camera and camera coordinate is OCXCYCZC
two wheels of the mobile robot must be controlled combine with the origin located at the midpoint of origin of two
together to keep the target’s image in the space of the cameras. The photo frame is specified at the front and
camera's view. Moreover, the mobile robot must be perpendicular to the Z-axis at the center, the axis u, v parallel
controlled to reach to the target. To done this task, the to the axis X, Y of camera, respectively.
velocity of the target is calculated or measured, Besides the θ1 is the direction angle of the mobile robot. Pan angle is
parameters of the system are not entirely accurate then to θ2, its rotation around the axis Z1, Tilt angle is θ3, its rotation
control all the system is a complex problem. around the axis Z3. The feature point coordinates of the
target obtained from left camera’s image is (UL, VL) and the
*Research supported by the Vietnam National Foundation for Science right camera’s is (UR, VR) on two axes (u, v). Following the
and Technology Development (NAFOSTED). assumption 1, 2 cameras have the same height so the
Van Chung Le is with the University of information and
coordinates of obtained images are the same on V axis or
communication technology, Thai Nguyen, Viet Nam (phone: +84-0988-
311-658; email: lvchung@ictu.edu.vn) VR= VL= V. From Fig. 2, the coordinates of feature point on
Thuong Cat Pham is with the Institute of Information Technology, the left and right image frame are transformed to (X, Z) and
Vietnamese academy of science and technology, Ha Noi, Viet Nam (email:
ptcat@ioit.ac.vn).
531
last
m xr
(m)last J r Jp 2
last
J1 1 ; (12)
t yr 3
where k1 is the distance from the midpoint of the 2 robot's r1 cos 1 sin 1 0 xrd xr
wheels to the center of the target see on the plane X000Y0. εr sin 1 cos 0 yrd yr . (20)
r2 1
Assumption 2: The motion of robot wheels is non-slip.
The angular, angular and straight velocity of the two r3 0 0 1 1d 1
wheels denote as ( left , left , vleft ) and ( right , right , vright ). Taking the derivative of εr with time, yields:
The relationships between vleft , vright and straight velocity r1 r r2 vr vrd cos r 3
vr, and angular velocity ωr of robot are: εr r2 r r1 vrd sin r 3 (21)
vright 1 1 k/2 vr r3 rd r
v . (18)
vleft r 1 k/2 r where: , vrd xrd yrd .
rd 1d
532
T
The kinematic control law applied to find vm , m in and pan/tilt joint is vs [ R , L , 2 , 3 ] and desired vector
order to εr 0 when t . There is some method to find is vd [ Rd , Ld , 2d , 3d ] . The dynamics of integrated
it. Backstepping method is the most used to build a mobile robot – pan/tilt robot with no friction or effect of
kinematic controller to track a moving target with desire noise can be written as follows:
velocity. Here, the straight and angular velocity of mobile M(q) v s C(q, v s ) v s g(q) = τ. (30)
robot is chosen following [3] as: With M(q) is the (n x n) symmetric positive definite inertia
vrd cos k2 matrix. q is the (n x 1) vector of integrated robot wheels and
r3 r1
vm joints. C(q, v s ) is the (n x n) vector of centripetal and
sin r3 . (22)
m rd k3 r3 vrd r2
Coriolis effects. g (q) is the vector of gravitational torque
r3 and τ is the torque vector applied for robot wheels and
Substituting (14) into (13) result in: joints. In this system, n = 4.
k1 r1 Assumption 3: The initial matrix M(q) satisfies the
r1 m r2
equations and inequalities (t ) :
εr r2 m r1 vd sin r3 , (23) 2 T 2
m1 M q m(q) ; (31)
r3 k3 r3 vd r 2 (sin r3 / r3 ) T
M q 2C q, q 0 Rn ; (32)
where k2, k3 are positive coefficient. If r3 0, where m1 is a positive constant, m(q) is a known positive
(sin r3 ) / r3 1 , then ωm is bounded. Following function.
assumption 2, we have the desired angular velocity of the Assumption 4: C q, v s ; g(q) are bounded, the first and
two wheels of mobile robot as in (24): second derivatives of the element M(q); C q, v s ; g(q) ;
1
rightd r/2 r/2 vm C q, v s are exits and bounded. The desired variable of the
, (24)
leftd r/k r/k m system q d , q d , q d and desired moment τ d , τ d , τ d are exits
where r = d/2 is the radius of the wheel; and bounded.
k is the distance between two wheels. The tracking error between the desired velocity and the
From kinematic controller (15) and (22), we are going to actual velocity of the integrated robot can be expressed as:
prove asymptotic stability system when t e vd vs . (33)
We choose the candidate Lyapunov function as follows:
B. Optimal Control Problem.
1 T
Vk εi εi εTr ε r . (25) If we choose a PD controller used Computed-Torque-
2 Controller (CTC) method, momentum τ is calculated as
We have Vk 0 when εi , ε r 0 ; Vk 0 if and only if follows:
εi εr 0 i=1,2,3; V when εi , ε r . Taking τ M(q)(vd kve k pe) C(q, vs )(vd kve) g(q) u. (34)
the derivative of Vk along time, yields: With u(t) is the optimal control components of the system. It
will be determined later. kv , k p are the gain coefficient of
Vk εTi εi εTr ε r Vk1 Vk 2 ; (26)
the controller CTC.
where: Substituting τ in (34) into (30) result in:
Vk1 εTi Kεi ; (27) M(q)(e kve k pe) C(q, v s )(e kve) u. (35)
and: T
Set the system state vector is s e e . We have the
Vk 2 er1er1 er 2 er 2 er 3er 3
. (28) equation of the system in the state-space equation form (35)
er1 ( vr vrd cos er 3 ) er 2 vrd sin er 3 er 3 (ωrd ωr ) following:
Substituting (22) into (28), yields: s A(q, v s )s B(q)u ; (36)
Vk 2 k2 er21 k3er23 . (29) where:
0n x n I 0n x n
From (27), (29), we have Vk 0 because K, k2, k3 are A(q, v s ) 1 1 , B(q) .
positive matrix and constant, the system is asymptotic k p I kv M C kv I M C M 1
stability, εr 0 when t . I, 0 are n x n identity and zero matrices. Q is 2n x 2n
symmetric positive definite weight matrix of state. R is n x n
positive definite weight matrix of the controller. Rewriting
III. OPTIMAL CONTROLLER FOR DYNAMIC SYSTEM
Q11 Q12
A. Dynamic model for mobile robot – pan/tilt system. Q in the following blocks: Q .
Q21 Q22
The measured vector of angular velocity of two wheels
533
C. Quadratic performance Optimal Control
1) Objective function
The problem of quadratic performance optimization is to
minimize an objective function J (u) :
1
J (u) sT Qs uT Ru dt . (37)
2 0
Using Hamilton-Jacobi equations-Bellman (HJB) to solve
[14], the tasks is finding a control signal u*(t) so the
objective function (37) of the system (36) reaches minimum. Figure 5. Structure of system
2) Constraints
Conditions for exists a solution if and only if there exists T
where s e e . Because P is a symmetric positive
the function V(s, t) so satisfy the HJB equation:
T definite matrix, then V > 0 when s ≠ 0; V= 0 if only if s = 0.
V (s, t ) V (s, t ) More over V is bounded function:
min H s, u, ,t 0; (38)
t u s s
2
V (s, t )
2
(50)
m M s , 0 m M .
where: Taking the derivative of V along time, yields:
1 T (39) 1 T 1 T 1 T
V (s, t )
s (t )P(s, t )s(t ). V s Ps s Ps s Ps . (51)
2 2 2 2
H is the Hamintonian function of opitimization defined as: Substituting (48) into (36) result in:
T
V (s, t ) 1 T 1 T V (s, t ) s ( A BR 1BT P)s . (52)
H s, u , ,t s Qs u Ru s. (40)
s 2 2 s Substituting P subtracted from (46) and s from (53) into
When the control signal is optimal u(t) = u*, objective (51) yields:
function (37) reaches minimum, Hamintonian functions is: 1 1 T
V (( A BR 1BT P)s)T Ps s P(A BR 1BT P)s
V (s, t )
T
V (s, t ) 2 2 (53)
H* min H s, u* , ,t . (41) 1 T T 1 T
u s t s ( PA A P PBR B P Q)s ;
2
3) Solving 1 T 1 T T
Let P(q) is 2n x 2n symmetric positive definite matrix: or: V s Qs s P B(R 1 )T BT Ps . (54)
2 2
T 0n x n Because Q and R-1 are positive definite matrix, it yields:
P(q) ; (42)
0n x n M 1 T (55)
V s Qs .
where T is n x n symmetric positive definite matrix. If 2
k p , kv , R, T in the equation (34), (37), (42) satisfy: Because Q is a symmetric positive definite matrix, from (55)
we have:
Q12 QT21 T k pM kv C ; (43) 1 2
V w1 s 0. (56)
Q11 0 ; (44) 2
1 Where w1 is the smallest value of Q.
R Q22 , (45)
We see that V is positive definite function. V 0 is
Then we have P(q) satisfies Riccati equation (38):
negative-semidefinite function. when t the set
P PA AT P PBR 1BT P Q 0 (46)
L e1 , e2 , e3 , e4 defined by V 0 contains only an
and the value of the V(s, t) function is:
equilibrium in 0(0, 0, 0, 0). Following the LaSalle invariance
1 T
V (s, t ) s Ps . (47) principle the equilibrium 0 is the asymptotically stable of the
2 system. In other words, e 0 , e 0 when t the
u*(t) satisfy (41), so the objective function (37) of the tracking error will asymptotic to 0.
system (36) reaches minimum [12] is: Flow chart of the algorithm can be seen in Fig. 6.
u* (t ) R 1BT Ps . (48)
Optimal control signal u* (48), control law (34) will IV. SIMULATION
make the dynamics in equation (36) tracking desire velocity Simulate the system with parameters: R = diag[0.5, 0.5, 0.5,
with objective function (37) reach minimum if parameters 0.5]; T = diag[10, 10, 10,10]; kp =500, kv = 10; The matrices
are satisfy (43, 44, 45). c11 c12 c13 c14
m11 m12 m13 0
D. Stability of system m21 m22 m23 0 c21 c22 c23 c24
We choose the candidate Lyapunov function as follows: M , C
m31 m32 m33 0 c31 c32 c33 c34
1 T (49) 0 0 0 m44 c41 c42 c43 0
V (s, t ) s (t )P(s, t )s(t );
2
534
Figure 7. Tracking target of stereo camera (left), mobile robot (right)
V. CONCLUSION
In the paper, the kinematic and dynamic models of the
Figure 6. Flow chart of the algorithm
integrated system are constructed. In addition, the authors
where: propose an quadratic performanceoptimal control for the
system. The pseudo-inverse matrix of J p is reversible at -π/2
m11 m22 0.25(m1 m2 m3 )r 2 mb r 2 Ib
cicle of the target movement, different from [6], it is not
0.5 I 3 y 1 cos(2 3 ) I 3 x 1 cos(2 3 ) (r / k ) 2 invertible, but it still cause of shock at about 35 seconds.
2Id I1z I2z (m2 m3 )l12 2m3l1l3c2 s3 (r / k ) 2 ; However, we will research about inverse property of it in
other paper.
m12 m21 0.25(m1 m2 m3 )r 2
0.5 I 3 y 1 cos(2 3 ) I 3 x 1 cos(2 3 ) (r / k ) 2 REFERENCES
[1] N.V. Tinh, P.T. Cat, P.M. Tuan, T.T. Quyen Bui, “Visual control of
2I d I1z I2z (m2 m3 )l12 2m3l1l3c2 s3 (r / k ) 2 ; integrated mobile robot-pan tilt-camera system for tracking a moving
target”, IEEE 2014 on robotics and biomimetics, pp 1566-1571.
m13 m31 m23 m32 I2 z m3l1l3 c2 s3 (r / k ) [2] Yang Fang, Wang Chao-Li, “Adaptive stabilization for uncertain
nonholonomic dynamic mobile robots based on visual servoing
0.5 I 3 y 1 cos(2 3 ) I3 x 1 cos(2 3 ) ( r / k ); feedback”, ACTA Automatic sinica, Vol. 37, No. 7, pp 857-864.
[3] T.H Thuan, N.V Tinh, P.M. Tuan, P.T Cat, “Trajectory tracking
m33 I2 z 0.5(1 cos(2 3 ))I3 y 0.5(1 cos(2 3 ))I3x ; control of a mobile robot by computed torque method with on-line
T learning neural network”, IEEE 2013-control and automation, pp
m44 I3 z m3l32 ; g(q) 0,0,0, m3 gl3 sin( 3 ) . 1184-1189.
[4] E.M. Guti´ errez-Arias, J.E. Flores-Mena, M.M. Morin-Castillo, and
c11 c22 c21 c12 H. Su´ arez-Ram´ ırez, “Design of an optimal control for an
0.5( I 3 x I 3 y ) sin(2 3 ) m3l1l3 ( s2 s3 c2 c3l1l3 3 ) (r / k ) 2 ; autonomous mobile robot”, Revista mexicada F’isica 57, pp. 75–83.
3 2
[5] T. Hu and S. Yang, “A novel tracking control method for a wheeled
mobile robot”, 2nd Workshop on Conference. Korea, 2001,pp.104-116
c13 c23 0.5( I 3 x I 3 y ) sin(2 3 ) 3 (r / k ) [6] Le Van Chung, Pham Thuong Cat, “A new control method for stereo
visual servoing system with pan-tilt platform”, Journal of Computer
m3l1l3 s2 s3 ( 1 2 ) c2 c3 3 (r / k ) Science and Cybernetics, vol 31, N.2, pp 107-122.
[7] Aleksey A.kabanov,“Optimal control of mobile robot’s trajectory
c14 c24 c41 c42 movement, “Wseas transactions on system and control”, vol 9, 2014,
(0.5( I3 x I3 y )sin(2 3 )( ) m3l1l3c2 c3 1 )(r / k ), pp. 398-404.
1 2
[8] Mohamad Shahab, Amar Khoukhi, Fouad Al-Sunni, “Time-energy
c31 c32 (0.5( I 3 x I 3 y )sin(2 3 ) 3 m3l1l3 s2 s3 q3 )(r / k ) 2 , optimal control of a mobile robot using direct numerical method,
[9] Widodo Budiharto, Ari Santoso, Djoko Purwanto, Achmad Jazidie,
c34 c43 0.5( I 3 x I 3 y )sin(2 3 )( “A navigation system for service robot using stereo vision and kalman
1 2) m3l1l3 c2 c3 1 ,
filtering”, International Conference on Control 2011. pp.1771 – 1776.
c33 0.5( I 3 x I 3 y )sin(2 3 ) 3 , [10] Ae-Il Kim, Wook Bahn, Changhun Lee, Tae-jae Lee, Muhammad
Muneeb Shaikh, Kwang-soo Kim. “Vision System for Mobile Robots
The target move in space following trajectory: when t = 0 (x, for Tracking Moving Targets, Based on Robot Motion and Stereo
y, z) = (4, 2, 0) in O0 coordinates. Process variability of Vision Information” SI International 2011 IEEE, pp. 634-639.
[11] Wook Bahn, Jaehong Park, Chang-hun Lee, Tae-il Kim, Teajae Lee,
straight and angular velocity with time is: zt = 0.1t. Muhammad Muneeb Shaikh, Kwang-soo Kim, and Dong-il Dan Cho,
0≤ t <5: v=0.5m/s. ω = 0 rad/s. “A motion-information-based vision-tracking system with a stereo
5≤ t <10: v = 0.5 + 0.15sin((t-5)π/10); ω=0.15sin((t-5)π/10); camera on mobile robots”, 2011 IEEE, pp. 252-257.
10≤ t <15: v=0.75m/s. ω = 0 rad/s. [12] Keith Dupree, Parag M.Patre, Zachary D.Wilcox,Warren E.Dixon,
“Asymptotic optimal control of uncertain nonlinear Euler–Lagrange
15≤ t <20: v = 0.75+0.15sin((t-15)π/10); ω= -0.15sin((t-15) systems”, Automatica 47(2011), pp. 99–107
π/10). t = 20-25s: v=0.75m/s. ω = -0.15 rad/s. [13] Kim, Y.H & Lewis, F.L. (2000). “Optimal design of CMAC neural-
t = 25-30s: v=0.75m/s. ω =-0.15 - 0.15sin((t-25)π/10); network controller for robot manipulators”, IEEE Transactions on
t > 30s: v=0.5m/s. ω = -0.3 rad/s; Systems, 30(1), pp. 22–31.
[14] Lewis, F.L & Syrmos, V.L, “Optimal control”, Wiley, 1995.
535