A novel approach to extract hand gesture feature in depth images

This paper proposes a novel approach to extract human hand gesture features in real-time from RGB-D images based on the earth mover’s distance and Lasso algorithms. Firstly, hand gestures with hand edge contour are segmented using a contour length information based de-noise method. A modified finger earth mover’s distance algorithm is then applied applied to locate the palm image and extract fingertip features. Lastly and more importantly, a Lasso algorithm is proposed to effectively and efficiently extract the fingertip feature from a hand contour curve. Experimental results are discussed to demonstrate the effectiveness of the proposed approach.

The authors would like to acknowledge support from DREAM project of EU FP7-ICT 611391 and Research Project of State Key Laboratory of Mechanical System and Vibration China MSV201508.

Authors and Affiliations

1. University of Portsmouth, Portsmouth, UK

   Zhaojie Ju

2. State Key Laboratory of Mechanical System and Vibration, Shanghai, China

   Zhaojie Ju

3. University of Portsmouth, Portsmouth, UK

   Dongxu Gao

4. Liaoning Shihua University, Fushun, China

   Jiangtao Cao

5. University of Portsmouth, Portsmouth, UK

   Honghai Liu

Corresponding author

Correspondence to Honghai Liu.

Appendix

Algorithm 1 Hand Edge Contour Generation

Require: Pixel[r] {Pixel is a 4 dimensional dataset with n pixels in the RGB-D image and r = {1,…,n}, n is the total number of the pixels in the image}

Require: Hand_centre {Hand_centre is the location of the centre of the tracked hand}

Require: initial_depth {initial_depth is the average depth of the tracked hand}

mini_depth = initial_depth-offset {offset is the user chosen threshold}

max_depth = initial_depth + offset

for all i such that 0≤i≤n do

         if mini_depth≤depth(i)≤max_depth and |pos(i)-Hand_centre| ≤0.5*body_length {body_length is the preset length of the body in pixels; depth(i) returns the depths of pixcel[i]; pos() gets the position of pixel[i],} then

         is_hand[i] ← true {is_hand[] is an array that indicates hand pixels}

else

         is_hand[i] false

end if

end for

for all i such that 0≤i≤n do

         if Is_hand[i]==true then

         K {k1,…,k8| |pos(k _r ) − pos(i)| < 2} {get the eight nearby pixels around the ith pixel, 0≤r≤n}

   if \( \prod_{j=1}^8 Is\_ hand\left[{k}_j\right]== true \) then

         Is_contour[i] ← true {Is_contour[] is an array that indicates the hand contour}

else

   Is_contour[i] ← false

end if

  else

is_contour[i] ← false

  end if

end for

return is_contour

Algorithm 2 Lasso Algorithm

struct m_signature {structure of fingertip feature}

{

         int a,b {the coordinate values of the fingers}

         bool R_suspended,L_suspended {indicates if the lasso is suspended}

         double rope_length {the length of the lasso}

}

Dim m_signature_num As int {number of the fingertip signature}

Dim L_cur As int {left cursor of the lasso}

Dim R_cur As int {right cursor of the lasso}

Dim R_cur_move As bool

Dim L_cur_move As bool {if the cursor should be moved}

Dim distance As int {the dynamic length of the lasso}

Dim pre_distance As int {the former dynamic length of the lasso}

Rope(m_signature[],m_signature_num,L_cur,R_cur,R_cur_move,distance,pre_distance) {function of the lasso algorithm}

{

For all i such that 0≤i≤m_signature_num {m_signature_num is the number of fingertip features} then

         While distance<m_signature[i].rope_length {if the length of the lasso is longer than the preset value of the width of the finger,that means the lasso has clamped completely, the lasso procedure will come to an end}

         Do

If (R_suspended) AND (R_cur_move) then

      R_cur<−R_cur+1 {move R_cur right}

      calculate R_suspended

         If R_suspended==ture {if the right side of the lasso is suspended, then its left side will move left} then

         R_cur_move ← false

      calculate distance

      end if

         If (distance > pre_distance) AND (!L_suspended) {if the length of the lasso become longer, then its left side will move left} then

         R_cur_move ← false

  pre_distance ← distance

      Else If (!L_suspended)AND(L_cur_move) then

         L_cur ← L_cur+1

  calculate L_suspended

      end if

         If L_suspended==true {if the left side of the lasso is suspended, then move the right side towards the right next} then

         R_cur_move ← true

  calculate distance

         end if

         If (distance>pre_distance&&!R_suspended) {if the length of the lasso become longer,then move the right side towards the right} then

         R_cur_move ← true

         pre_distance ← distance

         end if

         end if

         end while

  m_signature[i].a ← L_cur

  m_signature[i].b ← R_cur

end for

Reprints and permissions