research-article

Towards hand gesture based writing support system for blinds

Authors:

Gourav Modanwal,

Kishor SarawadekarAuthors Info & Claims

Pattern Recognition, Volume 57, Issue C

Pages 50 - 60

https://doi.org/10.1016/j.patcog.2016.03.026

Published: 01 September 2016 Publication History

Abstract

In this digital age, personal computers are widely used. But the blind people cannot handle it easily. Human-computer interaction (HCI) may play vital role in overcoming this obstacle. Computer keyboard contains about 104 keys, whereas six-dot Braille keyboard supports only 63 symbols. To bridge this gap, a new dactylology is proposed in this article. Additionally, a new feature extraction technique, Reduced Shape Signature (RSS), is introduced in this article which is rotation, translation and scale invariant. Further, RSS is compact as it reduces the number of feature-sets by 35%. To discriminate intra-class gestures difference angle and polygonal area are computed along with RSS. The symbols recognition rate of the proposed algorithm is 97.53%. A new dactylology is proposed to achieve functionality similar to a standard keyboard.A new feature extraction technique called as reduced shape signature (RSS) is introduced.Use of difference angle and polygonal area is proposed to discriminate Intra-class gestures.Symbols are entered in the Microsoft Word application with the proposed experimental setup and real time data entry support is verified.

References

[1]

Y. Wu, T.S. Huang, Vision-based gesture recognition: a review, Springer Berlin, Heidelberg, 1999.

[2]

C. Shan, T. Tan, Y. Wei, Real-time hand tracking using a mean shift embedded particle filter, Pattern Recognit., 40 (2007) 1958-1970.

Digital Library

[3]

P. Priyal, P.K. Bora, A robust static hand gesture recognition system using geometry based normalizations and Krawtchouk moments, Pattern Recognit., 46 (2013) 2202-2219.

Digital Library

[4]

Annual Report 2015: distribution of eligible students based on the federal quota census of January 6, American printing house for the blind, 2015.

[5]

B.X. William Ivanich, Systems and methods for hand gesture control of an electronic device. Patent US20140229845 A1, 14 August, 2014.

[6]

N.M.O. Andrew D. Wilson, Architecture for controlling a computer using hand gestures. Patent US8745541 B2, 3 June, 2014.

[7]

M.E. Ayadi, M.S. Kamel, F. Karray, Survey on speech emotion recognition: features, classification schemes, and databases, Pattern Recognit., 44 (2011) 572-587.

Digital Library

[8]

S. Gao, Y. Wang, X. Gao, B. Hong, Visual and auditory brain-computer interfaces, IEEE Trans. Biomed. Eng., 61 (2014) 1436-1447.

[9]

J. Lin, Y. Ding, A temporal hand gesture recognition system based on hog and motion trajectory, Opt. Int. J. Light. Electron Opt., 124 (2013) 6795-6798.

[10]

V. Pavlovic, R. Sharma, T. Huang, Visual interpretation of hand gestures for human-computer interaction: a review, Pattern Anal. Mach. Intell., 19 (1997) 677-695.

Digital Library

[11]

A. El-Sawah, N.D. Georganas, E. Petriu, A prototype for 3-D hand tracking, IEEE Trans. Instrum. Meas., 57 (2008) 1627-1636.

[12]

X. Teng, B. Wu, W. Yu, C. Liu, A hand gesture recognition system based on local linear embedding, J. Vis. Lang. Comput., 16 (2005) 442-454.

Digital Library

[13]

F. Dominio, M. Donadeo, P. Zanuttigh, Combining multiple depth-based descriptors for hand gesture, Pattern Recognit. Lett. (2013) 101-111.

[14]

D.-L. Dinh, S. Lee, T.-S. Kim, Hand number gesture recognition using recognized hand parts in depth images, Multimed. Tools Appl., 75 (2016) 1333-1348.

Digital Library

[15]

Y.T. Li, J.P. Wachs, HEGM: a hierarchical elastic graph matching for hand gesture recognition, Pattern Recognit., 47 (2014) 80-88.

Digital Library

[16]

Z. Ren, J. Yuan, J. Meng, Z. Zhang, Robust part-based hand gesture recognition using kinect sensor, IEEE Trans. Multimed., 15 (2011) 1110-1120.

Digital Library

[17]

Y.T. Li, J.P. Wachs, Recognizing hand gestures using the weighted elastic graph matching (WEGM) method, Image Vision. Comput., 31 (2013) 649-657.

Digital Library

[18]

N. Pugeault, R. Bowden, Spelling It out: real-time asl fingerspelling recognition, In: Proceedings of the Computer Vision Workshops (ICCV), 2011.

[19]

L. Yun, Z. Lifeng,¿Z. Shujun, A hand gesture recognition method based on multi-feature fusion and template matching, In: Proceedings of the International Workshop on Information and Electronics Engineering, 2012.

[20]

A. Gupta, V.K. Sehrawatb, M. Khoslac, FPGA based real time human hand gesture recognition system, In: International Conference on Communication, Computing & Security, 2012.

[21]

P. Hays, R. Ptucha, R. Melton, Mobile device to cloud co-processing of asl finger spelling to text conversion, In: Image Processing Workshop, 2013.

[22]

K. Otiniano-Rodr¿ıguez, G. C¿amara-Ch¿avez, Finger spelling recognition from RGB-D information using kernel descriptor, In: Conference on Graphics, Patterns and Images, 2013.

Digital Library

[23]

H. El Hayek, J. Nacouzi, A. Kassem, M. Hamad, Sign to letter translator system using a hand gloves, In: International Conference on e-Technologies and Networks for Development, 2014.

[24]

M. Panwar, P.S. Mehra, Hand gesture recognition for human computer, In: International Conference on Image Information Processing (ICIIP), 2011.

[25]

K. Silanon, N. Suvonvorn, Finger-spelling recognition system using fuzzy finger shape and hand appearance features, In: Digital Information and Communication Technology and it's Applications (DICTAP), 2014.

[26]

A. Kuznetsova, L. Leal-Taixe, B. Rosenhahn, Real-time sign language recognition using a consumer depth camera, In: Computer Vision Workshops (ICCVW), 2014.

Digital Library

[27]

M. Panwar, Hand gesture recognition based on shape parameters, Computing, Communication and Applications (ICCCA), Dindigul, Tamilnadu, 2012.

[28]

S. Bourennane, C. Fossati, Comparison of shape descriptors for hand posture recognition in video, Signal Image Video Process., 6 (2012) 147-157.

[29]

American manual alphabet, Wikimedia Foundation, Inc., 11 June 2014. Online. Available: {http://en.wikipedia.org/wiki/American_manual_alphabet}. Accessed 11 March 2015.

[30]

G. Buchsbaum, A spatial processor model for object colour perception, J. Frankl. Inst., 310 (1980) 1-26.

[31]

H. Rein-Lien, M. Abdel-Mottaleb, A.K. Jain, Face detection in color images, Pattern Anal. Mach., 24 (2002) 696-706.

Digital Library

[32]

J.A.M. Basilio, G.A. Torres, Explicit image detection using YCbCr space color model as skin detection In: Proceedings of the 2011 American Conference on Applied Mathematics, 2011.

Digital Library

[33]

R.C. Gonzalez, R.E. Woods, S.L. Eddins, Filtering in frequency domain, McGraw Hill Education (India), Gatesmark,LLC, 2010.

[34]

C. Solomon, T. Breckon, John Wiley & Sons, Ltd, Chichester, West Sussex, 2011.

[35]

A. El-ghazala, O. Basira, Saeid Belkasimb, Farthest point distance: a new shape signature for fourier descriptors, Signal Process.: Image Commun., 24 (2009) 572-586.

Digital Library

[36]

D. Zhang, G. Lu, Review of shape representation and description techniques, Pattern Recognit., 37 (2004) 1-19.

[37]

D.M. Tsai, M.F. Chen, Object recognition by linear weight classifier, Pattern Recognit. Lett., 16 (1995) 591-600.

Digital Library

Cited By

Kumar SChandra Trivedi MChauhan A(2023)Real Time Air-Written Mathematical Expression Recognition for Children’s Enhanced LearningNeural Processing Letters10.1007/s11063-022-11012-355:3(3355-3375)Online publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1007/s11063-022-11012-3
Elboushaki AHannane RAfdel KKoutti L(2020)Improving articulated hand pose detection for static finger sign recognition in RGB-D imagesMultimedia Tools and Applications10.1007/s11042-020-09370-y79:39-40(28925-28969)Online publication date: 7-Aug-2020
https://dl.acm.org/doi/10.1007/s11042-020-09370-y
Modanwal GMishra PSarawadekar K(2018)A Robust Algorithm for Hand-forearm SegmentationProceedings of the 2018 International Conference on Image and Graphics Processing10.1145/3191442.3191469(102-105)Online publication date: 24-Feb-2018
https://dl.acm.org/doi/10.1145/3191442.3191469

Index Terms

Towards hand gesture based writing support system for blinds
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
        Scene understanding
  2. Machine learning
    1. Machine learning algorithms
      1. Feature selection

Recommendations

Discriminating features learning in hand gesture classification

The advent and popularity of Kinect provides a new choice and opportunity for hand gesture recognition (HGR) research. In this study, the authors propose a discriminating features extraction for HGR, in which features from red, green and blue (RGB) images ...
Hand tracking and gesture recognition system for human-computer interaction using low-cost hardware

Human-Computer Interaction (HCI) exists ubiquitously in our daily lives. It is usually achieved by using a physical controller such as a mouse, keyboard or touch screen. It hinders Natural User Interface (NUI) as there is a strong barrier between the ...
Hand Gesture Recognition Based on Fingertip Detection
GCIS '13: Proceedings of the 2013 Fourth Global Congress on Intelligent Systems

To improve the hand gesture recognition accuracy based on Hu moment features, a new recognition algorithm was developed based on the fingertip structure detection. Firstly, the geometric features, the areas of skin region and the image, were designed to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Pattern Recognition

Pattern Recognition Volume 57, Issue C

September 2016

198 pages

ISSN:0031-3203

Issue’s Table of Contents

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 September 2016

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kumar SChandra Trivedi MChauhan A(2023)Real Time Air-Written Mathematical Expression Recognition for Children’s Enhanced LearningNeural Processing Letters10.1007/s11063-022-11012-355:3(3355-3375)Online publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1007/s11063-022-11012-3
Elboushaki AHannane RAfdel KKoutti L(2020)Improving articulated hand pose detection for static finger sign recognition in RGB-D imagesMultimedia Tools and Applications10.1007/s11042-020-09370-y79:39-40(28925-28969)Online publication date: 7-Aug-2020
https://dl.acm.org/doi/10.1007/s11042-020-09370-y
Modanwal GMishra PSarawadekar K(2018)A Robust Algorithm for Hand-forearm SegmentationProceedings of the 2018 International Conference on Image and Graphics Processing10.1145/3191442.3191469(102-105)Online publication date: 24-Feb-2018
https://dl.acm.org/doi/10.1145/3191442.3191469

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents