Abstract
Human-computer interaction has become increasingly easy and popular using widespread smart devices. Gestures and sketches as the trajectory of hands in 3D space are among the popular interaction media. Therefore, their recognition is essential. However, diversity of human gestures along with the lack of visual cues make the sketch recognition process challenging. This paper aims to develop an accurate sketch recognition algorithm using Block Sparse Bayesian Learning (BSBL). Sketches are acquired from three datasets using a Wii-mote in a virtual-reality environment. We evaluate the performance of the proposed sketch recognition approach (MATRACK) on diverse sketch datasets. Comparisons are drawn with three high accuracy classifiers namely, Hidden Markov Model (HMM), Principle Component Analysis (PCA) and K-Nearest Neighbour (K-NN). MATRACK, the developed BSBL based sketch recognition system, outperforms k-NN, HMM and PCA. Specifically, for the most diverse dataset, MATRACK reaches the accuracy of 93.5%, where other three classifiers approximately catches 80% accuracy.
Similar content being viewed by others
References
Akl A, Valaee S (2010) Accelerometer-based gesture recognition via dynamic-time warping, affinity propagation, & compressive sensing. in Acoustics Speech and Signal Processing (ICASSP), 2010 I.E. International Conference on, p 2270–2273
Álvaro F, Sánchez J-A, Benedí J-M (2014) Recognition of on-line handwritten mathematical expressions using 2d stochastic context-free grammars and hidden markov models. Pattern Recognition Letters 35:58–67
Andreopoulos A, Tsotsos JK (2013) 50 Years of object recognition: directions forward. Comput Vis Image Underst 117:827–891
Bae S-H, Balakrishnan R, Singh K (2008) ILoveSketch: as-natural-as-possible sketching system for creating 3d curve models. in Proceedings of the 21st annual ACM symposium on User interface software and technology, p 151–160
Baum LE, Petrie T (1966) Statistical inference for probabilistic functions of finite state Markov chains. Ann Math Stat 37:1554–1563
Billinghurst M, Piumsomboon T, Bai H (2014) Hands in space: gesture interaction with augmented-reality interfaces. IEEE Comput Graph Appl 34:77–80
Boyali A and Hashimoto N (2014) "Block-Sparse Representation Classification based gesture recognition approach for a robotic wheelchair," in Intelligent Vehicles Symposium Proceedings, 2014 IEEE, pp. 1133–1138.
Boyali A, Kavakli M (2012) A robust gesture recognition algorithm based on sparse representation, random projections and compressed sensing. in Industrial Electronics and Applications (ICIEA), 2012 7th IEEE Conference on, p 243–249
Braga-Neto UM, Dougherty ER (2004) Is cross-validation valid for small-sample microarray classification?, Bioinformatics 20:374–380
Candè EJ, Wakin MB (2008) "an introduction to compressive sampling," Signal Processing Magazine. IEEE 25:21–30
Candes EJ, Tao T (2006) "near-optimal signal recovery from random projections: universal encoding strategies?," Information Theory. IEEE Transactions 52:5406–5425
Candès EJ, Romberg J, Tao T (2006) "robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information," Information Theory. IEEE Transactions 52:489–509
Company P, Contero M, Varley P, Aleixos N, Naya F (2009) Computer-aided sketching as a tool to promote innovation in the new product development process. Comput Ind 60:592–603
Cunningham P, Delany SJ (2007) k-Nearest neighbour classifiers. Multiple Classifier Systems, pp. 1–17
Dix A (2009) Human-computer interaction. Springer, Berlin
Donoho DL (2006) "compressed sensing," Information Theory. IEEE Trans 52:1289–1306
Donoho DL, Elad M (2003) Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization. Proc Natl Acad Sci 100:2197–2202
Eitz M, Hays J (2011) Learning to classify human object sketches. in ACM SIGGRAPH 2011 Talks, p 30
Eitz M, Hildebrand K, Boubekeur T, Alexa M (2011) "sketch-based image retrieval: benchmark and bag-of-features descriptors," Visualization and Computer Graphics. IEEE Trans 17:1624–1636
Eitz M, Hays J, Alexa M (2012) How do humans sketch objects? ACM Trans Graph 31:44
Fariman HJ, Ahmad SA, Marhaban MH, Ghasab MA, Chappell PH (2015) Hand movements classification for myoelectric control system using adaptive resonance theory. Australas Phys Eng Sci Med 39:1–18
Freina L, Ott M (2015) A literature review on immersive virtual reality in education: state of the art and perspectives. in The International Scientific Conference eLearning and Software for Education, p. 133
Ghorbel A, Anquetil E, Camillerapp J, Lemaitre A (2014) IMISketch: an interactive method for sketch recognition. Pattern Recogn Lett 35:78–90
Ghorbel A, Lemaitre A, Anquetil E, Fleury S, Jamet E (2015) Interactive interpretation of structured documents: application to the recognition of handwritten architectural plans. Pattern Recogn 48:2446–2458
Hammond TA, Logsdon D, Paulson B, Johnston J, Peschel JM, Wolin A, et al. (2010) A sketch recognition system for recognizing free-hand course of action diagrams, in IAAI
Hu R, Collomosse J (2013) A performance evaluation of gradient field hog descriptor for sketch based image retrieval. Comput Vis Image Underst 117:790–806
Johnson G, Gross MD, Hong J, Yi-Luen Do E (2009) Computational support for sketching in design: a review. Found Trends on Hum-Comput Interact 2:1–93
Karam M (2006) Investigating user tolerance for errors in vision-enabled gesture-based interactions. in Proceedings of the working conference on Advanced visual interfaces, p 225–232
Kavakli M (2008) Gesture recognition in virtual reality. International Journal of Arts and Technology 1:215–229.
Kavakli M, Boyali A (2012) De-SIGN: robust gesture recognition in conceptual design, sensor analysis and synthesis. in Advances in Robotics and Virtual Reality, ed: Springer, p 201–225
Kavakli M, Jayarathna D (2005) Virtual hand: an interface for interactive sketching in virtual reality. in Computational intelligence for modelling, control and automation, 2005 and international conference on intelligent agents, Web Technologies and Internet Commerce, International Conference on, p 613–618
Kim KI, Jung K, Kim HJ (2002) "face recognition using kernel principal component analysis." Signal Processing Letters. IEEE 9:40–42.
Kim KS, Choi HH, Moon CS, Mun CW (2011) Comparison of k-nearest neighbor, quadratic discriminant and linear discriminant analysis in classification of electromyogram signals based on the wrist-motion directions. Curr Appl Phys 11:740–745
Landay J, Myers B (2001) Sketching interfaces: toward more human interface design. IEEE Journal of Computer 34:56–64
LaViola JJ Jr (2015) Context aware 3D gesture recognition for games and virtual reality. in ACM SIGGRAPH 2015 Courses, p 10
Li Y, Hospedales TM, Song Y-Z, Gong S (2015) Free-hand sketch recognition by multi-kernel feature learning. Comput Vis Image Underst 137:1–11
Liu N (2003) Lovell BC. Gesture classification using hidden markov models and viterbi path counting. in VIIth Digital Image Computing, Techniques and Applications
Mäntyjärvi J, Kallio S, Korpipää P, Kela J, Plomp J (2005) Gesture interaction for small handheld devices to support multimedia applications. Journal of Mobile Multimedia 1:92–111
Mitra S, Acharya T (2007) "gesture recognition: a survey," Systems, Man, and Cybernetics, Part C: Applications and Reviews. IEEE Transactions 37:311–324
Pantic M, Nijholt A, Pentland A, Huanag TS (2008) Human-Centred intelligent human? Computer interaction (HCI2): how far are we from attaining it? International Journal of Autonomous and Adaptive Communication Systems 1:168–187
Rahman MH, Afrin J (2013) Hand gesture recognition using multiclass support vector machine. Int J Comput Appl 74:39
Ramani K (2015) A gesture-free geometric approach for mid-air expression of design intent in 3D virtual pottery. Comput Aided Des 69:11–24
Rautaray SS, Agrawal A (2015) Vision based hand gesture recognition for human computer interaction: a survey. Artif Intell Rev 43:1–54
Rehm M, Bee N, André E (2008) Wave like an Egyptian: accelerometer based gesture recognition for culture specific interactions. in Proceedings of the 22nd British HCI Group Annual Conference on People and Computers: Culture, Creativity, Interaction-Volume 1, p 13–22
Seddati O, Dupont S, Mahmoudi S (2015) Deepsketch: deep convolutional neural networks for sketch recognition and similarity search. in Content-Based Multimedia Indexing (CBMI), 2015 13th International Workshop on, p 1–6
Su M-C, Hsio T-H, Hsieh Y-Z, Lin S-C, Chou C-H (2012) A neural-network-based sketch recognition system. in Intelligent Signal Processing and Communications Systems (ISPACS), 2012 International Symposium on, p 420–423
Tang X, Wang X (2004) "face sketch recognition," Circuits and Systems for Video Technology. IEEE Transactions 14:50–57
Tseng K-Y, Lin Y-L, Chen Y-H, Hsu WH (2012) Sketch-based image retrieval on mobile devices using compact hash bits. in Proceedings of the 20th ACM international conference on Multimedia, p 913–916
Wiethoff A, Schneider H, Rohs M, Butz A, Greenberg S (2012) Sketch-a-TUI: low cost prototyping of tangible interactions using cardboard and conductive ink. in Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction, p 309–312
Wilson AD, Bobick AF (2001) Hidden Markov models for modeling and recognizing gesture under variation. Int J Pattern Recognit Artif Intell 15:123–160
Wobbrock JO, Wilson AD, Li Y (2007) Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes. in Proceedings of the 20th annual ACM symposium on User interface software and technology, p 159–168
Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2009) "robust face recognition via sparse representation," Pattern Analysis and Machine Intelligence. IEEE Transactions 31:210–227
Yang K, Li Z, Ye J (2012) Freely-drawn sketches interpretation using SVMs-chain modeling. Engineering Applications of Artificial Intelligence 25:392–403
Zhang Z, Rao BD (2012) Recovery of block sparse signals using the framework of block sparse Bayesian learning. in Acoustics, Speech and Signal Processing (ICASSP), 2012 I.E. International Conference on, p 3345–3348
Zhang Z, Rao BD (2013) Extension of SBL algorithms for the recovery of block sparse signals with intra-block correlation. IEEE Trans Signal Process 61:2009–2015
Acknowledgements
This paper has been funded by iMQRES scholarship from Macquarie University, Sydney, NSW, Australia.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Jahani-Fariman, H., Kavakli, M. & Boyali, A. MATRACK: block sparse Bayesian learning for a sketch recognition approach. Multimed Tools Appl 77, 1997–2012 (2018). https://doi.org/10.1007/s11042-017-4368-8
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-017-4368-8