research-article

A New Design Based-SVM of the CNN Classifier Architecture with Dropout for Offline Arabic Handwritten Recognition

Authors:

Mohamed Elleuch,

Monji KherallahAuthors Info & Claims

Procedia Computer Science, Volume 80, Issue C

Pages 1712 - 1723

https://doi.org/10.1016/j.procs.2016.05.512

Published: 01 June 2016 Publication History

Abstract

In this paper we explore a new model focused on integrating two classifiers; Convolutional Neural Network (CNN) and Support Vector Machine (SVM) for offline Arabic handwriting recognition (OAHR) on which the dropout technique was applied. The suggested system altered the trainable classifier of the CNN by the SVM classifier. A convolutional network is beneficial for extracting features information and SVM functions as a recognizer. It was found that this model both automatically extracts features from the raw images and performs classification. Additionally, we protected our model against over-fitting due to the powerful performance of dropout. In this work, the recognition on the handwritten Arabic characters was evaluated; the training and test sets were taken from the HACDB and IFN/ENIT databases. Simulation results proved that the new design based-SVM of the CNN classifier architecture with dropout performs significantly more efficiently than CNN based-SVM model without dropout and the standard CNN classifier. The performance of our model is compared with character recognition accuracies gained from state-of-the-art Arabic Optical Character Recognition, producing favorable results.

References

[1]

H. Lee, R. Grosse, R. Ranganath, A.Y. Ng, Unsupervised learning of hierarchical representations with convolutional deep belief networks, Communications of the ACM, 54 (2011) 95-103.

Digital Library

[2]

G.-B. Huang, H. Zhou, X. Ding, R. Zhang, Extreme Learning Machine for Regression and Multiclass Classification, IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics, 42 (2012) 513-529.

Digital Library

[3]

H. Lee, P.T. Pham, Y. Largman, A.Y. Ng, Unsupervised feature learning for audio classification using convolutional deep belief networks, Advances in Neural Information Processing Systems (NIPS) (2009) 1096-1104.

Digital Library

[4]

Y. Ren, Y. Wu, Convolutional Deep Belief Networks for Feature Extraction of EEG Signal, International Joint Conference on Neural Networks (IJCNN) (2014) 2850-2853.

[5]

D.C. Ciresan, U. Meier, J. Schmidhuber, Transfer Learning for Latin and Chinese Characters with Deep Neural Networks, In Proceedings of International Joint Conference on Neural Networks, 2012 .

[6]

D.C. Ciresan, J. Schmidhuber, Multi-Column Deep Neural Networks for Offline Handwritten Chinese Character Classification, In Proceedings of CoRR. 2013 .

[7]

X.-X. Niu, Ching Y. Suen, Novel hybrid CNNSVM classifier for recognizing handwritten digits, Pattern Recognition, vol. 45, pp. 1318-1325, 2012 .

Digital Library

[8]

M. Elleuch, N. Tagougui and M. Kherallah, A Novel Architecture of CNN based on SVM Classifier for Recognizing Arabic Handwritten, International Journal of Intelligent Systems Technologies and Applications, vol. 15, 2016, in press.

Digital Library

[9]

G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. R. Salakhutdinov, Improving neural networks by preventing co-adaptation of feature detectors, arXiv preprint arXiv:1207.0580, 2012 .

[10]

Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition, Proceedings of the IEEE, vol. 86(11), pp. 2278-2324, 1998 .

[11]

D. C. Ciresan, U. Meier, J. Masci, L.M. Gambardella, and J. Schmidhuber, High performance neural networks for visual object classi_cation, Technical Report IDSIA-01-11, Dalle Molle Institute for Artificial Intelligence, 2011 .

[12]

Y. LeCun, B. Boser, J.S. Denker, D. Henderson, R.E. Howard, W. Hubbard, and L. D. Jackel, Backpropagation applied to handwritten zip code recognition, Neural Computation, vol. 1, pp. 541-551, 1989 .

Digital Library

[13]

V. Vapnik, Statistical Learn Theory, John Wiley, New York, 1998 .

[14]

H. Byun, S.-W. Lee, A survey on pattern recognition applications of Support Vector Machines, International Journal of Pattern Recognition and Artificial Intelligence, vol. 17, pp. 459-486, 2003 .

[15]

D. Gorgevik, D. Cakmakov, V. Radevski, Handwritten digit recognition by combining support vector machines using rule-based reasoning, Proc. 23rd Int. Conf. Information Technology Interfaces (ITI), pp. 139-144, 2001 .

[16]

G. Guo, S.Z. Li, K. Chan, Face Recognition by Support Vector Machines, Proc. 4th IEEE Intl. Conf. on Face and Gesture Recognition, pp. 196-201, 2000 .

Digital Library

[17]

MNIST: http://yann.lecun.com/exdb/mnist/.

[18]

P. J. Grother, Nist special database 19 - handprinted forms and characters database, National Institute of Standards and Thechnology (NIST), Tech. Rep., 1995 .

[19]

D. C. Ciresan, U. Meier, L.M. Gambardella, J. Schmidhuber, Convolutional Neural Network Committees For Handwritten Character Classification,11th International Conference on Document Analysis and Recognition (ICDAR), 2011 .

Digital Library

[20]

B. Thodore, N. Hermann, K. Christopher, Tandem HMM with convolutional neural network for handwritten word recognition, In: 38th International Conference on Acoustics Speech and Signal Processing (ICASSP), pp. 2390-2394, 2013 .

[21]

U. V. Marti and H. Bunke, The IAM-database: an English sentence database for offline handwriting recognition, International Journal on Document Analysis and Recognition, vol. 5, no. 1, pp. 39-46, 2002 .

[22]

E. Augustin, M. Carr, E. Grosicki, J.-M. Brodin, E. Geoffrois, F. Preteux, RIMES evaluation campaign for handwritten mail processing, In Workshop on Frontiers in Handwriting Recognition, 2006 (1) .

[23]

H. Byun, S.-W. Lee, Applications of Support Vector Machines for Pattern Recognition: A Survey,In Proceedings of the First International Workshop, SVM 2002, pp. 213-236, 2002 .

Digital Library

[24]

J. Chen, H. Cao, R. Prasad, A. Bhardwaj, P. Natarajan, Gabor features for offline arabic handwriting recognition, In Proceedings of the 9th IAPR International Workshop on Document Analysis Systems (DAS) (2010) 53-58.

Digital Library

[25]

M. Elleuch, M. Kherallah, An Improved Arabic Handwritten Recognition System using Deep Support Vector Machines, International Journal of Multimedia Data Engineering and Management, 7 (2016) 1-14.

Digital Library

[26]

N. Srivastava, G.E. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov, Dropout: A simple way to prevent neural networks from overfitting, The Journal of Machine Learning Research, 15 (2014) 1929-1958.

Digital Library

[27]

R. Maalej, N. Tagougui and M. Kherallah, Online Arabic Handwriting Recognition with Dropout Applied in Deep Recurrent Neural Networks, In 12th IAPR International Workshop on Document Analysis Systems (DAS), 2016, in press.

[28]

V. Pham, T. Bluche, C. Kermorvant, and J. Louradour, Dropout improves recurrent neural networks for handwriting recognition, ar**Xiv preprint ar**Xiv:1312.4569, 2013 .

[29]

Y. LeCun, F.J. Huang, L. Bottou, Learning methods for generic object recognition with invariance to pose and lighting, Proc. Computer Vision and Pattern Recognition Conference (CVPR), IEEE Press, 2004 .

Digital Library

[30]

M. Ranzato, F. Huang, Y. Boureau, Y. LeCun, Unsupervised learning of invariant feature hierarchies with applications to object recognition, Proc. Computer Vision and Pattern Recognition Conference (CVPR), IEEE Press, 2007 .

[31]

C. Cortes, V. Vapnik, Support vector networks, Machine Learning, vol. 20, pp. 273-297, 1995 .

[32]

C. Burges, A tutorial on support vector machines for pattern recognition, Data Mining Knowledge Discovery, vol. 2(2), pp. 121-167, 1998 .

Digital Library

[33]

A. Lawgali, M. Angelova, A. Bouridane, HACDB: Handwritten Arabic characters database for automatic character recognition, EUropean Workshop on Visual Information Processing (EUVIP), pp. 255-259, 2013 .

[34]

M. Pechwitz, S.S. Maddouri, V. Mrgner, N. Ellouze and H. Amiri, IFN/ENIT database of handwritten Arabic words, In: Colloque International Francophone sur lEcrit et le Document (CIFED), pp. 127-136, 2002 .

[35]

P. Simard, D. Steinkraus, J.C. Platt, Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis, International Conference on Document Analysis and Recognition (ICDAR) (2003) 958-962.

Digital Library

[36]

C.C. Chang, C.J. Lin, LIBSVM: A Library for Support Vector Machines, Software Available at http://www.csie.ntu.edu.tw/cjlin/libsvm, 2001 .

[37]

M. Elleuch, N. Tagougui,and M. Kherallah, Towards Unsupervised Learning for Arabic Handwritten Recognition Using Deep Architectures,Neural Information Processing - 22nd International Conference, ICONIP 2015, Istanbul, Turkey, part. (1), pp. 363-372, 2015 .

[38]

A. Lawgali, A. Bouridane, M. Angelova, Z. Ghassemlooy, Handwritten Arabic character recognition: Which feature extraction method?., International Journal of Advanced Science and Technology, 34 (2011) 1-8.

[39]

S.A. Azeem, H. Ahmed, Effective technique for the recognition of offline Arabic handwritten words using hidden Markov models, International Journal on Document Analysis and Recognition (IJDAR), 16 (2013) 399-412.

Digital Library

[40]

P. J. Grother, Nist special database 19 - handprinted forms and characters database, National Institute of Standards and Thechnology (NIST), Tech. Rep., 1995 .

[41]

Applied Media Analysis, Arabic-Handwritten-1.0, http://appliedmediaanalysis.com/Datasets.htm. 2007 .

Cited By

Hakim BAhror B(2025)Deep Learning for Accurate Recognition of Arabic Handwritten Words in Historical Documents.Procedia Computer Science10.1016/j.procs.2024.10.178244:C(57-65)Online publication date: 7-Jan-2025
https://dl.acm.org/doi/10.1016/j.procs.2024.10.178
Sahoo PPattnaik B(2025)Automated Arrhythmia Detection Using War Strategy Optimization Enabled with Archimedes Optimization Algorithm and Rule-Based ClassifiersSN Computer Science10.1007/s42979-024-03604-86:1Online publication date: 7-Jan-2025
https://dl.acm.org/doi/10.1007/s42979-024-03604-8
Farady ILin CChang M(2024)PreAugNet: improve data augmentation for industrial defect classification with small-scale training dataJournal of Intelligent Manufacturing10.1007/s10845-023-02109-035:3(1233-1246)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1007/s10845-023-02109-0
Show More Cited By

A New Design Based-SVM of the CNN Classifier Architecture with Dropout for Offline Arabic Handwritten Recognition
1. Applied computing
  1. Document management and text processing
    1. Document capture
2. Computing methodologies
  1. Machine learning

Recommendations

A novel architecture of CNN based on SVM classifier for recognising Arabic handwritten script

Convolutional neural network CNN, as a deep learning algorithms being developed for years, have been successfully applied in various domains of computer vision and pattern recognition. Recently, support vector machine SVM classifier has received more ...
Offline arabic handwritten text recognition: A Survey

Research in offline Arabic handwriting recognition has increased considerably in the past few years. This is evident from the numerous research results published recently in major journals and conferences in the area of handwriting recognition. Features ...
Offline Arabic handwritten character recognition: from conventional machine learning system to deep learning approaches

Researchers have made great strides in the area of Arabic handwritten character recognition in the last decades especially with the fast development of deep learning algorithms. The characteristics of Arabic manuscript text pose several problems for a ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Procedia Computer Science

Procedia Computer Science Volume 80, Issue C

June 2016

2452 pages

ISSN:1877-0509

EISSN:1877-0509

Issue’s Table of Contents

Copyright © The Authors.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 01 June 2016

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hakim BAhror B(2025)Deep Learning for Accurate Recognition of Arabic Handwritten Words in Historical Documents.Procedia Computer Science10.1016/j.procs.2024.10.178244:C(57-65)Online publication date: 7-Jan-2025
https://dl.acm.org/doi/10.1016/j.procs.2024.10.178
Sahoo PPattnaik B(2025)Automated Arrhythmia Detection Using War Strategy Optimization Enabled with Archimedes Optimization Algorithm and Rule-Based ClassifiersSN Computer Science10.1007/s42979-024-03604-86:1Online publication date: 7-Jan-2025
https://dl.acm.org/doi/10.1007/s42979-024-03604-8
Farady ILin CChang M(2024)PreAugNet: improve data augmentation for industrial defect classification with small-scale training dataJournal of Intelligent Manufacturing10.1007/s10845-023-02109-035:3(1233-1246)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1007/s10845-023-02109-0
Humayun MSiddiqi RUddin MKandhro IAbdelhaq MAlsaqour R(2024)A novel methodology for offline English handwritten character recognition using ELBP-based sequential (CNN)Neural Computing and Applications10.1007/s00521-024-10206-136:30(19139-19156)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1007/s00521-024-10206-1
Tsegaye MShashi M(2024)A hybrid convolutional neural network and support vector machine classifier for Amharic character recognitionNeural Computing and Applications10.1007/s00521-024-09657-336:27(16839-16856)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s00521-024-09657-3
Khan SNazir SKhan H(2023)Analysis of Cursive Text Recognition Systems: A Systematic Literature ReviewACM Transactions on Asian and Low-Resource Language Information Processing10.1145/359260022:7(1-30)Online publication date: 20-Jul-2023
https://dl.acm.org/doi/10.1145/3592600
Utku A(2023)Deep learning based hybrid prediction model for predicting the spread of COVID-19 in the world's most populous countriesExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.120769231:COnline publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1016/j.eswa.2023.120769
Kaur HBansal SKumar MMittal AKumar K(2023)Worddeepnet: handwritten gurumukhi word recognition using convolutional neural networkMultimedia Tools and Applications10.1007/s11042-023-15527-282:30(46763-46788)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1007/s11042-023-15527-2
Naseri HMehrdad V(2023)Novel CNN with investigation on accuracy by modifying stride, padding, kernel size and filter numbersMultimedia Tools and Applications10.1007/s11042-023-14603-x82:15(23673-23691)Online publication date: 23-Feb-2023
https://dl.acm.org/doi/10.1007/s11042-023-14603-x
Gayathri RBabitha Lincy R(2022)Transfer learning based handwritten character recognition of tamil script using inception-V3 ModelJournal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology10.3233/JIFS-21237842:6(6091-6102)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.3233/JIFS-212378
Show More Cited By

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents