Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Optimized leaky ReLU for handwritten Arabic character recognition using convolution neural networks

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Object classification, such as handwritten Arabic character recognition, is a computer vision application. Deep learning techniques such as convolutional neural networks (CNNs) are employed in character recognition to overcome the processing complexity with traditional methods. Usually, a CNN is followed by an activation function such as a rectified linear unit (ReLU) or leaky ReLU to filter the extracted features. Most handwritten character recognition endures an imbalanced number of positive and negative vectors. This issue decreases CNN performance when adopting ReLU and leaky ReLU for the next deep layers in the architecture. Hence, this study proposed an optimized leaky ReLU to retain more negative vectors using a CNN architecture with a batch normalization layer to address this weakness. To evaluate the proposed method, four datasets are used: Arabic Handwritten Characters Dataset (AHCD), self-collected, Modified National Institute of Standards and Technology (MNIST), and AlexU Isolated Alphabet (AIA9K). The proposed method shows significant performance in terms of accuracy, precision, and recall measures compared to the state-of-art methods. The results showed outstanding improvement over the known leaky ReLU as follows: 99% for AHCD, 95.4% for self-collected data, 90% for HIJJA dataset and 99% for Digit MNIST. The proposed CNN architecture with the proposed optimized leaky ReLU showed a stable accuracy performance and error rates between the training, validation, and testing phases. This indicates that most samples are trained and classified correctly.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Agarap AF (2018) Deep learning using rectified linear units (ReLU). arXiv:1803.08375

  2. Ali AAA, Suresha M (2019) Arabic handwritten character recognition using machine learning approaches. In: 2019 fifth international conference on image information processing (ICIIP). IEEE.

  3. Alsheikh I, Mohd M, Warlina L (2020) A review of Arabic text recognition dataset. Asia-Pacific J Inf Technol Multimedia 09:69–81

    Article  Google Scholar 

  4. Altwaijry N, Al-Turaiki I (2020) Arabic handwriting recognition system using convolutional neural network. Neural Computing Appl 1–13.

  5. Arora R, et al (2016) Understanding deep neural networks with rectified linear units. arXiv:1611.01491

  6. Azmi M (2013) A novel feature from combinations of triangle geometry for digital Jawi paleography. Universiti Kebangsaan Malaysia

  7. Bengio Y (2009) Learning deep architectures for AI. Foundations and trends® in Machine Learning 2(1): 1–127.

  8. Boufenar C, Kerboua A, Batouche M (2018) Investigation on deep learning for off-line handwritten Arabic character recognition. Cogn Syst Res 50:180–195

    Article  Google Scholar 

  9. Clevert D-A, Unterthiner T, Hochreiter S (2015) Fast and accurate deep network learning by exponential linear units (elus). arXiv:1511.07289

  10. Chen Z (2017) Deep-learning approaches to object recognition from 3D Data. Case Western Reserve University http://www.ohiolink.edu/. p. 92.

  11. Dahou A et al (2019) Arabic sentiment classification using convolutional neural network and differential evolution algorithm. Comput Intell Neurosci 2019:16

    Article  Google Scholar 

  12. Deng J, et al (2009) Imagenet: a large-scale hierarchical image database. In IEEE conference on computer vision and pattern recognition. IEEE

  13. Donahue J, et al (2014) Decaf: a deep convolutional activation feature for generic visual recognition. in International conference on machine learning.

  14. Dubey AK, Jain V (2019) Comparative study of convolution neural network’s ReLU and leaky-ReLU activation functions. Singapore: Springer Singapore.

  15. El-Sawy A, Loey M, El-Bakry H (2017) Arabic handwritten characters recognition using convolutional neural network. WSEAS Transactions on Computer Research 5:11–19

    Google Scholar 

  16. Gao Z, Edirisinghe E, Chesnokov S (2019) Image super-resolution using CNN optimised by self-feature loss. In 2019 IEEE international conference on image processing (ICIP). IEEE.

  17. Gu J et al (2018) Recent advances in convolutional neural networks. Pattern Recogn 77:354–377

    Article  Google Scholar 

  18. Hahnloser RH et al (2000) Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit. Nature 405(6789):947–951

    Article  Google Scholar 

  19. Hasan AH, Omar K, Nasrudin MF (2018) Multi-classifier Jawi Handwritten sub-word recognition. Int J Adv Sci Eng Inf Technol 8(4–2):1528–1533

    Article  Google Scholar 

  20. He K, et al (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision.

  21. He K, et al (2016) Identity mappings in deep residual networks. In European conference on computer vision. Springer.

  22. Ibrahim MN, et al (2013) A framework of an online self-based learning for teaching Arabic as second language (TASL). In: 2013 fifth international conference on computational intelligence, modelling and simulation. IEEE.

  23. Jebril NA, Al-Zoubi HR, Al-Haija QA (2018) Recognition of handwritten Arabic characters using histograms of oriented gradient (HOG). Pattern Recognit Image Anal 28(2):321–345

    Article  Google Scholar 

  24. Khan A, et al (2019) A survey of the recent architectures of deep convolutional neural networks. arXiv:1901.06032

  25. LeCun Y et al (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1(4):541–551

    Article  Google Scholar 

  26. LeCun Y, Bengio Y (1995) Convolutional networks for images, speech, and time series. Handbook Brain Theory Neural Netw 3361(10):1995

    Google Scholar 

  27. Lee J, et al (2019) ProbAct: a probabilistic activation function for deep neural networks. arXiv:1905.10761

  28. Maas AL, Hannun AY, Ng AY (2013) Rectifier nonlinearities improve neural network acoustic models. In Proc. ICML.

  29. Memon J, Sami M, Khan RA (2020) Handwritten optical character recognition (OCR): a comprehensive systematic literature review (SLR). arXiv:2001.00139

  30. Nair V, Hinton GE (2010) Rectified linear units improve restricted Boltzmann machines. In ICML

  31. Najadat HM, Alshboul AA, Alabed AF (2019) Arabic handwritten characters recognition using convolutional neural network. In: 2019 10th international conference on information and communication systems (ICICS).

  32. Nwankpa C, et al (2018) Activation functions: comparison of trends in practice and research for deep learning. arXiv:1811.03378

  33. Pathak AR, Pandey M, Rautaray S (2018) Application of deep learning for object detection. Procedia Comput Sci 132:1706–1717

    Article  Google Scholar 

  34. Ramdan J et al (2013) Arabic handwriting data base for text recognition. Procedia Technol 11:580–584

    Article  Google Scholar 

  35. Russakovsky O, et al (2012) Object-centric spatial pooling for image classification. In European conference on computer vision. Springer.

  36. Russakovsky O et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vision 115(3):211–252

    Article  MathSciNet  Google Scholar 

  37. Sahu DK, Jawahar C (2015) Unsupervised feature learning for optical character recognition. In: 2015 13th international conference on document analysis and recognition (ICDAR). IEEE.

  38. Sarkhel R et al (2016) A multi-objective approach towards cost effective isolated handwritten Bangla character and digit recognition. Pattern Recogn 58:172–189

    Article  Google Scholar 

  39. Shrikumar A, et al (2016) Not just a black box: learning important features through propagating activation differences. arXiv:1605.01713

  40. Sulaiman A, Omar K, Nasrudin MF (2021) Two streams deep neural network for handwriting word recognition. Multimedia Tools Appl 80(4):5473–5494

    Article  Google Scholar 

  41. Tian Z, et al (2016) Detecting text in natural image with connectionist text proposal network. In European conference on computer vision. Springer.

  42. Torki M, et al (2014) Window-based descriptors for Arabic handwritten alphabet recognition: a comparative study on a novel dataset. arXiv:1411.3519

  43. Visin F, et al (2015) Renet: a recurrent neural network based alternative to convolutional networks. arXiv:1505.00393

  44. Xu B, et al (2015) Empirical evaluation of rectified activations in convolutional network. abs/1505.00853

  45. Wang B, et al (2018) Deep neural nets with interpolating function as output activation. In: Advances in neural information processing systems.

  46. Younis KS (2017) Arabic handwritten character recognition based on deep convolutional neural networks. Jordan J Computers Inf Technol (JJCIT) 3(3):186–200

    Google Scholar 

Download references

Acknowledgements

We would like to convey our gratitude to research team members at the Digital Forensic Lab and Medical and Health Informatics Lab at the Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia, who contributed to this project. Apart from that, we thank the Ministry of Higher Education, Malaysia, which supported this project under the Fundamental Research Grant Scheme (FRGS) FRGS/1/2019/ICT02/UKM/02/9 entitled “Convolution neural network enhancement based on adaptive convexity and regularization functions for fake video analytics.”

Author information

Authors and Affiliations

  1. Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia, 43600, Bangi, Selangor, Malaysia

    Bahera H. Nayef, Siti Norul Huda Sheikh Abdullah, Rossilawati Sulaiman & Zaid Abdi Alkareem Alyasseri

Authors
  1. Bahera H. Nayef
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Siti Norul Huda Sheikh Abdullah
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rossilawati Sulaiman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zaid Abdi Alkareem Alyasseri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bahera H. Nayef.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nayef, B.H., Abdullah, S.N.H.S., Sulaiman, R. et al. Optimized leaky ReLU for handwritten Arabic character recognition using convolution neural networks. Multimed Tools Appl 81, 2065–2094 (2022). https://doi.org/10.1007/s11042-021-11593-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11593-6

Keywords

Search

Navigation