Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

A new in-air handwritten persian characters recognition method based on inertial sensor position estimation and convolutional neural network

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

With advances in microelectromechanical systems (MEMS), researchers have now become interested in the systems operating based on inertial signals. In fact, inertial signals have proven useful in different areas due to advances in their manufacturing technology, availability, and inexpensiveness as well as the development of powerful processing methods such as deep learning techniques. Handwritten character recognition (HCR) is among such areas. This paper aimed to design, implement, and evaluate a novel system for the recognition of handwritten Farsi characters extracted from an inertial pen. For this purpose, a wireless inertial pen was designed. Its motion trajectory was then determined by combining the signals of its angular velocity and acceleration and using the concepts of navigation systems such as quaternion in order to estimate the position signals of characters. A convolutional neural network (CNN) was also employed to facilitate the extraction of high-level features and the classification of characters. The position signal was also extracted as an image used for model learning to enhance the classifier efficiency. The experimental results indicated the CNN-6 architecture outperformed the other CNN-n architectures in terms of character classification accuracy. According to the evaluation of the proposed method through test data, character recognition accuracies of Farsi letters and numbers were reported 91.06% and 94.52%, respectively. In comparison with the previous systems, the proposed method managed to improve the recognition of handwritten Farsi characters.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Akbarisanto R, Danar W, Purwarianti A (2016) Analyzing bandung public mood using Twitter data. In: 2016 4th international conference on information and communication technology (ICoICT), 25–27 May 2016, pp 1–6

  • Breuel TM, Ul-Hasan A, Al-Azawi MA, Shafait F (2013) High-performance OCR for printed English and Fraktur using LSTM networks. In: 2013 12th international conference on document analysis and recognition, 25–28 Aug 2013, pp 683–687

  • Choi S, Lee S (2012) 3D stroke reconstruction and cursive script recognition with magnetometer-aided inertial measurement unit. IEEE Trans Consum Electron 58(2):661–669. https://doi.org/10.1109/TCE.2012.6227474

    Article  Google Scholar 

  • Christian M, Uyanik C, Erdemir E, Kaplanoglu E, Bhattacharya S, Bailey R et al (2019) Application of deep learning to IMU sensor motion. In: 2019 SoutheastCon. 11–14 April 2019, pp 1–6

  • Euston M, Coote P, Mahony R, Kim J, Hamel T (2008) A complementary filter for attitude estimation of a fixed-wing UAV. In: 2008 IEEE/RSJ international conference on intelligent robots and systems, 22–26 Sept. 2008, pp 340–345

  • Gałka J, Mąsior M, Zaborski M, Barczewska K (2016) Inertial motion sensing glove for sign language gesture acquisition and recognition. IEEE Sens J 16(16):6310–6316. https://doi.org/10.1109/JSEN.2016.2583542

    Article  Google Scholar 

  • Hidalgo-Carrió J, Arnold S, Poulakis P (2016) On the design of attitude-heading reference systems using the allan variance. IEEE Trans Ultrason Ferroelectr Freq Control 63(4):656–665. https://doi.org/10.1109/TUFFC.2016.2519268

    Article  Google Scholar 

  • Hsu Y-L, Yang S-C, Chang H-C, Lai H-C (2018) Human daily and sport activity recognition using a wearable inertial sensor network. IEEE Access 6:31715–31728. https://doi.org/10.1109/access.2018.2839766

    Article  Google Scholar 

  • Jing L, Dai Z, Zhou Y (2017) wearable handwriting recognition with an inertial sensor on a finger nail. In: 2017 14th IAPR international conference on document analysis and recognition (ICDAR), 9–15 Nov. 2017, pp 1330–1337

  • Kamilaris A, Prenafeta-Boldú FX (2018) Deep learning in agriculture: a survey. Comput Electron Agric 147:70–90. https://doi.org/10.1016/j.compag.2018.02.016

    Article  Google Scholar 

  • Khomami SA, Shamekhi S (2021) Persian sign language recognition using IMU and surface EMG sensors. Measurement 168:108471. https://doi.org/10.1016/j.measurement.2020.108471

    Article  Google Scholar 

  • Kim M, Cho J, Lee S, Jung Y (2019) IMU sensor-based hand gesture recognition for human-machine interfaces. Sensors (basel). https://doi.org/10.3390/s19183827

    Article  Google Scholar 

  • Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: Paper presented at the proceedings of the 25th international conference on neural information processing systems, Volume 1, Lake Tahoe, Nevada

  • Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324. https://doi.org/10.1109/5.726791

    Article  Google Scholar 

  • LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444. https://doi.org/10.1038/nature14539

    Article  Google Scholar 

  • Liu ZT, Wong DPY, Chou PH (2020) An Imu-based wearable ring for on-surface handwriting recognition. In: 2020 international symposium on VLSI design, automation and test (VLSI-DAT), 10–13 Aug. 2020, pp 1–4

  • Lu C, Uchiyama H, Thomas D, Shimada A, Taniguchi R-I (2019) Indoor positioning system based on chest-mounted IMU. Sensors. https://doi.org/10.3390/s19020420

    Article  Google Scholar 

  • Mueller PA, Oppenheimer DM (2014) The pen is mightier than the keyboard: advantages of longhand over laptop note taking. Psychol Sci 25(6):1159–1168. https://doi.org/10.1177/0956797614524581

    Article  Google Scholar 

  • Pan T-Y, Kuo C-H, Liu H-T, Hu M-C (2018) Handwriting trajectory reconstruction using low-cost IMU. IEEE Trans Emerg Top Comput Intell. https://doi.org/10.1109/tetci.2018.2803777

    Article  Google Scholar 

  • Rangasamy K, As’ari MA, Rahmad NA, Ghazali NF (2020) Hockey activity recognition using pre-trained deep learning model. ICT Express 6(3):170–174. https://doi.org/10.1016/j.icte.2020.04.013

    Article  Google Scholar 

  • Schrapel M, Stadler M-L, Rohs M (2018) Pentelligence: combining pen tip motion and writing sounds for handwritten digit recognition. In: Proceedings of the 2018 CHI conference on human factors in computing systems, p Paper 131. Association for Computing Machinery. https://doi.org/10.1145/3173574.3173705

  • Sepahvand M, Abdali-Mohammadi F (2021a) A novel multi-lead ECG personal recognition based on signals functional and structural dependencies using time-frequency representation and evolutionary morphological CNN. Biomed Signal Process Control 68:102766. https://doi.org/10.1016/j.bspc.2021.102766

    Article  Google Scholar 

  • Sepahvand M, Abdali-Mohammadi F (2021b) A novel representation in genetic programming for ensemble classification of human motions based on inertial signals. Expert Syst Appl. https://doi.org/10.1016/j.eswa.2021.115624

    Article  Google Scholar 

  • Sepahvand M, Abdali-Mohammadi F, Mardukhi F (2017) Evolutionary metric-learning-based recognition algorithm for online isolated Persian/Arabic characters, reconstructed using inertial pen signals. IEEE Trans Cybern 47(9):2872–2884. https://doi.org/10.1109/TCYB.2016.2633318

    Article  Google Scholar 

  • Serrano JI, Lambrecht S, del Castillo MD, Romero JP, Benito-León J, Rocon E (2017) Identification of activities of daily living in tremorous patients using inertial sensors. Expert Syst Appl 83:40–48. https://doi.org/10.1016/j.eswa.2017.04.032

    Article  Google Scholar 

  • Shengli Z, Zhuxin D, Li WJ, Chung Ping K (2008) Hand-written character recognition using MEMS motion sensing technology. In: Paper presented at the 2008 IEEE/ASME international conference on advanced intelligent mechatronics

  • Tick DF, Satici AC, Shen J, Gans N (2012) Tracking control of mobile robots localized via chained fusion of discrete and continuous epipolar geometry, IMU and odometry, pp 2168–2275 (Electronic)

  • Wang J-S, Chuang F-C (2012) An accelerometer-based digital pen with a trajectory recognition algorithm for handwritten digit and gesture recognition. IEEE Trans Ind Electron 59(7):2998–3007. https://doi.org/10.1109/tie.2011.2167895

    Article  Google Scholar 

  • Weng Y, Xia C (2020) A new deep learning-based handwritten character recognition system on mobile computing devices. Mobile Netw Appl 25(2):402–411. https://doi.org/10.1007/s11036-019-01243-5

    Article  Google Scholar 

  • Yu-Liang H, Cheng-Ling C, Yi-Ju T, Jeen-Shing W (2015) An inertial pen with dynamic time warping recognizer for handwriting and gesture recognition. IEEE Sens J 15(1):154–163. https://doi.org/10.1109/jsen.2014.2339843

    Article  Google Scholar 

  • Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. Springer International Publishing, Cham, pp 818–833

    Google Scholar 

  • Zhang X, Yin F, Zhang Y, Liu C, Bengio Y (2018) Drawing and recognizing chinese characters with recurrent neural network. IEEE Trans Pattern Anal Mach Intell 40(4):849–862. https://doi.org/10.1109/TPAMI.2017.2695539

    Article  Google Scholar 

  • Zhang J, Bi H, Chen Y, Wang M, Han L, Cai L (2020) SmartHandwriting: handwritten chinese character recognition with smartwatch. IEEE Internet Things J 7(2):960–970. https://doi.org/10.1109/JIOT.2019.2947448

    Article  Google Scholar 

  • Zhao Y (2017) Applying time-differenced carrier phase in nondifferential GPS/IMU tightly coupled navigation systems to improve the positioning performance. IEEE Trans Veh Technol 66(2):992–1003. https://doi.org/10.1109/TVT.2016.2558206

    Article  Google Scholar 

  • Žumer J, Reynaerts D, Boltežar M (2012) An advanced nonlinear model of a low-g MEMS accelerometer for a computer pen. Measurement 45(3):459–468. https://doi.org/10.1016/j.measurement.2011.10.027

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering and Information Technology, Razi University, Kermanshah, Iran

    Farzaneh Meshkat & Fardin Abdali-Mohammadi

Authors
  1. Farzaneh Meshkat
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fardin Abdali-Mohammadi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fardin Abdali-Mohammadi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Meshkat, F., Abdali-Mohammadi, F. A new in-air handwritten persian characters recognition method based on inertial sensor position estimation and convolutional neural network. J Ambient Intell Human Comput 14, 13097–13112 (2023). https://doi.org/10.1007/s12652-022-03770-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-022-03770-8

Keywords

Search

Navigation