Towards an IMU-based Pen Online Handwriting Recognizer

1. The digipen devkit. https://stabilodigital.com/devkit-demoapp-introduction/. Accessed 30 Jan 2021

2. The digipen hardware. https://stabilodigital.com/sensors-2021/. Accessed 30 Jan 2021

3. Abadi, M., et al.: Tensorflow: a system for large-scale machine learning. In: 12th \(\{\)USENIX\(\}\) Symposium on Operating Systems Design and Implementation (\(\{\)OSDI\(\}\) 16), pp. 265–283 (2016)

   Google Scholar 

4. Abd Alshafy, H.A., Mustafa, M.E.: Hmm based approach for online Arabic handwriting recognition. In: 2014 14th International Conference on Intelligent Systems Design and Applications, pp. 211–215. IEEE (2014)

   Google Scholar 

5. Amma, C., Georgi, M., Schultz, T.: Airwriting: hands-free mobile text input by spotting and continuous recognition of 3d-space handwriting with inertial sensors. In: 2012 16th International Symposium on Wearable Computers, pp. 52–59. IEEE (2012)

   Google Scholar 

6. Bengio, Y., LeCun, Y., Nohl, C., Burges, C.: LEREC: a NN/HMM hybrid for on-line handwriting recognition. Neural Comput. 7(6), 1289–1303 (1995)

   Article  Google Scholar 

7. Bersch, S.D., Azzi, D., Khusainov, R., Achumba, I.E., Ries, J.: Sensor data acquisition and processing parameters for human activity classification. Sensors 14(3), 4239–4270 (2014)

   Article  Google Scholar 

8. Carbune, V., et al.: Fast multi-language LSTM-based online handwriting recognition. In: International Journal on Document Analysis and Recognition (IJDAR), pp. 1–14 (2020)

   Google Scholar 

9. Chollet, F., et al.: Keras (2015). https://keras.io

10. Feng, Y., Zhang, Y., Xu, X.: End-to-end speech recognition system based on improved CLDNN structure. In: 2019 IEEE 8th Joint International Information Technology and Artificial Intelligence Conference (ITAIC), pp. 538–542. IEEE (2019)

    Google Scholar 

11. Gan, J., Wang, W.: In-air handwritten English word recognition using attention recurrent translator. Neural Comput. Appl. 31(7), 3155–3172 (2019)

    Article  Google Scholar 

12. Gan, J., Wang, W., Lu, K.: A unified CNN-RNN approach for in-air handwritten English word recognition. In: 2018 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE (2018)

    Google Scholar 

13. Gerth, S., et al.: Is handwriting performance affected by the writing surface? comparing tablet vs. paper. Frontiers in psychology 7 (2016)

    Google Scholar 

14. Graves, A., Fernández, S., Gomez, F., Schmidhuber, J.: Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 369–376 (2006)

    Google Scholar 

15. Graves, A., Fernández, S., Liwicki, M., Bunke, H., Schmidhuber, J.: Unconstrained online handwriting recognition with recurrent neural networks. In: Advances in Neural Information Processing Systems 20, NIPS 2008 (2008)

    Google Scholar 

16. Graves, A., Liwicki, M., Fernández, S., Bertolami, R., Bunke, H., Schmidhuber, J.: A novel connectionist system for unconstrained handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31(5), 855–868 (2008)

    Article  Google Scholar 

17. Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Netw. 18(5–6), 602–610 (2005)

    Article  Google Scholar 

18. Guyon, I., Schomaker, L., Plamondon, R., Liberman, M., Janet, S.: Unipen project of on-line data exchange and recognizer benchmarks. In: Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3-Conference C: Signal Processing (Cat. No. 94CH3440-5), vol. 2, pp. 29–33. IEEE (1994)

    Google Scholar 

19. Halder, A., Ramakrishnan, A.: Time delay neural networks for online handwriting recognition, June 2007. https://doi.org/10.13140/RG.2.2.25975.52641

20. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp. 448–456. PMLR (2015)

    Google Scholar 

21. Jaeger, S., Manke, S., Reichert, J., Waibel, A.: Online handwriting recognition: the NPEN++ recognizer. Int. J. Doc. Anal. Recogn. 3(3), 169–180 (2001)

    Article  Google Scholar 

22. Jäger, S., Liu, C.L., Nakagawa, M.: The state of the art in Japanese online handwriting recognition compared to techniques in western handwriting recognition. Doc. Anal. Recogn. 6(2), 75–88 (2003)

    Article  Google Scholar 

23. Jeen-Shing, W., Yu-Liang, H., Cheng-Ling, C.: Online handwriting recognition using an accelerometer-based pen device. In: 2nd International Conference on Advances in Computer Science and Engineering (CSE 2013), pp. 231–234. Atlantis Press (2013)

    Google Scholar 

24. Keysers, D., Deselaers, T., Rowley, H.A., Wang, L.L., Carbune, V.: Multi-language online handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1180–1194 (2016)

    Article  Google Scholar 

25. Kim, J.H., Sin, B.-K.: Online handwriting recognition. In: Doermann, D., Tombre, K. (eds.) Handbook of Document Image Processing and Recognition, pp. 887–915. Springer, London (2014). https://doi.org/10.1007/978-0-85729-859-1_29

    Chapter  Google Scholar 

26. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

27. Koellner, C., Kurz, M., Sonnleitner, E.: What did you mean? an evaluation of online character recognition approaches. In: 2019 International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob), pp. 1–6. IEEE (2019)

    Google Scholar 

28. Liu, C.L., Jaeger, S., Nakagawa, M.: ’online recognition of Chinese characters: the state-of-the-art. IEEE Trans. Pattern Anal. Mach. Intell. 26(2), 198–213 (2004)

    Article  Google Scholar 

29. Liu, Z.T., Wong, D.P., Chou, P.H.: An IMU-based wearable ring for on-surface handwriting recognition. In: 2020 International Symposium on VLSI Design, Automation and Test (VLSI-DAT), pp. 1–4. IEEE (2020)

    Google Scholar 

30. Liwicki, M., Bunke, H.: IAM-ONDB-an on-line English sentence database acquired from handwritten text on a whiteboard. In: Eighth International Conference on Document Analysis and Recognition (ICDAR 2005), pp. 956–961. IEEE (2005)

    Google Scholar 

31. Liwicki, M., Bunke, H., Pittman, J.A., Knerr, S.: Combining diverse systems for handwritten text line recognition. Mach. Vis. Appl. 22(1), 39–51 (2011)

    Article  Google Scholar 

32. Liwicki, M., Graves, A., Fernàndez, S., Bunke, H., Schmidhuber, J.: A novel approach to on-line handwriting recognition based on bidirectional long short-term memory networks. In: Proceedings of the 9th International Conference on Document Analysis and Recognition, ICDAR 2007 (2007)

    Google Scholar 

33. Mandal, S., Prasanna, S.M., Sundaram, S.: Exploration of CNN features for online handwriting recognition. In: 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 831–836. IEEE (2019)

    Google Scholar 

34. Musa, M.E.: Towards building standard datasets for Arabic recognition. Int. J. Eng. Adv. Res. Technol. (IJEART) 2(2), 16–19 (2016)

    Google Scholar 

35. Ott, F., Wehbi, M., Hamann, T., Barth, J., Eskofier, B., Mutschler, C.: The onhw dataset: Online handwriting recognition from imu-enhanced ballpoint pens with machine learning. In: Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 4, no. 3, pp. 1–20 (2020)

    Google Scholar 

36. Palacios, R., Gupta, A., Wang, P.S.: Handwritten bank check recognition of courtesy amounts. Int. J. Image Graphics 4(02), 203–222 (2004)

    Article  Google Scholar 

37. Plamondon, R., Srihari, S.N.: Online and off-line handwriting recognition: a comprehensive survey. IEEE Trans. Pattern Anal. Mach. Intell. 22(1), 63–84 (2000)

    Article  Google Scholar 

38. Priya, A., Mishra, S., Raj, S., Mandal, S., Datta, S.: Online and offline character recognition: a survey. In: 2016 International Conference on Communication and Signal Processing (ICCSP), pp. 0967–0970. IEEE (2016)

    Google Scholar 

39. Sainath, T.N., Vinyals, O., Senior, A., Sak, H.: Convolutional, long short-term memory, fully connected deep neural networks. In: 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4580–4584. IEEE (2015)

    Google Scholar 

40. Schrapel, M., Stadler, M.L., Rohs, M.: Pentelligence: Combining pen tip motion and writing sounds for handwritten digit recognition. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1–11 (2018)

    Google Scholar 

41. Shaikh Jahidabegum, K.: Character recognition system for text entry using inertial pen. Int. J. Sci. Eng. Technol. Res. (IJSETR) 4 (2015)

    Google Scholar 

42. Singh, A., Bacchuwar, K., Bhasin, A.: A survey of OCR applications. Int. J. Mach. Learn. Comput. 2(3), 314 (2012)

    Article  Google Scholar 

43. Srihari, S.N.: Recognition of handwritten and machine-printed text for postal address interpretation. Pattern Recogn. Lett. 14(4), 291–302 (1993)

    Article  Google Scholar 

44. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

45. Tlemsani, R., Belbachir, K.: An improved Arabic on-line characters recognition system. In: 2018 International Arab Conference on Information Technology (ACIT), pp. 1–10. IEEE (2018)

    Google Scholar 

46. Wang, J.S., Chuang, F.C.: An accelerometer-based digital pen with a trajectory recognition algorithm for handwritten digit and gesture recognition. IEEE Trans. Industr. Electron. 59(7), 2998–3007 (2011)

    Article  Google Scholar 

47. Wehbi, M., Hamann, T., Barth, J., Eskofier, B.: Digitizing handwriting with a sensor pen: a writer-independent recognizer. In: 2020 17th International Conference on Frontiers in Handwriting Recognition (ICFHR),pp. 295–300. IEEE (2020)

    Google Scholar 

48. Yaeger, L.S., Webb, B.J., Lyon, R.F.: Combining neural networks and context-driven search for online, printed handwriting recognition in the newton. AI Mag. 19(1), 73–73 (1998)

    Google Scholar 

Download references