Online Signature Biometrics for Mobile Devices
<p>SigNet network architecture diagram.</p> "> Figure 2
<p>An example of a genuine signature (<b>top</b>) and its skilled forgery (<b>bottom</b>).</p> "> Figure 3
<p>Number of samples for each class: genuine signatures (<b>left</b>) and skilled forgeries (<b>right</b>).</p> "> Figure 4
<p>Handwritten signatures represented as images (pixels correspond to pressure values), scaled to the SigNet input data size.</p> "> Figure 5
<p>Validation of SigNet, SigNetExt and VGG-16 on the TrainSet dataset. Averaged ROC curves for 20 rounds of cross-validation.</p> "> Figure 6
<p>Validation of SigNet, SigNetExt and VGG-16 on the ValSet dataset. Averaged ROC curves for 20 rounds of cross-validation.</p> "> Figure 7
<p>Testing of SigNet, SigNetExt, and VGG-16 on the TestSet dataset. Averaged ROC curves for 20 rounds of cross-validation.</p> ">
Abstract
:1. Introduction
- A method for preprocessing online signatures gathered on mobile phones into the valuable form for verification systems;
- Two custom-made classifiers (SigNet and SigNetExt) using convolutional neural networks for signature recognition on mobile phones;
- A comprehensive study comparing online signature recognition performance based on the commonly used pre-trained VGG-16 model for image recognition and our SigNet and SigNetExt models.
2. Related Work
- Nonparametric methods, e.g., Dynamic Time Warping (DTW);
- Parametric methods, e.g., Hidden Markov Models (HMM);
- Parametric methods with unknown parameter numbers, e.g., artificial neural networks.
- x and y coordinate differences;
- Linear classifier;
- Multilayer perceptron;
- Convolutional neural networks.
- Skilled forgeries: EER = 4.02% (five samples), EER = 2.72% (ten samples);
- Random forgeries: EER = 1.15% (five samples), EER = 0.44% (ten samples).
- Skilled forgeries: EER = 6.08% (five samples);
- Random forgeries: EER = 2.94% (five samples).
- An intra-device performance study;
- An intra-/inter-modality analysis where modalities depend on the data acquisition technique, i.e., stylus or finger-based.
3. Convolutional Neural Networks for Handwritten Signature Verification
3.1. Convolutional Neural Network (CNN)
3.2. VGG-16 Neural Network Architecture
- 64 filter kernels with a size of 3 × 3;
- 128 filter kernels with a size of 3 × 3;
- 256 filter kernels with a size of 3 × 3;
- 512 filter kernels with a size of 3 × 3.
3.3. SigNet Neural Network Architecture
- 1st convolutional layer with ReLU;
- Max pooling layer;
- 2nd, 3rd and 4th convolutional layer with ReLU, each followed by average pooling layer;
- One fully connected layer with ReLU;
- Softmax layer.
- 32 filter kernels with a size of 9 × 9;
- 32 filter kernels with a size of 7 × 7;
- 64 filter kernels with a size of 5 × 5;
- 64 filter kernels with a size of 6 × 16.
4. Training and Validation
4.1. Database of Online Signatures
4.2. Handwritten Signatures Preprocessing
4.3. Evaluation Metrics and Results
- SigNet: the SigNet network architecture trained on the dataset of 968 samples acquired using a custom-made application (the base training dataset).
- SigNetExt: the SigNet network architecture trained on the augmented dataset of 9680 samples. The base dataset was augmented by multiplying the base training dataset. Each sample was transformed with 5 random shifts and 20 rotations (from to ) for each shifted image.
- VGG-16: the VGG-16 network architecture trained on the augmented dataset of 968 samples. Each sample was transformed with shifts randomly selected from the range in x and y coordinates and pivots randomly selected from for each shifted image. The network was trained with slightly different images in each epoch. Thus, the size of the training dataset was not increased, but it was randomly modified.
- EER—a statistic commonly used in performance evaluation of verification and authorization systems. Equal error rate describes the point at which the false rejection rate (FRR) and false acceptance rate (FAR) are equal. The lower the equal error rate value, the higher the accuracy of the biometric system. The definitions of FAR and FRR are as follows:
- AUC—the area under the Receiver Operating Characteristic Curve (ROC). ROC is a plot that presents in the function of .
5. Performance Evaluation of Signature Biometrics
5.1. Experiment Setup
Algorithm 1 Classifiers testing process. |
|
5.2. Results of Experiments
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Abbreviations
EER | Equal Error Rate |
DTW | Dynamic Time Warping |
FAR | False Acceptance Rate |
FRR | False Rejection Rate |
PCA | Principal Component Analysis |
CNN | Convolutional Neural Network |
RNN | Recurrent Neural Network |
TA-RNN | Time-Aligned Recurrent Neural Network |
References
- Campisi, P.; Maiorana, E.; Neri, A. Signature Biometrics. In Encyclopedia of Cryptography and Security; van Tilborg, H.C.A., Jajodia, S., Eds.; Springer: Boston, MA, USA, 2011; pp. 1208–1210. [Google Scholar] [CrossRef]
- Sharif, M.; Raza, M.; Shah, J.H.; Yasmin, M.; Fernandes, S.L. An Overview of Biometrics Methods. In Handbook of Multimedia Information Security: Techniques and Applications; Singh, A.K., Mohan, A., Eds.; Springer International Publishing: Cham, Switzerland, 2019; pp. 15–35. [Google Scholar] [CrossRef]
- Hashim, Z.; Ahmed, H.; Alkhayyat, A. A Comparative Study among Handwritten Signature Verification Methods Using Machine Learning Techniques. Sci. Program. 2022, 2022, 8170424. [Google Scholar] [CrossRef]
- Test Report Force Sensing Testing with OPTOFIDELITY™ GOLDFINGER. Available online: https://cdn2.hubspot.net/hubfs/6347010/docs/2016/OptoFidelity_GoldFinger_Force-Sensing_testreport.pdf?hsCtaTracking=31d5e724-5949-428b-8475-1edcbf6d7a7f%7C6cc0bc53-20b8-4a5a-8806-4f0a5654d775 (accessed on 28 October 2017).
- Putz-Leszczyska, J. Online signature verification using dynamic time warping with positional coordinates. In Proceedings of the Photonics Applications in Astronomy, Communications, Industry, and High-Energy Physics Experiments 2006, Wilga, Poland, 29 May–4 June 2006; SPIE: Bellinghwm, WA, USA, 2006; Volume 6347, pp. 575–582. [Google Scholar]
- Sadak, M.S.; Kahraman, N.; Uludag, U. Handwritten Signature Verification System Using Sound as a Feature. In Proceedings of the 2020 43rd International Conference on Telecommunications and Signal Processing (TSP), Milan, Italy, 7–9 July 2020; pp. 365–368. [Google Scholar] [CrossRef]
- Martinez-Diaz, M.; Fierrez, J.; Freire, M.; Ortega-Garcia, J. On the effects of sampling rate and interpolation in HMM-based dynamic signature verification. In Proceedings of the Ninth International Conference on Document Analysis and Recognition (ICDAR 2007), Curitiba, Brazil, 23–26 September 2007; Volume 2, pp. 1113–1117. [Google Scholar]
- Ortega-Garcia, J.; Fierrez-Aguilar, J.; Simon, D.; Gonzalez, J.; Faundez-Zanuy, M.; Espinosa, V.; Satue, A.; Hernaez, I.; Igarza, J.J.; Vivaracho, C.; et al. MCYT baseline corpus: A bimodal biometric database. IEE Proc.-Vision Image Signal Process. 2003, 150, 395–401. [Google Scholar] [CrossRef]
- Fallah, A.; Jamaati, M.; Soleamani, A. A new online signature verification system based on combining Mellin transform, MFCC and neural network. Digit. Signal Process. 2011, 21, 404–416. [Google Scholar] [CrossRef]
- Yeung, D.Y.; Chang, H.; Xiong, Y.; George, S.; Kashi, R.; Matsumoto, T.; Rigoll, G. SVC2004: First international signature verification competition. In Proceedings of the Biometric Authentication: First International Conference, ICBA 2004, Hong Kong, China, 15–17 July 2004; Proceedings; Springer: Berlin/Heidelberg, Germany, 2004; pp. 16–22. [Google Scholar]
- Drott, B.; Hassan-Reza, T. On-Line Handwritten Signature Verification Using Machine Learning Techniques with a Deep Learning Approach. Master’s Thesis, Lund University, Lund, Sweden, 2015. [Google Scholar]
- Anoto Pen Stylus. Available online: http://www.anoto.com (accessed on 22 August 2018).
- Tolosana, R.; Vera-Rodriguez, R.; Fierrez, J.; Ortega-Garcia, J. DeepSign: Deep on-line signature verification. IEEE Trans. Biom. Behav. Identity Sci. 2021, 3, 229–239. [Google Scholar] [CrossRef]
- Sae-Bae, N.; Memon, N. Online signature verification on mobile devices. IEEE Trans. Inf. Forensics Secur. 2014, 9, 933–947. [Google Scholar] [CrossRef]
- Kholmatov, A.; Yanikoglu, B. SUSIG: An on-line signature database, associated protocols and benchmark results. Pattern Anal. Appl. 2009, 12, 227–236. [Google Scholar] [CrossRef]
- Fakhiroh, L.A.; Fariza, A.; Basofi, A. Mobile Based Offline Handwritten Signature Forgery Identification using Convolutional Neural Network. In Proceedings of the International Electronics Symposium (IES), Surabaya, Indonesia, 29–30 September 2021; pp. 423–429. [Google Scholar] [CrossRef]
- Blanco-Gonzalo, R.; Sanchez-Reillo, R.; Miguel-Hurtado, O.; Liu-Jimenez, J. Performance evaluation of handwritten signature recognition in mobile environments. IET Biom. 2014, 3, 139–146. [Google Scholar] [CrossRef]
- Wacom Pen Tablets. Available online: https://www.wacom.com/en/products/pen-tablets (accessed on 22 August 2018).
- Nerini, M.; Favarelli, E.; Chiani, M. Augmented PIN Authentication through Behavioral Biometrics. Sensors 2022, 22, 15. [Google Scholar] [CrossRef] [PubMed]
- Lis, K.; Niewiadomska-Szynkiewicz, E.; Dziewulska, K. Siamese Neural Network for Keystroke Dynamics-Based Authentication on Partial Passwords. Sensors 2023, 23, 22. [Google Scholar] [CrossRef] [PubMed]
- Siddiqui, N.; Dave, R.; Vanamala, M.; Seliya, N. Machine and Deep Learning Applications to Mouse Dynamics for Continuous User Authentication. Mach. Learn. Knowl. Extr. 2022, 4, 502–518. [Google Scholar] [CrossRef]
- Convolutional Neural Networks for Visual Recognition. Available online: http://cs231n.github.io/convolutional-networks/ (accessed on 22 August 2018).
- Simonyan, K.; Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv 2015, arXiv:1409.1556. [Google Scholar] [CrossRef]
- Dimiccoli, M.; Cartas, A.; Radeva, P. Activity recognition from visual lifelogs: State of the art and future challenges. In Multimodal Behavior Analysis in the Wild; Alameda-Pineda, X., Ricci, E., Sebe, N., Eds.; Computer Vision and Pattern Recognition; Academic Press: Cambridge, MA, USA, 2019; pp. 121–134. [Google Scholar] [CrossRef]
- Naveen, P.; Diwan, B. Pre-trained VGG-16 with CNN Architecture to classify X-Rays images into Normal or Pneumonia. In Proceedings of the International Conference on Emerging Smart Computing and Informatics (ESCI), Pune, India, 5–7 March 2021; pp. 102–105. [Google Scholar] [CrossRef]
- Montazeri, A.H.; Emami, S.K.; Zaghiyan, M.R.; Eslamian, S. Stochastic learning algorithms. In Handbook of Hydroinformatics; Eslamian, S., Eslamian, F., Eds.; Elsevier: Amsterdam, The Netherlands, 2023; pp. 385–410. [Google Scholar] [CrossRef]
- Bartuzi, E.; Roszczewska, K.; Trokielewicz, M.; Białobrzeski, R. Mobibits: Multimodal mobile biometric database. In Proceedings of the 2018 international conference of the biometrics special interest group (BIOSIG), Darmstadt, Germany, 26–28 September 2018; pp. 1–5. [Google Scholar]
SigNet | VGG-16 | |
---|---|---|
size of input data | 77 × 190 | 224 × 224 |
layers | 4 convolutional layers (with ReLU and pooling) 1 fully connected softmax | 13 convolutional layers with ReLU and ax pooling 3 fully connected softmax |
sizes of filters | , , , | |
number of filters | 32, 32, 64, 64 | 64, 128, 256, 512 |
optimization | SGD, | SGD, |
learning rate | 0.05–0.0005 | |
training | trained on online signatures | trained on natural images and fine-tuned using transfer learning on online signatures |
regularization | data augmentation | data augmentation 50% dropout |
Input Dataset | Classifier | Random Forgeries | Skilled Forgeries |
---|---|---|---|
SigNet | 0.01 ± 0.01% | 2.66 ± 0.41% | |
TrainSet | VGG-16 | 0.10 ± 0.10% | 9.77 ± 1.37% |
SigNetExt | 0.25 ± 0.12% | 2.88 ± 0.37% | |
SigNet | 2.81 ± 0.36% | 7.47 ± 0.67% | |
ValSet | VGG-16 | 0.63 ± 0.38% | 12.47 ± 1.39% |
SigNetExt | 1.14 ± 0.19% | 6.88 ± 0.76% |
CNN Architecture | Random Forgeries | Skilled Forgeries |
---|---|---|
SigNet | 2.70 ± 0.29% | 7.16 ± 0.75% |
VGG-16 | 0.63 ± 0.29% | 12.90 ± 1.26% |
SigNetExt | 1.14 ± 0.17% | 6.66 ± 0.65% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Roszczewska, K.; Niewiadomska-Szynkiewicz, E. Online Signature Biometrics for Mobile Devices. Sensors 2024, 24, 3524. https://doi.org/10.3390/s24113524
Roszczewska K, Niewiadomska-Szynkiewicz E. Online Signature Biometrics for Mobile Devices. Sensors. 2024; 24(11):3524. https://doi.org/10.3390/s24113524
Chicago/Turabian StyleRoszczewska, Katarzyna, and Ewa Niewiadomska-Szynkiewicz. 2024. "Online Signature Biometrics for Mobile Devices" Sensors 24, no. 11: 3524. https://doi.org/10.3390/s24113524