Efficient Fingerprint Extraction and Matching
Using Smartphone Camera
Saksham Gupta, Sukhad Anand1 , and Atul Rai2
arXiv:1708.00884v1 [cs.CV] 2 Aug 2017
1
Department of Computer Science, Delhi Technological University, Rohini,Delhi
110042
sakshamgupta 2k14@dtu.ac.in , sukhad.anand@gmail.com
2
Staqu Technologies,Gurgoan, Haryana 11004515
atual.rai@staqu.com
Abstract. In the previous decade, there has been a considerable rise in
the usage of smartphones.Due to exorbitant advancement in technology,
computational speed and quality of image capturing has increased considerably. With an increase in the need for remote fingerprint verification,
smartphones can be used as a powerful alternative for fingerprint authentication instead of conventional optical sensors. In this research, we
propose a technique to capture finger-images from the smartphones and
pre-process them in such a way that it can be easily matched with the optical sensor images.Effective finger-image capturing, image enhancement,
fingerprint pattern extraction, core point detection and image alignment
techniques have been discussed. The proposed approach has been validated on FVC 2004 DB1 & DB2 dataset and the results show the efficacy
of the methodology proposed. The method can be deployed for real-time
commercial usage.
Keywords: Finger-image, Gabor Filter, Core point, Histogram, Remote
Verification, FVC
1
Introduction
Over the past decade, fingerprint verification has been used irresistibly in various domains including criminal identification, legal matters and user authentication.With advancement in automated fingerprint identification systems, accompanied by an increasing need for reliable authentication has resulted in
widespread deployment of fingerprint authentication in broad applications such
as background checks of employees, access to secure facilities, user access authentication in smartphones and remote verification.
Remote verification includes verifying the authenticity of a user’s located at
access regions. Identity proofing can be used for diverse applications including
secure fund transfer,medical-identity proofing and ATM security.The need for remote verification has been immensely growing especially in developing countries
like India which need to verify users located at remote regions for governmentrelated services. One prime example is remote e-KYC(Know your customer Service) started by Indian Government. Generally, these services require fingerprint
2
Gupta et al.
authentication which is generally done with optical sensors, difficult to be made
accessible in rural areas. With the dawn of the age of smartphones, the overhead
of providing optical sensors at outlying regions can be considerably reduced with
the help of smartphones.
We propose an approach to use smartphone camera to extract fingerprints
of a user which can be further used for authentication by matching with images
available in any database consisting of optical sensor images.
2
Related Work
Substantial research has been done in the field of finger-photo recognition.Some
of the considerable work include Derawi et al. [1] who proposed finger-image
recognition with two different smartphones.Neurotechnology, Verifinger 6.0 Extended SDK commercial minutia extractor has been used for the feature extraction.The verification results of the Neurotechnology algorithm on the processed images are used for comparison.Stein et al.[2] proposed a technique for
authentication of users on smartphones using finger-image authentication. The
proposed method provides algorithms for the capture process to ensure reliable
finger-image recognition. Various preprocessing techniques have been described
to enhance the recognition rates. In 2015 Anush et. al.[3] proposed Scatnet based
feature representation[4] and matching algorithm for finger-image images.
The above approaches fail to produce reliable results which could be used for
remote verification. None of the above-discussed approaches have dealt matching
between finger-image captured using a smartphone camera and optical sensor
images.
[5] gives a detailed description of various techniques used for Fingerprint
Recognition.The paper discusses major techniques used in past for feature extraction. These techniques include Minutiae Based Technique, Pattern Matching
or Ridge Feature Based Techniques, Correlation Based Technique and imagebased technique.The paper also describes various enhancement techniques used
in the past.
3
Approach
As shown in Fig.1 the proposed fingerprint verification algorithm involves effective image capturing to get a clear and enhanced image.After a clear image has
been captured the image has to be preprocessed which involves histogram equalization and removing superfluous noise in the image.After the finger-image has
been preprocessed, the fingerprint pattern consisting of frictional ridges has been
extracted so that the finger-image can be further matched with optical sensor
images. After the fingerprint pattern has been extracted, the core point of the
finger has to be located so that the finger-image can be properly cropped and
aligned with the optical sensor images available.Finally, the image is matched
with the database of optical sensor images available.
Efficient Fingerprint Extraction and Matching Using Smartphone Camera’s
3
Fig. 1. Overview of the proposed methodology
3.1
Image Capturing
A custom UI application was developed over Open camera application[6] for
effective image capturing.The UI developed is shown in the Fig.2. The application ensures quality image capturing. Features of the UI include a custom ROI
to constrain the location of the fingerprint in the image which eliminates the
need for background-separation and finger-image alignment with optical sensor
images.
Multiple Images are captured in quick succesion by varying exposure during
each capture.Finally, an HDR image is extracted which is processed further.
Region of Intereset(ROI) It is a rectangular bounding box which constrains
the user to capture finger-image in a particular orientation resulting in desired
image alignment.
High Dynamic Range(HDR) A well-known problem in images is due to
varied lighting and to ensure that every detail, be it well lit or be it in shadow
is captured by the camera we use HDR technique. HDR ensure finger-image
quality and capture every minuscule detail. We use a five-bracket system in
which we vary the exposure of the camera and capture the image at each of
these exposures. These five photos are then combined to form a single high
detail image.
3.2
Finger-image Pre-processing
For precise and unerring authentication using fingerprint matching, the minutiae
have to be extracted with an extraordinary amount of precision.For voracious
4
Gupta et al.
Fig. 2. Custom UI built upon Open Camera Application
minutiae extraction, the noise from the image has to be completely eliminated
i.e the image has to be enhanced. CLAHE(Contrast limited adaptive histogram
equalization) is superior in terms of accuracy as compared to adaptive histogram
equalization because it overcomes its problem of superfluous noise amplification
which would have produced infelicitous results. ClAHE captures and improves
the local edge information which is essential to the process of fingerprint extraction.
The image is converted from RGB format to a grayscale image and is partitioned into small non-overlapping regions of equal size called tiles. Histogram
equalization is applied to each tile separately. Histogram equalization improves
the contrast of the image by stretching out the intensity range at points where
pixels have been clustered.
The contrast enhancement can be restricted by restricting the slope of cumulative distribution graph which depends on the slope of the transformation function.This eventually reduces the value of histogram at that pixel value. CLAHE
achieves this by clipping the histogram at predefined value which limits the slope
of cumulative distribution
Superfluous noise in the resultant image is minimized by setting up a clip
limit.A clip limit reduces the maximum height of a histogram equalization graph
of pixel vs intensity which in turn reduces the amplified noise.After histogram
equalization the adjacent tiles are joined together using bilinear interpolation to
Efficient Fingerprint Extraction and Matching Using Smartphone Camera’s
5
completely purge the artificially induced boundaries. The results obtained after
applying CLAHE to the image have been shown in Fig.3.
Fig. 3. (a)Original Image captured by Custom UI (b)Image after application of preprocessing
Fig. 4. (a)Histogram of Captured image (b) Histogram of Captured Image after application of CLAHE.
3.3
Finger-image Pattern Extraction
To facilitate feature extraction and matching we need to transform our fingerimage. Feature Extraction only works well when the image has clear demarcation
6
Gupta et al.
of ridge-valley structure. To highlight this ridge-valley structure in the fingerimage multiple convolutions of 2D Gabor filters varied over range of theta values
are used. A detailed explanation of each process is described below:Gabor Filter A 2D Gabor filter is defined by sinusoidal plane wave multiplied
by a Gaussian function. Discrete domain Gabor filter is given in Eq.1,2.
Gc [i, j] = Be−
Gs [i, j] = Ce−
(i2 +j 2 )
2ϕ
(i2 +j 2 )
2ϕ
cos(2πf (i cos Ω + j sin Ω))
(1)
sin(2πf (i cos Ω + j sin Ω))
(2)
where B and C are normalising factors, f is frequency in the texture and Ω
gives texture oriented in particular direction, ϕ is the size of the image.Above
equations can be derived from 1D Gabor Kernels given by
Real
g(a, b; φ, Ω, ψ, ϕ, γ) = exp(−
a′
a′2 + γ 2 b′2
) cos(2π + ψ)
2
2ϕ
φ
(3)
a′2 + γ 2 b′2
a′
) sin(2π + ψ)
2
2ϕ
φ
(4)
Imaginary
g(a, b; φ, Ω, ψ, ϕ, γ) = exp(−
where
a′ = a cos Ω + b sin Ω
(5)
b′ = −a sin Ω + b cos Ω
(6)
and
where φ is the wavelength of the sinusoidal factor, ψ is the phase offset,ϕis the
standard deviation of the Gaussian envelope and γ represents the spatial aspect
ratio and specifies the ellipticity of the support of the Gabor function.
Variation in Kernel’s Multiple kernels are formed by altering the value of
theta in Eq.1,2 above over a range(generally from 0 to pie in steps of pi/16).
Fig. 5.
Gabor
Convolutions
right:k=15,18,19,20,24,28,30,35
for
different
Kernel
size,left
to
Efficient Fingerprint Extraction and Matching Using Smartphone Camera’s
7
Convolutions Convolution is performed using the kernels generated.Eq.7 describe the convolution model used
H(x, y) =
M
j −1
i −1 M
X
X
i=0
I(x + i − ai , y + j − aj )K(i, j)
(7)
j=0
Here I is input image K is kernel, ai and aj is the value of anchor point in the
kernel and Mi and Mj gives kernel size.
3.4
Core-point Detection
A crucial determinant in fast fingerprint authentication or verification is the
relative alignment of the two fingerprints being matched. A precise alignment
reduces computation by preventing the comparison of each minutiae features
set.Additionally the cropping of finger-image is centered around the core-point.
In the proposed work, the latest technique for core point detection proposed
by G. A. Bhagat et al. [7] has been used. As the fingerprint consists of frictional
ridges.The proposed approach uses ridge orientation map for localizing the core
point.
The accuracy has been amplified by using an adaptive smoothing technique
as described in [8]. The smoothed orientation map is obtained by
P
1
(m,n)ǫΩ(s) sin(2θo (m, n))
)
(8)
θs (a, b) = arctan( P
2
(m,n)ǫΩ(s) cos(2θo (m, n))
where θs (a, b) is the smoothed orientation of the block (i, j),Ω(s) is the surrounding neighborhood of the block and s is the consistency level and θo (k, l)
provides the initial orientation i.e the angle made by ridges passing through
neighbourhood centred at (m,n). The computation time has been minimized by
dividing the image into smaller blocks of size r ∗ r where r is slightly larger than
the width of the frictional ridges.
A binary set aligned in square shape is presented which scans the segmented
smoothed orientation map. The core point is finally detected using the method
proposed by[7].
Fig. 6. Oveview of Core-point Detection Algorithm
8
3.5
Gupta et al.
Finger-image Alignment
After the core point of the fingerprint has been determined, the image has to
be properly aligned with the images of the optical sensor images for accurate
matching.
The image also has to be cropped about the core point. Cropping is necessary
because if complete image is given for matching it would result in comparison
of large number of non-useful minutiae sets which would ultimately result in
increased overhead time and will also result in improper matching.
The images obtained from the optical sensor and the finger-image have a
phase difference of 1800 .Hence the images captured using smartphones have to
be flipped so that the images can be precisely matched.
3.6
Feature Extraction and Matching
For the purpose of feature extraction and matching, We used open source application SouceAFIS[9]. The software is based on MINDTCT template extractor[10]
and Bozorth3 [11] matching technique. SourceAFIS allow modification in FAR(False
Accept Rate) and FRR(False Rejection Rate) so that ERR(Equal Error Rate)
can be computed.
4
Results
As our approach is based on heterogeneous matching of fingerprints i.e. matching
between images prints acquired from the optical sensor and smartphone camera
we used the standard optical sensor database FVC 2004(DB1,DB2)[12] . Fingerimages were captured using a smartphone having a 16 MP camera and resolutio
of 1920x1080.
A custom android application was developed for accurate image acquisition. The application was built upon Open-Camera[6].Fig2 depicts the application.The bounding box represents the ROI required to enforce alignment.To
ensure image quality 5 frames are captured with different exposures in the burst
mode and finally an HDR image is generated.
Experiments were performed by transfiguring various parameters, which have
been described below:
Gabor kernel size As described above in section multiple Gabor filter convolutions are applied to highlight the ridge-valley structure of the print. Gabor filter
is applied to the images in the form of a 2-D convolution kernel. Experiments
were made upon different Gabor kernel sizes to ensure maximum matching accuracy. The curve shown in Fig 7 depicts the relationship between various Kernel
sizes and the matching scores obtained using SourceAFIS matcher.
From the curve, we conclude that the matching score achieve its maxima for
a kernel size of 21 with a score of 74.23. Thus a Gabor filter of square kernel
size of 21 supplies best results for our experiments. Further results are computed
using this kernel size only.
Efficient Fingerprint Extraction and Matching Using Smartphone Camera’s
9
Fig. 7. Matching Score for different kernel sizes of 2D Gabour filters
Effective area of interest around the core point After core point detection,
different dimensions for cropping area around the core point were considered to
find its dependence on matching score.Fig 8(a) represents the desired dependency
relationship.
Fig. 8. (a)Matching Score(SourceAFIS) varied over area of cropping(b)Processing
Time required to find Gabor convolutions over multiple threads.
From the curve, we can ascertain that the optimal dimensions of the effective
region of interest to achieve maximum accuracy are 800x700 pixels around the
core point detected.
10
Gupta et al.
Threading Our approach consists of convolutions of multiple Gabor filters
which is a computationally expensive activity. For a multi-processor system,
applying threading can significantly speed up the convolution process. Fig 8(b)
depicts the time of execution for different number of threads. From the graph it
is evident that computation time decreases drastically as the number of threads
are increased and saturates around a thread count of 5.
Accuracy The ROC curve of the experiments performed with different thresholds of the matcher is shown in Fig.9.As the value of the threshold is increased,
both true positive rate and the false positive rate decrease and vice versa. ERR
for the proposed technique on the FVC 2004 database lies between the range of
6% - 15%, thus making the approach feasible for commercial usage in real-time.
Fig. 9. ROC curve obtained by varying threshold values of the matcher
5
Conclusion and Future Work
By this study, we conclude that fingerprint authentication can be performed
using Smartphone cameras. We were able to match the finger-images obtained
using the camera with the fingerprints obtained using the optical sensor. The
EER was 6% - 15% which is well within the acceptable limits. Thus this technique
can be efficiently used by people not having easy access to fingerprint sensors.
The technique can be greatly improved by adjusting the Gabor filter parameters
optimal to the setting of finger-image capturing. Further better minutiae extractor and matcher can be used to accurately match the prints. As the technology
evolves more powerful smartphone cameras shall be able to capture higher resolution images which will ultimately lead to better extraction of the ridge-valley
structure.
Efficient Fingerprint Extraction and Matching Using Smartphone Camera’s
11
References
1. Derawi, Mohammad Omar; Yang, Bian; Busch, Christoph, ”Fingerprint Recognition
with Embedded Cameras on Mobile Phones”, ”Security and Privacy in Mobile Information and Communication Systems: Third International ICST Conference, MobiSec 2011, Aalborg, Denmark, May 17-19, 2011, Revised Selected Papers”,2012,pp.
136-147,doi:10.1007/978-3-642-30244-2 12
2. Fingerphoto Recognition with Smartphone Cameras Chris Stein, Claudia Nickel,
Christoph Busch Hochschule Darmstadt - CASED Mornewegstrae 32 D-64295
Darmstadt
3. On Smartphone Camera based Fingerphoto Authentication Anush Sankaran,
Aakarsh Malhotra, Apoorva Mittal, Mayank Vatsa, Richa Singh IIIT Delhi, India
4. L. Sifre and S. Mallat. Rotation, scaling and deformation invariant scattering for
texture discrimination. In CVPR, pages 12331240, 2013. 2,
5. J. Rajharia, PC. Gupta, A. Sharma, ”Fingerprint Based Identification System: A
Survey”, International Journal of Computer Technology and Electronics Engineering(IJCTEE) Vol 1 Issue 3, 2012
6. Mark Harman, Open Camera, Retrieved from https://opencamera.sourceforge.net/
7. G A Bhagat, A. H. Khalil and N. S. Abdel Kader, Fast and accurate algorithm for
core point detection in fingerprint images, Journal of Egyptian Informatic, vol. 14,
no.1, pp. 15-25, 2013.
8. M. Liu, X. Jiang, A.C. Kot,”Fingerprint reference point detection”,EURASIP J
Appl Signal Process,4 (2002), pp. 498 - 509
9. Robert Vazan,”Fingerprint Recognition Toolkit”, 2015, url:http://sourceafis.org/
10. C. I. Watson, M. D. Garris, E. Tabassi, C. L. Wilson, ”User’s Guide to NIST
Biometric Image Software”, National Institute of Standards and Technology, url:
http://ws680.nist.gov/publication/get pdf.cfm?pub id=51097
11. C. I. Watson, M. D. Garris, E. Tabassi, C. L. Wilson, ”User’s Guide to NIST
Biometric Image Software”, National Institute of Standards and Technology, url:
http://ws680.nist.gov/publication/get pdf.cfm?pub id=51096
12. Fingerprint Verification Competition, http://bias.csr.unibo.it/fvc2004/, 2004