Face Recognition on a Smart Image Sensor Using Local Gradients
<p>Proposed method and hardware accelerator. The left hand side illustrates the steps of our face recognition algorithm. The right hand side shows the elements of our SIS, namely smart-pixel array, pattern generator, and digital coprocessor, which execute the stages of the algorithm.</p> "> Figure 2
<p>Examples of LBP and RLBP operators on a 3 × 3-pixel window. (<b>a</b>) LBP operator with label 11001100. (<b>b</b>) RLBP operator with label 01011100.</p> "> Figure 3
<p>Ahonen’s algorithm using uniform RLBP.</p> "> Figure 4
<p>Architecture of the proposed SIS. An array of smart pixels outputs either the pixel value or the difference between horizontally adjacent pixels. An RLBP generator (RPG) reads pixel values and creates an 8-bit RLBP for each pixel in the image. The digital coprocessor computes histograms of RLBP patterns to construct the feature vector, executes the LDA projection on each vector, and selects the nearest neighbor from a stored set of projected vectors using the Euclidean distance.</p> "> Figure 5
<p>The smart pixel consists of an analog input-select multiplexer, a configurable CTIA, and a row-select switch. All the smart pixels in the array share the control signals placed above in the figure, and all the pixels in the same column share the column output signal. The input to the CTIA can be selected from the photodetector in the local or adjacent pixel.</p> "> Figure 6
<p>Schematic diagram of the configurable CTIA. The CTIA integrates the photodetector currents and outputs a voltage that represents either the pixel value or the difference between horizontally-adjacent pixels.</p> "> Figure 7
<p>Smart pixel in conventional mode: the input-select switches pass the current from PD1, <span class="html-italic">sw1</span> and <span class="html-italic">sw4</span> are closed to integrate the current, and <span class="html-italic">sw2</span> and <span class="html-italic">sw3</span> stay open.</p> "> Figure 8
<p>Simplified view of positive and negative integration. During positive integration: <span class="html-italic">sw1</span> and <span class="html-italic">sw4</span> stay closed, <span class="html-italic">sw2</span> and <span class="html-italic">sw3</span> stay open. During positive integration: <span class="html-italic">sw2</span> and <span class="html-italic">sw3</span> stay closed, <span class="html-italic">sw1</span> and <span class="html-italic">sw4</span> stay open.</p> "> Figure 9
<p>Architecture of the RPG. An input comparator compares the local gradient value for each pixel to a reference voltage. The digital comparator outputs are sequentially stored in an array of <math display="inline"><semantics> <mrow> <mn>3</mn> <mo>×</mo> <mn>3</mn> </mrow> </semantics></math> flip-flops, organized as three shift registers. The RPG outputs an 8-bit RLBP with the output of all the flip-flops except for the one at the center.</p> "> Figure 10
<p>Architecture of the digital coprocessor. The processor receives a stream of RLBPs from the RPG, and simultaneously builds the histogram vector and projects it using LDA. A memory controller retrieves the LDA coefficients from RAM. The Euclidean distance between the projected vector and the contents of database of stored faces is used for classification with the nearest neighbor criterion.</p> "> Figure 11
<p>Architecture of the LDA projection module. The module transforms the 8-bit RLBP into a 6-bit uniform RLBP (uRP). For each uRP value received, the module accumulates the value of its corresponding LDA coefficient, thus performing histogram computation and LDA projection in a single step.</p> "> Figure 12
<p>Euclidean distance module. It normalizes and centers the input vector and computes the distance between vectors <span class="html-italic">p</span> and <span class="html-italic">q</span> as <math display="inline"><semantics> <mrow> <mo>∑</mo> <msubsup> <mi>p</mi> <mi>i</mi> <mn>2</mn> </msubsup> <mo>−</mo> <mn>2</mn> <mo>∑</mo> <mrow> <msub> <mi>p</mi> <mi>i</mi> </msub> <msub> <mi>q</mi> <mi>i</mi> </msub> </mrow> <mo>+</mo> <mo>∑</mo> <msubsup> <mi>q</mi> <mi>i</mi> <mn>2</mn> </msubsup> </mrow> </semantics></math>.</p> "> Figure 13
<p>Classification module. The module implements a nearest neighbor criterion by selecting the face label that corresponds to the minimum distance computed between the input image and the stored database of know faces.</p> "> Figure 14
<p>Experimental setup to test the face recognition algorithm. An FPGA board receives IR images from a FLIR Tau 2 camera core and uses a HOG algorithm to detect face locations. The FPGA emulates the smart pixel array and the digital coprocessor. A monitor connected to the FPGA displays the image acquired by the smart pixel array, and the location and number of identified faces. The FPGA sends the labels of the recognized faces to a remote computer via Ethernet.</p> "> Figure 15
<p>Layout of the smart pixel. We used the design shown in <a href="#sensors-21-02901-f006" class="html-fig">Figure 6</a>, implemented on the TMSC 0.35 μm mixed-signal process. The opamp and integration capacitors are implemented using two poly layers.</p> "> Figure 16
<p>Postlayout simulation of five pixels in the SIS operating in local gradient mode. The graph shows the voltage across the integration capacitor of the CTIA in <a href="#sensors-21-02901-f007" class="html-fig">Figure 7</a> during the positive and negative integration phases shown in <a href="#sensors-21-02901-f008" class="html-fig">Figure 8</a>.</p> "> Figure 17
<p>Postlayout simulation of the RPG input comparator while reading multiple pixels. The plot shows the integration phase for the first and third pixel, the voltage input to the comparator, the reference voltage, and the voltage output. Gradient values are read every 50 ns.</p> "> Figure 18
<p>Effect of V<math display="inline"><semantics> <msub> <mrow/> <mi>ref</mi> </msub> </semantics></math> in the RLBP values generated by the RPG for 3 images acquired using a FLIR Tau 2 thermal IR core. (<b>a</b>) original IR image, (<b>b</b>) RLBP image generated by software, (<b>c</b>–<b>e</b>) RLBP images generated by the RPG for V<math display="inline"><semantics> <msub> <mrow/> <mi>ref</mi> </msub> </semantics></math> values of 1.665 V, 1.650 V and 1.645 V, respectively.</p> "> Figure 19
<p>Classification accuracy as a function of the value of the comparator input <math display="inline"><semantics> <msub> <mi>V</mi> <mrow> <mi>r</mi> <mi>e</mi> <mi>f</mi> </mrow> </msub> </semantics></math> in <a href="#sensors-21-02901-f009" class="html-fig">Figure 9</a>.</p> ">
Abstract
:1. Introduction
2. Related Work
3. Methods
Algorithm 1:Proposed method using RLBP + LDA. |
Algorithm 2:Uniform RLBP computation. |
4. SIS Architecture
4.1. Smart Pixel
4.2. RLBP Generator
4.3. Digital Coprocessor
5. Results
5.1. Smart Pixel and RPG Implementation
5.2. FPGA Implementation of the Digital Coprocessor
5.3. Method Classification Performance
5.4. SIS Classification Performance
6. Conclusions
Author Contributions
Funding
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Rui, Z.; Yan, Z. A Survey on Biometric Authentication: Toward Secure and Privacy-Preserving Identification. IEEE Access 2019, 7, 5994–6009. [Google Scholar] [CrossRef]
- Sharif, M.; Raza, M.; Shah, J.H.; Yasmin, M.; Fernandes, S.L. An Overview of Biometrics Methods. In Handbook of Multimedia Information Security: Techniques and Applications; Singh, A.K., Mohan, A., Eds.; Springer International Publishing: Cham, Switzerland, 2019; pp. 15–35. [Google Scholar] [CrossRef]
- Adjabi, I.; Ouahabi, A.; Benzaoui, A.; Taleb-Ahmed, A. Past, Present, and Future of Face Recognition: A Review. Electronics 2020, 9, 1188. [Google Scholar] [CrossRef]
- Oloyede, M.O.; Hancke, G.P.; Myburgh, H.C. A review on face recognition systems: Recent approaches and challenges. Multimed. Tools Appl. 2020, 79, 27891–27922. [Google Scholar] [CrossRef]
- Ambeth Kumar, V.D.; Ramya, S.; Divakar, H.; Kumutha Rajeswari, G. A Survey on Face Recognition in Video Surveillance. In Proceedings of the International Conference on ISMAC in Computational Vision and Bio-Engineering 2018 (ISMAC-CVB); Series Title: Lecture Notes in Computational Vision and Biomechanics; Pandian, D., Fernando, X., Baig, Z., Shi, F., Eds.; Springer International Publishing: Cham, Switzerland, 2019; Volume 30, pp. 699–708. [Google Scholar] [CrossRef]
- Pujol, F.A.; Pujol, M.J.; Rizo-Maestre, C.; Pujol, M. Entropy-Based Face Recognition and Spoof Detection for Security Applications. Sustainability 2020, 12, 85. [Google Scholar] [CrossRef] [Green Version]
- Galterio, M.G.; Shavit, S.A.; Hayajneh, T. A Review of Facial Biometrics Security for Smart Devices. Computers 2018, 7, 37. [Google Scholar] [CrossRef] [Green Version]
- Pandey, S.; Chouhan, V.; Mahapatra, R.P.; Chhettri, D.; Sharma, H. Real-Time Safety and Surveillance System Using Facial Recognition Mechanism. In Intelligent Computing and Applications; Dash, S.S., Das, S., Panigrahi, B.K., Eds.; Springer: Singapore, 2021; pp. 497–506. [Google Scholar]
- Son, N.T.; Anh, B.N.; Ban, T.Q.; Chi, L.P.; Chien, B.D.; Hoa, D.X.; Thanh, L.V.; Huy, T.Q.; Duy, L.D.; Hassan Raza Khan, M.; et al. Implementing cctv-based attendance taking support system using deep face recognition: A case study at fpt polytechnic college. Symmetry 2020, 12, 307. [Google Scholar] [CrossRef] [Green Version]
- Muthumari, M.; Sah, N.K.; Raj, R.; Saharia, J. Arduino based Auto Door unlock control system by Android mobile through Bluetooth and Wi-Fi. In Proceedings of the 2018 IEEE International Conference on Computational Intelligence and Computing Research (ICCIC), Madurai, India, 13–15 December 2018; IEEE: Madurai, India, 2018; pp. 1–4. [Google Scholar] [CrossRef]
- Das, A.; Galdi, C.; Han, H.; Ramachandra, R.; Dugelay, J.L.; Dantcheva, A. Recent Advances in Biometric Technology for Mobile Devices. In Proceedings of the 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS), Redondo Beach, CA, USA, 22–25 October 2018; IEEE: Redondo Beach, CA, USA, 2018; pp. 1–11. [Google Scholar] [CrossRef] [Green Version]
- Pirbhulal, S.; Zhang, H.; Wu, W.; Mukhopadhyay, S.C.; Zhang, Y.T. Heartbeats Based Biometric Random Binary Sequences Generation to Secure Wireless Body Sensor Networks. IEEE Trans. Biomed. Eng. 2018, 65, 2751–2759. [Google Scholar] [CrossRef]
- Kaur, B.; Singh, D.; Roy, P.P. Age and gender classification using brain–computer interface. Neural Comput. Appl. 2019, 31, 5887–5900. [Google Scholar] [CrossRef]
- Taheri, S.; Yuan, J.-S. A Cross-Layer Biometric Recognition System for Mobile IoT Devices. Electronics 2018, 7, 26. [Google Scholar] [CrossRef] [Green Version]
- Bong, K.; Choi, S.; Kim, C.; Han, D.; Yoo, H.J. A Low-Power Convolutional Neural Network Face Recognition Processor and a CIS Integrated With Always-on Face Detector. IEEE J. Solid-State Circuits 2018, 53, 115–123. [Google Scholar] [CrossRef]
- Kim, C.; Bong, K.; Hong, I.; Lee, K.; Choi, S.; Yoo, H.J. An ultra-low-power and mixed-mode event-driven face detection SoC for always-on mobile applications. In Proceedings of the ESSCIRC 2017—43rd IEEE European Solid State Circuits Conference, Leuven, Belgium, 11–14 September 2017; pp. 255–258. [Google Scholar] [CrossRef]
- Kim, J.H.; Kim, C.; Kim, K.; Yoo, H.J. An Ultra-Low-Power Analog-Digital Hybrid CNN Face Recognition Processor Integrated with a CIS for Always-on Mobile Devices. In Proceedings of the 2019 IEEE International Symposium on Circuits and Systems (ISCAS), Sapporo, Japan, 26–29 May 2019; pp. 1–5. [Google Scholar] [CrossRef]
- Jia, S.; Guo, G.; Xu, Z. A survey on 3D mask presentation attack detection and countermeasures. Pattern Recognit. 2020, 98, 107032. [Google Scholar] [CrossRef]
- Jia, S.; Guo, G.; Xu, Z.; Wang, Q. Face presentation attack detection in mobile scenarios: A comprehensive evaluation. Image Vis. Comput. 2020, 93, 103826. [Google Scholar] [CrossRef]
- Popa, D.; Ali, S.Z.; Hopper, R.; Dai, Y.; Udrea, F. Smart CMOS mid-infrared sensor array. Opt. Lett. 2019, 44, 4111–4114. [Google Scholar] [CrossRef]
- Bhattacharjee, S.; Mohammadi, A.; Marcel, S. Spoofing Deep Face Recognition with Custom Silicone Masks. In Proceedings of the 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS), Redondo Beach, CA, USA, 22–25 October 2018; pp. 1–7. [Google Scholar] [CrossRef] [Green Version]
- Baweja, Y.; Oza, P.; Perera, P.; Patel, V.M. Anomaly Detection-Based Unknown Face Presentation Attack Detection. In Proceedings of the 2020 IEEE International Joint Conference on Biometrics (IJCB), Houston, TX, USA, 28 September–1 October 2020; pp. 1–9. [Google Scholar] [CrossRef]
- Kowalski, M.; Grudzień, A. High-resolution thermal face dataset for face and expression recognition. Metrol. Meas. Syst. 2018, 25. [Google Scholar]
- Seo, J.; Chung, I.J. Face liveness detection using thermal face-CNN with external knowledge. Symmetry 2019, 11, 360. [Google Scholar] [CrossRef] [Green Version]
- George, A.; Mostaani, Z.; Geissenbuhler, D.; Nikisins, O.; Anjos, A.; Marcel, S. Biometric Face Presentation Attack Detection With Multi-Channel Convolutional Neural Network. IEEE Trans. Inf. Forensics Secur. 2020, 15, 42–55. [Google Scholar] [CrossRef] [Green Version]
- Jo, H.; Kim, W.Y. NIR Reflection Augmentation for DeepLearning-Based NIR Face Recognition. Symmetry 2019, 11, 1234. [Google Scholar] [CrossRef] [Green Version]
- Ohta, J. Smart CMOS Image Sensors and Applications, 2nd ed.; Optical Science and Engineering; CRC Press: Boca Raton, FL, USA, 2020. [Google Scholar]
- Yang, J.; Shi, C.; Cao, Z.; Han, Y.; Liu, L.; Wu, N. Smart image sensing system. In Proceedings of the 2013 IEEE SENSORS, Baltimore, MD, USA, 3–6 November 2013; pp. 1–4. [Google Scholar] [CrossRef]
- Jin, M.; Noh, H.; Song, M.; Kim, S.Y. Design of an Edge-Detection CMOS Image Sensor with Built-in Mask Circuits. Sensors 2020, 20, 3649. [Google Scholar] [CrossRef] [PubMed]
- Yin, C.; Hsieh, C.C. A 0.5 V 34.4 uW 14.28 kfps 105 dB smart image sensor with array-level analog signal processing. In Proceedings of the 2013 IEEE Asian Solid-State Circuits Conference (A-SSCC), Singapore, 11–13 November 2013; pp. 97–100. [Google Scholar] [CrossRef]
- Choi, J.; Lee, S.; Son, Y.; Kim, S.Y. Design of an Always-On Image Sensor Using an Analog Lightweight Convolutional Neural Network. Sensors 2020, 20, 3101. [Google Scholar] [CrossRef] [PubMed]
- Xie, S.; Prouza, A.A.; Theuwissen, A. A CMOS-Imager-Pixel-Based Temperature Sensor for Dark Current Compensation. IEEE Trans. Circuits Syst. II Express Briefs 2020, 67, 255–259. [Google Scholar] [CrossRef]
- Zhou, T.; Zhao, J.; He, Y.; Jiang, B.; Su, Y. A Readout Integrated Circuit (ROIC) employing self-adaptive background current compensation technique for Infrared Focal Plane Array (IRFPA). Infrared Phys. Technol. 2018, 90, 122–132. [Google Scholar] [CrossRef]
- Lee, K.; Park, S.; Park, S.Y.; Cho, J.; Yoon, E. A 272.49 pJ/pixel CMOS image sensor with embedded object detection and bio-inspired 2D optic flow generation for nano-air-vehicle navigation. In Proceedings of the IEEE 2017 Symposium on VLSI Circuits, Kyoto, Japan, 5–8 June 2017; pp. C294–C295. [Google Scholar]
- Zhong, X.; Yu, Q.; Bermak, A.; Tsui, C.Y.; Law, M.K. A 2pJ/pixel/direction MIMO processing based CMOS image sensor for omnidirectional local binary pattern extraction and edge detection. In Proceedings of the 2018 IEEE Symposium on VLSI Circuits, Honolulu, HI, USA, 18–22 June 2018; pp. 247–248. [Google Scholar]
- Choi, J.; Park, S.; Cho, J.; Yoon, E. A 3.4-$\mu$W Object-Adaptive CMOS Image Sensor With Embedded Feature Extraction Algorithm for Motion-Triggered Object-of-Interest Imaging. IEEE J. Solid-State Circuits 2014, 49, 289–300. [Google Scholar] [CrossRef]
- Hsu, T.H.; Chen, Y.R.; Liu, R.S.; Lo, C.C.; Tang, K.T.; Chang, M.F.; Hsieh, C.C. A 0.5-V Real-Time Computational CMOS Image Sensor With Programmable Kernel for Feature Extraction. IEEE J. Solid-State Circuits 2020. [Google Scholar] [CrossRef]
- Massari, N.; Gottardi, M. A 100 dB Dynamic-Range CMOS Vision Sensor With Programmable Image Processing and Global Feature Extraction. IEEE J. Solid-State Circuits 2007, 42, 647–657. [Google Scholar] [CrossRef]
- Zhang, W.; Fu, Q.; Wu, N.J. A Programmable Vision Chip Based on Multiple Levels of Parallel Processors. IEEE J. Solid-State Circuits 2011, 46, 2132–2147. [Google Scholar] [CrossRef]
- Hasler, J. Analog Architecture Complexity Theory Empowering Ultra-Low Power Configurable Analog and Mixed Mode SoC Systems. J. Low Power Electron. Appl. 2019, 9, 4. [Google Scholar] [CrossRef] [Green Version]
- Soto, J.E.; Valenzuela, W.E.; Diaz, S.; Saavedra, A.; Figueroa, M.; Ghasemi, J.; Zarkesh-Ha, P. An intelligent readout integrated circuit (iROIC) with on-chip local gradient operations. In Proceedings of the 2017 24th IEEE International Conference on Electronics, Circuits and Systems (ICECS), Batumi, Georgia, 5–8 December 2017; pp. 360–362. [Google Scholar] [CrossRef]
- Sanchez-Fernandez, A.J.; Romero, L.F.; Peralta, D.; Medina-Pérez, M.A.; Saeys, Y.; Herrera, F.; Tabik, S. Asynchronous Processing for Latent Fingerprint Identification on Heterogeneous CPU-GPU Systems. IEEE Access 2020, 8, 124236–124253. [Google Scholar] [CrossRef]
- Zhang, S.; Wang, X.; Lei, Z.; Li, S.Z. Faceboxes: A CPU real-time and accurate unconstrained face detector. Neurocomputing 2019, 364, 297–309. [Google Scholar] [CrossRef]
- Šušteršič, T.; Peulić, A. Implementation of Face Recognition Algorithm on Field Programmable Gate Array (FPGA). J. Circuits Syst. Comput. 2019, 28, 1950129. [Google Scholar] [CrossRef]
- Bonny, T.; Rabie, T.; Hafez, A.H.A. Multiple histogram-based face recognition with high speed FPGA implementation. Multimed. Tools Appl. 2018, 77, 24269–24288. [Google Scholar] [CrossRef]
- Ahmed, M.T.; Sinha, S. Design and Development of Efficient Face Recognition Architecture Using Neural Network on FPGA. In Proceedings of the 2018 Second International Conference on Intelligent Computing and Control Systems (ICICCS), Madurai, India, 14–15 June 2018; pp. 905–909. [Google Scholar] [CrossRef]
- Qu, X.; Wei, T.; Peng, C.; Du, P. A Fast Face Recognition System Based on Deep Learning. In Proceedings of the 2018 11th International Symposium on Computational Intelligence and Design (ISCID), Hangzhou, China, 8–9 December 2018; Volume 1, pp. 289–292. [Google Scholar] [CrossRef]
- Mahale, G.; Mahale, H.; Goel, A.; Nandy, S.K.; Bhattacharya, S.; Narayan, R. Hardware Solution for Real-Time Face Recognition. In Proceedings of the 2015 28th International Conference on VLSI Design, Bangalore, India, 3–7 January 2015; pp. 81–86. [Google Scholar] [CrossRef]
- Soto, J.E.; Figueroa, M. An embedded face-classification system for infrared images on an FPGA. In Proceedings of the Electro-Optical and Infrared Systems: Technology and Applications XI, Amsterdam, The Netherlands, 22–25 September 2014; Huckridge, D.A., Ebert, R., Eds.; International Society for Optics and Photonics, SPIE: Bellingham, WA, USA, 2014; Volume 9249, pp. 159–170. [Google Scholar]
- Hameed, R.; Qadeer, W.; Wachs, M.; Azizi, O.; Solomatnikov, A.; Lee, B.C.; Richardson, S.; Kozyrakis, C.; Horowitz, M. Understanding sources of inefficiency in general-purpose chips. In Proceedings of the 37th Annual International Symposium on Computer Architecture, Saint-Malo France, 19–23 June 2010; pp. 37–47. [Google Scholar]
- Shin, M.S.; Kim, J.B.; Kim, M.K.; Jo, Y.R.; Kwon, O.K. A 1.92-megapixel CMOS image sensor with column-parallel low-power and area-efficient SA-ADCs. IEEE Trans. Electron Devices 2012, 59, 1693–1700. [Google Scholar] [CrossRef]
- Gottardi, M.; Lecca, M. A 64×64 Pixel Vision Sensor for Local Binary Pattern Computation. IEEE Trans. Circuits Syst. I Regul. Pap. 2019, 66, 1831–1839. [Google Scholar] [CrossRef]
- Hermosilla Vigneau, G.; Verdugo, J.L.; Farias Castro, G.; Pizarro, F.; Vera, E. Thermal Face Recognition Under Temporal Variation Conditions. IEEE Access 2017, 5, 9663–9672. [Google Scholar] [CrossRef]
- Nixon, M.; Aguado, A. Feature Extraction and Image Processing for Computer Vision; Academic Press: Cambridge, MA, USA, 2019. [Google Scholar]
- Lahaw, Z.B.; Essaidani, D.; Seddik, H. Robust Face Recognition Approaches Using PCA, ICA, LDA Based on DWT, and SVM Algorithms. In Proceedings of the 2018 41st International Conference on Telecommunications and Signal Processing (TSP), Athens, Greece, 4–6 July 2018; pp. 1–5. [Google Scholar] [CrossRef]
- Shi, L.; Wang, X.; Shen, Y. Research on 3D face recognition method based on LBP and SVM. Optik 2020, 220, 165157. [Google Scholar] [CrossRef]
- Ayyad, M.; Khalid, C. New fusion of SVD and Relevance Weighted LDA for face recognition. Procedia Comput. Sci. 2019, 148, 380–388. [Google Scholar] [CrossRef]
- Bai, S.; Liu, Z.; Yao, C. Classify vehicles in traffic scene images with deformable part-based models. Mach. Vis. Appl. 2018, 29, 393–403. [Google Scholar] [CrossRef]
- Anwer, R.M.; Khan, F.S.; Laaksonen, J. Two-Stream Part-Based Deep Representation for Human Attribute Recognition. In Proceedings of the 2018 International Conference on Biometrics (ICB), Gold Coast, QLD, Australia, 20–23 February 2018; pp. 90–97. [Google Scholar] [CrossRef]
- Annalakshmi, M.; Roomi, S.M.M.; Naveedh, A.S. A hybrid technique for gender classification with SLBP and HOG features. Clust. Comput. 2019, 22, 11–20. [Google Scholar] [CrossRef]
- Islam, K.T.; Wijewickrema, S.; Raj, R.G.; O’Leary, S. Street Sign Recognition Using Histogram of Oriented Gradients and Artificial Neural Networks. J. Imaging 2019, 5, 44. [Google Scholar] [CrossRef] [Green Version]
- Cheng, D.; Tang, S.; Feng, C. Extended HOG-CLBC for pedstrain detection. Opto-Electron. Eng. 2018, 45, 180111. [Google Scholar]
- Gonzalez-Lopez, J.; Ventura, S.; Cano, A. Distributed nearest neighbor classification for large-scale multi-label data on spark. Future Gener. Comput. Syst. 2018, 87, 66–82. [Google Scholar] [CrossRef]
- Al-Dabagh, M.Z.N.; Alhabib, M.; Al-Mukhtar, F. Face recognition system based on kernel discriminant analysis, k-nearest neighbor and support vector machine. Int. J. Res. Eng. 2018, 5, 335–338. [Google Scholar] [CrossRef] [Green Version]
- Shankar, K.; Lakshmanaprabu, S.; Gupta, D.; Maseleno, A.; De Albuquerque, V.H.C. Optimal feature-based multi-kernel SVM approach for thyroid disease classification. J. Supercomput. 2020, 76, 1128–1143. [Google Scholar] [CrossRef]
- Almabdy, S.; Elrefaei, L. Deep convolutional neural network-based approaches for face recognition. Appl. Sci. 2019, 9, 4397. [Google Scholar] [CrossRef] [Green Version]
- Jain, D.K.; Shamsolmoali, P.; Sehdev, P. Extended deep neural network for facial emotion recognition. Pattern Recognit. Lett. 2019, 120, 69–74. [Google Scholar] [CrossRef]
- Ahonen, T.; Matas, J.; He, C.; Pietikäinen, M. Rotation invariant image description with local binary pattern histogram fourier features. In Scandinavian Conference on Image Analysis; Springer: New York City, NY, USA, 2009; pp. 61–70. [Google Scholar]
- Yohanes, B.W.; Airlangga, R.D.; Setyawan, I. Real Time Face Recognition Comparison using Fisherfaces and Local Binary Pattern. In Proceedings of the IEEE 2018 4th International Conference on Science and Technology (ICST), Yogyakarta, Indonesia, 7–8 August 2018; pp. 1–5. [Google Scholar]
- Eminoglu, S.; Isikhan, M.; Bayhan, N.; Gulden, M.A.; Incedere, O.S.; Soyer, S.T.; Kocak, S.; Yalcin, C.; Ustundag, M.C.B.; Turan, O.; et al. A 1280×1024-15 m CTIA ROIC for SWIR FPAs. In Proceedings of the IEEE Infrared Technology and Applications XLI; Andresen, B.F., Fulop, G.F., Hanson, C.M., Norton, P.R., Eds.; International Society for Optics and Photonics, SPIE: Bellingham, WA, USA, 2015; Volume 9451, pp. 218–230. [Google Scholar]
- Murari, K.; Etienne-Cummings, R.; Thakor, N.V.; Cauwenberghs, G. A CMOS In-Pixel CTIA High-Sensitivity Fluorescence Imager. IEEE Trans. Biomed. Circuits Syst. 2011, 5, 449–458. [Google Scholar] [CrossRef] [Green Version]
- Berkovich, A.; Castro, A.; Islam, M.; Choa, F.; Barrows, G.; Abshire, P. Dark current reduction by an adaptive CTIA photocircuit for room temperature SWIR sensing. In Proceedings of the 2017 IEEE International Symposium on Circuits and Systems (ISCAS), Baltimore, MD, USA, 28–31 May 2017; pp. 1–4. [Google Scholar] [CrossRef]
- Zhai, Y.; Ding, R. Design of a ROIC with high dynamic range for LWIR FPAs. In Infrared, Millimeter-Wave, and Terahertz Technologies III; Zhang, C., Zhang, X.C., Tani, M., Eds.; International Society for Optics and Photonics, SPIE: Bellingham, WA, USA, 2014; Volume 9275, pp. 160–167. [Google Scholar]
- Borniol, E.D.; Guellec, F.; Castelein, P.; Rouvié, A.; Robo, J.A.; Reverchon, J.L. High-performance 640 × 512 pixel hybrid InGaAs image sensor for night vision. In Infrared Technology and Applications XXXVIII; Andresen, B.F., Fulop, G.F., Norton, P.R., Eds.; International Society for Optics and Photonics, SPIE: Bellingham, WA, USA, 2012; Volume 8353, pp. 88–95. [Google Scholar]
- Blerkom, D.A.V. Analysis and simulation of CTIA-based pixel reset noise. In Infrared Technology and Applications XXXVII; Andresen, B.F., Fulop, G.F., Norton, P.R., Eds.; International Society for Optics and Photonics, SPIE: Bellingham, WA, USA, 2011; Volume 8012, pp. 159–168. [Google Scholar]
- Zarkesh-Ha, P. An intelligent readout circuit for infrared multispectral remote sensing. In Proceedings of the 2014 IEEE 57th International Midwest Symposium on Circuits and Systems (MWSCAS), College Station, TX, USA, 3–6 August 2014; pp. 153–156. [Google Scholar]
- Berkovich, A.; Lecca, M.; Gasparini, L.; Abshire, P.A.; Gottardi, M. A 30 μW 30 fps 110 × 110 Pixels Vision Sensor Embedding Local Binary Patterns. IEEE J. Solid-State Circuits 2015, 50, 2138–2148. [Google Scholar] [CrossRef]
- Hermosilla, G.; Ruiz-del-Solar, J.; Verschae, R.; Correa, M. A comparative study of thermal face recognition methods in unconstrained environments. Pattern Recognit. 2012, 45, 2445–2459. [Google Scholar] [CrossRef]
- Li, S.Z.; Chu, R.; Liao, S.; Zhang, L. Illumination Invariant Face Recognition Using Near-Infrared Images. IEEE Trans. Pattern Anal. Mach. Intell. 2007, 29, 627–639. [Google Scholar] [CrossRef]
- Shoja Ghiass, R. Face Recognition Using Infrared Vision. Ph.D. Thesis, Université Laval, Quebec, QC, Canada, 2014. [Google Scholar]
- Shoja Ghiass, R.; Bendada, H.; Maldague, X. Université Laval Face Motion and Time-Lapse Video Database (UL-FMTV). In Proceedings of the 2018 International Conference on Quantitative InfraRed Thermography, Berlin, Germany, 25–29 June 2018; QIRT Council: Quebec City, QC, Canada, 2018. [Google Scholar] [CrossRef]
- Georghiades, A.; Belhumeur, P.; Kriegman, D. From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose. IEEE Trans. Pattern Anal. Mach. Intell. 2001, 23, 643–660. [Google Scholar] [CrossRef] [Green Version]
- Sun, Z.; Yu, Y. Fast Approximation for Sparse Coding with Applications to Object Recognition. Sensors 2021, 21, 1442. [Google Scholar] [CrossRef]
SIS | Technology | Pixel Pitch μm×μm | Fill Factor | Tested Spectrum | Type of Integrator |
---|---|---|---|---|---|
Proposed RLBP+LDA face recognition | 0.35 μm TMSC | 32×32 | 34% | Visible Thermal IR NIR | CTIA |
0.18 μm TMSC | 32×32 | 76% | Visible Thermal IR NIR | CTIA | |
Edge detection [34] | 0.18 μm 1P4M CMOS | 31×31 | 19% | Visible | 2T-integrator, CTIA at column level |
LBP Edge detection [35] | 0.18 μm CMOS | 7.9× 7.9 | 55% | Visible | 2T-integrator |
Spatial contrast LBP [77] | 0.35 μm | 26 × 26 | 23% | Visible | 2T-integrator |
4-neighbor LBP [52] | 0.35 μm CMOS | 64×64 | 15% | Visible | 2T-integrator |
Slice LUTs | Distributed Memory | Block RAMs | DSP Slices | |
---|---|---|---|---|
Total used | 4345 | 1298 | 2 | 6 |
Available | 53,200 | 17,400 | 140 | 220 |
Percentage | 8.2% | 7.6% | 1.4% | 2.7% |
Database | Spectrum | Image Size (Pixels) | Number of Subjects | Images per Subject | Face Positions and Conditions |
---|---|---|---|---|---|
UCH-TF | Thermal IR | 53 | 28 | Rotations and expressions | |
CBSR NIR | Near IR | 197 | 20 | Frontal, with and without glasses | |
UL-FMTV | Thermal IR | 238 | Short video sequence | Rotations | |
YaleFace B | Visible range | 38 | 64 | Frontal, expressions and light variations |
UCH-TF | CBSR NIR | UL-FMTV | YaleFaceB | |
---|---|---|---|---|
RLBP+LDA | 96.7% | 96.0% | 75–95.9% | 76.4% |
LBP+LDA | 98.5% | 98.1% | 79–97.1% | 82.9% |
Method | RLBP+LDA | LBP+LDA | GJD | WLD | LBP |
---|---|---|---|---|---|
Accuracy | 96.7% | 98.5% | 96.6% | 94.4% | 92.0% |
Method | RLBP+LDA | LBP+LDA | NIRFaceNet+Aug | NIRFaceNet | FaceNet |
---|---|---|---|---|---|
Accuracy | 96.0% | 98.2% | 96.6% | 94.8% | 84.1% |
Method | RLBP+LDA* | LBP+LDA* | Sun’s Kernel | CRC | ELM | Tanh |
---|---|---|---|---|---|---|
Accuracy | 76.4% | 82.9% | 98.33% | 96.82% | 96.44% | 96.34% |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Valenzuela, W.; Soto, J.E.; Zarkesh-Ha, P.; Figueroa, M. Face Recognition on a Smart Image Sensor Using Local Gradients. Sensors 2021, 21, 2901. https://doi.org/10.3390/s21092901
Valenzuela W, Soto JE, Zarkesh-Ha P, Figueroa M. Face Recognition on a Smart Image Sensor Using Local Gradients. Sensors. 2021; 21(9):2901. https://doi.org/10.3390/s21092901
Chicago/Turabian StyleValenzuela, Wladimir, Javier E. Soto, Payman Zarkesh-Ha, and Miguel Figueroa. 2021. "Face Recognition on a Smart Image Sensor Using Local Gradients" Sensors 21, no. 9: 2901. https://doi.org/10.3390/s21092901