Class-specific kernel linear regression classification for face recognition under low-resolution and illumination variation conditions

In this paper, a novel class-specific kernel linear regression classification is proposed for face recognition under very low-resolution and severe illumination variation conditions. Since the low-resolution problem coupled with illumination variations makes ill-posed data distribution, the nonlinear projection rendered by a kernel function would enhance the modeling capability of linear regression for the ill-posed data distribution. The explicit knowledge of the nonlinear mapping function can be avoided by using the kernel trick. To reduce nonlinear redundancy, the low-rank-r approximation is suggested to make the kernel projection be feasible for classification. With the proposed class-specific kernel projection combined with linear regression classification, the class label can be determined by calculating the minimum projection error. Experiments on 8 × 8 and 8 × 6 images down-sampled from extended Yale B, FERET, and AR facial databases revealed that the proposed algorithm outperforms the state-of-the-art methods under severe illumination variation and very low-resolution conditions.

Numerous studies [1] have been greatly proposed for face recognition recently. In realistic situations such as video surveillance, face recognition may encounter many great challenges, especially low-resolution problems, caused by the cameras at a distance. Additionally, the low-resolution (LR) problems might be coupled with other effects such as illumination variations. Therefore, it is desired to devise a face recognition method for both very low-resolution and illumination variation problems.

In the literature, numerous researches based on the subspace projection methods have been proposed to achieve successful face recognition. The principle component analysis (PCA) [2–6] and independent component analysis (ICA) [7–9] have been reported to be robust in noisy conditions. The linear discriminant analysis (LDA) [3–6] yields better results in clean conditions and lighting changes than the PCA. Moreover, the kernel variants such as kernel PCA (KPCA) [10–12] and kernel LDA (KLDA) [12–14] have been presented to achieve better performance by nonlinearly mapping the data from the original space to a very high-dimensional feature space, which is called the reproducing kernel Hilbert space (RKHS). Therefore, KPCA and KLDA by nonlinearly mapping could be improved from high-order statistics, whereas the PCA and LDA only utilize the first- and second-order statistics. Thus, for highly nonlinear data distribution, these kernel methods are more suitable for low-resolution and illumination variation conditions. Moreover, some geometrically motivated approaches such as locality preserving projection (LPP) [15] and neighboring preserving embedding (NPE) [16] have been shown effectiveness for face recognition.

Recently, the spare representation classification (SRC) [17, 18] and a linear regression classification (LRC) algorithms [19] have been proposed for face recognition. Although SRC-based approaches perform very well in many situations, the execution time in SRC-based approaches is more than that in LRC-based approaches. For pursuing the accuracy and speed, the LRC would be a good choice for further investigation. The LRC is based on that face images from a specific class are known to lie on a class-specific linear subspace [3, 20]. The regression coefficients can be estimated by using the least-square method, and the decision can be determined by the minimum reconstruction error. Experiments reported have shown that the down-sampled low-resolution face image could be used for face classification directly. However, as the results reported, the LRC could not withstand severe illumination variations. In addition, a robust linear regression classification algorithm (RLRC) [21] has been introduced to address the problem of robust face recognition. However, low-resolution problems did not be addressed.

The performances achieved by the existing methods, such as PCA, LDA, LPP, and NPE methods, decrease under low-resolution condition because of the loss of high-frequency information [22]. Boom et al. in [23] showed that low-resolution face images below 32 × 32 pixels degraded the performances of the PCA and LDA seriously. In [24], the face images with 20 × 20 and 10 × 10 resolution dramatically deteriorated the recognition performance compared with those with 40 × 40 pixels in video-based face recognition systems. In [25], face resolution below 36 × 48 reduced the expression recognition performance.

To overcome the problem of low-resolution face recognition, several works have been presented to resolve this problem by using the super-resolution (SR) method [26–30]. One is to train relationship between the low-resolution (LR) and its corresponding high-resolution (HR) face images [29]. The other uses the canonical correlation analysis to compute the coherent features between the LR and its HR face images [30]. Here, we can observe that the LR and HR image pairs are needed for SR methods.

1.1 Problem statements

In realistic situations, it may be a case that only the LR face images are available in training set for identifying criminals, so it is imperative to overcome the problem that some training individuals may not be the same as gallery. In other words, the HR face images for specific persons are not available for modeling the relationship and computing the similarity in SR approaches [26–30]. Thus, how to perform LR face recognition directly without the HR information is a critical and practical topic.

To conquer the illumination variation problems, several approaches [3, 31–35] have been proposed. For instance, numerous preprocessing methods, such as the histogram equalization, gamma correction, and logarithm transform, are widely used for the illumination normalization. Other methods including the gradient operation, Gabor filters, and LDA-based approaches are well-known illumination invariant methods. However, these methods would fail because the important features in high-frequency details for face recognition are lost under the LR problems.

1.2 Contributions

We propose a novel face recognition algorithm to improve the limitation of the LRC [15] by embedding the kernel method into the linear regression. The key of the proposed method is to apply a nonlinear mapping function to twist the original space into a higher dimensional feature space for better linear regression. Moreover, in order to make the proposed kernel projection feasible, a constrained low-rank approximation [36–38] is proposed to obtain low-rank-r singular value approximation. The low-rank approximation is a rank reduction method which minimizes the difference between a given matrix and an approximation matrix. Simulations carried on the extended Yale B, FERET, and AR facial databases reveal that the proposed kernel linear regression classification (KLRC) can achieve good performance for LR face recognition under variable lighting changes without any preprocessing. At the same time, the proposed algorithm can reconstruct the very low-resolution face image under illumination variations with high quality measured by quality assessments.

1.3 Paper outline

The rest of this paper is organized as follows. Section 2 reviews the LRC approach and presents the motivations. Section 3 formulates the proposed KLRC method with a constrained low-rank approximation algorithm. Section 4 shows the comparisons with the related work. Section 5 gives experimental results. Finally, we draw conclusions in Section 6.

2.1 Linear regression classification (LRC)

Assume we have N subjects with p _i training images from the ith class, i = 1, 2,…,N. Each gray scale training image is in size of a × b pixels and is represented as v _i,j  ∈ ℜ ^a × b, i = 1, 2,…,N and j = 1, 2,…,p _i . Then, each training image is transformed to a column vector as w _i,j  ∈ ℜ ^q × 1, where q = a × b. For applying the linear regression to estimate the class-specific model, we stack all column vectors w _i,j regarding the class-membership. Hence, for the ith class, we have

where each vector w _i,j is a column vector of W _i . Thus, in the training phase, the ith class is represented by a vector space W _i , which is called the regressor for each subject.

If y belongs to the ith class, it can be represented as a linear combination of the training images from the ith class and can be defined as

where \( {\boldsymbol{\beta}}_i\in {\Re}^{p_i\times 1} \) is the vector of regression parameters and e is an error vector whose elements are independent random variables with zero mean and variance σ ². The goal of the regression is to find \( {\tilde{\boldsymbol{\beta}}}_i \), which minimizes the residual errors as

The regression coefficients can be solved through the least-square estimation and can be written as a matrix form as

The vector of estimated parameters, \( {\tilde{\boldsymbol{\beta}}}_i \), and the predictor, W _i , are used to predict the response vector ỹ _i for the ith class as

By substituting (4) for \( {\tilde{\boldsymbol{\beta}}}_i \) in (5), the optimal prediction in the least-square sense becomes

Theoretically, we can treat the above equation as a class-specific projection as [39],

where ỹ _i is the projection of y onto the subspace of the ith class by the projection matrix, \( {\boldsymbol{H}}_i={\boldsymbol{W}}_i{\left({\boldsymbol{W}}_i^T{\boldsymbol{W}}_i\right)}^{-1}{\boldsymbol{W}}_i^T \). It is noted that the projection matrix is a symmetric and idempotent matrix.

The LRC is developed based on the minimum reconstruction error. In other words, if the original vector belongs to the subspace of class i, the predicted response vector ỹ _i will be the closest vector to the original vector. The identity i* could be determined by calculating the Euclidean distance measure between the predicted response vectors and the original vector as

2.2 Motivations

The LRC has been developed based on the concept that samples from a specific person are known to lie on a class-specific linear subspace and demonstrated that it could achieve good performance for the low-resolution face images, but not good for severe illumination variations. This is because illumination variations make the data distribution more complicated. So the face images captured under variable lighting conditions may cause the linear subspace approaches inappropriate. In other words, the linear subspace methods would fail when they violate the Lambertian assumption regarding the illumination problem. Especially, when the low-resolution problem is coupled with illumination variations, the linear subspace methods, such as the PCA and LDA, and the linear regression classification (LRC) could not counteract the problem. In this paper, a kernel linear regression classification (KLRC) with a constrained low-rank approximation is proposed for low-resolution face recognition under illumination variations. The KLRC with the nonlinear mapping function can evaluate the LRC in the higher dimensional feature space and can achieve good results.

Assume the original input space can be always mapped to some higher dimensional feature space where the data set is distributed linearly. As shown in Fig. 1, the left figure shows that it is difficult to fit the data by a regression line because of nonlinear data distribution, whereas the right figure shows that it is easy to fit the data by a regression plane because of data distribution linearly by a mapping function from R ² to R ³. Thus, it can be expected that a nonlinear mapping prior to linear regression could improve the limitation of the LRC under severe illumination variations. In order to formulate a general equation and solve the problem systematically, we will discuss it in details later. Here, we first introduce the kernel linear regression classification (KLRC) method. The KLRC is also developed based on the theory that samples from a specific class are known to lie on a linear subspace by a nonlinear mapping. The key is to apply a nonlinear mapping function to the input space and then evaluate it by the LRC in the higher dimensional feature space. The dimension of the resulting feature space could be very large. Fortunately, this explicit knowledge of the nonlinear mapping function can be computationally avoided by using the kernel trick [40].

Illustration of a mapping from R ² to R ³ . The left figure shows it is difficult to fit the data by a regression line because of nonlinear data distribution, whereas the right figure shows it can fit the data by a regression plane because of linear data distribution in the higher dimensional space

Full size image

For the following theoretical derivation, a vector space should be defined as

where each vector z _i,j is a row vector of Z _i . Specifically, each row vector in Z _i is projected from the original space \( {\Re}^{p_i} \) to a high-dimensional space ℜ ^f by a nonlinear mapping function \( \begin{array}{cc}\hfill \boldsymbol{\varPhi} \left({\boldsymbol{z}}_{i,j}\right):{\Re}^{p_i}\to {\Re}^f,\hfill & \hfill f>{p}_i\hfill \end{array} \). Therefore, now, ℜ ^f is the space spanned by Φ(z _i,j ). The projected row vectors can be used for linear regression as

Because of the increase in dimensionality, the mapping function Φ(z _i,j ) is made implicitly by using the kernel function satisfying Mercer’s theorem. Furthermore, by using the dual representation β _i  = Φ(Z _i )^T α _i , the linear regression stated in (10) becomes

where the kernel matrix K _i is positive semi-definite. Typically, kernel functions include the polynomial kernel and Gaussian kernel, which satisfy Mercer’s theorem.

We first perform singular value decomposition (SVD) on the kernel matrix K _i as

where U and V ^T are left and right SVD orthonormal matrices and S = diag{λ ₁, λ ₂, …, λ _g } is an rectangular diagonal matrix with the descend-sorted singular values on the diagonal with λ ₁ ≥ … ≥ λ _k  ≥ … ≥ λ _g  ≥ 0. To achieve a robust estimation, we propose a constrained rank-r approximation of K _i defined as

where

by discarding (g − r) least SVD components. The number of principle SVD components, r, is determined by

where μ is a selected control factor, λ _g is the smallest singular value, and \( {\lambda}_{{}_{\mathrm{median}}} \), which is the median of all singular values, is expressed as

After the construction of the constrained low-rank approximation stated in (13), we could obtain a kernel linear regression model for the ith class as

Then, the kernel linear regression aims to minimize the residual errors as

The above solution can be also solved by the least-square estimation since it has the same form as stated in (2). After the low-rank approximation, we can use the pseudo-inverse of \( {\boldsymbol{K}}_i^r \) to obtain the least-square solution as

where the pseudo-inverse of \( {\boldsymbol{K}}_i^r \) is expressed by

with

Since \( {\boldsymbol{K}}_i^r{\left({\boldsymbol{K}}_i^r\right)}^{-}\ne \boldsymbol{I} \), it will be feasible to compute the minimum reconstruction error between the original vector and projected vector for determining the classification results.

In the classification phase, the response vector ỹ _i for the ith class can be predicted by

By substituting (19) in (22), we can obtain

and obtain a class-specific kernel projection matrix as

where ỹ _i is the projection of y onto the kernel subspace of the ith class by the class-specific kernel projection matrix, P _i . It is noted that \( {\boldsymbol{K}}_i^r{\left({\boldsymbol{K}}_i^r\right)}^{-}\ne \boldsymbol{I} \) is necessary for the KLRC computation.

The KLRC is also developed based on the minimum reconstruction error. So in the recognition phase, the identity i* could be determined by calculating the Euclidean distance measure between the predicted response vectors and the original vector as

4.1 Analysis of the regression parameter

To simplify the analysis, we assume that W _i is a square matrix. We have W _i  = U _i D _i V _i ^T by SVD with U _i ^T U _i  = I, because W _i is a square matrix, U _i ^T = U _i ^− 1, and U _i U _i ^T = I. Similarly, V _i V _i ^T = V _i ^T V _i  = I. In addition, the linear kernel k(z _ij , z _ij ) = < z _ij , z _ij  > = z _ij z _ij ^T is used in the KLRC for the theoretical analysis below.

4.1.1 LRC

The goal of the LRC is to find \( {\tilde{\boldsymbol{\beta}}}_i \), which minimizes the residual errors. Statistically, the linear regression model is an unbiased estimate. Also, the variance of the regression parameter vector \( {\tilde{\boldsymbol{\beta}}}_i \) in the linear regression model is expressed as

where W _i  = U _i D _i V _i ^T by SVD, v _ij is the jth column eigenvector of V _i , and d _ij is the jth eigenvalue corresponding to the v _ij .

4.1.2 KLRC

The goal of the KLRC is to find \( {\tilde{\boldsymbol{\alpha}}}_i \), which minimizes the residual errors. Statistically, the kernel linear regression model is also an unbiased estimator since the kernel linear regression model in (17) has the same form as in (2). On the other side, the variance of the regression parameter vector in the kernel linear regression model is expressed as

where r < J, \( {\boldsymbol{W}}_i^r={\boldsymbol{U}}_i^r{\boldsymbol{D}}_i^r{{\boldsymbol{V}}_i^r}^T \) by the SVD, and d _ij is the jth eigenvalue corresponding to the jth eigenvector. Compared (26) with (27), the variance of the regression parameter vector in the KLRC is smaller than that in the LRC. Therefore, it can be expected that the KLRC can provide more reliable regression parameter to the regression model for classification.

For verifications, we examine the proposed algorithms on the facial images, which are down-sampled from the extended Yale B (EYB) [41], AR [42], and FERET [43] face databases. In the experiments, we evaluate the proposed method against low-resolution problems coupled with illumination variations. In this section, all experimental results report the top 1 recognition accuracy (%).

The experiments are designed to evaluate the effectiveness of the proposed method in coping with unseen lighting changes under the LR condition. In the experiments, we compare the proposed methods, KLRC-p and KLRC-g, with the PCA+Euclidean, PCA+Mahalanobis [44], KPCA-p, KPCA-g, LDA+Euclidean, LDA+Mahalanobis, KLDA-p, KLDA-g, LRC, RLRC, SRC, LPP, NPE, improved principal component regression (IPRC) [45], unitary regression classification (URC) [46], linear discriminant regression classification (LDRC) [47], and local binary pattern (LBP) [48], where p and g denote the polynomial kernel and Gaussian kernel, respectively. The PCA-based and LDA-based approaches utilize 85 % dimensionality for experiments. It should be noted that this paper assumes the corresponding HR face images for the LR face images are not available as stated in Section 1.1. Hence, the existing face recognition systems with the SR method are not suitable for this problem.

5.1 Experiments on EYB

The EYB contains images of 38 subjects with 9 poses and 64 illuminations per pose. The frontal face images of all subjects with 64 different illuminations are used for evaluation. The EYB is divided into five subsets based on the angle of the light source directions. As a result, there are total 2432 images: 266 (7 images per person), 456 (12 images per person), 456 (12 images per person), 532 (14 images per person), and 722 (19 images per person) images in subsets 1 to 5, respectively. All images are cropped, lowpassed, and resized to low-resolution images in size of 8 × 8 pixels, as shown in Fig. 2. Subset 1 is conducted for training, and the remaining subsets (subsets 2 to 5) are used for testing.

10 samples with the low-resolution problem which are obtained from five subsets of two persons from the EYB face database

Full size image

First of all, we investigate the performances under different image resolutions. Figure 3 shows the average recognition rate over the remaining subsets. Results reveal that the proposed KLRC-p and KLRC-g outperform the other methods consistently. Low-resolution face images with 8 × 8 pixels degrade the performances significantly. Nonetheless, the proposed KLRC algorithms still could perform well under the very low-resolution condition. Also, the results show that the performances achieved by the PCA-based, LDA-based, and the other subspace projection methods are drastically reduced for resolution below 16 × 16 pixels. Moreover, it is interesting to point out that the KLRC-p and KLRC-q under 8 × 8 pixels outperform the PCA-based and LDA-based approaches under 32 × 32 pixels significantly and achieve comparable recognition rate by the LRC-based approaches under 32 × 32 pixels. LBP [48] is a local feature method which can effectively defense the illumination variations under 32 × 32 pixels. However, LBP cannot perform very well under very low-resolution situation since the facial information is not enough for local feature extraction. This success should be attributed to perform linear regression in the higher dimensional feature space.

The recognition rate under different image resolutions on the EYB face database

Full size image

Then, we further focus on the low-resolution images. As shown in Table 1, the KLRC-p and KLRC-g outperform the LRC, RLRC, IPCR, URC, LDRC, PCA+Euclidean, PCA+Mahalanobis, kernel-based PCA, LDA+Euclidean, LDA+Mahalanobis, kernel-based LDA, SRC, LPP, NPE, and LBP for the low-resolution face recognition under illumination variations. The PCA-based methods are the worst ones. Note that although it is widely accepted that the discriminant-based approaches offer higher robustness to lighting variations than the PCA-based approaches [3], the discriminant-based approaches still cannot withstand the low-resolution problem coupled with illumination variations in our work. It is because the low-resolution image contains insufficient high-frequency components containing the discriminative information for discriminant analysis. In [45], Huang et al. had shown that the regression-based method can perform better than the discriminant-based approach for face recognition under illumination variations. The proposed KLRC-p and KLRC-q have gained improvement significantly. In addition, although the RLRC performs better than the LRC, the RLRC could not obtain satisfactory performance because of performing regression in the original linear space.

5.2 Experiments on AR

For further verifications, we conducted experiments on AR face database. The AR database, built by Martinez and Benavente, totally contains 3510 mug shots of 135 subjects (76 males and 59 females) with different facial expressions, lighting changes, and partially occlusions. Each subject contains 26 images in two sessions. The first session, containing 13 images, includes the neutral expression, smile, anger, screaming, different lighting changes, and two realistic partial occlusions with lighting changes. The second session duplicates the first session in the same way 2 weeks later.

To evaluate the effectiveness of the proposed approach in coping with variable illuminations, only the face images with illumination variations were considered in the experiments. All color face images in the AR are converted to gray levels, cropped, and down-sampled to the size of 8 × 6 pixels. Note that no face alignment is done in the copped face images. As shown in Fig. 4, 120 subjects with 8 face images under illumination variations, including no lighting, left lighting, right lighting, and full lighting, are chosen for evaluation. Training is conducted on the images with no lighting, and the remaining lighting conditions (left lighting, right lighting, and full lighting) are used for testing.

Eight samples in size of 8 × 6 pixels from the lighting subset of one person from AR database

Full size image

The experimental results are tabulated in Table 2, which reflects that the proposed KLRC-p and KLRC-g can attain higher recognition rate than the LRC, RLRC, IPCR, URC, LDRC, PCA+Euclidean, PCA+Mahalanobis, kernel-based PCA, LDA+Euclidean, LDA+Mahalanobis, kernel-based LDA, NPP, NPE, and LBP for low-resolution face recognition under illumination variations. From the experimental results, we can observe that SRC performs well as the proposed KLRC, especially under low ill-posed situation. However, the execution time in SRC-based approaches generally is more than that in LRC-based approaches [49]. The discriminant-based approaches, LRC, RLRC, IPCR, URC, and LDRC, could not work well for low-resolution face recognition under illumination variations.

5.3 Experiments on FERET

We further conduct experiments on the FERET face database to evaluate the performance on the illumination and expression variations since the different facial expression is inevitable to face recognition. The FERET includes 250 people with four frontal view images from each subject. These 1000 face images with illumination and expression variations are resized to 8 × 6 pixels. Two images of each person are randomly selected for training, and the other two images are for testing. The experimental results are tabulated in Table 3, which has reported that the KLRC-p and KLRC-g can achieve higher recognition rate than the LRC, RLRC, IPCR, URC, LDRC, PCA+Euclidean, PCA+Mahalanobis, kernel-based PCA, LDA+Euclidean, LDA+Mahalanobis, kernel-based LDA, SRC, NPP, NPE, and LBP. As the results shown, we can observe that the KLRC-p and KLRC-g can work well for low-resolution problems with illumination and expression variations. It is reasonable because the LR face image will lose facial expression information [25]. On the other hand, in FERET, the lighting variations are slight, so the improvement is limited.

In this paper, the statistical analyses and experiment results verified that the proposed class-specific kernel linear regression classification performs the best for the low-resolution face recognition under illumination variations. With the kernel trick, the nonlinear and increased-dimension mapping function enhances the modeling capability for low-resolution and illumination variations. Furthermore, the constrained low-rank approximation has been proposed to perform low-rank approximation automatically to make the kernel projection feasible for classification. The comparisons with the state-of-the-art methods indicate a comparable performance for the proposed KLRC-p and KLRC-g. We have demonstrated that the proposed KLRC-p and KLRC-g perform better than the PCA+Euclidean, PCA+Mahalanobis, KPCA-p, KPCA-g, LDA+Euclidean, LDA+Mahalanobis, KLDA-p, KLDA-g, LRC, RLRC, SRC, LPP, NPE, IPRC, URC, LDRC, and LBP for low-resolution face recognition under variable lighting. In summary, the KLRC-p and KLRC-g dramatically improve the LRC to possess good robustness for very low-resolution face recognition under severe illumination variations.

This work was supported in part by National Science Council of Taiwan, under Grant NSC 103-2221-E-006-109 -MY3.

Authors and Affiliations

1. Institute of Computer and Communication EngineeringDepartment of Electrical Engineering, National Cheng Kung University, Tainan, Taiwan

   Yang-Ting Chou, Shih-Ming Huang & Jar-Ferr Yang

Corresponding author

Correspondence to Jar-Ferr Yang.

Competing interests

The authors declare that they have no competing interests.

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions