Review article on PCA and LDA

Abstract—Machine learning has been a fast growing field reason is that LDA has the small sample size problem in which
with various algorithms being developed to solved real-world dataset selected should have larger samples per class for good
cases. In this paper an attempt is made to review a wide range of discriminating features extraction.Thus implementing LDA
methods used for face recognition comprehensively. This include directly resulted in poor extraction of discriminating features.
PCA, LDA, ICA, SVM, Gabor wavelet soft computing tool like In the proposed method Gabor filter is used to filter frontal face
ANN for recognition and various hybrid combination of this images and PCA is used to reduce the dimension of filtered
techniques. This review investigates all these methods with feature vectors and then LDA is used for feature extraction.
parameters that challenges face recognition like illumination,
The performances of appearance based statistical methods such
pose variation, facial expressions.
as PCA,LDA and ICA are tested and compared for the
recognition of colored faces images in. PCA is better than LDA
and ICA under different illumination variations but LDA is
better than ICA. LDA is more sensitive than PCA and ICA on
I. INTRODUCTION partial occlusions, but PCA is less sensitive to partial
Face recognition is an important part of the capability of occlusions compared to LDA and ICA. PCA is used as a
human perception system and is a routine task for humans, dimension reduction technique in and for modeling expression
while building a similar computational model of face deformations in .PCA can outperform over many other
recognition. The computational model not only contribute to techniques when the size of database is small. Despite the good
theoretical insights but also to many practical applications like results of PCA, this technique has the disadvantage of being
automated crowd surveillance, access control, design of human computationally expensive and complex with the increase in
computer interface(HCI), content based image database database size, since all the pixels in the image are necessary to
management,criminal identification and so on. During the past obtain the representation used to match the input image with all
decades, face recognition has received increased attention and others in the database. Different dimensionality reduction
has advanced technically. Many commercial systems for still techniques such as PCA, Kernel PCA, LDA, Locality
face recognition are now available. Recently, significant preserving Projections and Neighborhood Preserving
research efforts have been focused on video-based face embedding were selected and applied in order to reduce the
modeling/tracking,recognition and system integration. New loss of classification performance due to changes in facial
databases have been created and evaluations of recognition appearance. The performance of recognition while using PCA
techniques using these databases have been carried out. Now, as well as LDA for dimensionality reduction seems to be equal
the face recognition has become one of the most active in terms of accuracy.But it was observed that LDA requires
applications of pattern recognition,image analysis and very long time for processing more number of multiple face
understanding. images even for small databases. In case of Locality Preserving
Projections (LPP) and NPE methods, the recognition rate was
II. FACE RECOGNITION ALGORITHMS very less if increasing number of face images were used as
compared to that of PCA and KPCA methods.
A. Principal Component Analysis (PCA)
B. Linear Discriminant Analysis (LDA)
PCA also known as Karhunen-Loeve method is one of the
popular methods for feature selection and dimension reduction. LDA is a powerful method for face recognition. It yields an
Recognition of human faces using PCA was first done by Turk effective representation that linearly transforms the original
and Pentland and reconstruction of human faces was done by data space into a low-dimensional feature space where the data
Kirby and Sirovich. The recognition method, known as is well separated. However, the within-class scatter matrix
eigenface method defines a feature space which reduces the (SW) becomes singular in face recognition and the classical
dimensionality of the original data space. This reduced data LDA cannot be solved which is the undersampled problem of
space is used for recognition. But poor discriminating power LDA (also known as small sample size problem). A subspace
within the class and large computation are the well known analysis method for face recognition called kernel discriminant
common problems in PCA method. This limitation is overcome locality preserving projections (MMDLPP) was proposed in
by Linear Discriminant Analysis (LDA). LDA is the most based on the analysis of LDA, LPP and kernel function. A non
dominant algorithms for feature selection in appearance based linear subspace which can not only preserves the local facial
methods.But many LDA based face recognition system first manifold structure but also emphasizes discriminant
used PCA to reduce dimensions and then LDA is used to information.
maximize the discriminating power of feature selection. The
 Combined with maximum margin criterion (MMC) a new determine the validity of this assumption, project the
method called maximizing margin and discriminant locality image onto the face space, and examine difference
preserving projections (MMDLPP) was proposed in to find the between the projected image and Γ. The image is
subspace that best discriminates different face change and projected by computing Φ = Γ − Ψ and projecting onto
preserving the intrinsic relations of the local neighbourhood in Φf = PM0 i=1 ωiui . The distance ﾤ 2 = ||Φ − Φf ||2
the same face class according to prior class label determines the distance between the face space. be the
information.The proposed method was compared with PCA as use of English units as identifiers in trade, such as
well as locality preserving projections (LPP) ORL ,YALE, “3.5-inch disk drive.”
YALEB face database and authors had shown that it provides a
better representation of class information and achieved better C. Location and Detection
recognition accuracy. Illumination adaptive linear discriminant The previous sections assume a centered face image that is
analysis (IALDA) was proposed in to solve illumination the same size as the faces in the training images. The system
variation problems in face recognition. The recognition cannot operate correctly if the faces are not in the same
accuracy of the suggested method (IALDA), far higher than location with approximately the same size. Turk et al. use
that of PCA method and LDA method. The recognition frame differencing to track motion against a static background.
accuracy of the suggested method was lower than that the The filtered image is thresholded to produce a number of
Logarithmic Total Variation (LTV) algorithm. motion blobs. Heuristics such as the small blob of the large
blob is a face and the face must all move contiguously can be
III. EIGENFACES FOR RECOGNITION used to determine the location of the head. The size of this
”Eigenfaces for Recognition” seeks to implement a system ’blob’ can be used to scale estimate the size of the face, to
capable of efficient, simple, and accurate face recognition in a perhaps scale it to fit the size of the faces in the face space.
constrained environment (such as a household or an office). Alternatively the size can be used in a multi-scale eigenfaces
The system does not depend on 3-D models or intuitive approach. If there are a number of ’blobs’ to select from, the
knowledge of the structure of the face (eyes, nose, mouth). face can be located by examining a fixed sized subregion and
Classification is instead performed using a linear combination determining the ’faceness’ of that region. If the distance ﾤ
of characteristic features (eigenfaces).Previous works cited by between the region and the face space can be used to analyze
Turk et al. fall into three major categories: feature based face the image for faces.
recognition, connectionist based face recognition and
geometric face recognition. Feature based recognition uses the D. Other Issues
position, size and relationship of facial features (eyes, nose, A significantly different background will
mouth) to perform face recognition. adversely affect recognition, as the algorithm
The connectionist approach recognizes faces using a cannot distinguish between face and background.
general 2-D pattern of the face. Geometric recognition models To reduce this problem, the authors use a 2
the 3-D image of the face for recognition. dimensional Gaussian centered at the face to reduce
the background intensity. The size of the face may
also play a major role in the recognition rate. The
A. Eigenfaces tracking algorithm gives an idea of the face size,
Eigenfaces seeks to answer this by using principal but may not always be correct. The solution may be
component analysis of the images of the faces. This analysis to train using different scale faces, then use the
reduces the dimensionality of the training set, leaving only eigenfaces at various scale to estimate size. Head
those features that are critical for face recognition.Eigenvectors tilt will also affect the recognition rate, face
and eigenvalues are computed on the covariance matrix of the symmetry measures are used to determine tilt and
training images. The M highest eigenvectors are kept. to rotate to a standardized orientation.
The mean of the training images (Γ1, Γ2, ...ΓM) is the ”average E. Experiments
face” Ψ = 1 M PM n=1 Γn. Each training image differs from
the average face by Φi = Γi − Ψ. Experiments were conducted using recognition under
various lighting, scale and orientation. The experimental
B. Recognition using Eigenfaces database is over 2500 face images of the 16 subjects using all
combination of 3 head orientations, 3 head sizes, and 3 lighting
The new image Γ is projected into the face space conditions. A six level Gaussian pyramid was constructed
using ωk = u T k (Γ − Ψ). The weights form a vector Ω resulting in resolutions from 512 x 512 to 16 x 16. 2 The first
T = [ω1, ω2, ..., ωM]. The euclidean distance : ﾤ 2 k = experiment sets an infinite threshold. The system is trained
||Ω − Ωk||2 measures the distance between the new using 1 image per class at a fixed orientation, size and lighting.
image and a class of faces k. Note that if there are All of the remaining images are assigned to different classes.
more than one examples of the face, these weights are This experiment shows 96% recognition over varied lighting,
averaged among all of the examples. If the distance 85% over varied orientation and 64% over varied size.
measure, ﾤ k is less then a threshold Θ ﾤ , the face is
assigned to recognized, and assigned to class k. This
threshold is assigned empirically. The distance function References
previously assumed that new image Γ is a face. To
 [2] (K. Etemad, R. Chellappa, Discriminant Analysis for Recognition of
Human Face Images, Journal of the Optical Society of America A, Vol.
14, No. 8, August 1997, pp. 1724-1733.
[1] M. Turk, A. Pentland, Eigenfaces for Recognition, Journal of Cognitive
Neurosicence, Vol. 3, No. 1, 1991, pp. 71-86.