Factor Analysis and Structural Equations Modelling: Statistics For Psychology
Factor Analysis and Structural Equations Modelling: Statistics For Psychology
Factor Analysis and Structural Equations Modelling: Statistics For Psychology
10
Factor
Analysis
and
Structural
Equa9ons
Modeling
Outline
Model
Data
and
Rota9on
Interpret
Input
to
Number
Chose
Collect
Data
Extrac9on
of
Factors
Interpret
on
Large
FA
Method
Rota9on
rotated
Determine
Method
Factor
Number
of
Correla9on
Select
Number
of
(Orthogonal
Variables
Matrix
or
Method
of
or
Oblique)
PaJern/
(k)
and
on
Factors
for
Covariance
Extrac9on:
and
Rotate
Factor
Large
Rota9on
(r
Matrix
PCA
or
Selected
Structure
Sample
<
k)
Common
Factors
matrix
Factor
Factor
Analysis
Jargon
Correla9on
matrix
(or
covariance
matrix)
Extrac9on:
Process
of
transforming
variance
from
a
correla9on
matrix
into
factors
or
components.
Communali9es:
Es9mates
of
shared
variance.
Rota9on:
Transforming
selected
factors
to
make
them
interpretable.
Factor
PaUern
matrix:
Variables
by
factors
matrix;
elements
are
loadings
(weights).
Factor
Structure
Matrix:
Variables
by
factors
matrix;
elements
are
correla9ons.
Phi
matrix:
Inter-factor
correla9ons
matrix.
Purpose
of
Exploratory
Factor
Analysis
Understanding
the
number
of
factors
that
can
describe
the
exis9ng
correla9on
matrix.
Understanding
the
theore9cal
meaning
of
those
factors.
The
EFA
can
be
carried
out
to
evaluate
the
theory
(CFA
is
a
beJer
op9on
than
EFA).
EFA
factors
can
be
saved
and
used
as
variables
in
analyses.
Specic
Issues
in
EFA
Sample
size
Normality,
linearity,
outliers,
and
mul9collinearity
Factorability
of
correla9on
matrix
BartleJs
test
2k + 5
2 = n 1 ln R
6
KaiserMeyerOlkin
(KMO)
n
rij2
i j i
KMO = n n
r
i j i
2
ij + aij2
i j i
Basic
Equa9ons
PCA
Model
z j = a j1F1 + a j 2 F2 + + a jk Fk for, j = 1, 2,..., k
where,
z j is standardized form of variable j
F1,F2 ,...,Fk are k unrelated components such that each component one after
another that makes maximum contribution to variance of k variables;
a j1 is a weight for the variable j for the factor F1, and so on
Basic
Equa9ons
Common
Factor
MZodel
z j = a j1F1 + a j 2 F2 + + a jm Fm + d jU j for, j = 1, 2,..., k
where,
m < k that is common factors,
U j is unique j factors,
d j is weight associated with U j
Variance
Component
If
factors
are
uncorrelated,
then
S j = 1 = a j1 + a j 2 + + a jm + d j
2 2 2 2 2
Communality
of
a
variable
h 2j = a 2j1 + a 2j 2 + + a 2jm for, j = 1, 2,..., k
Unique
factors
can
be
decomposed
into
z = a F + a F + + a F + b S + e E for, j = 1, 2,..., k
j j1 1 j2 2 jm m j j j j
S = 1 = a + a + + a + b + e
2
j
2
j1
2
j2
2
jm
2
j
2
j
S =1= h +b + e
2
j
2
j
2
j
2
j
rj = h 2j + b 2j
Matrix
Equa9ons
for
EFA
R kk correlation matrix among the variables
obtained from Z nk data matrix
L = V RV
Vkk eigenvectors
L principal diagonal of L are the eigenvalues
factor pattern is
A=V L
Extrac9on
Models
Component
Model
o Keeps
uni9es
in
the
diagonal
of
correla9on
matrix
o Maximum
variance
Common
Factor
Model
o Uses
some
es9mate
of
common
factors
in
the
diagonal
of
correla9on
matrix.
o Only
common
variance
is
condensed.
o Common
factors
are
those
which
contribute
to
two
or
more
variables.
o Es9mates
of
communality
are:
(i)
highest
correla9on,
(ii)
reliability
coecients,
(iii)
squared
mul9ple
correla9on
(SMC),
(iv)
itera9ve
methods.
Methods
of
Extrac9on
Principal
Component
Analysis
(PCA)
Extract
maximum
variance
HoJeling
(1933)
Full
model
is
seldom
used.
Truncated
component
solu9on
is
usually
used.
Accounts
for
maximum
available
variance.
Orthogonal
(uncorrelated,
independent)
with
all
prior
factors.
Full
solu9on
(as
many
factors
as
variables),
that
is,
accounts
for
all
the
variance.
Methods
of
Extrac9on
Principal
Axis
Factoring
(PAF)
Common
factor
model
Uses
an
es9mate
of
communali9es.
PAF
extracts
smaller
variance
than
PCA.
SMC
is
used
as
star9ng
values
and
communali9es
are
es9mated
using
an
itera9ve
process.
Factors
are
latent
variables
and
have
causal
meaning
associated
with
them.
Methods
of
Extrac9on
Maximum
Likelihood
Extrac9on
method
based
on
sta9s9cal
considera9ons.
Lawley
and
Maxwell,
and
further
rened
by
Jreskog
(1967).
Logic:
Es9mate
popula9on
values
of
factor
loading
that
maximizes
the
likelihood
of
obtaining
the
sample
correla9on
matrix
from
popula9on.
Chi-square
sta9s9cs
used
to
assess
goodness-of-t
of
a
model
to
the
data
ML
extrac9ons
are
over-factoriza9on.
Methods
of
Extrac9on
Minimum
Residual
Analysis
(MinRes)
MinRes
minimizes
the
residuals
sum-of-squares
o-diagonal
loadings
are
smaller
Unweighted
Least
Squares
Minimizes
the
squared
dierence
between
the
observed
correla9on
matrix
and
the
reproduced
correla9on
matrix
Generalized
(weighted)
Least
Squares
GLS
uses
weights
for
variables
while
compu9ng
dierence
Image
Factoring
Distributes
the
variance
of
the
variable
reected
by
other
variables
into
factors
Alpha
Factoring
Alpha
reliability
of
the
factors
is
maximized
Number
of
Factors
Full
solu9on
not
useful
Truncated
solu9on
required
First
few
components/factors
are
retained
Eigenvalues
Percentage
of
variance
Eigenvalue
Percentage of variance = 100
k
How
to
decide
number
of
factors?
Methods
to
Deciding
Number
of
Factors
GuJmanns
eigenvalue
above
one
criteria
Scree
plot
Parallel
analysis
and
eigenvalue
larger
than
Monte
Carlo
eigenvalues
Percentage
of
variance
The
sta9s9cal
test
Use
of
guiding
theory
Interpretability
of
dierent
solu9on
Why
is
Rota9on
Required?
Theore9cal
interpretability
What
is
Rota9on?
Figure'10.3'A:'The'Unrotated'Solution' Figure'10.3'B:'Graphically'rotated'
Solution'
' '
'
Types
of
Rota9on
Two
Types:
Orthogonal
and
Oblique
Orthogonal:
Uncorrelated
Factors
Oblique:
Correlated
Factors
Orthogonal
Methods
Quar9max
Varimax
Transvarimax
Equamax
Parsimax
Oblique
Rota9ons
Oblimin
Oblimax,
Promax
Factor
Structure
and
Factor
PaUern
The
factor
paJern
matrix
(A)
is
a
k
!
r
matrix.
=
C = A
Factor
Scores
B = R -1A
F = ZB
Conrmatory
Factor
Analysis
CFA
is
a
special
case
of
structural
equa9ons
model
(SEM).
SEM
has
two
types
of
models:
one,
measurement
model
and
two,
full
SEM
model.
CFA
is
a
measurement
model
of
SEM
Falsica9on
of
a
hypothesis
Tests
of
hypothesis
CFA
in
that
sense
is
a
scien9c
method
Conrmatory
Factor
Analysis
Basic
Idea
Measured
variable
(also
called
as
observed
variables,
manifest
variables
or
indicators)
are
imperfect
indicators
of
latent
variables.
Latent
variable
is
an
unobservable,
underlying
construct
that
causes
the
observable
and
measured
variables
(also
called
as
manifest
variables).
Observed
variable
is
known
as
the
imperfect
indicator
of
the
latent
variables.
For
example,
the
intelligence
test
score
(observed)
is
the
func9on
of
ability
intelligence
(latent).
Observed
Variable
=
Latent
Variable
+
Random
Error
Steps
in
CFA
(i)
Have
theory
(ii)
Get
data
(iii)
Specify
model
(iv)
Test
for
iden9ca9on
(v)
Es9mate
model
parameters
(vi)
Sta9s9cal
test
and
Fit
indices
(vii)
Compare
dierent
models
(viii)
Interpret
and
conclude
The
Graphical
Model
Latent
Varibale
Observed
Variable
Error
Meaning
of
the
CFA
Symbols
Sign% Description%% Represents%
! Circle! Latent!variable!(unobservable,!not!
! measured)!
!
! Square!! Observed!variable!(manifest!variable!or!
! Indicator).!Directly!measured.!
!
! Straight!Arrow! Causal!Direction!or!Influence!path!
! Curved!double! Correlation/Covariance!between!
! headed!arrow! exogenous!(latent)!variables!or!errors.!!
!
!
CFA
Model
X = +
X is observed variables
matrix of structural coefficients associated with each
of the manifest variables for the .
latent variables
error associated with each of the manifest variables
Matrix
representa9on
X1 11 0 1
X2 21 0 2
X3 31 0 1 3
= +
X4 0 42 2 4
X5 0 52 5
X6 0 62 6
X 61 = 62 21 + 61
Assump9ons
(A)$ E ( X ) = E ( ) = 0 .$The$mean$of$the$observed$and$latent$variables$is$zero.$$
(B)$The$relationships$between$the$observed$ ( X ) and$latent$variables$ ( ) is$$linear.$$
(C)$There$are$assumptions$about$measurement$error:$$
$ (i)$ E ( ) = 0 .$The$errors$have$mean$zero.$
$ (ii)$The$errors$have$constant$variance$across$observations$
$ (iii)$The$errors$are$independent$across$observations$
(iv)$ E ( ) = E ( ) = 0 $.$The$errors$are$uncorrelated$with$latent$variables$
$
Unrestricted
and
Model-implied
Covariance
Matrix
Sample
variance
covariance
matrix
(S)
Popula9on
variancecovariance
matrix
of
the
observed
variables
().
Also
called
as
unrestricted
variancecovariance
matrix
= (XX )
( ) = XX
( ) = ( + )( + )
( ) = ( ) + E ( ) + E ( ) + E ( )
( ) = ( ) + E ( )
Iden9ca9on
Iden9ca9on
problem
for
the
CFA
model
is
whether
unique
solu9on
exists
for
each
parameter.
Nonredundant
elements
of
correla9on
matrix
c = k ( k + 1) 2
Necessary
but
not
a
sucient
condi9on
for
iden9ca9on
is
that
the
number
of
parameters
es9mated,
p,
are
smaller
than
or
equal
to
c.
Bollen
(1989)
discussed
three
indicators
rule
and
two
indicators
rule
for
the
iden9ca9on
as
a
sucient
condi9on.
Es9ma9on
( ) model implied variance covariance matrix
The maximum likelihoodestimation ( ML )
FML = log ( ) + tr S -1 ( ) log S p
Unweighted Least Squares (ULS )
FULS
1
{
= tr S ( )
2
2
}
Generalized Least Squares ( GLS )
FGLS
1
{
= tr I ( ) S
2
-1 2
}
Model
Evalua9on
Evalua9on
of
Null
hypothesis
H 0 : = ( )
H A : ( )
2 = ( n 1) F S ,
( )
Chi-square
test
is
a
very
sensi9ve
test.
Signicance
of
chi-square
is
not
used.
So
t
measures
are
used.
Fit
Measures
Goodness-of-Fit
(GFI)
and
Adjusted
Goodness-of-
Fit
Index
(AGFI)
(
tr S I
) ()
2
1
F S ,
GFI = 1 = 1
F S , ( 0 )
tr S
( )
2
1
c
AGFI = 1 (1 GFI )
df h
Normed
Fit
Index
(NFI)
and
Non-normed
Fit
Index
(NNFI)
i2 h2 Fh
NFI = = 1
i2
Fi
NNFI =
( i
2
dfi ) ( df h )
2
h
dfi 1
i
2
Compara9ve
Fit
Index
(CFI)
lh
FI = 1
li
l1
CFI = 1
l2
Parsimonious
Fit
Indices
df h
PGFI = GFI
df n
df h
PNFI = NFI
dfi
Comparing
Models
AIC
( )
LR = 2 log L R log L U
( )
Likelihood
Ra9o
Test
(LR)
LR = R U
2 2
X = X + !
Steps
The
Model
Specica9on
Iden9ca9on
Obtaining Data