Advanced Statistics W2: Matrix Algebra Basics

Why matrix?
Advanced Statistics
W2: Matrix Algebra Basics
This meeting introduces the basic elements of

matrix algebra used in multivariate statistics.
l
l
Titis Wijayanto
Industrial Engineering, UGM
Why matrices? Lots of data and calculation

Why algebra? Allows you to find unknowns.
To do matrix algebra you need to use the matrix manipulations youve
will lear, e.g.
l
l
l
l
Why matrix?
l
Matrix algebra is the fundamental notational technique used in

multiple correlation, MANOVA, factor analysis, structural
equation modeling (SEM), etc.
Although it is possible to use statistical programs without

understanding matrix algebra, it is much harder to do so.
Matrix algebra is a convenient notational system that allows us

to think about data at a higher (broader) level rather than data
point by data point.
Addition and subtraction

Multiplication
Division
Powers
e.g. Inner / outer

e.g. Inverse
e.g. Transpose
Notation and basic definitions:

Matrix
A matrix is a rectangular or square array of
numbers/variables arranged in rows and
columns.
a11 ,, a1n
a 21 ,, a 2n
A=
= {Aij}

am1 ,, amn

Column Vector

Row Vector
[m x 1] matrix
a1
a 2
a = = {ai}

am

Square Matrix
Same number of rows and columns
5 4 7
B = 3 6 1
2 1 3
[1 x n] matrix
a ' = [a1 a 2 , , an ] = {aj}

We use lowercase letter with to
indicate row vector as the transpose of
column vector

Equal Matrix
If A = [aij ]mn , B = [bij ]mn
Then A = B if and only if aij = bij 1 i m, 1 j n
Ex 1: (Equal matrix)
1 2
A=
3 4
a b
B=
c d
If A = B
Then a = 1, b = 2, c = 3, d = 4

Symmetric Matrix

Symmetric Matrix
Symmetric matrix:
Ex:
A square matrix A is symmetric if A = AT

Skew-symmetric matrix:
A square matrix A is skew-symmetric if AT = A
is a skew-symmetric, find a, b, c?
0 1 2
A = a 0 3
b c 0
0 a b
A = 1 0 c
2 3 0
Sol:
Ex:
Sol:
0 1 2
If A = a 0 3
b c 0
1 2 3
If A = a 4 5
b c 6
is symmetric, find a, b, c?
1 2 3
1 a b
T
A = a 4 5 A = 2 4 c
b c 6
3 5 6
A = AT a = 1, b = 2, c = 3
A = AT
a = 2, b = 3, c = 5

Diagonal matrix and trace
If a matrix contains zeros in all off diagonal positions, it
is said to be a diagonal matrix.
d1
0
A = diag (d1 , d 2 ,!, d n ) =
"
0
Trace:
If A = [aij ]nn
Then Tr ( A) = a11 + a22 + ! + ann
0
d2
"
0
0
! 0
M nn
# "
! d n

Identity Matrix
Square matrix with ones on the
diagonal and zeros elsewhere.
1
0
I =
0
0 0 0
1 0 0
0 1 0
0 0 1

Transpose Matrix
Rows become columns and columns become rows

Transpose Matrix
Properties of transposes:
a11 a 21 ,, am1
a12 a 22 ,, am 2
A' =

a1n a 2n ,, amn
(1) ( AT )T = A
(2) ( A + B)T = AT + BT
(3) (cA)T = c( AT )
(4) ( AB)T = BT AT
The transpose of a matrix A, denoted by A, is

obtained from A by interchanging rows and
columns.
Matrix operations
Matrix operations
Scalar multiplication:
Matrix addition:
If A = [aij ]mn , B = [bij ]mn

Then A + B = [aij ]mn + [bij ]mn = [aij + bij ]mn
Ex 2: (Matrix addition)
1 2 1 3 1 + 1 2 + 3 0 5
0 1 + 1 2 = 0 1 1 + 2 = 1 3
1 1
3 + 3 =

2 2
1 1 0
3 + 3 = 0

2 + 2 0
If A = [aij ]mn , c : scalar

Then cA = [ca ij ]mn
Matrix subtraction:
A B = A + (1) B
Ex 3: (Scalar multiplication and matrix subtraction)
1 2 4
A = 3 0 1
2 1 2
0 0
2
B = 1 4 3
3 2
1
Find (a) 3A, (b) B, (c) 3A B
Matrix operations
Matrix operations
Ex 4: (Find AB)
Matrix multiplication:
3
1
A = 4 2
0
5
If A = [aij ]mn , B = [bij ]n p

Then AB = [aij ]mn [bij ]n p = [cij ]m p
Size of AB
where
Sol:
cij = aik bkj = ai1b1 j + ai 2b2 j + ! + ainbnj

k =1
a11 a12 ! a1n b ! b

! b1n
11
1j
"
"
"

b
"
b
!
b
21
2j
2n
=
ai1 ai 2 ! ain
"
"
"
" ci1 ci 2 ! cij ! cin
"
"
"
b ! bnj ! bnn
an1 an 2 ! ann n1
Notes: (1) A+B = B+A, (2)
3 2
B=
4 1
AB BA
(1)(3) + (3)(4) (1)(2) + (3)(1)

AB = (4)(3) + (2)(4) (4)(2) + (2)(1)
(5)(2) + (0)(1)
(5)(3) + (0)(4)
9 1
= 4 6
15 10
Matrix operations
Determinants
Matrix form of a system of linear equations:
a11 x1 + a12 x2 + ! + a1n xn = b1

a x + a x +!+ a x = b
21 1 22 2
2n n
2
"
am1 x1 + am 2 x2 + ! + amn xn = bm
m linear equations
The determinant of an nn matrix A is defined as the sum of

all n! possible products of n elements such that
1. each product contains one element from every row and
every column, and
a11 a12 " a1n x1 b1

a

21 a22 " a2 n x2 = b2
!
!
!
! ! !

am1 am 2 " amn xn bm
=
Single matrix equation

Ax =b
m n n 1
m 1
2. the factors in each product are written so that the column

subscripts appear in order of magnitude and each product
is then preceded by a plus or minus sign according to
whether the number of inversions in the row subscripts is
even or odd.
The Inverse of a Matrix

Determinants
Inverse matrix:
The determinant of A is a scalar denoted by |A| or by

det(A).
Consider
A M nn
If there exists a matrix B M nn such that AB = BA = I n ,

Then (1) A is invertible (or nonsingular)
(2) B is the inverse of A
Note:
A matrix that does not have an inverse is called noninvertible (or
Positive
Negative
singular).
The Inverse of a Matrix

Eigenvalues and eigenvectors
The inverse of a matrix is unique)

If B and C are both inverses of the matrix A, then B = C.
Pf:
AB = I
Let A be an nxn matrix. A scalar is called an eigenvalue

of A if there is a nonzero vector x such that
Ax = x.
C ( AB ) = CI
(CA) B = C
IB = C
B=C
such vector x is called a eigenvector A corresponding to
Consequently, the inverse of a matrix is unique.

Notes:
(1) The inverse of A is denoted by
(2) AA1 = A1 A = I
A1

2
3
Show that x = 1 is an eigenvector of A =
Corresponding to
3 2

When A is squared, the eigenvectors stay the same.
The eigenvalues are squared
=4
Is it true that
?
Ax = x
If is an eigenvalue of A and x is an eigenvector
belonging to , any nonzero multiple of x will be an
eigenvector

to find and x, we use the following equation:
(A I)x = 0
The product of the n eigenvalues equals the determinant.
The sum of the n eigenvalues equals the sum of the n diagonal
entries (trace).

Example:
Find the eigenvalues and eigenvectors of
2 1
A =
1 2

Try next:
Find the eigenvalues and eigenvectors of A2, A-1, and (A+4I)
Quiz
1. Find the eigenvalues of these matrices. All powers have the same
eigenvectors.
2. Find the eigenvalues of A and B and AB and BA
Matrix in R
Matrix in R
Now open your R software!
Matrix operation
Making a Matrix in R
A <matrix(data=c(7,18,-2,22,-16,3,55,1,9,-4,0,31),byrow=
TRUE,nrow=3,ncol=4)
#check the dimensions
dim(A)
Making an Identity Matrix (I) in R
id <- function(n)diag(c(1),nrow=n,ncol=n)
# To create an identity matrix of order 12
I12 <- id(12)
Transposition
At <- t(A)
# t() is the transpose function
Diagonals
diag(A)
Addition/Substraction of matrix
A+A
#addition of matrix A and A
Multiplication of matrices
A%*$At
#matrix At multiplied by A

Advanced Statistics W2: Matrix Algebra Basics

Uploaded by

Copyright:

Available Formats

Advanced Statistics W2: Matrix Algebra Basics

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Advanced Statistics W2: Matrix Algebra Basics

Uploaded by

Copyright:

Available Formats

Why matrix?

This meeting introduces the basic elements of

Why matrices? Lots of data and calculation

Matrix algebra is the fundamental notational technique used in

Although it is possible to use statistical programs without

Matrix algebra is a convenient notational system that allows us

Addition and subtraction

e.g. Inner / outer

Notation and basic definitions:

Notation and basic definitions:

Notation and basic definitions:

Notation and basic definitions:

a ' = [a1 a 2 , , an ] = {aj}

Notation and basic definitions:

Notation and basic definitions:

Notation and basic definitions:

A square matrix A is symmetric if A = AT

Notation and basic definitions:

Then Tr ( A) = a11 + a22 + ! + ann

Notation and basic definitions:

Notation and basic definitions:

Notation and basic definitions:

The transpose of a matrix A, denoted by A, is

If A = [aij ]mn , B = [bij ]mn

If A = [aij ]mn , c : scalar

Find (a) 3A, (b) B, (c) 3A B

If A = [aij ]mn , B = [bij ]n p

cij = aik bkj = ai1b1 j + ai 2b2 j + ! + ainbnj

a11 a12 ! a1n b ! b

(1)(3) + (3)(4) (1)(2) + (3)(1)

Matrix form of a system of linear equations:

a11 x1 + a12 x2 + ! + a1n xn = b1

The determinant of an nn matrix A is defined as the sum of

a11 a12 " a1n x1 b1

Single matrix equation

2. the factors in each product are written so that the column

The Inverse of a Matrix

The determinant of A is a scalar denoted by |A| or by

If there exists a matrix B M nn such that AB = BA = I n ,

The Inverse of a Matrix

The inverse of a matrix is unique)

Let A be an nxn matrix. A scalar is called an eigenvalue

such vector x is called a eigenvector A corresponding to

Consequently, the inverse of a matrix is unique.

Eigenvalues and eigenvectors

Eigenvalues and eigenvectors

Eigenvalues and eigenvectors

Eigenvalues and eigenvectors

Eigenvalues and eigenvectors

2. Find the eigenvalues of A and B and AB and BA

Now open your R software!

You might also like