ELEC-8900 Special Topics:

Advanced Energy Storage Systems

Lecture 01: Review of Linear Algebra
Instructor: Dr. Balakumar Balasingam
(Link to video from last year: https://tinyurl.com/y579tpaf)
May 20, 2022

Contents
1 Linear System of Equations 2

2 Matrices 3
2.1 Transpose of a Matrix . . . . . . . . . . . . . . . . . . . . . . . . 3
2.2 Symmetric Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.3 Diagonal Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.4 Identity Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.5 Upper Triangular Matrix . . . . . . . . . . . . . . . . . . . . . . 5
2.6 Lower Triangular Matrix . . . . . . . . . . . . . . . . . . . . . . . 5
2.7 Banded Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3 Matrix Operations 6
3.1 Addition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.2 Subtraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.3 Multiplication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.4 Determinant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.5 Inverse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

4 Solving Linear Algebraic Equations 8

4.1 Graphical Method . . . . . . . . . . . . . . . . . . . . . . . . . . 8
4.2 Cramer’s Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4.3 Matrix Inverse Method . . . . . . . . . . . . . . . . . . . . . . . . 11
4.4 Gaussian Elimination . . . . . . . . . . . . . . . . . . . . . . . . . 11
4.5 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

5 Questions 14

1
1 Linear System of Equations
Consider the following linear system of equations

a11 x1 + a12 x2 + . . . + a1n xn = b1

a21 x1 + a22 x2 + . . . + a2n xn = b2
.. (1)
.
an1 x1 + an2 x2 + . . . + ann xn = bn

where, the a’s are constant coefficients, the b’s are constants, and n is the number
of equations. The above equation can be written in matrix form as follows

Ax = b (2)

where      
a11 ... a1n x1 b1
 a21 ... a2n   x2   b2 
A= x =  b =  (3)
     
.. .. .. .. .. 
 . . .   .   . 
an1 ... ann xn bn
Example 1. Convert the following linear system of equations into matrix form.

x1 + 2x2 + x3 = 2
x1 − x2 + 2x3 = 5 (4)
2x1 + 2x2 + 2x3 = 12

Solution: The linear system of equations in (4) can be written in matrix form
as follows

Ax = b

where      
1 2 1 x1 2
A= 1 −1 2  x =  x2  b =  5  (5)
2 2 2 x3 12
There are several methods to solve linear system of equations similar to those
shown in (4). Such as
• Algebraic method. Done by hand, becomes harder with the number of
variables.
• Graphical method. Nearly impossible for number of variables n > 3.
• Matrix inverse method. Computationally very intense when the matrix
size becomes large.
• Cramer’s method. Computationally very intense when the matrix size
becomes large.

2
 • Gaussian elimination. Better than all previous methods in terms of com-
putation.
• Gaussian elimination with pivoting. Makes Gaussian elimination method
more stable.
• Gaussian elimination with scaling. Makes Gaussian elimination method
more stable.

2 Matrices
A matrix A is a two dimensional array. The (i, j)th element of the matrix A is
denoted as aij . Horizontal set of matrix elements are called a row and vertical
set is called a column. A matrix with n rows and m columns is said to have a
dimension of n by m or (n × m).

2.1 Transpose of a Matrix

Consider an m × n matrix A whose (i, j)th element is denoted by aij , i =
1, . . . , m, j = 1, . . . , n. The transpose of A, denoted AT is written as

B = AT (6)

where B is an n × m matrix such that

bji = ai,j i = 1, . . . , m, j = 1, . . . , n (7)

Exercise 1.
Write the transpose of
 
7 3 1 6
A= 3 9 4 5  (8)
1 4 10 2

Solution.

 
7 3 1
T
 3 9 4 
B=A =
 1
 (9)
4 10 
6 5 2

2.2 Symmetric Matrix

A symmetric matrix A has the property that

aij = aji ∀ i, j (10)

3
Exercise 2 (Symmetric matrix). A 4 by 4 matrix (referred hereafter as 4 × 4
matrix) is A is given below
 
7 3 1 6
 3 9 4 5 
A=
 1 4 10
 (11)
2 
6 5 2 8

Write down the matrix B which is the transpose of A, i.e. B = AT .

Solution.

 
7 3 1 6
3 9 4 5 
B = AT = 

 1
 (12)
4 10 2 
6 5 2 8

We notice that B = AT = A That is, a symmetric matrix has the property

that its transpose is the same, i.e., AT = A

2.3 Diagonal Matrix

A diagonal matrix is a square matrix where all elements off the main diagonal
are equal to zero, i.e.,

aij = 0 ∀i 6= j (13)

Example 2. A 3 by 3 diagonal matrix A is given by

 
7 0 0
A= 0 9 0  (14)
0 0 10

2.4 Identity Matrix

An identity matrix is a diagonal matrix where all elements on the main diagonal
are equal to 1. Formally, we can write

0 if i 6= j
aij = (15)
1 if i = j

Example 3. A 3 by 3 identity matrix is written as

 
1 0 0
I= 0 1 0  (16)
0 0 1

4
2.5 Upper Triangular Matrix
An upper triangular matrix is one where all the elements below the main
diagonal are zero, i.e.,

aij = 0 if i > j (17)

Example 4. A 4 × 4 upper triangular matrix is A is written as

 
7 3 1 6
 0 9 4 5 
A=  0 0
 (18)
10 2 
0 0 0 8

2.6 Lower Triangular Matrix

An lower triangular matrix is one where all the elements above the main
diagonal are zero, i.e.,

aij = 0 if i < j (19)

Example 5. A 4 × 4 lower triangular matrix is A is written as

 
7 0 0 0
 1 4 0 0 
A=  1 5 10 0 
 (20)
2 0 2 8

2.7 Banded Matrix

A banded matrix has non-zero elements along its diagonal and few adjacent
off-diagonal elements, i.e.,

aij = 0 if |i − j| > 0 (21)

• A tridiagonal matrix has a bandwidth of 3

• A penta-diagonal matrix has a bandwidth of 5
and is given a special name - the tridiagonal matrix.

Example 6. (A 4 × 4 tridiagonal matrix)

 
7 3 0 0
 3 9 4 0 
A=  0 4 10
 (22)
2 
0 0 2 8

5
Example 7. (A 6 × 6 penta-diagonal matrix)
 
7 3 5 0 0 0
6 2 3 9 0 0
 
9 2 7 4 9 0
A= 0 7 1 6 2
 (23)
 8
0 0 6 4 9 2
0 0 0 2 2 8

3 Matrix Operations
3.1 Addition
Addition of two matrices say A and B of equal size is accomplished by adding
corresponding terms in each matrix, i.e.,

C = A + B =⇒ cij = aij + bij (24)

where the matrix C is the result of adding A and B.

3.2 Subtraction
Subtraction of one matrix (B) from another (A) of equal size is accomplished
by subtracting corresponding terms in each matrix, i.e.,

C = A − B =⇒ cij = aij − bij (25)

where the matrix C is the result of subtracting B from A.

3.3 Multiplication
Multiplication of two matrices can be performed only if the first matrix has as
many columns as the number of rows in the second matrix. Product of an m × n
matrix A and and n × k matrix B is represented as
n
X
C = A ∗ B =⇒ cij = aik ∗ bkj (26)
k=1

where the resulting matrix C is m × k.

3.4 Determinant
For a 2 × 2 matrix A, the determinant D is defined as follows
 
a11 a12
D= = a11 a22 − a21 a12 (27)
a21 a22

6
For an n × n matrix A, the determinant D is defined as follows
n
X
D = |A| = (−1)(j+1) |Mj | a1j (28)
j=1

where |Mj | denotes the determinant of the minor of A at the j th column. The
minor of A at the j th column is a new matrix formed by removing the first row
and the j th column of the matrix A.

Exercise 3. Write the determinant of

 
a11 a12 a13
A =  a21 a22 a23  (29)
a31 a32 a33

Solution. Using the formula in equation (28),

3
X
D = |A| = (−1)(j+1) |Mj | a1j (30)
j=1

where
 
a22 a23
M1 =
a32 a33
 
a21 a23
M2 = (31)
a31 a33
 
a21 a22
M3 =
a31 a32

Now, the determinant of A is

D = a11 |M1 | + a12 |M2 | + a13 |M3 |
a22 a23 a21 a23 a21 a22
D = a11 − a12 + a13
a32 a33 a31 a33 a31 a32

3.5 Inverse
If a matrix A is non-singular and square, then its inverse A−1 satisfies the
following

AA−1 = A−1 A = I (32)

Inverse of an n × n matrix A can be written as

1 T
A−1 = C (33)
|A|

7
where |A| is the determinant of A and C is the matrix of cofactors. The (i, j)th
element of C is written as

cij = (−1)i+j |Mij | (34)

where the matrix Mij (known as the minor) is formed by removing the ith row
and the j th column of the matrix A.
Example 8 (Inverse of a 2 × 2 matrix). Given that
 
a11 a12
A= (35)
a21 a22

the definition (33) can be used to write its inverse as follows

 
−1 1 a22 −a12
A = (36)
D −a21 a11

where the determinant can be written as D = a11 a22 − a12 a21 .

Exercise 4. Use (33) to find the inverse of a 3 × 3 matrix A given by
 
2 −1 4
1 1 1 (37)
4 1 −2

Solution. Left as an exercise. 

Exercise 5. Write the pseudo code to compute the inverse of an n × n matrix
A.
Solution. Left as an exercise. 

4 Solving Linear Algebraic Equations

4.1 Graphical Method
Exercise 6. Solve the following system of linear equations through a graphical
approach

3x1 + 2x2 = 18 (38)

−x1 + 2x2 = 2 (39)

Solution.
All we have to do is plot the two equations separately and find the coordi-
nates of the intersection. Fig. 1 shows the lines represented by equations (38)
and (39) above. We can see that the lines intersect at (4, 3), hence the solution
to the above linear system of equations is x1 = 4 and x2 = 3.

8
Figure 1: Graphical solution of two linear equations. The coordinates of
their intersection represent the solution of the equations.

4.2 Cramer’s Rule

Cramers Rule states that each unknown in a system of linear algebraic equations
may be expressed as a fraction of two determinants. Consider the following
system of equations

Ax = b (40)

The Cramer’s method showed that the ith element of x can be obtained as
|Bi |
xi = (41)
|A|

where Bi is obtained by replacing the ith column of A with b.

9
Exercise 7. Consider the following system of equations
Ax = b (42)
where it is given that A is 3 × 3. Use Cramer’s method to find the solution for
x.
Solution.
Let us denote the elements of A as aij where i, j ∈ 1, 2, 3.
b1 a12 a13
b2 a22 a23
b3 a32 a33
x1 = (43)
|A|
 
a11 b1 a13
 a21 b2 a23 
a31 b3 a33
x2 = (44)
|A|
 
a11 a12 b1
 a21 a22 b2 
a31 a32 b3
x3 = (45)
|A|
Exercise 8. Let us solve the following system of equations using Cramer’s
method
3x1 + 2x2 = 18 (46)
−x1 + 2x2 = 2 (47)
Solution.
First, let us write the system of equations in the matrix form Ax = b, i.e.,
    
3 2 x1 18
= (48)
−1 2 x2 2
Next, let us find out the determinant of A
3 2
D= = 3 ∗ 2 − (−1) ∗ 2 = 8 (49)
−1 2
Now, we can use Cramer’s method to write the solutions as follows
18 2
2 2 18 ∗ 2 − 2 ∗ 2
x1 = = =4
 D  8 (50)
3 18
−1 2 3 ∗ 2 − (−1) ∗ 18
x2 = = =3
D 8

10
4.3 Matrix Inverse Method
Let us write the general linear system of equations in matrix form

Ax = b (51)

Now, let us post multiply (51) by A−1

−1 −1
| {z A} x = A b
A
I
(52)
Ix = A−1 b
x = A−1 b

Exercise 9. Let us solve the following system of equations using inverse method

3x1 + 2x2 = 18 (53)

−x1 + 2x2 = 2 (54)

Solution.
First, let us write the system of equations in the matrix form Ax = b, i.e.,
    
3 2 x1 18
= (55)
−1 2 x2 2

Next, let us find out the inverse of A

 
−1 1 2 −2
A =
(3 ∗ 2 − (−1) ∗ 2) 1 3
  (56)
2/8 −2/8
=
1/8 3/8

Now, the solution for x is

x = A−1 b
  
2/8 −2/8 18
=
1/8 3/8 2 (57)
 
4
=
3

4.4 Gaussian Elimination

Cramer’s approach and matrix inversion approach will work for linear systems
of any dimension. However, they are computationally intensive.
Gaussian Elimination method consists of two phase:
• Forward elimination. Here, the objective is to make the system into Ãx =
b̃ such that à is upper diagonal.

11
 • Back substitution. Once we have a system that is in the upper diagonal
form, it is easy to solve for x using back substitution.
The above two steps are achieved through row operations. The row operations
consists of the following that does not alter the solution of the system:

• Multiplying any row by a constant

• Adding two rows
• Subtracting one row from another
Figure 2 provides a graphical illustration of forward elimination and back sub-
stitution.

Figure 2: Gaussian elimination approach consists of two phases: forward

elimination and back substitution.

12
4.5 Procedure
Consider the folllowing linear system of equations

a11 x1 + a12 x2 + . . . + a1n xn = b1 ... (r1 )

a21 x1 + a22 x2 + . . . + a2n xn = b2 ... (r2 )
.. (58)
.
an1 x1 + an2 x2 + . . . + ann xn = bn ... (rn )

where (r1 ), (r2 ) and (rn ) indicate the row numbers.

The forward elimination progresses in the following manner
• Eliminate x1 from (r2 ), (r3 ), . . . , (rn )
• Eliminate x2 from (r3 ), (r4 ), . . . , (rn )
.
• ..
.
• ..
• Eliminate xn−1 from (rn )
After the above steps are performed, the linear system in (58) will be in the
upper diagonal form like follows

a11 x1 + a12 x2 + a13 x3 + ... + a1n xn = b1

(1) (1) (1) (1)
a22 x2 + a23 x3 + ... + a2n xn = b2
(2) (2) (2)
a23 x3 + ... + a2n xn = b3 (59)
..
.
(n−1)
ann xn = b(n−1)
n

Now, we solve for xn , xn−1 , . . . , x1 through back substitution as follows

(n−1)
bn
xn = (n−1)
(60)
ann

(n−2) (n−2) b(n−1)

bn−1 − an−1n n
(n−1)
ann
xn−1 = (n−2)
(61)
an−1n−1

and so on.
In general
(i−2) Pn (i−1)
bi−1 − j=i+1 aij xj
xi = (i−1)
(62)
aii

13
 Example:
Let us solve the following system of equation using Gaussian elimination

x1 + x2 − x3 = −3 ... (r1 )
6x1 + 2x2 + 2x3 = 2 ... (r2 ) (63)
−3x1 + 4x2 + x3 = 1 ... (r3 )

First, let us do the forward elimination

• Eliminate x1 from (r2 ), (r3 )

r2 ← r2 − 6 ∗ r1
r3 ← r3 + 3 ∗ r1

x1 + x2 − x3 = −3 ... (r1 )
−4x2 + 8x3 = 20 ... (r2 ) (64)
7x2 + −2x3 = −8 ... (r3 )

• Eliminate x2 from (r3 )

r2 ← r2 /(−4)
r3 ← r3 − 7 ∗ r1

x1 + x2 − x3 = −3 ... (r1 )
x2 − 2x3 = −5 ... (r2 ) (65)
+12x3 = 27 ... (r3 )

• Which results in the following system in the upper diagonal form

x1 + x2 − x3 = −3
x2 − 2x3 = −5 (66)
x3 = 2.25

• Back substitution
x2 = −5 + 2x3 = −5 + 2 ∗ 2.25 = −0.5
x1 = −3 − x2 + x3 = −3 − (−0.5) + 2.25 = −0.25

5 Questions
Question 1 (Matrices).

14
 Answer the following questions based on the matrices given below.
     
4 7 4 3 7 3
A= 1 2  B= 1 2 7  C= 6 
5 6 2 0 4 1

 
  1 5 8  
3 2 3 0 1
D= E= 0 2 3  F=
−1 2 1 7 3
0 0 6

 
  1 0 0
G= 7 6 4 H= 5 2 0 
8 6 3

(a) What are the dimensions of each matrices?

(b) Identify the square, symmetric, column, row, upper-diagonal, lower-diagonal
matrices
(c) What are the values of the elements a12 , b23 , d32 , e22 , f12 , g12 ?
(d) Perform the following matrix operations
1. E+B
2. A×F
3. B−E
4. 7B
5. E×B
6. CT
7. BT
8. D−1
9. B−1
10. H × GT

Question 2.
Answer the following questions based on the matrices given below.
   
4 −2 1 4 3 1
A= 1 2 0  B= 1 2 0  (67)
−3 1 1 2 0 4

 
3  
3 2
C= 6  D= (68)
−1 2
1

(a) Perform the following matrix operations: (i) A × B (ii) A × B × C

15
 (b) Someone tells you that
 
1/2 −3/4 −1/8
B−1 =  −1/4 7/8 5/80  (69)
−1/4 3/8 5/16

Verify if it is true.

Question 3 (Linear system of equations).

(a) Write the following set of equations in matrix form Ax = b.

6x3 + 2x2 = 8
2 − x1 = x3 (70)
5x2 + 8x1 = 13

(b) Write the matrix B = AT

(c) Find the matrix C1 = AAT and C2 = AT A
(d) What is the relationship between C1 and C2 ?
(e) Find A−1

Question 4 (Linear system of equations).

Consider the following system of equations

3x − y = 7
(71)
x+y =9

(a) Write the above equations in matrix form Ax = b.

(b) Solve the above system using algebraic manipulations
(c) Solve the above system using graphical method
(d) Solve the above system using Cramer’s method
(e) Find A−1 and solve the above system as x = A−1 b

Question 5 (Linear System of Equations).

Use Gaussian elimination to solve the following system of equations for the
x’s showing all the steps involved.

2x1 − x2 + 4x3 = 12
x1 + x2 + 2x3 = 6 (72)
4x1 + x2 − 2x3 = 0

(a) Write the row operations in the same orders in which they were applied
(b) Write the resulting system in the following form Ax = b where A is an
upper diagonal matrix
(c) Write the obtained solutions for x3 , x2 and x1 through back-substitution.

16
Question 6 (Linear System of Equations).
Consider the set of equations

2x2 + 5x3 = 9
2x1 + x2 + x3 = 9 (73)
3x1 + x2 = 10

(a) Compute the determinant

(b) Use Cramer’s rule to solve for the x’s
(c) Substitute the values back into the equation to verify the results
Question 7.
Consider the following system of equations

3x − y = 1
(74)
x+y = 3

(a) Write the above equations in matrix form Ax = b.

(b) Solve the above system using algebraic manipulations
(c) Solve the above system using graphical method
(d) Solve the above system using Cramer’s method
(e) Find A−1 and solve the above system by finding A−1 b

Question 8 (Gaussian elimination).

Consider the following system of equations

x1 + x2 − x3 = −3
6x1 + 2x2 + 2x3 = 2 (75)
−3x1 + 4x2 + x3 = 1

Perform the following row operations on the above system of equations

(a) r2 ← r2 − 6r1 ; r3 ← r3 + 3r1
(b) r2 ← r2 /(−4); r3 ← r3 − 7r2
(c) r3 ← r3 /12
(d) Use back-substitution to solve for x1 , x2 , and then x1
Question 9 (Gaussian elimination).
Perform Gaussian elimination on the following system to answer the ques-
tions below
8x1 + 2x2 − 2x3 = −2
10x1 + 2x2 + 4x3 = 4 (76)
12x1 + 2x2 + 2x3 = 6

17
 (a) Write the row operations in the same orders in which they were applied
(b) Write the resulting system in the following form Ax = b where A is an
upper diagonal matrix
(c) Write the obtained solutions for x1 , x2 and x3
Question 10 (Naive Gaussian Elimination).

Write a pseudocode (in terms of row-operations) to solve the following system

of equations through Gaussian Elimination (without considering pivoting or
scaling).

a11 x1 + a12 x1 + . . . + a1n xn = b1

a21 x1 + a22 x1 + . . . + a2n xn = b2
.. (77)
.
an1 x1 + an2 x1 + . . . + ann xn = bn

Question 11.
Use Gaussian elimination to solve the following system of equations for the
x’s showing all the steps involved.

2x1 − x2 + 4x3 = 7
x1 + x2 + 2x3 = 5 (78)
4x1 + x2 − 2x3 = −3

(a) Write the row operations in the same orders in which they were applied
(b) Write the resulting system in the following form Ax = b where A is an
upper diagonal matrix
(c) Write the obtained solutions for x3 , x2 and x1 through back-substitution.

References
[1] G. Strang, Introduction to linear algebra. Wellesley-Cambridge Press Welles-
ley, MA, 2016.

18