18.

06 - Linear Algebra Cheatsheet

February 22, 2019

Remark: This is not intended as a definitive list of everything you are meant to memorize for 18.06.
Some of these ideas will be familiar, some of them less so. This is a reference document for you to look
up definitions if you come across something that seems unfamiliar/confusing.

1 Vectors
1. When we talk about vectors in 18.06, we are usually referring to column vectors.

2. A two-dimensional vector v is defined by its two components, v1 and v2 . We write the vector as
 
v
v= 1
v2

3. The set of all two-dimensional vectors is referred to as R2

4. In general, a vector v can have n components, and would then be an n-dimensional vector (a n × 1
array):
 
v1
 v2 
v= . 
 
 .. 
vn

The set of all n-dimensional vectors is referred to as Rn .

5. We can always multiply vectors by scalars. We can also add two vectors, provided they have the
same dimensions.

6. The zero, n-dimensional vector 0 is a vector where every component is 0.

7. The length (or magnitude) of a vector v is written kvk. It is given by the following formula:

n
X
kvk2 = v12 + v22 + ... + vn2 = vi2
i=1

8. A unit vector n is a vector with length knk = 1.

 9. Suppose you have two n-dimensional vectors, u and v:
   
u1 v1
 u2   v2 
u =  . , v =  . 
   
.
 .   .. 
un vn

Then the dot product (or inner product) of these two vectors, u·v, is given by the following formula:

u · v = u1 v1 + u2 v2 + ... + un vn

10. The angle between two vectors u and v is given by the following formula:

u·v
cos θ =
kukkvk

11. We say that a nonzero vector u is parallel to a nonzero vector v if u = av for some scalar a 6= 0.
We sometimes say that u and v are in the same direction.

12. We say that a vector u is perpendicular, or orthogonal, to a vector v if u · v = 0 .

13. The transpose of a vector is an n-dimensional row vector (a 1 × n array):

v T = v1 v2 · · ·


vn

14. We can multiply a n-dimensional row vector by a n-dimensional column vector. The order of
multiplication matters:

• uT v has dimensions 1 × 1, i.e. it is a scalar. In fact uT v = vT u:

 
v1

 v2 

uT v = u1 u2 · · · un  .  = u1 v1 + u2 v2 + ... + un vn

 .. 
vn

We see then that really uT v = u · v, i.e. the dot product of u and v.

• uvT has dimensions n × n. It is called an outer product.
2 Matrices
2.1 General properties
Remark. The most basic rule that you should remember: row-column . It shows the order in which
you write or compute, e.g.:

• The first index denotes the row, the second number the column.

• You multiply a row by a column to get a number.

• An m × n matrix has m rows and n columns.

1. A matrix is an m × n array of numbers. An m × n matrix has m rows and n columns. A matrix

is square if m = n. Examples:
 
1 2
• A= is a 2 × 2 square matrix.
−1 3
 
1 0 1
• B= is a 2 × 3 matrix.
−1 2 3
 
1 4
• C = 0 −3 is a 3 × 2 matrix.
1 1
2. Suppose A is a m × n matrix and B is a p × q matrix. We can only multiply these matrices if the
dimensions make sense. We can multiply AB only if n = p; we can multiply BA only if m = q.

3. Suppose A and B are two n × n square matrices. In general AB 6= BA . Matrix multiplication

does not commute.

4. A diagonal matrix is a matrix which only has entries along its diagonal.

5. The identity matrix I is a square, diagonal matrix. The entries along the diagonal are all equal to
1. The identity matrix is the only matrix for which IA = A for all square matrices A.

6. We can compute the product of three matrices ABC either as (AB)C (multiply AB and then
multiply on the right by C), or as A(BC) (multiply BC and then multiply on the left by A).
Matrix multiplication is associative.

7. Matrix multiplication is distributive. This means that (A + B)C = AC + BC.

2.2 Linear systems

1. Often we will be interested in solving equations of the form Ax = b , where A is an m × n matrix,
x is a n-dimensional vector, and b is a m-dimensional vector. This is usually called a linear system.

2. If A is a square matrix, then it might have an inverse A−1 , so that AA−1 = A−1 A = I. The
unique solution of the linear system Ax = b in this case is x = A−1 b.

3. If A and B are square matrices with inverses A−1 and B −1 , then (AB)−1 = B −1 A−1 .
 4. The inverse
 of a square matrix does not always exist. We have already seen that for a 2 × 2 matrix
a b
A= , we have the following formula
c d
 
−1 1 d −b
A =
ad − bc −c a

This exists if and only if ad − bc 6= 0 .

5. Rectangular matrices for which m 6= n will not have inverses.

6. A general linear system Ax = b, where A is a rectangular m × n matrix, may have a unique

solution. It may also have infinitely many solutions, or no solutions at all.

2.3 Transposes
1. The transpose of a matrix A is denoted by AT . The transpose is the matrix formed by taking the
columns of A and making them the rows of AT .

2. If A has components aij , then AT has components aji .

3. If A is m × n, then AT is n × m.

4. AT A and AAT can always be computed. They are both square matrices, even if A is rectangular.
If A is n × m, then AT A is m × m and AAT is n × n.

5. (AB)T = B T AT .

6. If A is a square matrix and A−1 exists, then (AT )−1 = (A−1 )T .

2.4 Orthogonal matrices

1. A n × n square matrix Q is orthogonal if QT Q = I . A rectangular matrix for which QT Q = I we
will usually refer to as a tall skinny orthogonal matrix.

2. An equivalent definition: suppose a matrix Q has n columns given by the vectors q1 , ..., qn . Then
Q is orthogonal if the column vectors are orthonormal. This means that qiT qj = 0 for i 6= j, and
kqi k = 1.

3. If Q is square and orthogonal, then we also have that QQT = I, and so Q−1 = QT .

4. If Q obey QT Q = I, but is not square, then QQT 6= I generally!

3 Planes, hyperplanes and surfaces
3.1 Planes
1. The general equation of a plane in R3 is

ax + by + cd = f,

where a, b, c, d ∈ R are scalars. The plane contains the origin if and only if f = 0.
 
a
2. The normal to this plane is given by the vector w = b .

c
 
x
3. The equation of the plane may equivalently be written as w · x = wT x = f , where x = y .
z

3.2 Hyperplanes
 
x1
1. A hyperplane in Rn is the set of points with position vectors x =  ...  obeying the equation
 

xn

w1 x1 + w2 x2 + ...wn xn = f

for scalars w1 , w2 , ..., wn ∈ R.

 
w1
2. The normal to this plane is given by the vector w =  ... .
 

wn

3. The equation of the hyperplane may equivalently be written as w · x = wT x = f .

3.3 Surfaces
• A surface in three dimensions is described by an equation f (x, y, z) = Const.

• The normal to a surface is n = ∇f .