UNIT – 1 Linear Algebra

LECTURE-6
VECTOR SPACES, INNER
PRODUCT AND
ORTHOGONALITY
 Vector
Vectors as something that has both magnitude and direction
where the length of the vector is the magnitude and the
orientation of the vector is the direction.

⮚Speed is a scalar quantity, but velocity is a vector because it

has both magnitude and direction.
Solutions to systems of linear equations can be points in a plane, if the equations have
two variables, points in three-space if they are equations in three variables,
points in four-space if they have four variables, and so on.
The solutions make up subsets of the larger spaces.
Any quantity having n-components is called a vector of order n. Therefore, the
coefficients in a linear equation or the elements in a row or column matrix will form a
vector. Thus any n numbers x1, x2,….., xn, written in a particular order, constitute a vector
x.
Vectors in Computer Science
A vector is an ordered finite list of numbers. Vectors are usually written as vertical arrays,
surrounded by square or curved brackets, as in

Or

The size (also called dimension or length) of the vector is the number of elements it contains.
Magnitude and Direction of Vectors

If the coordinates of the initial point and the end point of a vector is given, the Distance Formula can be
used to find its magnitude.
 Direction of a Vector

The direction of a vector is the measure of the angle it makes with a horizontal line .

One of the following formulas can be used to find the direction of a vector:
 Operations on Vectors: Addition and Scalar Multiplication

 Vector Addition :

Sum of the vetors (4, 1) and (2, 3).

 Addition of vectors geometric description

Vector addition can be visualized in two ways.

 Draw the two vectors a and b to be added so that the tail of one of
the vectors, say b, is at the head of the other. Then the vector sum a
+ b may be represented by an arrow whose tail is at the tail of a and
whose head is at the head of b.

The second way (parallelogram law)

 If a and b are nonparallel vectors drawn with their tails emanating
from the same point, then a + b may be represented by the arrow that
runs along a diagonal of the parallelogram determined by a and b .
 Scalar Multiplication of vectors

Remark: 1. In general, if u is a vector in any vector space, and c a non-zero scalar,

the direction of cu will be the same as the direction of u, if c > 0 and the opposite
direction to u, if c < 0.
2. The length of cu is 𝑐 times the length of u.

Ex 1: The scalar multiple of the vector (3, 2) by 2.

⮚ 2 is a nonzero scalar.
 Real Vectors
 If the scalars are real numbers, then vectors are said to be real vectors.
(Occasionally other types of scalars arise, for example, complex numbers, in which
case we refer to the vector as a complex vector.)
 The set of all real numbers is written as R,
 The set of all real n-vectors is denoted Rn ,
So,
a ∈ Rn
is another way to say that a is an n-vector with real entries or a ∈ Rn means that a is an
element of the set Rn
Definition of Points in Space

a point a vector
 Vector: Ordered list of n-real numbers.
Rn : The set of all real n-vectors be a sequence of n real numbers.
• The elements of Rn can be interpreted as points in n-space or as position vectors in n-space.
• Let (v1,v2,…..,vn) be a sequence of n real numbers. The set of all such sequences is called n-space and vi’s are
denoted is the components of its.

The set of all ordered pairs of real numbers is denoted by R2

 Note the significance of “ordered” here;
 for example, the point (5, 3) is not the same point as (3, 5). The order is important.

The set of ordered triples, denoted by R3.

 Elements of this set such as (2, 4, 3) can be interpreted in two ways: as the location of a point in three space
relative to an xyz coordinate system, or as a position vector.

We now generalize these concepts. Let (u1,u2,…..,un) be a sequence of n real numbers. The set of all such
sequences is called n-space and is denoted by Rn. u1 is the first component of (u1,u2,…..,un) and u2 is the second
and so on.

For example: R4 is the set of sequence of 4 real numbers; (1,2,3,4) and (-1,2,5,0) are in R4. R5 is the sequences of 5
real numbers; (-1,2,0,3,9) is in this set.
  These interpretations are shown in Figure

Plane Equation Example: If you want a plane

that passes through the point (2,4,3), we could use
v=2i+4j+3k the general plane equation:
Ax+By+Cz=D
where A, B,C, and D are constants.
To ensure the plane passes through (2,4,3), we
would solve for D using these coordinates. For
instance, if we choose A=1, B=2, and C=−1, then
substituting (2,4,3) into the equation:
1⋅2+2⋅4−1⋅3=D
D=7
So, the plane equation passing through the point
(2,4,3)could be:
x+2y−z=7
The position vector v in component form: v=2i+4j+3k
where i, j, and k are the unit vectors in the x, y, and z
directions, respectively.
EXAMPLE 2 a. Compute the inner product of the vectors 𝑢=(1,2,3) and 𝑣=(4,−1,0) in 𝑅3 .
b. Prove that two vectors are orthogonal if and only if their inner product is zero.
Properties
Orthogonal Vectors
 Definition: Two vectors are orthogonal to each other if their inner
product is zero. That means that the projection of one vector onto the
other "collapses" to a point. So the distances from to or from to
should be identical if they are orthogonal (perpendicular) to each
other. Two vectors 𝑢 and 𝑣 in 𝑅𝑛 are orthogonal if and only if their
dot product is zero, i.e., 𝑢⋅𝑣=0.
Example:

[4/3 -1 2/3] = 20/3-6 - 2/3=0

Hence c and d are orthogonal to each other.

EXAMPLE 1: Determine if and are orthogonal

EXAMPLE 2: Are the vectors 𝑢=(1,2,3), 𝑣=(−2,1,0), and 𝑤=(0,0,1) orthogonal to each other in 𝑅3? Justify
your answer using the dot product.
Linear Combinations of Vectors
DEFINITION The sum of cv and dw is a linear combination of v and w
Four special linear combinations are: sum, difference, zero, and a scalar multiple cv:
1v + 1w = sum of vectors
1v- l w = difference of vectors
0v+0w = zero vector
cv+0w =vector cv in the direction of v v =
Let v1,v2,…..,vm be vectors in Rn. The vector v in Rn is a linear combination of v1,v2,…..,vm
if there exist scalars c1,c2,…..,cm such that v can be written
v= c1v1+c2v2+…..+ cm vm

Example 1: The vector v = (−7, −6) is a linear combination of the vectors v1 = (−2, 3) and
v2 = (1, 4), since v = 2 v1 − 3 v2.
The zero vector is also a linear combination of v1 and v2, since 0 = 0 v1 + 0 v2.
EXAMPLE 1: Determine the linear combination 2u - 3v + w. If u = (2, 5, -3), v = (-4, 1, 9), w = (4, 0, 2)

EXAMPLE 2: Determine whether the vector (8, 0, 5) is a linear combination of the vectors (1, 2, 3),
(0, 1, 4), and (2, -1, 1).
 Pictorial View

 Linear combinations of two vectors u and v.

u–2v 3u

2u+0.5v

–v u 2u+2v
0 v
Vector Space
Vector space is a collection of vectors, on which we have two
operations: vector addition and scalar multiplication.

It Means
We can add any two vectors, and we can multiply all vectors by
scalars.
⮚ In other words, we can take linear combinations.
⮚ These combinations fill a vector space

A vector space can be any set of elements (this could be lists,

functions, or other objects) but these elements must follow some
rules:
Vector Space Definition
A vector space consists of a set of V (elements of V are called vectors), a field F
(elements of F are scalars) and the two operations
 Vector addition is an operation that takes two vectors u, v ∈ V, and it produces the
third vector u + v ∈ V
 Scalar Multiplication is an operation that takes a scalar c ∈ F and a vector v ∈ V
and it produces a new vector c.v ∈ V.
Where both the operations must satisfy the following condition
Conditions for Vector Addition
An operation vector addition ‘ + ‘ must satisfy the following conditions:
1. Closure : If x and y are any vectors in the vector space V, then x + y
belongs to V (Closed under Addition)
2. Commutative Law : For all vectors x and y in V, then x + y = y + x
3. Associative Law : For all vectors x, y and z in V, then x + (y + z) = (x + y) + z
4. Additive Identity : For any vector x in V, the vector space contains the additive
identity element and it is denoted by ‘ 0 ‘such that 0 + x = x and x + 0 = x
(Property of zero vector)
5. Additive inverse : For each vector x in V, there is an additive inverse -x to get a
solution in V. -x + x = 0= x +(-x) (Property of negative vector)
Condition for Scalar Multiplication
An operation scalar multiplication is defined between a scalar and a vector and it
should satisfy the following condition :
6. Closure: If x is any vector and c is any real number in the vector space V, then
x. c belongs to V (Closed under scalar Multi.)
7. Associative Law: For all real numbers c and d, and the vector x in V, then
c.(d. x) = (c. d).x
8. Distributive law: For all real numbers c and d, and the vector x in V,
(c + d).x = c.x + d.x
9. Distributive law: For all real numbers c and the vectors x and y in V,
c.(x + y) = c. x + c. y
10. Unitary Law : For all vectors x in V, then 1.v = v.1 = v

Then, if every such possible linear combination gives a object inside the set, then its a vector space.
 1
Ex: n = 1 R = 1-space
= set of all real number
2
n=2 R = 2-space
= set of all ordered pair of real numbers ( x1 , x2 )
3
n=3 R = 3-space
= set of all ordered triple of real numbers ( x1 , x2 , x3 )
4
n=4 R = 4-space
= set of all ordered quadruple of real numbers ( x1 , x2 , x3 , x4 )
x1 , x2  x1 , x2 

0,0
a point a vector

Example: Rn is an example of a vector space.

Example: Show that ℝ3 is a vector space
ℝ3={v1,v2,v3,…….}
v1,v2,v3,… is in the form of (x1,x2,x3)

1. Closure under Addition:

v1,v2 ℝ3 ⇒ v1+v2 ℝ3
Let v1=(1,3,9) & v2=(-2,-3,4)
v1+v2= (-1,0,13) ℝ3
ℝ3 is closed under addition
2. Closure under Multiplication:
If v1 ℝ3 & c is constant

⇒ c. v1 ℝ3
If c=3, v1=(1,3,9)
⇒ c. v1= 2. (1,3,9)
(2,6,18)  ℝ3 satisfied.
3. Commutative Property:
v1+v2=v2+v1
(1,3,9) + (-2,-3,4) = (-2,-3,4) +(1,3,9)
(-1,0,13) = (-1,0,13) Commutative holds
4. Associative Property:
v1+(v2+v3)=(v1+v2)+v3
(1,3,9) + [(-2,-3,4)+ (-1,0,13)] = [(-2,-3,4) +(1,3,9)]+ (-1,0,13)
(-2,0,26) = (-2,0,26) Associative holds
5. Additive Identity:
v1+0 = 0+v1= v1
(1,3,9)+(0,0,0)=(0,0,0)+ (1,3,9)
(1,3,9) = (1,3,9)
6. Inverse Identity:
v1+(-v1) = (-v1)+v1 = 0
(1,3,9) - (1,3,9) = -(1,3,9) +(1,3,9)
(0,0,0) = (0,0,0)
7. Associative Property:
Let c=2, d=3
c.(d.v1) = (c. d).v1
2.[3.(1,3,9)] = (2.3).(1,3,9)
2. (3,9,27) = (6).(1,3,9)
(6,18, 54) = (6,18, 54) Associative holds
8. Distributive Property(scalar):
c.(v1+v2) = c. v1+c. v2 Let c=2
2. [(1,3,9)+(-2,-3,4) ] = 2. (1,3,9) + 2. (-2,-3,4)
2.(-1, 0,13) = (2,6,18)+(-4,-6,8)
(-2, 0, 26) = (-2, 0, 26)
9. Distributive Property(vector):
(c+d)(v1) = c. v1 + d. v1
Let c=2, d=3
(2+3). (1,3,9) = 2. (1,3,9) + 3. (1,3,9)
5. (1, 3,9) = (2,6,18)+(3,9,27)
(5, 15, 45) = (5, 15, 45)

10. Scalar Distributive Property (Unitary Law):

1.(v1) = v1 =(v1).1

1. (1, 3,9) = (1, 3,9)

Ex.3. Pn be the set of all polynomials of
degree n or less, then Pn is a Vector space.

For n=3,
P3={a0+a1X+a2X2+a3X3}, ai are real no.
 SUBSPACES
▪ Definition: A subspace of a vector space V is a subset W of V that has three properties:
1.1. The zero vector of V is in W.
2.2. W is closed under vector addition. That is, for each u and v in W, the sum u + v is
3. also in W.
4.3. W is closed under scalar multiplication . That is, for each u in W and each scalar c,
5. the vector cu is also in W.
6.OR
Definition. A subset W of a vector space V is a subspace if
7.

(1) W is non-empty
(2) For every 𝑣 , 𝑤 ∈ W and a, b ∈ F, a𝑣 + 𝑏𝑤 ∈ W.
 Span
The set of all linear combinations of a collection of vectors v1, v2,…, vr from Rn is called
the span of { v1, v2,…, vr }.
This set, denoted span { v1, v2,…, vr}, is always a subspace of Rn , since it is clearly
closed under addition and scalar multiplication (because it contains all linear
combinations of v1, v2,…, vr).
If V = span { v 1, v 2,…, v r }, then V is said to be spanned by v 1, v 2,…, v r .

Example: The span of the set {(2, 5, 3), (1, 1, 1)} is the subspace of R 3 consisting of all
linear combinations of the vectors v 1 = (2, 5, 3) and v 2 = (1, 1, 1). This defines a plane
in R 3.
 Example
7
Span of and
6

We therefore say that 2

1
2 3 4 5
Ex: Consider the subset V of R3 of vectors of the form (a, 2a, 3a), where the second
component is twice the first, and the third is three times the first. Show that V is a
subspace of R3.
Let (a, 2a, 3a) and (b, 2b, 3b) be two vectors in V, and let k be a scalar.
Then (a, 2a, 3a) + (b, 2b, 3b) = (a + b, 2a + 2b, 3a + 3b) = (a + b,2(a + b), 3(a + b))
This is a vector in V since the second component is twice the first, and the third is three
times the first. V is closed under addition.
Further, k(a, 2a, 3a) = (ka, 2ka, 3ka)
This vector is in V. V is closed under scalar multiplication.
V is a subspace of R3 .

Its Geometry: Consider the arbitrary vector u = (a, 2a, 3a).

We get u = (a, 2a, 3a) = a(1, 2,3)
The vectors of V are scalar multiples of the single vector (1, 2, 3).
These vectors form the line through the origin defined by the vector (1, 2, 3).
The vector (1, 2, 3) spans V.
Q. Let W be the set of vectors of the form (a, a2, b). Show that W is not a subspace of R3.

Solution:
W consists of all elements of for which the IInd component is the square of the Ist.
For example, the vector (2, 4, 3) is in W, whereas the vector (2, 5, 3) is not.

Thus k(a, a2, b) is not an element of W. W is not closed under scalar multiplication either.
The set of solutions to every homogeneous system of linear equations is a subspace.
Consider the following homogeneous system of linear equations

There are many solutions x1= 2r, x2 = 5r, x3 = r, We can write these solutions as vectors in R3 as ( 2r, 5r, r)
The set of solutions W thus consists of vectors for which the first component is twice the third, and the second component is five times the
third component.
The vector (4, 10, 2) for example is in W while (4, 9, 2) is not.
To show : W is closed under addition and under scalar multiplication.
Let u= (2r, 5r, r) and v=(2s, 5s, s) be arbitrary vectors in W. We get
In u + v and k u, the first component is twice the third, and the second component is five times the third. They are both in
W.
W is closed under addition and under scalar multi plication.
It is a subspace of R3.
Writing the vector (2r, 5r, r) in the form r(2, 5, 1), we see that the set of solutions is the line defined by the vector (2, 5, 1).
The vector (2, 5, 1) spans the set of solutions.
Basis and dimension:
A set of vectors is a basis for a vector space if the vectors
(1)span the space and
(2) are linearly independent.
There are usually many bases for a given vector space.
However the bases all have the same number of vectors. This number is called the
dimension of the space.

Ex: The set {(1, 0, ..., 0), (0, 1, ..., 0), ..., (0, 0, ..., 1)} of n vectors is the standard basis
of Rn.
The dimension of Rn is n.
Ex: (A spanning set for R3)

Sol:
Column Space:
The column space (also called the range or image) of a matrix 𝐴 is the subspace of 𝑅𝑛 (or whatever
field the vectors are over) spanned by its columns. In other words, it is the set of all possible linear
combinations of the columns of the matrix.
Definition:
Given a matrix 𝐴 with columns 𝑎1,𝑎2,…,𝑎𝑛 :
A= (𝑎1,𝑎2,…,𝑎𝑛)
The column space of 𝐴, denoted as Col(𝐴) is:
Col(𝐴)=span{𝑎1,𝑎2,…,𝑎𝑛}.
It represents all vectors that can be expressed as a linear combination of the columns of 𝐴.
Dimension of Column Space:
The dimension of the column space is called the rank of the matrix, and it tells us how many linearly
independent columns there are. The rank is denoted as:
Row Space:
The row space of a matrix A is the subspace of Rn spanned by its rows. Just like the column space,
the row space contains all possible linear combinations of the rows of the matrix.
Definition:
Given a matrix A with rows r1,r2,…, the row space of A, denoted as Row(A) is:

It represents all vectors that can be written as a linear combination of the rows of the matrix.
Dimension of Row Space:
The dimension of the row space is also equal to the rank of the matrix.
This means that:

For any matrix, the rank of the column space and the rank of the row space are always equal.