Chapter 3 inner product space (1)

Inner product spacesPhan Thi Khanh Van E-mail: khanhvanphan@hcmut.edu.vn May 13, 2021... Table of contents1 Inner product 2 The orthogonal/perpendicular complement 3 Vector projection 4

Inner product spaces

Phan Thi Khanh Van

E-mail: khanhvanphan@hcmut.edu.vn

May 13, 2021

Table of contents

1 Inner product

2 The orthogonal/perpendicular complement

3 Vector projection

4 Gram- Schmidt orthogonalization

Inner product

LetV be a vector space The inner product of2 vectorsu, v ∈ V is a real number satisfying 4following axioms:

2 ∀α ∈ K , u, v ∈ V : (αu, v ) = α(u, v )

3 ∀u, v ∈ V : (u, v ) = (v , u)

4 ∀u, v , w ∈ V : (u + v , w ) = (u, w ) + (v , w )

A vector space V + inner product = inner product space

Rn + inner product = Euclidean space

In Rn, the standard inner product (dot product) is:

(x , y ) = x1y1+ x2y2+ x3y3+ + xnyn

Example

1 In R2: (x , y ) = x1y1+ 2x2y1+ 2x1y2+ 5x2y2 is an inner product

2 In P2[R] : (p, q) =R01p(x )q(x )dx is an inner product

3 InR3 : (x , y ) = x1y1− x2y1− x1y2+ 3x2y2+ 5x3y3 is an inner product

Norm of a vector

Distance between 2 vectors

d (u, v ) = ku − v k

Angle between 2 vectors

cos(u, v ) =d kukkv k(u,v )

Orthogonal vectors

u ⊥ v ⇔u, v =d π2 ⇔ cos(u, v ) = 0d

A vector having norm1 is called a unit vector

Cauchy-Schwatz inequality: |(u, v )| ≤ kuk.kv k

Triangle inequality: ku + v k ≤ kuk + kv k

InR3 given an inner product: (x , y ) = x1y1+ x2y1+ x1y2+ 3x2y2+ 5x3y3,

x = (1, 2, −1), y = (−1, 0, 1) Determine:

1 (x , 3y + x )

2 kxk

3 The distance, angle between 2 vectors2x + y andx − y

4 a unit vector parallel to x

1 3y + x = 3(−1, 0, 1) + (1, 2, −1) = (−2, 2, 2)

⇒ (x, 3y + x) = −2 + 1.2 − 2.2 + 3.2.2 − 5.1.2 = −2

Remark

The inner product of x = (x1, x2, x3) andy = (y1, y2, y3) can be

represented as

(x , y ) =x1 x2 x3















y1

y2

x3



= x A.yT

So, we can compute: (x , 3y + x ) = x A.(3y + x )T = −2

2 kxk =p(x, x) =√x A.xT =√22

3 d (2x + y , x − y ) = k(2x + y ) − (x − y )k = kx + 2y k = k(−1, 2, 1)

The angle α between 2vectors: 2x + y = (1, 4, −1)and

x − y = (2, 2, −2):

cos α = k2x+y k.kx−y k(2x +y ,x −y ) = √46

62.44 = √23

682

α = arccos√23

682

4 Let u be a unit vector parallel tox

We have that u = αx andkuk = |α|.kxk = 1

⇒ α = ±kxk1

kxk = ±(1,2,−1)√

22 = ±(√1

22,√2

22, −√1

22)

In an inner product space V with the inner product(x , y )

kxk = 2, ky k = 3, \(x , y ) = π3 Findd (x − 3y , 2x + y )

d (x − 3y , 2x + y ) = kx − 3y − 2x − y k = k − x − 4y k

=p(x, x) + 4(x, y) + 4(y, x) + 16(y, y)

=

q

kxk2+ 8kx k.ky k cos \(x , y ) + 16ky k2

=

q

22+ 8.2.3.12+ 16.32 =√172

A vector orthogonal to a vector set M

A vector orthogonal to a vector space

A vectoru is called orthogonal to a vector spaceU if it is orthogonal to any vectorv in U: (u, v ) = 0, ∀v ∈ U

set ofU)

2 orthogonal vector sets

M ⊥ N ⇔ (u, v ) = 0, ∀u ∈ M, v ∈ N

2 orthogonal spaces

Given2 spaces U = span{M}, V = span{N},

⇔ 2spanning sets are orthogonal)

Orthogonal set

M is called an orthogonal set if: ∀u, v ∈ M : (u, v ) = 0 (mutually

orthogonal/perpendicular)

Property: An orthogonal set without vector 0is LI

Orthonormal set

M is called an orthonormal set if it is orthogonal andkeik = 1, ∀ei ∈ M

InR3given an inner product: (x , y ) = x1y1− x2y1− x1y2+ 4x2y2+ 3x3y3

a Let F = span{f1= (1, 1, −1), f2= (6, 0, 1)} andu = (1, 2, m) Findm

such that u ⊥ F

b Let W = {x ∈ R3|x1+ x2− x3= 0, 3x1+ x3 = 0} Findm such that

(

u ⊥ f1

( (u, f1) = 0 (u, f2) = 0















1 2 m



b Find the basis of W: x ∈ W ⇔ x = (−m, 4m, 3m)

A basis of W : {(−1, 4, 3)}













1 2 m



⇔ m = −299

The orthogonal complement

In an inner product space V, let U be a subspace The

orthogonal/perpendicular complement ofU is the set of all vectors orthogonal toU:

Theorem

1 For any vector u ∈ V,u can be uniquely represented as: u = x + y, where x ∈ U andy ∈ U⊥

Example 1

InR3 with the inner product (x , y ) = x1y1+ 3x2y2+ 4x3y3, let U be

U = span{(1, 2, −1), (0, 1, 2), (2, 3, −4)} Find the dimension and one basis

ofU⊥

x = (x1, x2, x3) ∈ U⊥ ⇔







x ⊥ u1

x ⊥ u2

x ⊥ u3

⇔





















x1

x2

x3











⇔











The general solution: x = (60m, −8m, 3m)

⇒ dim(U⊥) = 1 One basis forU⊥: {(60, −8, 3)}

Example 2

InR3 with the inner product (x , y ) = x1y1+ x1y2+ x2y1+ 4x2y2+ x3y3, find a basis, the dimension ofU⊥ where:

U = {x ∈ R3|x1+ x2− x3= 0, 2x1+ x2+ x3 = 0}

First, we will find a basis ofU:





⇔





General solution: x = (−2m, 3m, m)T, a basis ofU : {(−2, 3, 1)}

Next, we will find the general vectorx ∈ U⊥ ⇔ x ⊥ U

⇔ (x, (−2, 3, 1)) = 0













x1

x2

x3



General solutionx = m(−10, 1, 0) + n(−1, 0, 1)

One basis forU⊥: {(−10, 1, 0), (−1, 0, 1)},dim(U⊥) = 2

1 In R4 with the dot product, find the dimension, a basis of U⊥ where:

U = {x ∈ R4|x1+ x2− x4 = 0, 2x1+ x2+ x3 = 0}

2 In R4 with the dot product, find the dimension, a basis of U⊥ where:

U =< (1, −2, 1, 0), (1, 2, 3, 4) >

3 In R3 with the inner product:

(x , y ) = x1y1+ x1y2+ x2y1+ 4x2y2+ x3y3, find the dimension, a basis

of U⊥ whereU =< (1, 2 − 3), (2, 1, 0) >

Vector projection

LetU be a subspace of an inner product space V Any vectorz in U can

be uniquely represented asz = x + y,x ∈ U, y ∈ U⊥

x is called the vector projection/ vector resolution ofz ontoU Denote: prU(z)

y is called the vector rejection ofz

The distance from z toU: d (z, U) = ky k = kz − prU(z)k

InR3 with the inner product (x , y ) = x1y1+ 2x2y2+ 3x3y3, given

U = span{(1, 2, −1), (2, 1, 0)}, z = (2, −3, 4) FindprU(z), d (z, U)

Letz = x + y , x ∈ U, y ∈ U⊥, e1 = (1, 2, −1), e2 = (2, 1, 0)

x ∈ U ⇒ x = αe1+ βe2

( (y , e1) = 0 (y , e2) = 0

Then, we have

( (z, e1) = α(e1, e1) + β(e2, e1) (z, e2) = α(e1, e2) + β(e2, e2)

or

(

−22 = 12α + 6β

(

α = −103

β = 3

Therefore,prU(z) == −103(1, 2, −1) + 3(2, 0, 1) = (83, −113,103)

Andd (z, U) = kz − prU(z)k = k(−23 ,203,23)k =

√ 816 3

1 In R3 with the dot product, given

U = {x ∈ R3|x1+ x2− x3 = 0}, z = (1, 2, 1) Find prU(z)

2 In R3 given an inner product

(x , y ) = x1y1+ x2y1+ x1y2+ 3x2y2− 2x2y3− 2x3y2+ 3x3y3, a

subspace U =< (1, 1, 2), (2, −1, 1) >, a vectorz = (−8, 7, 3) Find

prU(z), d (z, U)

Orthogonal basis

In an inner product space V, a basisE = {e1, e2, , en}is called an

orthogonal basis forV, if it is mutually orthogonal (ei ⊥ ej ∀i , j)

Orthonormal basis

In an inner product space V, a basisE = {e1, e2, , en}is called an

orthonormal basis forV, if it is orthogonal (ei ⊥ ej), and keik = 1,∀i , j Gram-Schmidt orthogonalization

In an inner product space V,E = {e1, e2, , en}is a basis of V We will find an orthogonal basisF fromE using Gram-Schmidt algorithm:

f1 = e1

f2 = e2− prf1(e2) = e2− (e2 ,f 1 )

(f 1 ,f 1 )f1

f3 = e3− prf1(e3) − prf2(e3) = e3−(e3 ,f 1 )

(f 1 ,f 1 )f1−(e3 ,f 2 )

(f 2 ,f 2 )f2

fn = en− prf1(en) − prf2(en) − prfn−1(en)

Normalizing: gi = fi

kfik, ∀i = 1 n, we obtain an orthonormal basis

InR3 with the dot product, let E = {e1= (1, 1, 1), e2= (1, 2, −1)} be a basis Find an orthonormal basisF fromE

Orthogonalizing E using Gram-Schmidt algorithm:

f1 = e1= (1, 1, 1)

f2 = e2− (e2 ,f 1 )

(f 1 ,f 1 )f1 = (1, 2, −1) − 23(1, 1, 1) = (13,43, −53) Choose

f2 = (1, 4, −5)

An orthogonal basisF = {(1, 1, 1), (1, 4, −5)}

An orthonormal basis

F = {(1/√3, 1/√3, 1/√3), (1/√42, 4/√42, −5/√42)}

Thank you for your attention!