a concise text on advanced linear algebra pdf

For example, the notion ofdeterminant is shown to appear from calculating the index of a vector field which leads to a self-contained proof of the Fundamental Theorem of Algebra; the Cay

A Concise Text on Advanced Linear Algebra

This engaging textbook for advanced undergraduate students and beginning graduatescovers the core subjects in linear algebra The author motivates the concepts bydrawing clear links to applications and other important areas

The book places particular emphasis on integrating ideas from analysis whereverappropriate and features many novelties in its presentation For example, the notion ofdeterminant is shown to appear from calculating the index of a vector field which leads

to a self-contained proof of the Fundamental Theorem of Algebra; the

Cayley–Hamilton theorem is established by recognizing the fact that the set ofcomplex matrices of distinct eigenvalues is dense; the existence of a real eigenvalue of

a self-adjoint map is deduced by the method of calculus; the construction of the Jordandecomposition is seen to boil down to understanding nilpotent maps of degree two;and a lucid and elementary introduction to quantum mechanics based on linear algebra

is given

The material is supplemented by a rich collection of over 350 mostly proof-orientedexercises, suitable for readers from a wide variety of backgrounds Selected solutionsare provided at the back of the book, making it ideal for self-study as well as for use as

a course text

Cambridge University Press is part of the University of Cambridge.

It furthers the University’s mission by disseminating knowledge in the pursuit of education, learning and research at the highest international levels of excellence.

www.cambridge.org Information on this title: www.cambridge.org/9781107087514

c

 Yisong Yang 2015 This publication is in copyright Subject to statutory exception

and to the provisions of relevant collective licensing agreements,

no reproduction of any part may take place without the written

permission of Cambridge University Press.

First published 2015 Printed in the United Kingdom by Clays, St Ives plc

A catalogue record for this publication is available from the British Library

Library of Congress Cataloguing in Publication data

Yang, Yisong.

A concise text on advanced linear algebra / Yisong Yang, Polytechnic School

of Engineering, New York University.

pages cm Includes bibliographical references and index.

ISBN 978-1-107-08751-4 (Hardback) – ISBN 978-1-107-45681-5 (Paperback)

1 Algebras, Linear–Textbooks 2 Algebras, Linear–Study and teaching (Higher)

3 Algebras, Linear–Study and teaching (Graduate) I Title.

II Title: Advanced linear algebra.

QA184.2.Y36 2015

512 .5–dc23 2014028951ISBN 978-1-107-08751-4 Hardback ISBN 978-1-107-45681-5 Paperback Cambridge University Press has no responsibility for the persistence or accuracy

of URLs for external or third-party internet websites referred to in this publication, and does not guarantee that any content on such websites is, or will remain,

accurate or appropriate.

For Sheng,Peter, Anna, and Julia

2.5 Linear mappings from a vector space into itself 55

3.2 Definition and properties of determinants 883.3 Adjugate matrices and Cramer’s rule 1023.4 Characteristic polynomials and Cayley–Hamilton

4.1 Scalar products and basic properties 115

4.2 Non-degenerate scalar products 1204.3 Positive definite scalar products 1274.4 Orthogonal resolutions of vectors 1374.5 Orthogonal and unitary versus isometric mappings 142

5.3 Positive definite quadratic forms, mappings, and matrices 1575.4 Alternative characterizations of positive definite matrices 1645.5 Commutativity of self-adjoint mappings 170

6 Complex quadratic forms and self-adjoint mappings 1806.1 Complex sesquilinear and associated quadratic forms 180

9.1 Vectors inCnand Dirac bracket 248

9.3 Non-commutativity and uncertainty principle 2579.4 Heisenberg picture for quantum mechanics 262

This book is concisely written to provide comprehensive core materials for

a year-long course in Linear Algebra for senior undergraduate and beginninggraduate students in mathematics, science, and engineering Students who gainprofound understanding and grasp of the concepts and methods of this coursewill acquire an essential knowledge foundation to excel in their future aca-demic endeavors

Throughout the book, methods and ideas of analysis are greatly emphasizedand used, along with those of algebra, wherever appropriate, and a delicatebalance is cast between abstract formulation and practical origins of varioussubject matters

The book is divided into nine chapters The first seven chapters embody atraditional course curriculum An outline of the contents of these chapters issketched as follows

In Chapter 1 we cover basic facts and properties of vector spaces Theseinclude definitions of vector spaces and subspaces, concepts of linear dep-endence, bases, coordinates, dimensionality, dual spaces and dual bases,quotient spaces, normed spaces, and the equivalence of the norms of a finite-dimensional normed space

In Chapter 2 we cover linear mappings between vector spaces We start fromthe definition of linear mappings and discuss how linear mappings may be con-cretely represented by matrices with respect to given bases We then introducethe notion of adjoint mappings and quotient mappings Linear mappings from

a vector space into itself comprise a special but important family of mappingsand are given a separate treatment later in this chapter Topics studied thereinclude invariance and reducibility, eigenvalues and eigenvectors, projections,nilpotent mappings, and polynomials of linear mappings We end the chapterwith a discussion of the concept of the norms of linear mappings and use it

to show that being invertible is a generic property of a linear mapping and

then to show how the exponential of a linear mapping may be constructed andunderstood.

In Chapter 3 we cover determinants As a non-traditional but highlymotivating example, we show that the calculation of the topological degree

of a differentiable map from a closed curve into the unit circle inR2

involvescomputing a two-by-two determinant, and the knowledge gained allows us toprove the Fundamental Theorem of Algebra We then formulate the definition

of a general determinant inductively, without resorting to the notion of tations, and establish all its properties We end the chapter by establishing theCayley–Hamilton theorem Two independent proofs of this important theoremare given The first proof is analytic and consists of two steps In the first step,

permu-we show that the theorem is valid for a matrix of distinct eigenvalues In thesecond step, we show that any matrix may be regarded as a limiting point of asequence of matrices of distinct eigenvalues Hence the theorem follows again

by taking the limit The second proof, on the other hand, is purely algebraic

In Chapter 4 we discuss vector spaces with scalar products We start from themost general notion of scalar products without requiring either non-degeneracy

or positive definiteness We then carry out detailed studies on non-degenerateand positive definite scalar products, respectively, and elaborate on adjointmappings in terms of scalar products We end the chapter with a discussion

of isometric mappings in both real and complex space settings and noting theirsubtle differences

In Chapter 5 we focus on real vector spaces with positive definite scalarproducts and quadratic forms We first establish the main spectral theorem forself-adjoint mappings We will not take the traditional path of first using theFundamental Theorem of Algebra to assert that there is an eigenvalue and thenapplying the self-adjointness to show that the eigenvalue must be real Instead

we shall formulate an optimization problem and use calculus to prove directlythat a self-adjoint mapping must have a real eigenvalue We then present aseries of characteristic conditions for a symmetric bilinear form, a symmetricmatrix, or a self-adjoint mapping, to be positive definite We end the chapter

by a discussion of the commutativity of self-adjoint mappings and the ness of self-adjoint mappings for the investigation of linear mappings betweendifferent spaces

useful-In Chapter 6 we study complex vector spaces with Hermitian scalar productsand related notions Much of the theory here is parallel to that of the real spacesituation with the exception that normal mappings can only be fully understoodand appreciated within a complex space formalism

In Chapter 7 we establish the Jordan decomposition theorem We start with

a discussion of some basic facts regarding polynomials We next show how

In Chapter 8 we present four selected topics that may be used as als for some optional extra-curricular study when time and interest permit Inthe first section we present the Schur decomposition theorem, which may beviewed as a complement to the Jordan decomposition theorem In the secondsection we give a classification of skewsymmetric bilinear forms In the thirdsection we state and prove the Perron–Frobenius theorem regarding the prin-cipal eigenvalues of positive matrices In the fourth section we establish somebasic properties of the Markov matrices.

materi-In Chapter 9 we present yet another selected topic for the purpose ofoptional extra-curricular study: a short excursion into quantum mechanicsusing gadgets purely from linear algebra Specifically we will useCn as thestate space and Hermitian matrices as quantum mechanical observables to for-mulate the over-simplified quantum mechanical postulates including Bohr’sstatistical interpretation of quantum mechanics and the Schrödinger equationgoverning the time evolution of a state We next establish Heisenberg’s uncer-tainty principle Then we prove the equivalence of the Schrödinger descriptionvia the Schrödinger equation and the Heisenberg description via the Heisen-berg equation of quantum mechanics

Also provided in the book is a rich collection of mostly proof-orientedexercises to supplement and consolidate the main course materials Thediversity and elasticity of these exercises aim to satisfy the needs and inter-ests of students from a wide variety of backgrounds

At the end of the book, solutions to some selected exercises are presented.These exercises and solutions provide additional illustrative examples, extendmain course materials, and render convenience for the reader to master thesubjects and methods covered in a broader range

Finally some bibliographic notes conclude the book

This text may be curtailed to meet the time constraint of a semester-longcourse Here is a suggested list of selected sections for such a plan: Sec-tions 1.1–1.5, 2.1–2.3, 2.5, 3.1.2, 3.2, and 3.3 (present the concept of adjugatematrices only), Section 3.4 (give the second proof of the Cayley–Hamilton the-orem only, based on an adjugate matrix expansion), Sections 4.3, 4.4, 5.1, 5.2

(omit the analytic proof that a self-adjoint mapping must have an eigenvaluebut resort to Exercise 5.2.1 instead), Sections 5.3, 6.1, 6.2, 6.3.1, and 7.1–7.4.Depending on the pace of lectures and time available, the instructor maydecide in the later stage of the course to what extent the topics in Sections7.1–7.4 (the Jordan decomposition) can be presented productively.

The author would like to take this opportunity to thank Patrick Lin, ThomasOtway, and Robert Sibner for constructive comments and suggestions, andRoger Astley of Cambridge University Press for valuable editorial advice,which helped improve the presentation of the book

Notation and convention

We useN to denote the set of all natural numbers,

N = {0, 1, 2, },

andZ the set of all integers,

Z = { , −2, −1, 0, 1, 2, }.

We use i to denote the imaginary unit √

−1 For a complex number c =

a + ib where a, b are real numbers we use

We use t to denote the variable in a polynomial or a function or the transpose

operation on a vector or a matrix

When X or Y is given, we use X ≡ Y to denote that Y , or X, is defined to

be X, or Y , respectively.

Occasionally, we use the symbol∀ to express ‘for all’

Let X be a set and Y, Z subsets of X We use Y \ Z to denote the subset of elements in Y which are not in Z.

Vector spaces

In this chapter we study vector spaces and their basic properties and structures

We start by stating the definition and a discussion of the examples of vectorspaces We next introduce the notions of subspaces, linear dependence, bases,coordinates, and dimensionality We then consider dual spaces, direct sums,and quotient spaces Finally we cover normed vector spaces

The scalars to operate on vectors in a vector space are required to form a field,

which may be denoted by F, where two operations, usually called addition,denoted by ‘+’, and multiplication, denoted by ‘·’ or omitted, over F are per-formed between scalars, such that the following axioms are satisfied

(1) (Closure) If a, b ∈ F, then a + b ∈ F and ab ∈ F.

(2) (Commutativity) For a, b ∈ F, there hold a + b = b + a and ab = ba (3) (Associativity) For a, b, c ∈ F, there hold (a + b) + c = a + (b + c) and

(4) (Distributivity) For a, b, c ∈ F, there hold a(b + c) = ab + ac.

(5) (Existence of zero) There is a scalar, called zero, denoted by 0, such that

a + 0 = a for any a ∈ F.

(6) (Existence of unity) There is a scalar different from zero, called one,

denoted by 1, such that 1a = a for any a ∈ F.

(7) (Existence of additive inverse) For any a∈ F, there is a scalar, denoted by

−a or (−a), such that a + (−a) = 0.

(8) (Existence of multiplicative inverse) For any a∈ F \ {0}, there is a scalar,

denoted by a−1, such that aa−1= 1

It is easily seen that zero, unity, additive and multiplicative inverses are allunique Besides, a field consists of at least two elements

With the usual addition and multiplication, the sets of rational numbers, realnumbers, and complex numbers, denoted byQ, R, and C, respectively, are allfields These fields are infinite fields However, the set of integers,Z, is not afield because there is a lack of multiplicative inverses for its non-unit elements

Let p be a prime (p = 2, 3, 5, ) and set pZ = {n ∈ Z | n = kp, k ∈ Z}.

ClassifyZ into the so-called cosets modulo pZ, that is, some non-overlapping

subsets ofZ represented as [i] (i ∈ Z) such that

[i] = {j ∈ Z | i − j ∈ pZ}. (1.1.1)

It is clear thatZ is divided into exactly p cosets, [0], [1], , [p − 1] Use

Zp to denote the set of these cosets and pass the additive and multiplicativeoperations inZ over naturally to the elements in Zpso that

[i] + [j] = [i + j], [i][j] = [ij]. (1.1.2)

It can be verified that, with these operations,Zpbecomes a field with its ous zero and unit elements,[0] and [1] Of course, p[1] = [1] + · · · + [1] (p

obvi-terms)= [p] = [0] In fact, p is the smallest positive integer whose cation with unit element results in zero element A number of such a property

multipli-is called the charactermultipli-istic of the field Thus,Zp is a field of characteristic p.

ForQ, R, and C, since no such integer exists, we say that these fields are of

characteristic 0.

1.1.2 Vector spaces

LetF be a field Consider the set of n-tuples, denoted by F n

, with elementscalled vectors arranged in row or column forms such as

The setFn, modeled over the fieldF and equipped with the above operations,

is a prototype example of a vector space

More generally, we say that a set U is a vector space over a field F if U is non-empty and there is an operation called addition, denoted by ‘+’, between the elements of U , called vectors, and another operation called scalar mul-

following axioms hold

(1) (Closure) For u, v ∈ U, we have u + v ∈ U For u ∈ U and a ∈ F, we have au ∈ U.

(2) (Commutativity) For u, v ∈ U, we have u + v = v + u.

(3) (Associativity of addition) For u, v, w ∈ U, we have u + (v + w) =

(u + v) + w.

(4) (Existence of zero vector) There is a vector, called zero and denoted by 0,

such that u + 0 = u for any u ∈ U.

(5) (Existence of additive inverse) For any u ∈ U, there is a vector, denoted

as ( −u), such that u + (−u) = 0.

(6) (Associativity of scalar multiplication) For any a, b ∈ F and u ∈ U, we have a(bu) = (ab)u.

(7) (Property of unit scalar) For any u ∈ U, we have 1u = u.

(8) (Distributivity) For any a, b ∈ F and u, v ∈ U, we have (a+b)u = au+bu and a(u + v) = au + av.

As in the case of the definition of a field, we see that it readily follows fromthe definition that zero vector and additive inverse vectors are all unique in

a vector space Besides, any vector multiplied by zero scalar results in zero

vector That is, 0u = 0 for any u ∈ U.

Other examples of vector spaces (with obviously defined vector addition andscalar multiplication) include the following

(1) The set of all polynomials with coefficients inF defined by

P = {a0 + a1t + · · · + a n t n | a0, a1, , a n ∈ F, n ∈ N}, (1.1.6)

where t is a variable parameter.

(2) The set of all real-valued continuous functions over the interval[a, b] for

a, b ∈ R and a < b usually denoted by C[a, b].

(3) The set of real-valued solutions to the differential equation

a ndn x

dt n + · · · + a1

dx

dt + a0x = 0, a0, a1, , a n ∈ R. (1.1.7)(4) In addition, we can also consider the set of arrays of scalars inF consisting

of m rows of vectors inFn or n columns of vectors inFmof the form

matrix and each a ij is called an entry or component of the matrix The set

of all m × n matrices with entries in F may be denoted by F(m, n) In

particular,F(m, 1) or F(1, n) is simply F morFn Elements inF(n, n) are also called square matrices.

Of course, A t ∈ F(n, m) Simply put, A t is a matrix obtained from taking the

row (column) vectors of A to be its corresponding column (row) vectors For A ∈ F(n, n), we say that A is symmetric if A = A t , or skew-symmetric

or anti-symmetric if A t = −A The sets of symmetric and anti-symmetric

matrices are denoted byFS (n, n)andFA (n, n), respectively

It is clear that (A t ) t = A.

(1) (Commutativity) u · v = v · u for any u, v ∈ F n

That is, the dot product of u and v may be viewed as a matrix product of the

1× n matrix u t and n × 1 matrix v as well.

Matrix product (or matrix multiplication) enjoys the following properties.

(1) (Associativity of scalar multiplication) a(AB) = (aA)B = A(aB) for any a ∈ F and any A ∈ F(m, k), B ∈ F(k, n).

(2) (Distributivity) A(B + C) = AB + AC for any A ∈ F(m, k) and B, C ∈ F(k, n); (A + B)C = AC + BC for any A, B ∈ F(m, k) and C ∈ F(k, n) (3) (Associativity) A(BC) = (AB)C for any A ∈ F(m, k), B ∈ F(k, l),

then, formally, we have

Theorem 1.1 For A ∈ F(m, k) and B ∈ F(k, n), there holds

(AB) t = B t A t (1.1.15)The proof of this basic fact is assigned as an exercise

Other matrices in F(n, n) having interesting properties include the

following

(1) Diagonal matrices are of the form A = (a ij ) with a ij = 0 whenever

(2) Lower triangular matrices are of the form A = (a ij ) with a ij = 0

when-ever j > i The set of lower triangular matrices is denoted asFL (n, n)

(3) Upper triangular matrices are of the form A = (a ij ) with a ij = 0

when-ever i > j The set of upper triangular matrices is denoted asFU (n, n).There is a special element in F(n, n), called the identity matrix, or unit

diagonal entries are all 1 (unit scalar) and off-diagonal entries are all 0 For

any A ∈ F(n, n), we have AI = IA = A.

1.1 Vector spaces 7

Definition 1.2 A matrix A ∈ F(n, n) is called invertible or nonsingular if there

is some B ∈ F(n, n) such that

In this situation, B is unique (cf Exercise 1.1.7) and called the inverse of A and denoted by A−1.

If A, B ∈ F(n, n) are such that AB = I, then we say that A is a left inverse

of B and B a right inverse of A It can be shown that a left or right inverse is simply the inverse In other words, if A is a left inverse of B, then both A and

Bare invertible and the inverses of each other

If A ∈ R(n, n) enjoys the property AA t = A t A = I, then A is called an

for taking the complex conjugate of A and use A†to denote taking the complex

conjugate of the transpose of A, A† = A t

, which is also commonly referred

to as taking the Hermitian conjugate of A If A ∈ C(n, n), we say that A is

or anti-Hermitian, if A† = −A If A ∈ C(n, n) enjoys the property AA† =

A†A = I, then A is called a unitary matrix We will see the importance of

these notions later

Exercises

1.1.1 Show that it follows from the definition of a field that zero, unit, additive,and multiplicative inverse scalars are all unique

1.1.2 Let p ∈ N be a prime and [n] ∈ Z p Find−[n] and prove the existence

of[n]−1when 5, find−[4] and [4]−1

1.1.3 Show that it follows from the definition of a vector space that both zeroand additive inverse vectors are unique

1.1.4 Prove the associativity of matrix multiplication by showing that

1.1.7 Prove that the inverse of an invertible matrix is unique by showing the

fact that if A, B, C ∈ F(n, n) satisfy AB = I and CA = I then B = C 1.1.8 Let A ∈ C(n, n) Show that A is Hermitian if and only if iA is anti-

Hermitian

1.2 Subspaces, span, and linear dependence

Let U be a vector space over a field F and V ⊂ U a non-empty subset of U.

We say that V is a subspace of U if V is a vector space over F with the

inherited addition and scalar multiplication from U It is worth noting that, when checking whether a subset V of a vector space U becomes a sub-

space, one only needs to verify the closure axiom (1) in the definition of avector space since the rest of the axioms follow automatically as a conse-quence of (1)

The two trivial subspaces of U are those consisting only of zero vector,{0},

and U itself A nontrivial subspace is also called a proper subspace.

Consider the subsetP n (n ∈ N) of P defined by

We also say that u is linearly spanned by u1, , u k or linearly dependent on

u1, , u k Therefore, zero vector 0 is linearly dependent on any finite set ofvectors

If U = Span{u1, , uk }, we also say that U is generated by the vectors

u1, , u k

For P n defined in (1.2.1), we have P n = Span{1, t, , t n} Thus

P n is generated by 1, t, , t n Naturally, for two elements p, q in

1.2 Subspaces, span, and linear dependence 9

P n , say p(t) = a0+ a1t + · · · + a n t n , q(t) = b0+ b1t + · · · + b n t n, we

identify p and q if and only if all the coefficients of p and q of like powers of

t coincide inF, or, a i = b i for all i = 0, 1, , n.

InFn, define

e1= (1, 0, 0, , 0), e2= (0, 1, 0, , 0), e n = (0, 0, , 0, 1).

(1.2.5)ThenFn = Span{e1, e2, , e n} and Fn is generated by e1, e2, , e n

Thus, for S0defined in (1.2.2), we have

(x1, x2, , x n ) = −(x2+ · · · + x n )e1+ x2e2+ · · · + x n e n

= x2(e2− e1) + · · · + x n (e n − e1), (1.2.6)

where x2, , x nare arbitrarily taken fromF Therefore

S0 = Span{e2− e1, , en − e1}. (1.2.7)For F(m, n), we define M ij ∈ F(m, n) to be the vector such that all its entries vanish except that its entry at the position (i, j ) (at the ith row and j th column) is 1, i = 1, , m, j = 1, , n We have

F(m, n) = Span{M ij | i = 1, , m, j = 1, , n}. (1.2.8)

The notion of spans can be extended to cover some useful situations Let U

be a vector space and S be a (finite or infinite) subset of U Define

Span(S)= the set of linear combinations

of all possible finite subsets of S. (1.2.9)

It is obvious that Span(S) is a subspace of U If U = Span(S), we say that U

is spanned or generated by the set of vectors S.

The above discussion motivates the following formal definition

Definition 1.3 Let u1, , u m be m vectors in the vector space U over a

fieldF We say that these vectors are linearly dependent if one of them may

be written as a linear span of the rest of them or linearly dependent on the rest

of them Or equivalently, u1, , u mare linearly dependent if there are scalars

overF with unknowns x1, , x n

Theorem 1.4 In the system (1.2.13), if m < n, then the system has a nontrivial

solution (x1, , x n )

The beginning situation is m+ n = 3 when m = 1 and n = 2 It is clear that

we always have a nontrivial solution

Assume that the statement of the theorem is true when m + n ≤ k where

Let m + n = k + 1 If k = 3, the condition m < n implies m = 1, n = 3 and the existence of a nontrivial solution is obvious Assume then k≥ 4 If all the

coefficients of the variable x1in (1.2.13) are zero, i.e a11 = · · · = a m1 = 0,

then x1 = 1, x2 = · · · = x n = 0 is a nontrivial solution So we may assume

one of the coefficients of x1is nonzero Without loss of generality, we assume

a11

first equation in (1.2.13) by a11 if necessary, we can further assume a11 = 1

Then, adding the (−a i1) -multiple of the first equation into the ith equation, in (1.2.13), for i = 2, , m, we arrive at

The system below the first equation in (1.2.14) contains m− 1 equations and

n − 1 unknowns x2, , x n Of course, m − 1 < n − 1 So, in view of the

1.2 Subspaces, span, and linear dependence 11

inductive assumption, it has a nontrivial solution Substituting this nontrivialsolution into the first equation in (1.2.14) to determine the remaining unknown

x1, we see that the existence of a nontrivial solution to the original system(1.2.13) follows

The importance of Theorem 1.4 is seen in the following

Theorem 1.5 Any set of more than m vectors in Span {u1, , u m } must be

linearly dependent.

Proof Let v1, , v n ∈ Span{u1, , u m } be n vectors where n > m.

Consider the possible linear dependence relation

1.2.1 Let U1and U2be subspaces of a vector space U Show that U1∪ U2is

a subspace of U if and only if U1⊂ U2or U2⊂ U1

1.2.2 LetP n denote the vector space of the polynomials of degrees up to n

over a fieldF expressed in terms of a variable t Show that the vectors

1, t, , t ninP nare linearly independent

1.2.3 Show that the vectors inFndefined in (1.2.5) are linearly independent

1.2.4 Show that S0defined in (1.2.2) may also be expressed as

S0= Span{e1− e n , , e n−1− e n }, (1.2.20)and deduce that, inRn

2n−1−1



, e1−e n , , e n−1− e n , (1.2.21)

are linearly dependent (n≥ 4)

1.2.5 Show thatFS (n, n),FA (n, n),FD (n, n),FL (n, n), andFU (n, n)are allsubspaces ofF(n, n).

1.2.6 Let u1, , u n (n ≥ 2) be linearly independent vectors in a vector

space U and set

v i−1= u i−1+ u i , i = 2, , n; v n = u n + u1. (1.2.22)

Investigate whether v1, , v nare linearly independent as well.1.2.7 Let F be a field For any two vectors u = (a1, , an ) and v =

(b1, , b n )inFn

(n ≥ 2), viewed as matrices, we see that the

ma-trix product A = u t vlies inF(n, n) Prove that any two row vectors of

A are linearly dependent What happens to the column vectors of A?

1.2.8 Consider a slightly strengthened version of the second part of Exercise

1.2.1 above: Let U1, U2 be subspaces of a vector space U , U1

1.2.10 For A ∈ F(m, n) and B ∈ F(n, m) with m > n show that AB as an

element inF(m, m) can never be invertible.

1.3 Bases, dimensionality, and coordinates 13

1.3 Bases, dimensionality, and coordinates

Let U be a vector space over a field F, take u1, , u n ∈ U, and set

V = Span{u1, , un} Eliminating linearly dependent vectors from the set

{u1, , u n } if necessary, we can certainly assume that the vectors u1, , u n

are already made linearly independent Thus, any vector u ∈ V may take

then, combining the above two relations, we have (a1 − b1)u1 + · · · +

(a n − b n )u n = 0 Since u1, , un are linearly independent, we obtain

a1= b1, , a n = b nand the uniqueness follows

Furthermore, if there is another set of vectors v1, , v m in U such that

Span{v1, , v m } = Span{u1, , u n }, (1.3.3)

then m ≥ n in view of Theorem 1.5 As a consequence, if v1, , vmare also

linearly independent, then m = n This observation leads to the following.

Definition 1.6 If there are linearly independent vectors u1, , u n ∈ U such that U = Span{u1, , u n }, then U is said to be finitely generated and the set

of vectors{u1, , un } is called a basis of U The number of vectors in any basis of a finitely generated vector space, n, is independent of the choice of the basis and is referred to as the dimension of the finitely generated vector space, written as dim(U ) = n A finitely generated vector space is also said to be

of finite dimensionality or finite dimensional If a vector space U is not finite dimensional, it is said to be infinite dimensional, also written as dim(U )= ∞

As an example of an infinite-dimensional vector space, we show that when

R is regarded as a vector space over Q, then dim(R) = ∞ In fact, recall that a real number is called an algebraic number if it is the zero of a polyno-

mial with coefficients inQ We also know that there are many non-algebraicnumbers in R, called transcendental numbers Let τ be such a transcenden- tal number Then for any n = 1, 2, the numbers 1, τ, τ2, , τ n are lin-early independent in the vector spaceR over the field Q Indeed, if there are

r0, r1, r2, , r n∈ Q so that

r0+ r1τ + r2τ2+ · · · + r n τ n = 0, (1.3.4)

and at least one number among r0, r1, r2, , r n is nonzero, then τ is the zero

of the nontrivial polynomial

which violates the assumption that τ is transcendental Thus R is infinitedimensional overQ

The following theorem indicates that it is fairly easy to construct a basis for

a finite-dimensional vector space

Theorem 1.7 Let U be an n-dimensional vector space over a field F Any n

linearly independent vectors in U form a basis of U

Proof Let u1, , u n ∈ U be linearly independent vectors We only need to show that they span U In fact, take any u ∈ U We know that u1, , un , u

are linearly dependent So there is a nonzero vector (a1, , a n , a) ∈ Fn+1

These scalars, a1, , a n are called the coordinates, and (a1, , a n )∈ Fnthe

coordinate vector, of the vector u with respect to the basis {u1, , un}

It will be interesting to investigate the relation between the coordinate tors of a vector under different bases

vec-LetU = {u1 , , u n } and V = {v1, , v n} be two bases of the vector space

U For u ∈ U, let (a1, , a n )∈ Fn and (b1, , b n )∈ Fnbe the coordinate

vectors of u with respect to U and V, respectively Thus

1.3 Bases, dimensionality, and coordinates 15

The n × n matrix A = (a ij ) is called a basis transition matrix or basis change

matrix Inserting (1.3.9) into (1.3.8), we have

⎛

⎜

⎝

a1

1.3.1 Let U be a vector space with dim(U ) = n ≥ 2 and V a subspace of U

with a basis{v1, , v n−1} Prove that for any u ∈ U \ V the vectors

u, v1, , v n−1form a basis for U

1.3.2 Show that dim(F(m, n)) = mn.

1.3.3 Determine dim(F S (n, n)) , dim(F A (n, n)) , and dim(F D (n, n))

1.3.4 LetP be the vector space of all polynomials with coefficients in a field

F Show that dim(P) = ∞.

1.3.5 Consider the vector space R3

and the bases U = {e1 , e2, e3} and

V = {e1 , e1+ e2, e1+ e2+ e3} Find the basis transition matrix A from

U into V satisfying V = UA Find the coordinate vectors of the given

vector (1, 2, 3) ∈ R3 with respect to the basesU and V, respectively,

and relate these vectors with the matrix A.

1.3.6 Prove that a basis transition matrix must be invertible

1.3.7 Let U be an n-dimensional vector space over a field F where n ≥ 2

(say) Consider the following construction

(i) Take u1∈ U \ {0}.

(ii) Take u2∈ U \ Span{u1}

(iii) Take (if any) u3∈ U \ Span{u1, u2}

(iv) In general, take u i ∈ U \ Span{u1, , u i−1} (i ≥ 2).

Show that this construction will terminate itself in exactly n steps, that is, it will not be possible anymore to get u n+1, and that the vectors

u1, u2, , u n so obtained form a basis of U

1.4 Dual spaces

Let U be an n-dimensional vector space over a field F A functional (also called

a form or a 1-form) f over U is a linear function f : U → F satisfying

f (u + v) = f (u) + f (v), u, v ∈ U; f (au) = af (u), a ∈ F, u ∈ U.

(1.4.1)

Let f, g be two functionals Then we can define another functional called the sum of f and g, denoted by f + g, by

(f + g)(u) = f (u) + g(u), u ∈ U. (1.4.2)

Similarly, let f be a functional and a ∈ F We can define another functional

called the scalar multiple of a with f , denoted by af , by

It is a simple exercise to check that these two operations make the set of all

functionals over U a vector space over F This vector space is called the dual

n to denote the elements in U

correspond-ing to the vectors e1, , e ninFn

given by (1.2.5) Then we have

n are linearly independent and span Ubecause an

element f of Usatisfying (1.4.5) is simply given by

f = f1u

1+ · · · + f n u

In other words,{u1, , u

n } is a basis of U, commonly called the dual basis

of U with respect to the basis{u1, , u n } of U In particular, we have seen that U and Uare of the same dimensionality.

Let U = {u1 , , u n } and V = {v1, , v n} be two bases of the

vec-tor space U Let their dual bases be denoted by U = {u1, , u

which leads to a

ij = a j i (i, j = 1, , n) In other words, we have arrived at

the correspondence relation

⎛

⎜

⎝

u 1

Comparing the above results with those established in the previous section,

we see that, with respect to bases and dual bases, the coordinates vectors in U and Ufollow ‘opposite’ rules of correspondence For this reason, coordinate

vectors in U are often called covariant vectors, and those in Ucontravariant

vectors.

Using the relation stated in (1.4.8), we see that we may naturally view

u1, , u n as elements in (U) = U so that they form a basis of U dual

to{u

1, , u

n} since

i ) = a i (i = 1, , n), we have

u = a1u1+ · · · + a n u n (1.4.21)

In this way, we see that U may be identified with U In other words, we

have seen that the identification just made spells out the relationship

u(u) = u(u), u ∈ U, u∈ U, (1.4.24)simply says that the ‘pairing’·, · as given in (1.4.23) is symmetric:

It is clear that S0 is always a subspace of U regardless of whether S is a

subspace of U Likewise, for any nonempty subset S⊂ U, we can define the

annihilator of S, S0, as the subset

S0 = {u ∈ U | u, u  = 0, ∀u∈ S} (1.4.27)

of U Of course, S0is always a subspace of U

Exercises

1.4.1 LetF be a field Describe the dual spaces Fand (F2).

1.4.2 Let U be a finite-dimensional vector space Prove that for any vectors

1.4.3 Let U be a finite-dimensional vector space and f, g ∈ U For any v∈

U , f (v) = 0 if and only if g(v) = 0 Show that f and g are linearly

dependent

1.4.4 LetF be a field and

(ii) Find a basisB of U which is dual to B

1.4.6 Let U be an n-dimensional vector space and V be an m-dimensional subspace of U Show that the annihilator V0is an (n − m)-dimensional subspace of U In other words, there holds the dimensionality equation

dim(V ) + dim(V0) = dim(U). (1.4.31)

1.4.7 Let U be an n-dimensional vector space and V be an m-dimensional subspace of U Show that V00= (V0)0= V

1.5 Constructions of vector spaces

Let U be a vector space and V , W its subspaces It is clear that V ∩ W is also

a subspace of U but V ∪ W in general may fail to be a subspace of U The smallest subspace of U that contains V ∪ W should contain all vectors in U of the form v + w where v ∈ V and w ∈ W Such an observation motivates the

following definition

Definition 1.9 If U is a vector space and V , W its subspaces, the sum of V

and W , denoted by V + W, is the subspace of U given by

V + W ≡ {u ∈ U | u = v + w, v ∈ V, w ∈ W}. (1.5.1)

Checking that V + W is a subspace of U that is also the smallest subspace

of U containing V ∪ W will be assigned as an exercise.

1.5 Constructions of vector spaces 21

Now letB0 = {u1, , u k } be a basis of V ∩ W Expand it to obtain bases for V and W , respectively, of the forms

B V ={u1, , uk , v1, , v l }, B W = {u1, , uk , w1, , w m } (1.5.2) From the definition of V + W, we get

Theorem 1.10 The following general dimensionality formula

dim(V + W) = dim(V ) + dim(W) − dim(V ∩ W) (1.5.6)

is valid for the sum of any two subspaces V and W of finite dimensions in a vector space.

Of great importance is the situation when dim(V ∩W) = 0 or V ∩W = {0}.

In this situation, the sum is called direct sum, and rewritten as V ⊕ W Thus,

we have

dim(V ⊕ W) = dim(V ) + dim(W). (1.5.7)Direct sum has the following characteristic

Theorem 1.11 The sum of two subspaces V and W of U is a direct sum if and

v ∈ V and a unique vector w ∈ W.

Proof Suppose first V ∩ W = {0} For any u ∈ V + W, assume that it may

be expressed as

From (1.5.8), we have v1− v2= w2− w1which lies in V ∩ W So v1− v2=

w2− w1= 0 and the stated uniqueness follows

Suppose that any u ∈ V + W can be expressed as u = v + w for some unique v

zero vector 0 may be expressed as 0= x + (−x) with x ∈ V and (−x) ∈ W,

which violates the stated uniqueness since 0= 0 + 0 with 0 ∈ V and 0 ∈ W,

as well

Let V be a subspace of an n-dimensional vector space U and B V =

{v1, , vk , w1, , w l }, where k + l = n Define

W = Span{w1, , wl }. (1.5.9)

Then we obtain U = V ⊕ W The subspace W is called a linear complement,

or simply complement, of V in U Besides, the subspaces V and W are said to

be mutually complementary in U

We may also build up a vector space from any two vector spaces, say V and

W, over the same fieldF, as a direct sum of V and W To see this, we construct

vectors of the form

u = (v, w), v ∈ V, w ∈ W, (1.5.10)and define vector addition and scalar multiplication component-wise by

au = a(v, w) = (aw, av), v ∈ V, w ∈ W, a ∈ F. (1.5.12)

It is clear that the set U of all vectors of the form (1.5.10) equipped with the

vector addition (1.5.11) and scalar multiplication (1.5.12) is a vector space over

F Naturally we may identify V and W with the subspaces of U given by

˜V = {(v, 0) | v ∈ V }, W˜ = {(0, w) | w ∈ W}. (1.5.13)

Of course, U = ˜V ⊕ ˜ W Thus, in a well-understood sense, we may also rewrite

this relation as U = V ⊕ W as anticipated Sometimes the vector space U so constructed is also referred to as the direct product of V and W and rewritten

as U = V ×W In this way, R2

may naturally be viewed asR×R, for example

1.5 Constructions of vector spaces 23

More generally, let V1, , V k be any k subspaces of a vector space U In a

similar manner we can define the sum

among V1, , V k, is not sufficient anymore to ensure (1.5.15)

To illustrate this subtlety, let us consider V = F2

and take

V1= Span



10



, V2= Span



11

1.5.2 Consider the vector space of all n×n matrices over a field F, denoted by F(n, n) As before, use F S (n, n)andFA (n, n)to denote the subspaces ofsymmetric and anti-symmetric matrices Assume that the characteristic

ofF is not equal to 2 For any M ∈ F(n, n), rewrite M as

2(M + M t )+1

2(M − M t ). (1.5.20)Check that 1

2(M − M t )∈ FA (n, n) Usethis fact to prove the decomposition

F(n, n) = F S (n, n)⊕ FA (n, n). (1.5.21)What happens when the characteristic ofF is 2 such as when F = Z2?1.5.3 Show thatFL (n, n)∩ FU (n, n)= FD (n, n)

1.5.4 UseFL (n, n)andFU (n, n)inF(n, n) to give an example for the

dimen-sionality relation (1.5.6)

1.5.5 Let X = C[a, b] (a, b ∈ R and a < b) be the vector space of all

real-valued continuous functions over the interval[a, b] and

(ii) For a = 0, b = 1, and f (t) = t2+ t − 1, find the unique c ∈ R and

g ∈ Y such that f (t) = c + g(t) for all t ∈ [0, 1].

1.5.6 Let U be a vector space and V , W its subspaces such that U = V + W.

If X is subspace of U , is it true that X = (X ∩ V ) + (X ∩ W)?

1.5.7 Let U be a vector space and V , W, X some subspaces of U such that

(iii) There holds the dimensionality relation

dim(V ) = dim(V1)+ · · · + dim(V k ). (1.5.26)

represents the line passing through the origin and along (or opposite to) the

direction of v More generally, for any u∈ R2, the coset

[u] = {u + w | w ∈ V } = {x ∈ R2| x − u ∈ V } (1.6.2)

represents the line passing through the vector u and parallel to the vector v.

Naturally, we define[u1] + [u2] = {x + y | x ∈ [u1], y ∈ [u2]} and claim

[u1] + [u2] = [u1+ u2]. (1.6.3)

In fact, let z ∈ [u1] + [u2] Then there exist x ∈ [u1] and y ∈ [u2] such that

z = x + y Rewrite x, y as x = u1+ w1, y = u2+ w2for some w1, w2∈ V Hence z = (u1+ u2) + (w1+ w2) , which implies z ∈ [u1+ u2] Conversely,

if z ∈ [u1+ u2], then there is some w ∈ V such that z = (u1+ u2) + w =

(u1 + w) + u2 Since u1+ w ∈ [u1] and u2∈ [u2], we see that z ∈ [u1] + [u2].Hence the claim follows

From the property (1.6.3), we see clearly that the coset[0] = V serves as an

additive zero element among the set of all cosets

Similarly, we may also naturally define a [u] = {ax | x ∈ [u]} for a ∈ R where a

would be a single-point set consisting of zero vector only We claim

a [u] = [au], a ∈ R \ {0}. (1.6.4)

In fact, if z ∈ a[u], there is some x ∈ [u] such that z = ax Since x ∈ [u], there is some w ∈ V such that x = u + w So z = au + aw which implies z ∈ [au] Conversely, if z ∈ [au], then there is some w ∈ V such that z = au + w Since z = a(u + a−1w) , we get z ∈ a[u] So (1.6.4) is established.