Handbook of Industrial Automation - Richard L. Shell and Ernest L. Hall Part 2 ppsx

Matrix addition, scalar multiplication, and multiplication of matrices are de®ned as follows.. The rules for matrix addition and scalar multiplica-tion are summarized in the following th

2.2.4 Equivalence Relations

Now we concentrate our attention on the properties of

a binary relation < de®ned in a set X

1 < is called re¯exive in X, if and only if, for all

x 2 X, x<x

2 < is called symmetrical in X, if and only if, for

all x; y 2 X, x<y implies y<x

3 < is called transitive in X, if and only if, for all

x; y; z 2 X, x<y and y<z implies x<z

A binary relation < is called an equivalence relation

on X if it is re¯exive, symmetrical and transitive

As an example, consider the set Z of integer

num-bers and let n be an arbitrary positive integer The

congruence relation modulo n on the set Z is de®ned

by x  y (modulo n) if and only if x y ˆ kn for some

k 2 Z The congruence relation is an equivalence

rela-tion on Z

Proof

1 For each x 2 Z, x x ˆ 0n This means that x 

x (modulo n) which implies that the congruence

relation is re¯exive

2 If x  y (modulo n), x y ˆ kn for some k 2 Z

Multiplying both sides of the last equality by 1,

we get y x ˆ kn which implies that y  x

(modulo n) Thus, the congruence relation is

symmetrical

3 If x  y (modulo n) and y  z (modulo n), we

have x y ˆ k1n and y z ˆ k2n for some k1

and k2 in Z Writing x z ˆ x y ‡ y z, we

get x z ˆ …k1‡ k2†n Since k1‡ k22 Z, we

conclude that x  z (modulo n) This shows

that the congruence relation is transitive

From 1±3 it follows that the congruence relation

(modulo n) is an equivalence relation on the set Z of

integer numbers

In particular, we observe that if we choose n ˆ 2,

then x  y (modulo 2) means that x y ˆ 2k for some

integer k This is equivalent to saying that either x and

y are both even or both x and y are odd In other

words, any two even integers are equivalent, any two

odd integers are equivalent but an even integer can not

be equivalent to an odd one The set Z has been

divided into two disjoint subsets whose union gives

Z One such proper subset is the set of even integers

and the other one is the set of odd integers

2.2.4.1 Partitions and Equivalence RelationsThe situation described in the last example is quitegeneral To study equivalence relations in more detail

we need to introduce the concepts of partition andequivalence class

Given a nonempty set X, a partition S of X is acollection of nonempty subsets of X such that

1 If A; B 2 S; A 6ˆ B, then A \ B ˆ ;

2 SA2SA ˆ X

If < is an equivalence relation on a nonempty set X,for each member x 2 X the equivalence class associatedwith x, denoted x=<, is given by

x=< ˆ fz 2 X : x<xgThe set x=< is a subset of X and, consequently, anelement of the power set P…X† Thus, the set

The converse of this statement also holds; that is,each partition of X generates an equivalence relation

on X In fact, if S is a partition of a nonempty set X,

we can de®ne the relationX=S ˆ f…x; y† 2 X  X : x 2 s and y 2 s for some

s 2 SgThis is an equivalence relation on X, and the equiva-lent classes induced by it are precisely the elements ofthe partition S, i.e.,

X=…X=S† ˆ SIntuitively, equivalence relations and partitions aretwo different ways to describe the same collection ofsubsets

2.2.5 Order RelationsOrder relations constitute another common type ofrelations Once again, we begin by introducing severalde®nitions

A binary relation < in X is said to be trical if for all x; y 2 X; x<y and y<x imply

antisymme-x ˆ y

A binary relation < in X is asymmetrical if for any

x; y 2 X; x<y implies that y<x does not hold In

other words, we can not have x<y an y<x both

true

A binary relation < in X is a partial ordering of X if

and only if it is re¯exive, antisymmetrical, and

transitive The pair …X; <† is called and ordered

set

A binary relation in X is a strict (or total) ordering

of X if and only if it is asymmetrical and

It is a simple task to check that <1is a partial ordering

of the set X It requires a little extra thinking to realize

that now the least and the greatest elements of X have

been identi®ed

On the same set X, the binary relation de®ned by

<1ˆ f…x; y† : x; y 2 X and x < yg

ˆf…1; 2†; …1; 3†; …2; 3†g

is an example of a strict ordering of X

It is also possible to establish a correspondencebetween partial orderings and strict orderings of a set:

If <1 is a partial ordering of X, then the binaryrelation <2 de®ned in X by

x<2y if and only if x<1y and x 6ˆ y

3.1.1 Shapes and Sizes

Throughout this chapter, F will denote a ®eld The

four most commonly used ®elds in linear algebra are

Q ˆ rationals, R ˆ reals, C ˆ complex numbers and

Zpˆ the integers modulo a prime p We will also let

N ˆ f1; 2; g, the set of natural numbers

De®nition 1 Let m; n 2 N An m  n matrix A with

entries from F is a rectangular array of m rows and n

columns of numbers from F

The most common notation used to represent an m

 n (read ``m by n'') matrix A is displayed in Eq (1):

If A is the m  n matrix displayed in Eq (1), then the

®eld elements aij …i ˆ 1; ; m; j ˆ 1; ; n† are called

the entries of A We will also use ‰AŠij to denote the

i; jth entry of A Thus, aijˆ ‰AŠij is the element of F

which lies in the ith row and jth column of A By the

size of A, we will mean the expression m  n Thus, size

…A† ˆ m  n if A has m rows and n columns Notice

that the size of a matrix is a pair of positive integers

with a ``'' put between them Negative numbers and

zero are not allowed to appear in the size of a matrix

De®nition 2 The set of all m  n matrices with entriesfrom F will be denoted by Mmn…F†

Matrices of various shapes are given special names

in linear algebra Here is a brief list of some of themore famous shapes and some pictures to illustratethe de®nitions

an

0B

1C

size ˆ 1  1; 2  1; ; n  1 …2b†

3 An m  n matrix is called a row vector if m ˆ 1

…a†; …a; b†; ; …a1; ; an†size ˆ 1  1; 1  2; ; 1  n …2c†

4 An m  n matrix A is upper triangular if ‰AŠijˆ

0 whenever i > j

43

am1 am2 am3 amm 0 0

0BBBB

1CCCC

C if m ˆ n …2f†

7 A square matrix is symmetric (skew-symmetric)

if ‰AŠijˆ ‰AŠji… ‰AŠji† for all i; j ˆ 1; ; n

1CA;

skew-symmetric

…2h†

De®nition 3 A submatrix of A is a matrix obtained

from A by deleting certain rows and/or columns of A

A partition of A is a series of horizontal and vertical lines

drawn in A which divide A into various submatrices

The most important partitions of a matrix A are itscolumn and row partitions

1C

A 2 Mmn…F†

1 For each j ˆ 1; ; n, the m  1 submatrix

Colj…A† ˆ

a1j

amj

0B

1C

of A is called the jth column of A

2 A ˆ …Col1…A† j Col2…A† j j Coln…A†† is calledthe column partition of A

3 For each i ˆ 1; ; m, the 1  n submatrixRowi…A† ˆ …ai1; ; ain† of A is called the ithrow of A

1C

is called the row partition of A

We will cut down on the amount of space required

to show a column or row partition by employing thefollowing notation In De®nition 4, let jˆ Colj…A† for

j ˆ 1; ; n and let iˆ Rowi…A† for i ˆ 1; ; m.Then the column partition of A will be written A ˆ…1 j 2j j n† and the row partition of A will bewritten A ˆ …1; 2; ; m†

3.1.2 Matrix ArithmeticDe®nition 5 Two matrices A and B with entries from

F are said to be equal if size …A† ˆ size …B† and ‰AŠijˆ

‰BŠij for all i ˆ 1; ; m; j ˆ 1; ; n Here

m  n ˆ size …A†

If A and B are equal, we will write A ˆ B Notice

that two matrices which are equal have the same size

Thus, the 1  1 matrix (0) is not equal to the 1  2

matrix (0,0) Matrix addition, scalar multiplication,

and multiplication of matrices are de®ned as follows

De®nition 6

1 Let A, B 2 Mmn…F† Then A ‡ B is the m  n

matrix whose i,jth entry is given by ‰A ‡ BŠijˆ

‰AŠij‡ ‰BŠijfor all i ˆ 1; ; m and j ˆ 1; ; n

2 If A 2 Mmn…F† and x 2 F, then xA is the m  n

matrix whose i,jth entry is given by ‰xAŠijˆ

x‰AŠij for all i ˆ 1; ; m and j ˆ 1; ; n

3 Let A 2 Mmn…F† and C 2 Mnp…F† Then AC is

the m  p matrix whose i,jth entry is given by

‰ACŠijˆ Xn

kˆ1

‰AŠik‰CŠkjfor i ˆ 1; ; m; j ˆ 1; ; p

Notice that addition is de®ned only for matrices of

the same size Multiplication is de®ned only when the

number of columns of the ®rst matrix is equal to the

number of rows of the second matrix

The rules for matrix addition and scalar

multiplica-tion are summarized in the following theorem:

Theorem 1 Let A, B, C 2 Mmn…F† Let x, y 2 F

2 of De®nition 6

Theorem 1(2) implies matrix addition is associative

It follows from this statement that expressions of theform x1A1‡


‡ xrAr…Ai2 Mmn…F† and xi2 F† can

be used unambiguously Any placement of parentheses

in this expression will result in the same answer Thesum x1A1‡


‡ xrAris called a linear combination of

A1; ; Ar The numbers x1; ; xr are called the lars of the linear combination

6 x…AB† ˆ …xA†B ˆ A…xB†:

In Theorem 2(4), the zero denotes the zero matrix ofvarious sizes In Theorem 2(5), In denotes the n  nidentity matrix This is the diagonal matrix given by

‰InŠjjˆ 1 for all j ˆ 1; ; n Theorem 2 implies Mnn…F† is an associative algebra with identity [2, p 36] overthe ®eld F

Consider the following system of m equations inunknowns x1; ; xn:

A B ˆ

b1

bm

0B

@

1C

A X ˆ

x1

xn

0B

@

1CA

…8†

Using matrix multiplication, the system of linear

equations in Eq (7) can be written succinctly as

We will let Fndenote the set of all column vectors of

size n Thus, Fnˆ Mn1…F† A column vector  2 Fn is

called a solution to Eq (9) if A ˆ B The m  n matrix

A ˆ …aij† 2 Mmn…F† is called the coef®cient matrix of

Eq (7) The partitioned matrix …A j B† 2 Mm…n‡1†…F†

is called the augmented matrix of Eq (7) Matrix

mul-tiplication was invented to handle linear substitutions

of variables in Eq (7) Suppose y1; ; ypare new

vari-ables which are related to x1; ; xn by the following

set of linear equations:

A 2 Mnp…F†

Substituting the expressions in Eq (10) into Eq (7)

produces m equations in y1; ; yp The coef®cient

matrix of the new system is AC, the matrix product

of A and C

De®nition 7 A square matrix A 2 Mnn…F† is said to

be invertible (or nonsingular) if there exists a square

matrix B 2 Mnn…F† such that AB ˆ BA ˆ In

If A 2 Mnn…F† is invertible and AB ˆ BA ˆ In for

some B 2 Mnn…F†, then B is unique and will be

denoted by A 1 A 1 is called the inverse of A

j ˆ 1; ; m

A square matrix is symmetric (skew-symmetric) ifand only if A ˆ At… At† When the ®eld F ˆ C, thecomplex numbers, the Hermitian conjugate (or conju-gate transpose) of A is more useful than the transpose.De®nition 9 Let A 2 Mmn…C† The Hermitian conju-gate of A is denoted by A Ais the n  m matrix whoseentries are given by ‰AŠijˆ ‰ AŠjifor all i ˆ 1; ; n and

3.1.3 Block Multiplication of Matrices

Theorem 4 Let A 2 Mmn…F† and B 2 Mnp…F†

A and

B ˆ

B11 B1t

are partitions of A and Bsuch that size …Aij† ˆ mi nj

and size …Bjl† ˆ nj pl Thus, m1‡


‡ mrˆ m,

n1‡


‡ nkˆ n, and p1‡


‡ pt ˆ p For each i ˆ

Notice that the only hypothesis in Theorem 4 is that

every vertical line drawn in A must be matched with

the corresponding horizontal line in B There are four

special cases of Theorem 4 which are very useful We

collect these in the next theorem

Theorem 5 implies that the column space of A

con-sists of all linear combinations of the columns of A

RS…A† is all linear combinations of the rows of A.Using all four parts of Theorem 5, we have

CS…AB†  CS…A†

The column space of A is particularly important forthe theory of linear equations Suppose A 2 Mmn…F†and B 2 Fm Theorem 5 implies that AX ˆ B has asolution if and only if B 2 CS…A†:

3.1.4 Gaussian EliminationThe three elementary row operations that can be per-formed on a given matrix A are as follows:

…† Interchange two rows of A… † Add a scalar times one row of A to anotherrow of A

…† Multiply a row of A by a nonzero scalar

…14†There are three corresponding elementary columnoperations which can be preformed on A as well.De®nition 11 Let A1; A22 Mmn…F† A1 and A2 aresaid to be row (column) equivalent if A2can be obtainedfrom A1by applying ®nitely many elementary row (col-umn) operations to A1:

If A1 and A2 are row (column) equivalent, we willwrite A1 ~r A2…A1 ~c A2† Either one of these relations is

an equivalence relation on Mmn…F† By this, we mean

A1 ~r A2, A2 ~r A1 (~r is symmetric)

A1 ~r A2; A2 ~r A3) A1 ~r A3 (~r is transitive)

(15)Theorem 6 Let A, C 2 Mmn…F† and B, D 2 Fm.Suppose …A j B† ~r …C j D† Then the two linear systems

of equations AX ˆ B and CX ˆ D have precisely thesame solutions

Gaussian elimination is a strategy for solving a tem of linear equations To ®nd all solutions to thelinear system of equations

1 Set up the augmented matrix of Eq (16):

bm

1C

0B

2 Apply elementary row operations to …A j B† to

obtain a matrix …C j D† in upper triangular

form

3 Solve CX ˆ D by back substitution

By Theorem 6, this algorithm yields a complete set

0B

1C

0B

1C

... well.De®nition 11 Let A1; A2< /sub >2 Mmn…F† A1 and A2< /sub> aresaid to be row (column) equivalent if A2< /sub>can be obtainedfrom A1by... row (col-umn) operations to A1:

If A1 and A2< /sub> are row (column) equivalent, we willwrite A1 ~r A2< /small>…A1 ~c A2< /small>†... obtained from A

by multiplying row i of A by c

Thus, two m  n matrices A and B are row

equiv-alent if and only if there exist a ®nite number of

elementary matrices E1;