In this section, we continue discussing the solution sets of linear systems. First we introduce row equivalence of matrices, and use this to prove our assertion in the last two sections that the Gaussian elimination and Gauss-Jordan row reduction methods always produce the complete solution set for a given linear system. We also note that every matrix has a unique corresponding matrix in reduced row echelon form and
use this fact to define the rank of the matrix. We then introduce an important set of linear combinations of vectors associated with a matrix, called the row space of the matrix, and show it is invariant under row operations.
Equivalent Systems and Row Equivalence of Matrices
The first two definitions below involve related concepts. The connection between them will be shown in Theorem 2.3.
DefinitionTwo systems ofmlinear equations innvariables areequivalentif and only if they have exactly the same solution set.
For example, the systems
*2xy1 3xy9 and
* x4y14 5x2y 4 are equivalent, because the solution set of both is exactly{(2, 3)}.
Definition An (augmented) matrixDisrow equivalentto a matrixCif and only ifDis obtained from C by a finite number of row operations of types (I), (II), and (III).
For example,given any matrix,either Gaussian elimination or the Gauss-Jordan row reduction method produces a matrix that is row equivalent to the original.
Now, ifDis row equivalent toC, thenCis also row equivalent toD. The reason is that each row operation is reversible; that is, the effect of any row operation can be undone by performing another row operation. These reverse, or inverse, row operations are shown in Table 2.1. Notice a row operation of type (I) is reversed by using the reciprocal 1/cand an operation of type (II) is reversed by using the additive inversec. (Do you see why?)
Thus, ifDis obtained fromCby the sequence
C→R1 A1→R2 A2→ ãããR3 →Rn AnR→n1D,
thenCcan be obtained fromDusing the reverse operations in reverse order:
DR
1
→n1AnR
1n
→ An1 Rn11
→ ãããR→21A1 R11
→ C
(Ri 1represents the inverse operation of Ri,as indicated inTable 2.1.) These comments provide a sketch for the proof of the following theorem. You are asked to fill in the details of the proof in Exercise 13(a).
Table 2.1 Row operations and their inverses
Type of Operation Operation Reverse Operation
(I) i ←ci i ←1ci
(II)
j
←ci j
j
←ci j
(III) i ↔
j
i ↔ j
Theorem 2.2 If a matrixDis row equivalent to a matrixC, thenCis row equivalent toD.
The next theorem asserts that if two augmented matrices are obtained from each other using only row operations, then their corresponding systems have the same solution set. This result guarantees that the Gaussian elimination and Gauss-Jordan methods provided in Sections 2.1 and 2.2 are correct because the only steps allowed in those procedures were row operations. Therefore, a final augmented matrix produced by either method represents a system equivalent to the original — that is, a system with precisely the same solution set.
Theorem 2.3 LetAXBbe a system of linear equations. If[C|D]is row equivalent to [A|B], then the systemCXDis equivalent toAXB.
Proof. (Abridged) LetSArepresent the complete solution set of the systemAXB, and letSCbe the solution set ofCXD. Our goal is to prove that if[C|D]is row equivalent to [A|B], thenSASC. It will be enough to show that[C|D]row equivalent to[A|B]implies SA⊆SC. This fact, together with Theorem 2.2, implies the reverse inclusion,SC⊆SA(why?).
Also, it is enough to assume that[C|D]=R([A|B])for a single row operationRbecause an induction argument extends the result to the case where any (finite) number of row operations are required to produce[C|D]from[A|B]. Therefore, we need only consider the effect of each type of row operation in turn. We present the proof for a type (II) operation and leave the proofs for the other types as Exercise 13(b).
type (II) Operation: Suppose that the original system has the form
⎧⎪
⎪⎪
⎨
⎪⎪
⎪⎩
a11x1 a12x2a13x3ãããa1nxnb1 a21x1 a22x2a23x3ãããa2nxnb2 ... ... ... . .. ... ... am1x1am2x2am3x3ãããamnxnbm
and that the row operation used is j
←qi j
(whereij). When this row operation is applied to the corresponding augmented matrix, all rows except the jth row remain
unchanged. The newjth equation then has the form
(qai1aj1)x1(qai2aj2)x2ããã(qainajn)xnqbibj.
We must show that any solution(s1,s2,...,sn)of the original system is a solution of the new one. Now, since(s1,s2,...,sn)is a solution of both theith andjth equations in the original system, we have
ai1s1ai2s2ãããainsnbi and aj1s1aj2s2ãããajnsnbj. Multiplying the first equation byqand then adding equations yields
(qai1aj1)s1(qai2aj2)s2ããã(qainajn)snqbibj.
Hence,(s1,s2,...,sn)is also a solution of the newjth equation. And(s1,s2,...,sn)is certainly a solution of every other equation in the new system as well, since none of those have changed.
Rank of a Matrix
When the Gauss-Jordan method is performed on a matrix, only one final augmented matrix can result. This fact is stated in the following theorem, the proof of which appears in Appendix A:
Theorem 2.4 Every matrix is row equivalent to a unique matrix in reduced row echelon form.
While each matrix is row equivalent to exactly one matrix inreducedrow echelon form, there may be many matrices in row echelon form to which it is row equivalent.
This is one of the advantages of Gauss-Jordan row reduction over Gaussian elimination.
Because each matrix has a unique corresponding reduced row echelon form matrix, we can make the following definition:
Definition LetAbe a matrix. Then therankofAis the number of nonzero rows (that is,rows with nonzero pivot entries) in the unique reduced row echelon form matrix that is row equivalent toA.
Example 1
Consider the following matrices:
A
⎡
⎢⎣
2 1 4
3 2 5
0 1 1
⎤
⎥⎦ and B
⎡
⎢⎣
3 1 0 1 9
0 2 12 8 6
2 3 22 14 17
⎤
⎥⎦.
The unique reduced row echelon form matrices forAandBare, respectively,
⎡
⎢⎣
1 0 0
0 1 0
0 0 1
⎤
⎥⎦ (I3) and
⎡
⎢⎣
1 0 2 1 4
0 1 6 4 3
0 0 0 0 0
⎤
⎥⎦.
Therefore, the rank ofAis3since the reduced row echelon form ofAhas three nonzero rows (and hence three nonzero pivot entries). On the other hand, the rank ofBis 2 since the reduced row echelon form ofBhas two nonzero rows (and hence two nonzero pivot entries).
Homogeneous Systems and Rank
We can now restate our observations about homogeneous systems from Section 2.2 in terms of rank.
Theorem 2.5 LetAXObe a homogeneous system innvariables.
(1) If rank(A) <n, then the system has a nontrivial solution.
(2) If rank(A)n, then the system has only the trivial solution.
Note that the presence of a nontrivial solution when rank(A) <nmeans that the homogeneous system has an infinite number of solutions.
Proof. After the Gauss-Jordan method is applied to the augmented matrix[A|O], the num- ber of nonzero pivots equals rank(A). Suppose rank(A) <n. Then at least one of the columns is a nonpivot column, and so at least one of the nvariables on the left side of [A|O]is independent. Now, because this system is homogeneous, it is consistent. There- fore, the solution set is infinite, with particular solutions found by choosing arbitrary values for all independent variables and then solving for the dependent variables. Choosing a nonzero value for at least one independent variable yields a nontrivial solution.
On the other hand, suppose rank(A)n. Then, because A has ncolumns, every column on the left side of [A|O]is a pivot column, and each variable must equal zero.
Hence, in this case,AXOhas only the trivial solution.
The following corollary (illustrated by Example 4 in Section 2.2) follows immedi- ately from Theorem 2.5:
Corollary 2.6 Let AXO be a homogeneous system of m linear equations in nvariables. Ifm<n, then the system has a nontrivial solution.
Linear Combinations of Vectors
In Section 1.1, we introduced linear combinations of vectors. Recall that a linear combination of vectors is a sum of scalar multiples of the vectors.
Example 2
Leta1[4, 1, 2],a2[2, 1, 0], anda3[6,3,4]inR3. Consider the vector[18, 15, 16]. Because
[18, 15, 16]2[4, 1, 2]4[2, 1, 0]3[6,3,4],
the vector[18, 15, 16]is a linear combination of the vectorsa1,a2, anda3. This combination shows us how to reach the “destination”[18, 15, 16]by traveling in directions parallel to the vectorsa1,a2, anda3.
Now consider the vector[16,3, 8]. This vector is not a linear combination ofa1,a2, and a3. For if it were, the equation
[16,3, 8]c1[4, 1, 2]c2[2, 1, 0]c3[6,3,4]
would have a solution. But, equating coordinates, we get the following system:
⎧⎪
⎨
⎪⎩
4c12c26c3 16 c1 c23c33 2c1 4c3 8
first coordinates second coordinates third coordinates.
We solve this system by row reducing
⎡
⎢⎢
⎢⎣
4 2 6 1 1 3 2 0 4
16 3 8
⎤
⎥⎥
⎥⎦ to obtain
⎡
⎢⎢
⎢⎣ 1 0 2 0 1 1 0 0 0
113 2 3 46 3
⎤
⎥⎥
⎥⎦.
The third row of this final matrix indicates that the system has no solutions, and hence, there are no values ofc1,c2, andc3that together satisfy the equation
[16,3, 8]c1[4, 1, 2]c2[2, 1, 0]c3[6,3,4].
Therefore, [16,3, 8] is not a linear combination of the vectors [4, 1, 2], [2, 1, 0], and [6,3,4]. This means that it is impossible to reach the “destination”[16,3, 8]by traveling in directions parallel to the vectorsa1,a2, anda3.
The next example shows that a vectorxcan sometimes be expressed as a linear combination of vectorsa1,a2,...,akin more than one way.
Example 3
To determine whether[14,21, 7]is a linear combination of[2,3, 1]and[4, 6,2], we need to find scalarsc1andc2such that
[14,21, 7]c1[2,3, 1]c2[4, 6,2].
This is equivalent to solving the system
⎧⎪
⎨
⎪⎩
2c14c2 14 3c16c221 c12c2 7 .
We solve this system by row reducing
⎡
⎢⎣
2 4
3 6
1 2
14 21 7
⎤
⎥⎦ to obtain
⎡
⎢⎣
1 2
0 0
0 0
7 0 0
⎤
⎥⎦.
Becausec2is an independent variable, we may takec2to be any real value. Thenc12c27.
Hence, there are an infinite number of solutions to the system.
For example, we could letc21, which forcesc12(1)79, yielding [14,21, 7]9[2,3, 1]1[4, 6,2].
On the other hand, we could letc20, which forcesc17, yielding [14,21, 7]7[2,3, 1]0[4, 6,2].
Thus, we have expressed [14,21, 7]as a linear combination of[2,3, 1]and[4, 6,2]in more than one way.
In Examples 2 and 3 we saw that to find the coefficients to express a given vectorx as a linear combination of other vectors, we row reduce an augmented matrix whose rightmost column isx, and whose remainingcolumnsare the other vectors.
It is possible to have a linear combination of a single vector: any scalar multiple of ais considered a linear combination ofa. For example, ifa[3,1, 5], then2a [6, 2,10]is a linear combination ofa.
The Row Space of a Matrix
SupposeAis anmnmatrix. Recall that each of themrows ofAis a vector withn entries — that is, a vector inRn.
Definition LetA be anmn matrix. The subset ofRn consisting of all vectors that are linear combinations of the rows ofAis called therow spaceofA.
Recall that we consider a linear combination of vectors to be a“possible destination”
obtained by traveling in the directions of those vectors. Hence, the row space of a matrix is the set of “allpossible destinations” using the rows ofAas our fundamental directions.
Example 4 Consider the matrix
A
⎡
⎢⎣
3 1 2
4 0 1
2 4 3
⎤
⎥⎦.
We want to determine whether[5, 17,20]is in the row space ofA. If so,[5, 17,20]can be expressed as a linear combination of the rows ofA, as follows:
[5, 17,20]c1[3, 1,2]c2[4, 0, 1]c3[2, 4,3].
Equating the coordinates on each side leads to the following system:
⎧⎪
⎨
⎪⎩
3c14c22c3 5 c1 4c3 17 2c1 c23c320
, whose matrix row reduces to
⎡
⎢⎣
1 0 0
0 1 0
0 0 1
5 1 3
⎤
⎥⎦.
Hence,c15,c2 1, andc33, and
[5, 17,20]5[3, 1,2]1[4, 0, 1]3[2, 4,3].
Therefore,[5, 17,20]is in the row space ofA.
Example 4 shows that to check whether a vectorXis in the row space ofA, we row reduce the augmented matrix ATX
to determine whether its corresponding system has a solution.
Example 5
The vectorX[3, 5]is not in the row space ofB +
2 4
1 2 ,
because there is no way to express[3, 5]as a linear combination of the rows[2,4]and[1, 2]ofB. That is, row reducing
BTX
+
2 1
4 2
3 5 ,
yields +
1 12
0 0
3 2 11
, ,
thus showing that the corresponding linear system is inconsistent.
IfA is anymnmatrix, then[0, 0,..., 0]inRn is always in the row space ofA.
This is because the zero vector can always be expressed as a linear combination of the rows ofA simply by multiplying each row by zero and adding the results. Similarly, each individual row ofAis in the row space ofA, because any particular row ofAcan be expressed as a linear combination of all the rows ofAsimply by multiplying that row by 1, multiplying all other rows by zero, and summing.
Row Equivalence Determines the Row Space
The following lemma is used in the proof of Theorem 2.8:
Lemma 2.7 Suppose that x is a linear combination ofq1,...,qk, and suppose also that each of q1,...,qk is itself a linear combination ofr1,...,rl. Then x is a linear combination ofr1,...,rl.
If we create a matrixQwhose rows are the vectorsq1,...,qkand a matrixRwhose rows are the vectorsr1,...,rl, then Lemma 2.7 can be rephrased as
Ifxis in the row space ofQand each row ofQis in the row space ofR, thenxis in the row space ofR.
Proof. Becausex is a linear combination ofq1,...,qk, we can write xc1q1c2q2
ãããckqkfor some scalarsc1,c2,...,ck. But, since each ofq1,...,qkcan be expressed as a linear combination ofr1,...,rl, there are scalarsd11,...,dklsuch that
⎧⎪
⎪⎪
⎨
⎪⎪
⎪⎩
q1d11r1d12r2 ããã d1lrl
q2d21r1d22r2 ããã d2lrl
... ... ... . .. ... qkdk1r1dk2r2 ããã dklrl
.
Substituting these equations into the equation forx, we obtain x c1(d11r1d12r2 ããã d1lrl)
c2(d21r1d22r2 ããã d2lrl) ... ... . .. ... ck(dk1r1dk2r2 ããã dklrl)
.
Collecting allr1terms, allr2terms, and so on, we get
x (c1d11c2d21 ããã ckdk1)r1
(c1d12c2d22 ããã ckdk2)r2 ... ... . .. ... (c1d1lc2d2l ããã ckdkl)rl
.
Thus,xcan be expressed as a linear combination ofr1,r2,...,rl.
The next theorem illustrates an important connection between row equivalence and row space.
Theorem 2.8 Suppose thatAandBare row equivalent matrices. Then the row space ofAequals the row space ofB.
In other words, if A and Bare row equivalent, then any vector that is a linear combination of the rows ofAmust be a linear combination of the rows ofB, and vice versa. Theorem 2.8 assures us that we do not gain or lose any linear combinations of the rows when we perform row operations. That is, the same set of “destination vectors” is obtained from the rows of row equivalent matrices.
Proof. (Abridged) LetA andBbe row equivalentmnmatrices. We will show that if x is a vector in the row space ofB, thenx is in the row space ofA. (A similar argument can then be used to show that ifxis in the row space of A, thenxis in the row space ofB.)
First consider the case in whichBis obtained fromAby performing a single row oper- ation. In this case, the definition for each type of row operation implies that each row ofB is a linear combination of the rows ofA(see Exercise 19(a)). Now, supposexis in the row space ofB. Thenxis a linear combination of the rows ofB. But since each of the rows of Bis a linear combination of the rows ofA, Lemma 2.7 indicates thatxis in the row space ofA.By induction, this argument can be extended to the case whereBis obtained fromA by any (finite) sequence of row operations (see Exercise 20).
Example 6 Consider the matrix
A
⎡
⎢⎢
⎢⎣
5 10 12 33 19
3 6 4 25 11
1 2 2 11 5
2 4 1 10 4
⎤
⎥⎥
⎥⎦.
The reduced row echelon form matrix forAis
B
⎡
⎢⎢
⎢⎣
1 2 0 3 1
0 0 1 4 2
0 0 0 0 0
0 0 0 0 0
⎤
⎥⎥
⎥⎦.
Theorem 2.8 asserts that the row spaces ofAandBare equal. Hence, the linear combinations that can be created from the rows ofAare identical to those that can be created fromB. For example, the vectorx[4, 8,30,132,64]is in both row spaces:
x 1[5, 10, 12, 33, 19]3[3, 6,4,25,11]4[1, 2,2,11,5]2[2, 4,1,10,4],
which showsxis in the row space ofA. Butxis in the row space ofB, since x4[1, 2, 0,3,1]30[0, 0, 1, 4, 2].
The matrixA in Example 6 essentially has two unneeded, or “redundant” rows.
Thus, from the reduced row echelon form matrixBofA, we obtain a smaller num- ber of rows (those that are nonzero in B) producing the same row space. In other words,we can reach the same “destinations” using just the two vector directions of the nonzero rows ofBas we could using all four of the vector directions of the rows of A. In fact, we will prove in Chapter 4 that the rank ofAgives precisely the minimal number of rows ofAneeded to produce the same set of linear combinations.
Numerical Method:You have now covered the prerequisites for Section 9.1,
“Numerical Methods for Solving Systems.”
New Vocabulary
equivalent systems rank (of a matrix)
reverse (inverse) row operations
row equivalent matrices row space (of a matrix)
Highlights
■Two matrices are row equivalent to each other if one can be produced from the other using some finite series of the three allowable row operations.
■ If the augmented matrices for two linear systems are row equivalent, then the systems have precisely the same solution set (that is, the systems are equivalent).
■ Every matrix is row equivalent to a unique reduced row echelon form matrix.
■The rank of a matrix is the number of nonzero rows (number of pivot columns) in its corresponding reduced row echelon form matrix.
■ If the rank of the augmented matrix for a homogeneous linear system is less than the number of variables, then the system has an infinite number of solutions.
■We can determine whether a given vector is a linear combination of other vectors by solving an appropriate system whose augmented matrix consists of those vectors as its leftmost columns and the given vector as the rightmost column.
■The row space of a matrix is the set of all possible linear combinations of the rows of the matrix.
■ If two matrices are row equivalent, then their row spaces are identical (that is, each linear combination of the rows that can be produced using one matrix can also be produced from the other).