3.3 Matrix Rank and the Inverse of a Matrix
3.3.6 Full Rank Matrices and Matrix Inverses
We have already seen that full rank matrices have some important properties.
In this section, we consider full rank matrices and matrices that are their Cayley multiplicative inverses.
3.3.6.1 Solutions of Linear Equations
Important applications of vectors and matrices involve systems of linear equa- tions:
a11x1 +ã ã ã+a1mxm
=? b1
... ... ...
an1x1 +ã ã ã+anmxm
=? bn
(3.134) or
Ax=? b. (3.135)
In this system, A is called the coefficient matrix. An x that satisfies this system of equations is called asolution to the system. For givenA and b, a solution may or may not exist. From equation (3.84), a solution exists if and only if then-vectorbis in thek-dimensional column space ofA, wherek≤m.
A system for which a solution exists is said to beconsistent; otherwise, it is inconsistent.
We note that ifAx=b, for any conformabley,
yTAx= 0⇐⇒yTb= 0. (3.136)
3.3.6.2 Consistent Systems
A linear systemAn×mx=bis consistent if and only if
rank([A|b]) = rank(A). (3.137)
We can see this following the argument above thatb∈ V(A); that is, the space spanned by the columns ofA is the same as that spanned by the columns of Aand the vectorb. Thereforebmust be a linear combination of the columns of A, and furthermore, the linear combination is a solution to the system Ax=b. (Note, of course, that it is not necessary that it be a unique linear combination.)
Equation (3.137) implies the equivalence of the conditions
[A|b]y= 0 for somey= 0 ⇔ Ax= 0 for somex= 0. (3.138) A special case that yields equation (3.137) for anybis
rank(An×m) =n, (3.139)
and so ifAis of full row rank, the system is consistent regardless of the value ofb. In this case, of course, the number of rows ofAmust be no greater than the number of columns (by inequality (3.116)). A square system in whichAis nonsingular is clearly consistent. (The condition of consistency is also called
“compatibility” of the system; that is, the linear systemAx=bis said to be compatibleif it is consistent.)
A generalization of the linear systemAx=b is AX=B, where B is an n×kmatrix. This is the same as ksystemsAx1 =b1, . . . , Axk =bk, where thexi and the bi are the columns of the respective matrices. Consistency of AX=B, as above, is the condition for a solution inX to exist, and in that case the system is also said to be compatible.
It is clear that the system AX =B is consistent if each of the Axi =bi systems is consistent. Furthermore, if the system is consistent, then every linear relationship among the rows ofA exists among the rows ofB; that is, for anyc such thatcTA = 0, thencTB = 0. To see this, letc be such that cTA= 0. We then havecTAX=cTB= 0, and so the same linear relationship that exists among the rows ofAexists among the rows ofB.
As above forAx=b, we also see that the systemAX=B is consistent if and only if any of the following conditions hold:
V(B)⊆ V(A) (3.140)
V([A|B]) =V(A) (3.141)
rank([A|B]) = rank(A). (3.142)
These relations imply that if AX = B is consistent, then for any con- formable vectorc,
cTA= 0 ⇐⇒ cTB= 0. (3.143)
It is clear that this condition also implies thatAX=B is consistent (because right-hand implication of the condition implies the relationship (3.140)).
We discuss methods for solving linear systems in Sect.3.5and in Chap.6.
In the next section, we consider a special case of n×n (square) A when equation (3.139) is satisfied (that is, whenAis nonsingular).
3.3.6.3 Matrix Inverses
LetA be ann×nnonsingular matrix, and consider the linear systems Axi=ei,
where ei is the ith unit vector. For each ei, this is a consistent system by equation (3.137).
We can represent allnsuch systems as A'
x1| ã ã ã |xn
(='
e1| ã ã ã |en
( or
AX=In,
and this full system must have a solution; that is, there must be anX such thatAX=In. BecauseAX=I, we callX a “right inverse” ofA. The matrix X must be n×n and nonsingular (because I is); hence, it also has a right inverse, sayY, andXY =I. From AX=I, we haveAXY =Y, soA=Y, and so finallyXA=I; that is, the right inverse ofAis also the “left inverse”.
We will therefore just call it the inverseof A and denote it as A−1. This is the Cayley multiplicative inverse. Hence, for ann×nnonsingular matrixA, we have a matrixA−1such that
A−1A=AA−1=In. (3.144) The inverse of the nonsingular square matrixAis unique. (This follows from the argument above about a “right inverse” and a “left inverse”.)
We have already encountered the idea of a matrix inverse in our discussions of elementary transformation matrices. The matrix that performs the inverse of the elementary operation is the inverse matrix.
From the definitions of the inverse and the transpose, we see that
(A−1)T= (AT)−1, (3.145) and because in applications we often encounter the inverse of a transpose of a matrix, we adopt the notation
A−T to denote the inverse of the transpose.
In the linear system (3.135), if n=mand Ais nonsingular, the solution is
x=A−1b. (3.146)
For scalars, the combined operations of inversion and multiplication are equivalent to the single operation of division. From the analogy with scalar op- erations, we sometimes denoteAB−1 byA/B. Because matrix multiplication is not commutative, we often use the notation “\” to indicate the combined operations of inversion and multiplication on the left; that is,B\Ais the same asB−1A. The solution given in equation (3.146) is also sometimes represented asA\b.
We discuss the solution of systems of equations in Chap. 6, but here we will point out that when we write an expression that involves computations to
evaluate it, such asA−1borA\b, the form of the expression does not specify how to do the computations. This is an instance of a principle that we will encounter repeatedly:the form of a mathematical expression and the way the expression should be evaluated in actual practice may be quite different.
3.3.6.4 Nonsquare Full Rank Matrices: Right and Left Inverses SupposeA is n×m and rank(A) = n; that is,n ≤ m and A is of full row rank. Then rank([A|ei]) = rank(A), whereei is theith unit vector of length n; hence the system
Axi=ei
is consistent for eachei, and, as before, we can represent allnsuch systems as
A'
x1| ã ã ã |xn(
='
e1| ã ã ã |en( or
AX=In.
As above, there must be an X such that AX = In, and we call X a right inverseofA. The matrixX must bem×nand it must be of rankn(because Iis). This matrix is not necessarily the inverse ofA, however, becauseAand X may not be square. We denote the right inverse ofAas
A−R.
Furthermore, we could only have solved the systemAX ifA was of full row rank becausen≤mandn= rank(I) = rank(AX)≤rank(A). To summarize, Ahas a right inverse if and only ifAis of full row rank.
Now, suppose A isn×mand rank(A) =m; that is,m ≤nand A is of full column rank. WritingY A=Im and reversing the roles of the coefficient matrix and the solution matrix in the argument above, we have thatY exists and is aleft inverse ofA. We denote the left inverse ofAas
A−L.
Also, using a similar argument as above, we see that the matrixAhas a left inverse if and only ifA is of full column rank.
We also note that ifAAT is of full rank, the right inverse ofAis
A−R=AT(AAT)−1. (3.147) Likewise, ifATAis of full rank, the left inverse of Ais
A−L= (ATA)−1AT. (3.148)