Exercise 2.10 Two Point Boundary Value Problem; Computation)
3.5 The LU and LDU Factorizations
Gaussian elimination without row interchanges is one way of computing an LU factorization of a matrix. There are other ways that can be advantageous for certain kind of problems. Here we consider the general theory of LU factorizations. Recall thatA=LU is anLU factorizationofA∈Cn×nifL∈Cn×nis lower triangular andU ∈Cn×nis upper triangular , i.e.,
L=
⎡
⎢⎣
l1,1ã ã ã 0 ... . .. ... ln,1ã ã ãln,n
⎤
⎥⎦, U =
⎡
⎢⎣
u1,1ã ã ãu1,n ... . .. ... 0 ã ã ãun,n
⎤
⎥⎦.
To find an LU factorization there is one equation for each of then2elements inA, andLandUcontain a total ofn2+nunknown elements. There are several ways to restrict the number of unknowns ton2.
L1U: lii =1 alli, LU1: uii=1 alli,
LDU: A=LDU,lii=uii =1 alli,D=diag(d11, . . . , dnn).
3.5.1 Existence and Uniqueness
Consider the L1U factorization. Three things can happen. An L1U factorization exists and is unique, it exists, but it is not unique, or it does not exist. The 2×2 case illustrates this.
Example 3.5 (L1U of2×2 Matrix) Leta, b, c, d ∈ C. An L1U factorization of A=
[a bc d
must satisfy the equations a b
c d
= 1 0
l11
u1u2 0 u3
=
u1 u2 u1l1u2l1+u3
for the unknownsl1inLandu1, u2, u3inU. The equations are
u1=a, u2=b, al1=c, bl1+u3=d. (3.15) These equations do not always have a solution. Indeed, the main problem is the equational1=c. There are essentially three cases
1. a=0: The matrix has a unique L1U factorization.
2. a =c =0: The L1U factorization exists, but it is not unique. Any value forl1 can be used.
3. a=0,c=0: No L1U factorization exists.
Consider the four matrices A1:=
2 −1
−1 2
, A2:=
0 1 1 1
, A3:=
0 1 0 2
, A4:=
1 1 1 1
. From the previous discussion it follows thatA1has a unique L1U factorization,A2 has no L1U factorization,A3has an L1U factorization but it is not unique, andA4 has a unique L1U factorization even if it is singular.
In preparation for the main theorem about LU factorization we prove a simple lemma. Recall that
A[k]:=
⎡
⎢⎣
a11 ã ã ãak1 ... ... ak1ã ã ãakk
⎤
⎥⎦
is called a leading principal submatrix ofA.
Lemma 3.1 (L1U of Leading Principal Submatrices) SupposeA = LU is an L1U factorization ofA∈Cn×n. Fork=1, . . . , nletA[k],L[k],U[k]be the leading principal submatrices ofA,L,U, respectively. ThenA[k] = L[k]U[k]is an L1U factorization ofA[k]fork=1, . . . , n.
Proof Fork=1, . . . , n−1 we partitionA=LUas follows:
A[k]Bk Ck Fk
=
L[k] 0 Mk Nk
U[k] Sk 0 Tk
=
L[k]U[k] L[k]Sk MkU[k] MkSk+NkTk
, (3.16) whereFk,Nk,Tk ∈Cn−k,n−k. Comparing blocks we findA[k]=L[k]U[k]. Since L[k]is unit lower triangular andU[k]is upper triangular this is an L1U factorization
ofA[k].
The following theorem gives a necessary and sufficient condition for existence of a unique LU factorization. The conditions are the same for the three factorizations L1U, LU1 and LDU.
Theorem 3.4 (LU Theorem) A square matrixA∈Cn×nhas a unique L1U (LU1, LDU) factorization if and only if the leading principal submatricesA[k]ofAare nonsingular fork=1, . . . , n−1.
Proof SupposeA[k]is nonsingular fork =1, . . . , n−1. Under these conditions Gaussian elimination gives an L1U factorization (cf. Theorem3.2). We give another proof here that in addition to showing uniqueness also gives alternative ways to compute the L1U factorization. The proofs for the LU1 and LDU factorizations are similar and left as exercises.
We use induction onnto show thatAhas a unique L1U factorization. The result is clearly true forn = 1, since the unique L1U factorization of a 1×1 matrix is [a11] = [1][a11]. Suppose thatA[n−1] has a unique L1U factorizationA[n−1] = Ln−1Un−1, and thatA[1], . . . ,A[n−1]are nonsingular. By block multiplication
A=
A[n−1] cn rTn ann
=
Ln−10 lTn 1
Un−1 un 0 unn
=
Ln−1Un−1 Ln−1un lTnUn−1 lTnun+unn
, (3.17) if and only ifA[n−1]=Ln−1Un−1andln,un∈Cn−1andunn ∈Care determined from
UTn−1ln=rn, Ln−1un=cn, unn=ann−lTnun. (3.18) Since A[n−1] is nonsingular it follows that Ln−1 and Un−1 are nonsingular and therefore ln,un, and unn are uniquely given. Thus (3.17) gives a unique L1U factorization ofA.
Conversely, supposeAhas a unique L1U factorizationA=LU. By Lemma3.1 A[k] = L[k]U[k]is an L1U factorization of A[k] fork = 1, . . . , n−1. Suppose
A[k]is singular for somek≤n−1. We will show that this leads to a contradiction.
Letkbe the smallest integer so thatA[k]is singular. SinceA[j]is nonsingular for j ≤k−1 it follows from what we have already shown thatA[k]=L[k]U[k]is the unique L1U factorization ofA[k]. The matrixU[k]is singular sinceA[k]is singular andL[k]is nonsingular. By (3.16) we haveUT[k]MTk =CTk. This can be written as n−klinear systems for the columns ofMTk. By assumptionMTk exists, but since UT[k]is singularMkis not unique, a contradiction.
By combining the last two equations in (3.18) we obtain withk=n UTk−1lk=rk,
Lk−10 lTk 1
uk ukk
= ck
akk
.
This can be used in an algorithm to compute the L1U factorization. Moreover, ifA isd-banded then the firstk−dcomponents inrkandck are zero so bothLandU will bed-banded. Thus we can use the bandedrforwardsolveAlgorithm3.1 to solve the lower triangular systemUTk−1lk =rk for thekth rowlTk inLand the kth columnuk
ukk
inU fork =2, . . . , n. This leads to the following algorithm to compute the L1U factorization of ad-banded matrixAwithd ≥1. The algorithm will fail if the conditions in the LU theorem are not satisfied.
function [L,U]=L1U(A,d)
% [L,U]=L1U(A,d) n=length(A);
L=eye(n,n); U=zeros(n,n);U(1,1)=A(1,1);
for k=2:n
km=max(1,k-d);
L(k,km:(k-1))=rforwardsolve(U(km:(k-1) ...
,km:(k-1))’,A(k,km:(k-1))’,d)’;
U(km:k,k)=rforwardsolve(L(km:k,km:k),A(km:k,k),d);
end
Listing 3.4 L1U
For eachkwe essentially solve a lower triangular linear system of orderd. Thus the number of arithmetic operation for this algorithm isO(d2n).
Remark 3.1 (LU of Upper Triangular Matrix) A matrixA∈Cn×ncan have an LU factorization even ifA[k]is singular for somek < n. By Theorem4.1such an LU factorization cannot be unique. An L1U factorization of an upper triangular matrix AisA =I Aso it always exists even ifAhas zeros somewhere on the diagonal.
By Lemma2.5, if someakkis zero thenA[k]is singular and the L1U factorization is not unique. In particular, for the zero matrix any unit lower triangular matrix can be used asLin an L1U factorization.