Exercise 2.10 Two Point Boundary Value Problem; Computation)
4.3 Semi-Cholesky Factorization of a Banded Matrix
A positive semidefinite matrix has a factorization that is similar to the Cholesky factorization.
Definition 4.2 (Semi-Cholesky Factorization) A factorizationA=LL∗ofA∈ Cn×n, whereLis lower triangular with nonnegative diagonal elements is called a semi-Cholesky factorization.
Note that a semi-Cholesky factorization of a positive definite matrix is necessar- ily a Cholesky factorization. For ifAis positive definite then it is nonsingular and thenLmust be nonsingular. Thus the diagonal elements ofLcannot be zero.
Theorem 4.5 (Characterization, Semi-Cholesky Factorization) A matrix A ∈ Cn×n has a semi-Cholesky factorization A = LL∗ if and only if it is positive semidefinite.
Proof IfA=LL∗is a semi-Cholesky factorization thenAis Hermitian. Moreover, x∗Ax = L∗x22 ≥ 0 andAis positive semidefinite. For the converse we use induction onn. A positive semidefinite matrix of order one has a semi-Cholesky factorization since the only element in A is nonnegative. Suppose any positive semidefinite matrix of ordern−1 has a semi-Cholesky factorization and suppose A∈Cn×nis positive semidefinite. We partitionAas follows
A= αv∗
v B
, α∈C, v∈Cn−1, B ∈C(n−1)×(n−1). (4.8) There are two cases. Suppose firstα = e∗1Ae1 > 0. We claim that C := B − vv∗/αis positive semidefinite.Cis Hermitian sinceBis. To show thatCis positive semidefinite we consider anyy∈Cn−1and definex∗:= [−y∗v/α,y∗] ∈Cn. Then
0≤x∗Ax= [−y∗v/α,y∗] αv∗
v B
−v∗y/α y
= [0,−(y∗v)v∗/α+y∗B]
−v∗y/α y
= −y∗vv∗y/α+y∗By=y∗Cy.
(4.9)
SoC∈C(n−1)×(n−1)is positive semidefinite and by the induction hypothesis it has a semi-Cholesky factorizationC=L1L∗1. The matrix
L∗:=
β v∗/β 0 L∗1
, β :=√
α, (4.10)
is upper triangular with nonnegative diagonal elements and LL∗=
β 0 v/β L1
β v∗/β 0 L∗1
= αv∗
v B
=A is a semi-Cholesky factorization ofA.
If α = 0 then part 4 of Theorem4.3 implies that v = 0. Moreover, B ∈ C(n−1)×(n−1) in (4.8) is positive semidefinite and therefore has a semi-Cholesky
factorizationB = L1L∗1. But thenLL∗, whereL = 0 0∗
0L1
is a semi-Cholesky factorization ofA. Indeed,Lis lower triangular and
LL∗= 0 0∗
0L1 0 0∗ 0L∗1
= 00∗
0 B
=A.
Recall that a matrixAisd-banded ifaij =0 for|i−j|> d. A (semi-) Cholesky factorization preserves bandwidth.
Theorem 4.6 (Bandwidth Semi-Cholesky Factor) The semi-Cholesky factor L given by(4.10)has the same bandwidth asA.
Proof SupposeA∈Cn×nisd-banded. Thenv∗= [u∗,0∗]in (4.8), whereu∈Cd, and thereforeC:=B−vv∗/αdiffers fromBonly in the upper leftd×d corner. It follows thatChas the same bandwidth asBandA. By induction onn,C=L1L∗1, whereL∗1has the same bandwidth asC. But thenLin (4.10) has the same bandwidth
asA.
Consider now implementing an algorithm based on the previous discussion.
SinceAis Hermitian we only need to use the lower part ofA. The first column ofLis[β,v∗/β]∗ifα >0. Ifα=0 then by 4 in Theorem4.3the first column of Ais zero and this is also the first column ofL. We obtain
ifA(1,1) >0 A(1,1)=
A(1,1)
A(2:n,1)=A(2:n,1)/A(1,1) forj =2:n
A(j :n, j )=A(j :n, j )−A(j,1)∗A(j:n,1)
(4.11)
Here we store the first column ofLin the first column ofAand the lower part of C=B−vv∗/αin the lower part ofA(2:n,2:n).
The code can be made more efficient whenAis ad-banded matrix. We simply replace all occurrences ofnby min(i+d, n). Continuing the reduction we arrive at the following algorithm, which take a d-banded positive semidefinite A and d ≥ 1 as input, and returns a lower triangular matrixLso thatA = LL∗. This is the Cholesky factorization ofA if Ais positive definite and a semi-Cholesky factorization ofAotherwise. The algorithm uses the MATLAB commandtril:
function L=bandsemicholeskyL (A,d)
%L=bandsemicholeskyL (A,d) n=length(A);
for k=1:n
kp=min(n,k+d);
if A(k,k)>0
A(k,k)=sqrt(A(k,k));
A((k+1):kp,k)=A((k+1):kp,k)/A(k,k);
for j=k+1:kp
A(j:kp,j)=A(j:kp,j)-A(j,k)*A(j:kp,k);
end else
A(k:kp,k)=zeros(kp-k+1,1);
end end
L=tril(A);
end
Listing 4.3 bandsemicholeskyL
In the algorithm we overwrite the lower triangle ofAwith the elements of L.
ColumnkofLis zero for thosek wherekk = 0. We reduce round-off noise by forcing those rows to be zero. In the semidefinite case no update is necessary and we “do nothing”.
Deciding when a diagonal element is zero can be a problem in floating point arithmetic.
We end the section with some necessary and sufficient conditions for a matrix to be positive semidefinite.
Theorem 4.7 (Positive Semidefinite Characterization) The following is equiva- lent for a matrixA∈Cn×n.
1. Ais positive semidefinite.
2. Ais Hermitian with only nonnegative eigenvalues.
3. Ais Hermitian and all principal submatrices have a nonnegative determinant.
4. A=BB∗for someB∈Cn×n. Proof
1 ⇐⇒ 2: This follows from Lemma4.5.
1 ⇐⇒ 4: This follows from Theorem4.5
1 ⇐⇒ 3: We refer to page 567 of [15], where it is shown that 4 ⇒ 3 (and
therefore 1 ⇒ 3 since 1 ⇐⇒ 4) and 3 ⇒ 1.
Example 4.7 (Positive Semidefinite Characterization) Consider the symmetric matrixA:=
1 1 1 1
.
1. We havex∗Ax = x12+x22+x1x2+x2x1 = (x1+x2)2 ≥ 0 for allx ∈ R2 showing thatAis positive semidefinite.
2. The eigenvalues ofAareλ1=2 andλ2=0 and they are nonnegative showing thatAis positive semidefinite since it is symmetric.
3. There are three principal sub matrices, and they have determinants det([a11])= 1, det([a22]) = 1 and det(A) = 0 and showing again that A is positive semidefinite.
4. FinallyAis positive semidefinite sinceA=BB∗, whereB= 1 0
1 0
.
In part 4 of Theorem 4.7 we require nonnegativity of all principal minors, while only positivity of leading principal minors was required for positive definite matrices (cf. Theorem4.4). To see that nonnegativity of the leading principal minors is not enough consider the matrixA := 0 0
0−1
. The leading principal minors are nonnegative, butAis not positive semidefinite.