LU Decomposition and Doolittle Decomposition
The LU Decomposition algorithm is used to write a square matrix, A, as a product of a lower and an upper triangular matrix. The applications of both the LU and Cholesky Decomposition will be discussed later on in the lab.
Recall from Lab 2, that we can perform elementary row operations on a matrix by multiplying the matrix by elementary matrices. The steps of the LU Decomposition of matrixAare to
1. Determine elementary matrices to transformA into an upper triangular matrix,U. That is find E1, E2,ã ã ã , Ek such thatEkã ã ãE2E1A=U. 2. WriteA= (Ekã ã ãE2E1)−1U =LU whereL= (Ekã ã ãE2E1)−1.
If A is also symmetric, then LU = A = AT = (LU)T = UTLT. Thus U(LT)−1=L−1UT =D, andU =DLT, whereD is a diagonal matrix. Thus ifA is symmetric we can findL andD such that A=LDLT. This is called theDoolittle Decomposition ofA.
IfAis Hermitian (Lab 9), thenLU =A=AT =LUT =UTLT. Therefore U(LT)−1=L−1UT =D and we can writeA=LDLT.
Exercises:
a. DefineA=
2 1 1 2
. Find the LU Decomposition and the Doolittle De- composition of Ausing the algorithm above.
b. Define B =
2 1 + 2i
1−2i 4
. Find the LU Decomposition and the Doolittle Decomposition ofB.
The MATLAB command:
[L,U,P] = lu(The Name of the Matrix)
produces L a lower triangular matrix, U, and upper triangular matrix, and P, a permutation matrix, such thatLU =P A, whereAis the original matrix.
If all of the entries inDare positive (and thusAis positive definite) then we can find the Cholesky Decomposition ofA.
106 Exploring Linear Algebra Labs and Projects with MATLABR The Cholesky Decomposition Algorithm
IfA is symmetric with real entries or Hermitian and if Ais positive definite (Lab 19) then there exists a lower triangular matrix with nonnegative diago- nal entries such thatA=LL.Lis considered to be the square root ofA. This decomposition of the matrixAis called theCholesky Decomposition.
It is helpful to note that a matrix is positive definite if and only if all of its eigenvalues are positive.
The Cholesky Decomposition Algorithm:
1. WriteA=LLT as
a11 A21T
A21 A22
=
l11 0 L21 L22
l11 L21T
0 L22T
and l11=√a11.
2. In general for real matriceslii=q
aii−Pi−1 j=1l2ijand lik=lkk1 (aik−Pk−1
j=1lijlkj) fori > k.
For Hermitian Matriceslii=q
aii−Pi−1
j=1lijlijand lik=lkk1 (aik−Pk−1
j=1lijlkj) fori > k.
The Cholesky Decomposition is more efficient than the LU Decomposition Algorithm for symmetric matrices and is a modified form of Gaussian Elimi- nation.
Exercises: Let A=
2 1 1 2
andB =
2 1 + 2i
1−2i 4
.
a. Determine ifA is positive definite.
b. Use the algorithm above to find the Cholesky Decomposition ofA.
c. Determine if B is positive definite. If B is positive definite find the Cholesky Decomposition ofB.
d. What is the relationship between the Cholesky Decomposition and the Doolittle Decomposition?
To use MATLAB to produce a lower triangular matrix for the Cholesky Decomposition, type:
L = chol(The Name of the Matrix,’lower’).
Matrix Decomposition with Applications 107 Generating Random Correlated Data Using Cholesky Decom- position
The goal in this section is to see one way that decomposition of matrices ap- plies to statistics, particularly related to the generation of correlated data.
Thecovariance matrix captures the variance and linear correlation in mul- tivariable data.Covariance is a measure of how muchmdata sets (of sizeN) change together. The variance,σi, in the ith data set is shown on the main diagonal. The covariance matrix,
Cov =
Px21
N
Px1x2
N
Px1x3
N . . . Px1xm
N
Px2x1
N
Px22
N
Px2x3
N . . . Px2xm
N
Px3x1
N
Px3x2
N
Px23
N . . . Px3xm
.. N
. ... ... . .. ...
Pxmx1
N
Pxmx2
N
Pxmx3
N . . . Px2m N
.
Notice that the covariance matrix is a symmetric matrix.
Many times when you are analyzing data, it is common to have to work with lots of variables. Sometimes these variables are redundant (or related) and there are only a few true sources (or variables of relevance) of information in the data. It is an analyst’s job to determine those sources.
The correlation matrix is directly related to the covariance matrix, as it lists the correlation coefficients between two random variablesiand j in the ijth entry; however the main diagonal entries will have a value of 1 represent- ing a full positive linear correlation between a variable and itself.
Exercises:
a. The data that is in Table 5.1 is International Monetary Fund (IMF) data for 6 world regions related to GDP and distribution of GDP. Find the co- variance matrix related to the given variables. Typevar(data set name) to find the variance of a data set and to find the covariance matrix af- filiated with n data sets type cov(data set 1; data set 2;...;data set n).
b. The entries of the correlation matrix, Cor, are directly related to the co- variance matrix. Cor(i,j) = cov(i,j)σ
iσj . Find the correlation matrix for the data above.
The correlation between two variablesx andy, or correlation coefficient, will always be between -1 and 1. If the correlation coefficient is close to 1 then
108 Exploring Linear Algebra Labs and Projects with MATLABR TABLE 5.1
IMF Data
GDP Investment Gross Volume of Current % of GDP National Exports of
Prices Savings Goods
% of GDP % of GDP
Central and 1844.682 21.222 17.005 4.802
Eastern Europe
Commonwealth of 2658.841 24.362 27.591 4.081
Independent States
Developing Asia 12324.727 41.891 42.972 4.424
ASEAN 5 1935.796 29.827 30.624 2.822
Latin America 5765.563 21.442 19.549 3.456
and the Caribbean
Middle East, North 3422.987 24.964 36.157 3.885
Africa, Afghanistan, and Pakistan
we say that there is a positive linear correlation, that is, when one variable increases the other variable increases.
If the correlation coefficient is close to -1 then we say that there is a negative linear correlation, and thus when one of the variables increases the other will decrease. If the correlation coefficient is close to 0 then there is no linear correlation.
c. Using your correlation matrix from part b. find the correlation coefficient between the Investment% of the GDP and the Gross national savings % of the GDP and interpret the results.
d. Find the correlation coefficient between Investment%of GDP and Volume of exports of goods and interpret this result.
e. Find the Cholesky Decomposition of the correlation matrix from part b.
In the next few exercises, we will use the Cholesky Decomposition of the correlation matrix to find a list of correlated data. Note that the correlation of the data is also dependent on the strength of correlation among the variables in your matrix. Therefore, a set of variables which are significantly correlated would produce even more dramatic results than those from this example. We will start with a random set of data and then use the decomposition to corre- late it.
f. Generate a random table of 1200 real data points, example Type:
r = -1 + (1+1)*rand(6,200);
Matrix Decomposition with Applications 109 and plot the data by typing
x = linspace(1,800,800);
scatter(x,[r(1,:) r(2,:) r(3,:) r(4,:)]).
g. Using Matrix L, from part e., calculate C =L∗r to get your correlated data. To see a graph of this data Type:
scatter(x,[C(1,:) C(2,:) C(3,:) C(4,:)])
If the correlated dataC=L.ris actually more linearly correlated, you will see a less random and more linear behavior in this second plot.
110 Exploring Linear Algebra Labs and Projects with MATLABR