Linear system handout

DIRECT METHODS AND ITERATIVETECHNIQUES FOR SOLVING LINEAR SYSTEMS Dr.. Lê Xuân Đại HoChiMinh City University of Technology Faculty of Applied Science, Department of Applied Mathematics E

DIRECT METHODS AND ITERATIVE

TECHNIQUES FOR SOLVING LINEAR SYSTEMS

Dr Lê Xuân Đại

HoChiMinh City University of Technology Faculty of Applied Science, Department of Applied Mathematics

Email: ytkadai@hcmut.edu.vn

HCMC — 2019.

mathematics to the social sciences and the

quantitative study of business and economic

problems.

1 We consider a linear system of n

A = (a ij ) ∈ M n (K ) and detA 6= 0. So this

methods of approximating the solution

to linear systems using iterative

methods.

E LEMENTARY ROW OPERATIONS

We use 3 operations to simplify the linear system (1):

1 Equations can be transposed in order (r i ↔ r j ).

2 Equation can be multiplied by any non-zero

constant λ 6= 0(r i → λr i ).

3 Equation can be multiplied by any constant λ

and added to another equation (r i → r i + λr j ).

By a sequence of these operations, a linear system will be systematically transformed into a new linear system that is more easily solved and has the same solutions.

3 Rewrite the new system corresponding to matrix

of row echelon form.

4 Backward substitution can be performed to

solve the n th equation for x n , the (n − 1)st

equation for x n−1 Continuing this process, we obtain the unique solution.

The corresponding system is

LU D ECOMPOSITIONS

3 Doolittle’s Method LU factorization

when the diagonal elements of lower

L andU have the form

The entries of matrices L and U can be

defined by the formula

Multiplying 2 matrices L and U, we have











M ULTIPLE CHOICE E XERCISES

 Factor A into the LU

decomposition A = LU using Doolittle’s

Method Find the entry ` 32 of matrixL

 Factor A into the LU

decomposition A = LU using Doolittle’s

Method Find the sum u 11 + u 22 + u 33 of

T HEOREM 4.2

The square matrix A is positive definite if and only if A can be factored in the form A = B.B T , where B is lower triangular with nonzero diagonal entries

( b ii > 0, i = 1 n ) and is defined by the formula:

The system can be rewritten in matrix form







E XAMPLE 4.2

Determine matrix B in Cholesky

factorization of the positive definite matrix

p 7 7

2 p 21 7







5 The above answers

2 p

2 .

D EFINITION 5.1

A vector norm X ∈ R n is a function, denoted

by ||X||, from R n into R with the following

properties:

1 ∀X ∈ R n , ||X|| Ê 0,||X|| = 0 ⇔ X = 0

2 ∀X ∈ R n , ∀λ ∈ R,||λX|| = |λ|.||X||

3 ∀X, Y ∈ R n , ||X + Y || É ||X|| + ||Y ||.

We will need only two specific norms on R n :

D EFINITION

D EFINITION 6.1

A sequence (X (m) ) ∞ m=0 of vectors X (m) ∈ R n is

said to converge to X with respect to the

norm || · || as m → +∞ if and only if

||X (m) − X|| → 0 as m → +∞.

T HEOREM 6.1

The sequence of vectors (X (m) ) ∞ m=0 converges

(x (m) k ) converge tox k , ∀k = 1,2, ,n.

The system AX = B(det(A) 6= 0) has a unique

T HEOREM 6.2

A matrix A is well-conditioned if k(A) is close

to 1 , and is ill-conditioned when k(A) is

significantly greater than 1.

Now we consider the system A X = e e B where B = e

µ 3 3.1

¶

This system has unique solution X = e µ −17

10

¶ We can find k ∞ (A) = 1207.01 >> 1 Therefore, B ≈ e B, however X and X e are as much as different.

M ULTIPLE CHOICE EXERCISES

MatA x −1 ⇒ A −1 =

à 3 14

2 21

¯

¯

¯

¯ ,

An iterative technique to solve the n × n

´ ∞

m=0

defined by the formula

X (m) = TX (m−1) + C, m = 1,2, (2)

T HEOREM 6.3

If ||T|| < 1 then the sequence of vectors

³

X (m) ´ ∞

m=0 defined by the formula (2) will

converge to X , starting with an initial

approximation X (0) Then the error analysis is

D EFINITION 6.3

diagonally dominant when

n

X

j=1,j6=i

|a ij | < |a ii |, i = 1, 2, , n

Note If A is a strictly diagonally dominant

Consider the system (1) where Ais strictly diagonally dominant matrix Factorize the

Since a ii 6= 0, ∀i = 1, 2, , n so detD 6= 0.

J ACOBI ’ S M ETHOD

We have

AX = B ⇔ (DưLưU)X = B ⇔ (D)X = (L+U)X +B

⇔ X = D ư1 (L + U)X + D ư1 B.

Jacobi iterative method can give us

X (m) = T j X (mư1) + C j , m = 1,2,

For each i Ê 1, generate the components x i (m)

method always converges starting with

E XAMPLE 6.2

Use Jacobi’s iterative technique to find

approximation accurate to within 10 −4 ,

choosing norm infinity

|0.09| + | − 0.15| < |3|; |0.04| + | − 0.08| < |4| so

matrix Therefore, Jacobi’s method always converges Rewrite the given system in the



 and then evaluate X (1) , X (2) ,

||T j || ∞ = max

i=1,2,3

3 P

j=1 |t ij | = max{|0| + | − 0.06| + |0.02|,| − 0.03| + |0| + |0.05|,

1 − 0.08 × 5.48 × 10

−4

≈0.4765 × 10 −4 < 10 −4

4

à 0.390 0.170

!

⇒ Answer 3

G AUSS -S EIDEL ’ S METHOD

Rewrite the system (1) in matrix form

Let T g = (D − L) −1 U, C g = (D − L) −1 B, we receive iterative formula Gauss-Seidel of the form

X (m) = T g X (m−1) + C g , m = 1,2,

E XPLICIT ITERATIVE FORMULA G AUSS -S EIDEL

x (m) 1 = c 1 +

n

X

j=2 t1jx m−1 j ,

Gauss-Seidel’s method is a possible

improvement of Jacobi’s method, but it

these most recently calculated values

x (m) 1 , x 2 (m) , , x i−1 (m)

E XAMPLE 6.3

Use the Gauss-Seidel iterative technique to find approximate solutions to the following system accurate within 10 −4 Choose norm infinity

SOLUTION We have |0.24| + | − 0.08| < |4|;

|0.09| + | − 0.15| < |3|; |0.04| + | − 0.08| < |4| so

matrix Rewrite the system in the form



 and then evaluate X (1) , X (2) ,

x (1) 1 = c 1 + t 12 x 2 (0) + t 13 x (0) 3 ,

x (1) 2 = c 2 + t 21 x 1 (1) + t 23 x (0) 3 ,

x 3 (1) = c 3 + t 31 x (1) 1 + t 32 x (1) 2

A = (8 − 0.24B + 0.08C) ÷ 4 :

B = (9 − 0.09A + 0.15C) ÷ 3 :

C = (20 − 0.04A + 0.08B) ÷ 4

Press ”=” continuously until receiving the

Error analysis ||X (3)

−X (2) || ∞ = max

i=1,2,3 |x i (3) −x i (2) | = max{| − 1.499.10 −4 |, |0.123.10 −4 |, |0.017.10 −4 |} = 1.499 × 10 −4

1 − 0.08 × 1.499 × 10

−4

≈0.1303 × 10 −4 < 10 −4

4

à 0.759 1.093

!

A = (5 + 6B) ÷ 15 : B = (5 + 5A) ÷ 8

Press ”=” continuously until receiving the

à 0.755 1.096875

! ⇒ Answer

THANK YOU FOR YOUR ATTENTION