DSpace at VNU: Parallel regularized Newton method for nonlinear ill-posed equations

DSpace at VNU: Parallel regularized Newton method for nonlinear ill-posed equations tài liệu, giáo án, bài giảng , luận...

DOI 10.1007/s11075-011-9460-y

O R I G I N A L P A P E R

Parallel regularized Newton method for nonlinear

ill-posed equations

Pham Ky Anh · Cao Van Chung

Received: 7 September 2010 / Accepted: 21 March 2011 /

Published online: 9 April 2011

© Springer Science+Business Media, LLC 2011

Abstract We introduce a regularized Newton method coupled with the

paral-lel splitting-up technique for solving nonlinear ill-posed equations with smooth monotone operators We analyze the convergence of the proposed method and carry out numerical experiments for nonlinear integral equations

Keywords Monotone operator · Regularized Newton method ·

Parallel splitting-up technique

Mathematics Subject Classifications (2010) 47J06 · 47J25 · 65J15 ·

65J20· 65Y05

1 Introduction

The Kaczmarz method for a system of linear algebraic equations was in-vented more than 70 years ago It has proved to be quite efficient in many applications ranging from computer tomography to digital signal processing Recently, there has been a great interest in the so-called Kaczmarz meth-ods for solving ill-posed problems, namely, Landweber–Kaczmarz method,

P K Anh (B) · C V Chung

Department of Mathematics, Vietnam National University, 334 Nguyen Trai,

Thanh Xuan, Hanoi, Vietnam

e-mail: anhpk@vnu.edu.vn

C V Chung

e-mail: chungcv@vnu.edu.vn

Newton–Kaczmarz method and the steepest-descent-Kaczmarz method The main idea of Kaczmarz methods is to split the initial ill-posed problem into

a finite number of subproblems and to perform a cyclic iteration over the subproblems These methods not only reduce the overall computational effort but also impose less constraints on the nonlinearity of the operators Besides, experiments show that the Kaczmarz methods in some cases are performed better than standard iterative methods For a wide literature concerning Kaczmarz methods, please refer to [1 6] and references therein

Clearly, Kaczmarz methods are inherently sequential algorithms When the

number of equations N is large, the Kaczmarz like methods are costly on a

single processor In this paper we introduce a parallel regularized Newton method, which may be regarded as a counterpart of the regularizing Newton– Kaczmarz method Our idea consists of splitting the given ill-posed problem into subproblems and performing synchronously one iteration of the regular-ized Newton method to each subproblem The next approximate solution is determined as a convex combination of the obtained iterations for subprob-lems The benifit of our approach is clear Based on parallel computation we can reduce the overall computational effort under widely used conditions on smooth monotone operators (cf [7 10])

Let us consider a nonlinear operator equation

where F : H → H is a twice locally Frechet differentiable and monotone operator; f ∈ H is given and H is a real Hilbert space We assume that the set C ⊂ H of solutions of (1.1) is not empty, hence C is a convex and closed subset of H (see, e.g [7,11])

Several problems arising in Quantum Mechanics, Wiener-type filtering theory or in physics, where dissipation of energy occurs, can be reduced to equations involving monotone operators (see [7 10])

If F is not strongly monotone or uniformly monotone, problem (1.1) in general is ill-posed In that case a process known as Lavrentiev regularization consisting of solving the singularly perturbed operator equation

A (x) + α n x = 0, (α n > 0, n = 0, 1, 2, ) (1.2)

is recommended (see [7,11–13])

In [14] we proposed parallel iterative regularization methods for solving (1.1), combining the regularization and parallel splitting-up techniques In this paper we use the parallel splitting-up technique (see [15]) and the Newton method to solve (1.2) Throughout this paper we assume that for all x ∈ H,

F (x) − f =N

i=1F i (x) −N

i=1 f i and put A i (x) := F i (x) − f i (i = 1, N) Further,

we suppose that all F i : H → H are twice locally Frechet differentiable and

monotone operators Then, knowing the n-th approximation z nwe can

deter-mine the next approximation z n+1as follows



Ai (z n ) + α n

N + γ n



I



(z i

n − z n ) = −



A i (z n ) + α n

N z n



, i = 1, 2, , N,

(1.3)

z n+1= 1

N

N



i=1

where α n and γ n are positive numbers Due to the nonnegativity of the

derivative Ai (z n ) (see [7]), all the linear regularized equations (1.3) are well-posed Moreover, being independent from each other, they can be solved stably and synchronously by parallel processors

The article is outlined as follows In Section2we provide a convergence analysis of the proposed method in both exact data and noisy data cases In the final Section3we verify all the assumptions required and perform numerical experiments for a model problem

2 Convergence analysis

We begin with some notions and auxiliary results

Lemma 2.1 If α n is a converging to zero sequence of positive numbers, then for each n∈N, the regularized equation (1.2) has a unique solution x∗n and

x∗n → x†:= argmin

x ∈C x as n → +∞ Moreover, the following estimates hold

i x∗

n  ≤ x†;

ii x∗

n+1− x∗

n ≤ |α n+1−α n|

α n x† for all n.

The proof of this lemma can be found in [7,11]

We recall that an operator A : H → H is called c−1—inverse-strongly monotone, if

1

c A(x) − A(y)2, ∀x, y ∈ H, where c is some positive constant (see, e.g [16])

Obviously, every inverse-strongly monotone operator is monotone and not necessarily strongly-monotone

The following lemma shows that a system of inverse-strongly monotone operator equations may be reduced to a single equation involving the sum of these operators

Lemma 2.2 Suppose A i , i = 1, 2, , N are c i−1 -inverse-strongly monotone operators If the system of equations

is consistent, then it is equivalent to the operator equation

A (x) :=

N



i=1

Proof Obviously, any common solution of (2.1) is a solution of (2.2)

Con-versely, let x∗be a solution of (2.2) and y be a common solution of (2.1) Then,

0=

 N



i

A i (x∗), x∗− y

=

 N



i

[A i (x∗) − A i (y)], x∗− y

=

N



i

i (x∗) − A i (y), x∗− y ≥

N



i

c−1i A i (x∗) − A i (y)2≥ 0 Thus, A i (x∗) = A i (x∗) − A i (y) = 0, hence x∗is a solution of system (2.1). 

In what follows, we denote a closed ball centered at a ∈ H and with a radius r by B [a, r] The following result on the convergence rate of Lavrentiev

regularization method is needed for our further study

Lemma 2.3 Let x†= 0 be a minimal-norm solution of (1.1) and assume A i (x),

i = 1, N, are twice continuously Frechet dif ferentiable operators in B[0, r] with

a radius r > x† Moreover, let A

i (x) ≤ L i , i = 1, 2, , N for all x ∈ B[0, r] Then the following statements hold:

(a) If A i , i = 1, 2, , N, are monotone and there exists an element u ∈ H, such that

x†= A(x†)u =

N



i=1

Ai (x†)u and

N



i=1

L i u < 2. (2.3)

Then the regularized solution x∗n of (1.2) converges to x†at the rate

x∗

(b) If A i , i = 1, 2, , N, are c−1

i - inverse-strongly monotone and system (2.1)

is consistent Moreover, assume that there exist w i , i = 1, 2, , N, such that

x†=

N



i=1

A∗i (x†)w i and

N



i=1

L i w i  < 1. (2.5)

Then the following convergence rate holds

x∗

n − x† = O(α1/4

Proof The first estimate of Lemma 2.3 can be found in [11,13] For the second estimate we can process as in [17] Indeed, Lemma 2.2 ensures that x† is a common solution of equations (2.1) Further, we have

1

2x∗

n − x+

n2= 1

2{x∗

n2− x†2 †, x n − x†}

†, x∗

n − x† = −

N



i=1

∗

i (x†)w i , x∗

n − x†

=

N



i=1

i (x∗

n ) − A i (x†) − A

i (x†)(x∗

n − x†), w i −

N



i=1

i (x∗

n ), w i

=

N



i=1

1 0 1 0

t i (x†+ st(x∗

n − x†))(x∗

n − x†)2, w i dsdt

−

N



i=1

i (x∗

n ), w i

Thus, we obtain the estimate

1

2x∗

n − x†2≤ 1

2

N



i=1

L i w i x∗

n − x†2+

N



i=1

A i (x∗

n )w i (2.7)

Using the inverse-strong monotonicity of A i (x) we get

A i (x∗

n )2= A i (x∗

n ) − A i (x†)2≤ c i i (x∗

n ) − A i (x†), x∗

n − x†

= c i

⎧

⎨

⎩

N



j=1

j (x∗

n )− A j (x†), x∗

n −x†−

j =i

j (x∗

n )− A j (x†), x∗

n −x†

⎫

⎬

⎭

≤ − c i α n ∗

n , x∗

n − x† ≤ 2α n c i x†2.

From the last inequality we findA i (x∗

n ) ≤√2α n c i x† Taking into account the last estimate, from (2.7) we getx∗

n − x†2≤ 2(1 −N

i=1L i w i )−1√

2α n x†

N

i=1

√

Before stating and proving convergence theorems we observe that A i (x∗

n ) +

α n

N x∗n → A i (x†) as n → +∞, hence we can assume A i (x∗

n ) + α n

N x∗n  ≤ C A for

all n = 0, 1, and i = 1, 2, , N Furthermore, we suppose that the operators

A i , i = 1, 2, , N, are twice continuously Frechet differentiable and A

i (x) ≤

ϕ for all x ∈ B[0, r], where r = M√D∗, and M > 1, D∗ ≥ maxC2

A , x†2 are appropriately chosen constants Now we have the following convergence result

Theorem 2.1 Let {α n }, {γ n } be two sequences of positive numbers, α n  0,

γ n  +∞ as n → +∞, such that the following conditions hold for all n ∈N

and for some constants c1∈ (0, 1), c2> 0

γ n α4

n ≥ γ0α 4

0 ; γ n3(α n − α n+1) 2

α5

n

≤ c1γ03

α3 0

and γ n (γ n+1− γ n ) ≤ c2γ2

0.

(2.8)

Moreover, the initial values α0 , γ0 and M satisfy the following relations

N2 ≤ 4γ0α 2

0; D∗≤ √γ l α00 < (M − 1)2D∗; l > 0, (2.9)

where l:=



2γ0(1−√c1)

(2+c2)(2Nγ0+α0)−4Nc1γ3+(4c1 +2√c1+c2)α0γ2+2α0

2γ0α2

 2

ϕ2 Then starting from z0 = 0, the approximations z n of the parallel regularized Newton method (1.3) and (1.4) will converge to the minimal norm solution x†

of (1.1).

For the sake of clarity, we divide the proof of Theorem 2.1 into several steps

Firstly, we establish a recurrence estimate for the distance between z ndefined

by (1.3), (1.4) and the regularized solution x∗nof (1.2)

Lemma 2.4 The distances ||e n || := ||z n − x∗

n || satisfy the following inequality.

e n+12≤ 1+  n

N (1 + 2 n )

 1

2√γ n

N



i=1

A

i (ξ i

n )2e n4+ Ne n2+ NC A2

γ2

n

+ N (1 + 2 n )(α n+1− α n )2

 n α2

n

x†2



where  n:= α n

2N γ n

Proof Inserting x = z n ; h = −e n := x∗

n − z n ; y = e i

n := z i

n − x∗

n in the Taylor formula



A (x) + A(x)h + A(x + θh)h2



,

where x , y, h ∈ H and θ := θ(y) ∈ (0, 1), we get



A i (x∗

n ) + α n

N x

∗

n , z i

n − x∗

n



i (z n ), z i

n − x∗

i (z n )(x∗

n − z n ), z i

n − x∗

n

2



i (ξ i

n )(x∗

n − z n )2, z i

n − x∗

n + α n

N

∗

n , z i

n − x∗

Hereξ i

n = θ i

n (x∗

n − z n ) + z nandθ i

n ∈ (0; 1) depends on A i , z i

n , x∗n On the other hand, from (1.2) it follows

N



i=1



A i (x∗

n ) + α n

N x

∗

n , z i

n − x∗

n



=

N



i=1



A i (x∗

n ) + α n

N x

∗

n , z i

n − z n



+

N



i=1



A i (x∗

n ) + α n

N x

∗

n , z n − x∗

n



=

N



i=1



A i (x∗

n ) + α n

N x

∗

n , z i

n − z n



Combining (2.11) with (2.12) we find

N



i=1

i (z n ), z i

n − x∗

n +

N



i=1



i (z n )(x∗

n − z n ), z i

n − x∗

n +α n

N

N



i=1

∗

n , z i

n − x∗

n

+1

2

N



i=1



i (ξ i

n )(x∗

n − z n )2, z i

n − x∗

n −

N



i=1



A i (x∗

n ) + α n

N x

∗

n , z i

n − z n



= 0.

(2.13) Multiplying both sides of (1.3) by z i n − x∗

n and summing up for i from 1 to N,

we have

N



i=1

i (z n ), z i

n − x∗

n +

N



i=1



i (z n )(z i

n − z n ), z i

n − x∗

n

+α n

N + γ n

N

i=1

i

n − z n , z i

n − x∗

n +α n

N

N



i=1

n , z i

n − x∗

n = 0. (2.14)

Subtracting (2.13) and (2.14), after a short computation we get

1

γ n

N



i=1



i (z n )(z i

n − x∗

n ), z i

n − x∗

n +

N



i=1

i

n − z n , z i

n − x∗

n

+ α n

N γ n

N



i=1

 i

n − z n , z i

n − x∗

n n − x∗

n , z i

n − x∗

n

2γ n

N



i=1



i (ξ i

n )(x∗

n − z n )2, z i

n − x∗

n

−γ1

n

N



i=1



A i (x∗

n ) + α n

N x

∗

n , z i

n − z n



Since A i is monotone, Ai (x) is a linear positive operator for all x Therefore,

the first term in the left-hand side of (2.15) is nonnegative Hence, from (2.15)

we get

N



i=1

i

n − z n , z i

n − x∗

n + α n

N γ n

N



i=1

z i n − x∗

n2

2γ n

N



i=1



i (ξ i

n )(x∗

n − z n )2, z i

n − x∗

n −γ1

n

N



i=1



A i (x∗

n ) + α n

N x

∗

n , z i

n − z n



.

Using the above notations we can rewrite the last inequality as follows

N



i=1

i

n − e n , e i

n + α n

N γ n

N



i=1

e i

n2

2γ n

N



i=1



i (ξ i

n )(−e n )2, e i

n − 1

γ n

N



i=1



A i (x∗

n ) + α n

N x

∗

n , e i

n − e n



.

n − e n , e i

n = 1

2e i

n2−1

2e n2+1

2e i

n − e n2, the last inequality is equivalent to

N



i=1

e i

n2−

N



i=1

e n2+

N



i=1

e i

n − e n2+ 2α n

N γ n

N



i=1

e i

n2

≤ γ1

n

N



i=1



i (ξ i

n )(−e n )2, e i

n −γ2

n

N



i=1



A i (x∗

n ) + α n

N x

∗

n , e i

n − e n



γ n

N



i=1



i (ξ i

n )(−e n )2, e i

n + 1

γ2

n

N



i=1

A i (x∗

n ) + α n

N x

∗

n2+

N



i=1

e i

n − e n2

≤ 2√γ1 n

N



i=1

A

i (ξ i

n )2e n4+ 1

2

γ3

n

N



i=1

e i

n2+ NC A2

γ2

n

+

N



i=1

e i

n − e n2.

Here we used the Young’s inequality 1

γ n



i (ξ i

n )(−e n )2, e i

n ≤ 1 2√γn A

i (ξ i

n )2

e n4+ 1

2√

γ3 e i

n2 Thus,

1+4α n √γ n − N

2N γ n √γ n

N

i=1

e i

n2≤ 1

2√γ n

N



i=1

A

i (ξ i

n )2e n4+ Ne n2+NC A2

γ2

n

.

By the assumptions N2≤ 4α2 0γ0andα4

n γ n ≥ α4

0γ0, we have 2α2

n √γ n − N ≥ 0 for all n, therefore α n

Nγ n ≤ 4α2√γ n −N

2Nγ n √γ n Hence, from the last inequality, we find

1+ α n

N γ n

N

i=1

e i

n2≤ 1

2√γ n

N



i=1

A

i (ξ i

n )2e n4+ Ne n2+ NC A2

γ2

n

(2.16)

From (1.4) we gete n+1 = z n+1− x∗

n+1 ≤ z n+1− x∗

n  + x∗

n+1− x∗

n Using the estimatex∗

n+1− x∗

n ≤ |α n+1−α n|

α n x† in Lemma 1.1, we have

e n+1 = z n+1− x∗

n+1 ≤ z n+1− x∗

n +|α n+1− α n|

α n x†

N

N



i=1

e i

n + |α n+1− α n|

α n x†

≤ √1

N

N

i=1

e i

n2

1/2

+|α n+1− α n|

α n x†.

Applying the inequality(a + b)2≤ (1 +  n )(a2+ 1

 n b2) to the last relation with

 n:= α n

2Nγ n > 0 and using (2.16), we come to the desired estimate (2.10)  Now for completing the proof of Theorem 2.1, we use estimate (2.10) to

show that e n −→ 0, hence z n − x† = e n + x∗

n − x†−→ 0 as n −→ ∞.

Proof of Theorem 2.1 Setting v k := e k2/λ k and λ k:= α k

√γ k, we can rewrite (2.10) as

v n+1≤ 1+  n

N (1 + 2 n )

 λ2

n v2

n

2λ n+1√γ n

N



i=1

A

i (ξ i

n )2+ N λ n

λ n+1v n

+ NC A2

λ n+1γ 2

n

+ N (1 + 2 n )(α n+1− α n )2

 n α2

n λ n+1 x†2



By our assumptions, we have v0 ≤ l and ξ i

0 ≤ x† ≤ M√D∗ Assume by induction thatv n ≤ l and ξ i

n  ≤ M√D∗ for some n≥ 0, we will show that

v n+1≤ l and ξ i

n+1 < M√D∗

From (2.17) and the estimatesA

i (ξ i

n ) ≤ ϕ, v n ≤ l, we get

v n+1− l ≤ 1+  n

(1 + 2 n )

 λ2

n ϕ2

2λ n+1√γ n

l2+

n

λ n+1 −1+ 2 n

1+  n

l

+ C A2

λ n+1γ 2

n

+(1 + 2 n )(α n+1− α n )2

 n α2

n λ n+1 x†2



.

Taking into account the relation max

C2

A , x†2

≤ D∗≤ lα0

√γ0, from the last inequality we find

v n+1− l ≤ 1+  n

(1 + 2 n )λ n+1

λ2

n ϕ2

2√γ n

l2−



n λ n+1

1+  n − (λ n − λ n+1)



l+ l α0

γ2

n √γ0 +l α0(1 + 2 n )(α n+1− α n )2

√γ0n α2

n



≤ (1 +  n )α2

n l (1 + 2 n )λ n+1γ 3/2

n



ϕ2l

2 −α n+1√γ n

α n √γ n+1 · 1

2N + α n /γ n

+γ n (α n √γ n+1− α n+1√γ n )

α2

n √γ n+1 +√γ α00 ·√γ1n α2

n

+2α0(Nγ n + α n )γ3/2

n (α n − α n+1) 2

√γ0α 5

n



Besides, a straightforward calculation yields

γ n (α n √γ n+1− α n+1√γ n )

α2

n √γ n+1

=γ

n



(α n − α n+1)(√γn+1+ √γ n ) + (α n+1√γ n+1− α n √γ n )



α2

n √γ n+1

≤ 2γ n (α n − α n+1)

α2

n

+γ n (α n+1− α n )

α2

n

+γ n (√γ n+1− √γ n )

α n √γ n+1

≤ γ n (α n − α n+1)

α2

n

+γ n (γ n+1− γ n )

2γ n α n =γ n (α n − α n+1)

α2

n

+(γ n+1− γ n )

2α n

From the assumptions γ3(α n −α n+1)2

α5 ≤ c1γ3

α3 and γ n (γ n+1− γ n ) ≤ c2γ2

0, it follows

α n+1

α n ≥ 1 −

c1α3γ3

α3γ3 and γ n+1

γ n ≤ 1 +c2γ2

2γ2 Hence, by (2.18) and the last relation,

we get

v n+1− l ≤ (1 +  n )α2

n l (1 + 2 n )λ n+1γ 3/2

n

ϕ2

l

2 +γ n (α n − α n+1)

α2

n

+(γ n+1− γ n )

2α n

−α n+1√γ n

α n √γ n+1 · 1

2N + α n /γ n+√γ α00 ·√γ1n α2

n

+2α0(Nγ n + α n )γ3/2

n (α n − α n+1) 2

√γ0α 5

n



≤ (1 +  n )α2

n l (1 + 2 n )λ n+1γ 3/2

n



ϕ2l

!

c1γ 3

0α n

γ n α3 0

+ c2γ02

2γ n α n +√γ0α0 √γ n α2

n

+2Nc1 γ

5 0

α2√γ n +2c1α n γ

5 0

α2 0

γ3

n

−

2N+α n

γ n

−1

×

1+c2γ02

2γ2

n

−1"

1−

!

c1α3

n γ3 0

α3γ3

n

#$

.