Strong convergence of subgradient viscosity methods for the bilevel Ky Fan inequality

In this paper, building upon subgradient techniques and viscositytype approximations, we propose a simple projection algorithm for solving the bilevel Ky Fan inequality in a real Hilbert space, where the second level of the problem is variational inequalities. By choosing suitable regularization parameters, strong convergence of proposed iteration sequences is established under minimal assumptions of the cost bifunctions. Preliminary computational experience is also reported

Strong convergence of subgradient-viscosity

Pham N Anh†, Le Q Thuy‡and Tran T.H Anh §

Abstract In this paper, building upon subgradient techniques and viscosity-type ap-proximations, we propose a simple projection algorithm for solving the bilevel Ky Fan inequality in a real Hilbert space, where the second level of the problem is variational inequalities By choosing suitable regularization parameters, strong convergence of pro-posed iteration sequences is established under minimal assumptions of the cost bifunctions Preliminary computational experience is also reported

Keyword: Ky Fan inequalities, variational inequalities, monotone, subgradient, viscosity AMS 2010 Mathematics subject classification: 65 K10, 90 C25

1 Introduction

Let H be a real Hilbert space with inner product and induced norm h·, ·i and k · k,

respec-tively Let C be a nonempty closed convex subset of H The bilevel Ky Fan inequality is

formulated as the follows:

Find x∗ ∈ C such that g(x∗, x) ≥ 0 ∀x ∈ Sol(f, C), where g : C × C → R, Sol(f, C) denotes the set of all solutions of the Ky Fan inequality:

Find y∗ ∈ C such that f (y∗, y) ≥ 0 ∀y ∈ C,

and f : C × C → R As usual, the bifunctions f and g are called to be the cost bifunctions.

In this paper, we consider an important special case of the bilevel Ky Fan inequality, shortly BKF (g, F, C), that f (x, y) = hF (x), y − xi for all x, y ∈ C, where the mapping

F : C → H So, Problem BKF (g, F, C) is a simple bilevel problem However, it covers the

Ky Fan inequality [9, 14], bilevel minimization problems [11, 12, 22, 24], bilevel variational inequalities [3, 4, 6, 13, 26], mathematical programs with equilibrum constraints [16], the

∗ This work was completed at the Vietnam Institute for Advanced Study in Mathematics and funded by Vietnam National Foundation for Science and Technology Development (NAFOSTED) under grant number ”101.02-2013.03”.

† Department of Scientific Fundamentals, Posts and Telecommunications Institute of Technology, Hanoi, Vietnam (anhpn@ptit.edu.vn).

‡ School of Applied Mathematics and Informatics, Ha Noi University of Science and Technology, Vietnam.

§ Department of Mathematics, Haiphong University, Vietnam.

minimum-norm problem [27] and many other problems in science and engineering (see [5, 10, 21])

It is well-known that the viscosity method is a fundamental method for solving vari-ational inequalities with the constraint set is the fixed point set of a nonexpansive map-ping (shortly, the hierarchical variational inequality) That is the problem of finding

x∗∈ F ix(T ) (the fixed point set of a nonexpansive mapping T : C → C) such that

h(I − V )(x∗), x − x∗i ≥ 0 ∀x ∈ F ix(T ), where V : C → C is nonexpansive and I is the identity mapping In this iteration method

in a real Hilbert space, there are two schemes to solve it: One implicit and one explicit as the follows:

xt= tf (xt) + (1 − t)T (xt) and

xk+1= λkf(xk) + (1 − λk)T (xk), where f is contraction on C, t ∈ (0, 1) and {λk} ⊂ [0, 1]

In [19], Mainge and Moudafi considered the viscosity method for the hierarchical variational inequality as the follows:

x0 ∈ C, xk+1:= λkf(xk) + (1 − λk)[αkV(xk) + (1 − λk)T (xk)],

where f : C → C is a contraction Under certain appropriate conditions onto param-eters αk and λk, they showed that the iteration sequence {xk} converges strongly to a solution of the hierarchical variational inequality It should be noticed that the method can be regarded as a generalized version Halpern’s algorithm Upon the idea of this, Lu

et al in [15] investigated other hybrid viscosity approximation methods for solving the hierarchical variational inequality By combining viscosity approximation methods with projected subgradient techniques, Mainge in [17] established a strong convergence theo-rem for optimization with variational inequality constraints in H Xu in [25] introduced

a viscosity method for hierarchical fixed point approach to variational inequalities Here, the author showed that the sequence defined by the implicit hierarchical scheme converges strongly in norm to a solution of the hierarchical variational inequality Recently, the viscosity method has been studied to develop iteration algorithms for variational inequal-ities, Ky Fan inequalinequal-ities, and other problems (see [8, 18]) Methods for solving Problem BKF(g, F, C) also have been studied extensively (see [1, 2, 14, 20])

It is the purpose of the present article to extend the above viscosity approxima-tion method, with suitable modificaapproxima-tions, by using subgradient techniques to Problem BKF(g, F, C) By this way, we obtain convergent algorithms for solving the problem in

a real Hilbert space H The iteration algorithm is simple and efficient At each iteration,

we only require computing the projection of a point onto the domain

This paper is organized as follows In Section 2, we recall some definitions, known lemmas with the corresponding references and our main results Finally, we present some numerical experiments to illustrate the behavior of the proposed algorithms

2 The algorithm and its convergence

Let us recall some definitions and results that will be used in the sequel Let C be a nonempty closed convex subset of H, for every x ∈ H, there exists an unique element

P rC(x), defined by

P rC(x) = arg min {ky − xk : y ∈ C}

It is also known that P rC is firmly nonexpansive, or 1-inverse strongly monotone, i.e.,

hP rC(x) − P rC(y), x − yi ≥ kP rC(x) − P rC(y)k2 ∀x, y ∈ C

Besides, we recall some other properties as the follows:

kx − PC(x)k2+ kPC(x) − yk2 ≤ kx − yk2 ∀x, y ∈ C; (2.1)

hx − P rC(x), P rC(x) − yi ≥ 0 ∀x, y ∈ C; (2.2)

kz − P rC(x − y)k2≤ kx − zk2

− 2hx − z, yi + 5kyk2

∀x, z ∈ C, y ∈ H (2.4)

A mapping F : C → H is called monotone on C, if

hF (x) − F (y), x − yi ≥ 0 ∀x, y ∈ C;

paramonotone on C, if F is monotone on C and

hF (x) − F (y), x − yi = 0 ⇒ F (x) = F (y);

weakly closed on C, if

n {xk} ⊂ C, xk⇀ x and F (xk) ⇀ wo⇒ w = T x

A bifunction g : C × C → R is called monotone on C, if

g(x, y) + g(y, x) ≤ 0 ∀x, y ∈ C;

ρ-strongly monotone on C, if

g(x, y) + g(y, x) ≤ −ρkx − yk2 ∀x, y ∈ C

Throughout this paper, we consider the bifunction g, the mapping F and regulariza-tion parameters with the following properties:

(A1) g is ρ-strongly monotone and for each x ∈ C, g(x, ·) is weakly continuous and convex

on the domain C, ∂2g(·, ·) is upper semicontinuous and g(x, x) = 0 for all x ∈ C; (A2) F : C → H is paramonotone and weakly closed on C, and Lipschitz continuous on the domain C;

(A3) the set SV I(F, C) := {x ∈ C : hF (x), y − xi ≥ 0 ∀y ∈ C} is nonempty;

(A4) µ ∈ (0, ∞), for each k ≥ 0, αk and λk are nonnegative real numbers such that

∞

X

k=0

λk= ∞,

∞

X

k=0

λ2k<∞, lim

k→∞λk= 0, lim

k→∞αk= 0,

∞

X

k=0

αk= ∞,

∞

X

k=0

αkλk= ∞

It is easy to check again that condition (A4) is satisfied by

λk:= 1

(k + 1)λ, αk:= 1

(k + 1)α, with λ ∈ 1

2,1

 and α ∈ (0, 1 − λ)

We establish a series of preliminary results needed for our convergence analysis Lemma 2.1 [17] Let {bn} be a sequence of real numbers that does not decrease at infinity,

in the sense that there exists a subsequence {bn j}j≥0 of {bn} which satisfies bn j < bn j +1

for all j ≥ 0 Also consider the sequence of integers {τn}n≥0 defined by

τn= max{k ≤ n : bk< bk+1}

Then, {τn}n≥n 0 is a nondecreasing sequence verifying lim

n→∞τn = ∞, and for all n ≥ 0, it

holds that

bτn < bτn+1 and bn< bτn+1 Lemma 2.2 [17] Let {λn} and {βn} be nonnegative sequences such that

∞

X

n=0

λn= ∞,

∞

X

n=0

λ2n<∞ and

∞

X

n=0

λnβn<∞

Then, the following two results hold:

(i) There exists a subsequence {βnk} of {βn} such that lim

k→∞βnk = 0

(ii) If {λn} and {βn} are also such that βn+1− βn < θλn (for some positive θ), then

lim

n→∞βn= 0

Lemma 2.3 [18] Let φ be the functional defined, for any x in C, by φ(x) := hT x, x − qi, where q ∈ H and T : C → H is monotone and weakly closed on C, and Lipschitz continuous

on bounded subset of C Then, φ is weakly lower semicontinuous on C.

In order to solve Problem BKF (g, F, C), we investigate the convergence analysis of the sequence {xk} given by the following iteration:

Algorithm 2.4 Initialization Take two sequences of nonnegative real numbers {αk}

and {λk}, µ ∈ (0, +∞), x0

∈ C.

Step 1 Choose wk ∈ ∂2g(xk, xk).

Step 2 Set dk = F (xk) + αkwk, ηk= max{µ, kdkk} and xk+1 = P rC



xk−λk

η kdk.

k:= k + 1 and go to Step 1.

We now turn to analyse the convergence of Algorithm 2.4

Theorem 2.5 Let the above assumptions (A1)−(A4) hold Then, the sequence {xk}

gen-erated by Algorithm 2.4 converges strongly to the unique solution of Problem BKF (g, F, C) Proof We divide the proof into several claims.

3 Examples and numerical results

In this section, we illustrate the proposed algorithms by a class of the bilevel Ky Fan inequality as defined by BKF (g, F, C), where C is a polyhedral convex set given by

C:= {x ∈ Rn: Ax ≤ b}, and the bifunction g : C × C → R is of the form

g(x, y) := hG(x) + Qy + q, y − xi, where G : C → Rn, Q ∈ Rn×n is a symmetric positive semidefinite matrix and q ∈ Rn Since Q is symmetric positive semidefinite, g(x, ·) is convex for each fixed x ∈ C The cost mapping F is defined more detail in the following examples It is well-known that

if G is ξ-strongly monotone on C and ξ > kQk, then g is strongly monotone on C × C with constant ξ − kQk (see [23]) As usual, we can say that xk is an ǫ-solution of Problem BKF(g, F, C) if kxk+1− xkk ≤ ǫ

Example 3.1 Consider the mapping F : R3

→ R3

given in [24] which is paramonotone (not strictly monotone) of the form

F(x) :=





2 2 0

1 2 0

0 0 0



x, G(x) := P x and P :=





2 5.6 0





In this example, let C, Q and q be defined by

C :=x ∈ R3

: xi ≥ 0 ∀i = 1, 2, 3, x1+ x2+ x3 ≤ 10 ,

and

Q:=





1 1.6 0



, q:=





1

−3 4





Example 3.2 Consider the mapping F : R4

→ R4

described in [7] which is paramono-tone of the form

F(x) = (x1− 2x2,−2x1+ 4x2, x3− 2x4,−2x3+ 4x4)T and

G(x) := P x, P :=









7 2.5 0 0 2.5 5.5 0 0









Take C, Q and q as the follows:

C:=x ∈ R4

: xi≥ 0 ∀i = 1, · · · , 4, x1+ x2+ x3+ x4≤ 9 ,

and

Q:=















 , q :=









−1 2 3 5









From the preliminary numerical results reported in the tables, we observe that (a) Similar to other methods for Ky Fan inequalities such as the proximal point algorithm,

or the auxiliary principle for equilibrium problems, the rapidity of the algorithm depends very much on the starting point

(b) The algorithm is quite sensitive to the choice of the parameters λk and αk

References

[1] Anh, P.N.: An interior proximal method for solving pseudomonotone nonlipschitzian multivalued variational inequalities Nonlinear Analysis Forum 14, 27-42 (2009) [2] Anh, P.N., Kim, J.K.: Outer approximation algorithms for pseudomonotone equilib-rium problems Computers and Mathematics with Applications 61, 2588-2595 (2011) [3] Anh, P.N., Kim, J.K., Muu, L.D.: An extragradient method for solving bilevel vari-ational inequalities Journal of Global Optimization 52, 627-639 (2012)

[4] Anh, P.N., Muu, L.D., Strodiot, J.J.: Generalized projection method for non-Lipschitz multivalued monotone variational inequalities ACTA Mathematica Viet-namica 34, 67-79 (2009)

[5] Anh, P.N., Muu, L.D., Hien, N.V., Strodiot, J.J.: Using the Banach contraction prin-ciple to implement the proximal point method for multivalued monotone variational inequalities Journal of Optimization Theory and Applications 124, 285-306 (2005) [6] Baiocchi, C., Capelo, A.: Variational and quasivariational inequalities, applications

to free boundary problems NewYork, (1984)

[7] Bao, X-B., Liao, L-Z., Qi, L.: Novel Neural Network for Variational inequalities with Linear and Nonlinear Constraints IEEE Transactions on Neural Network 16, 1305-1317 (2005)

[8] Ceng, L.C., Yao, J.C.: Relaxed viscosity approximation methods for fixed point prob-lems and variational inequality probprob-lems Nonlinear Analysis 69, 3299-3309 (2008) [9] Daniele, P., Giannessi, F., Maugeri, A.: Equilibrium problems and variational models Kluwer, (2003)

[10] Facchinei, F., Pang, J.S.: Finite-dimensional variational inequalities and complemen-tary problems Springer-Verlag, NewYork, (2003)

[11] Glackin, J., Ecker, J.G., Kupferschmid, M.: Solving bilevel linear programs using multiple objective linear programming Journal Optimization Theory Applications

140, 197-212 (2009)

[12] Iiduka, H.: Strong convergence for an iterative method for the triple-hierarchical constrained optimization problem Nonlinear Analysis 71, e1292-e1297 (2009) [13] Kalashnikov, V.V., Kalashnikova, N.I.: Solving two-level variational inequality Jour-nal of Global Optimization 8, 289-294 (1996)

[14] Konnov, I.V.: Combined relaxation methods for variational inequalities Springer-Verlag, Berlin, (2000)

[15] Lu, X.W., Xu, H.K., Yin, X.M.: Hybrid methods for a class of monotone variational inequalities Nonlinear Analysis 71, 1032-1041 (2009)

[16] Luo, Z.Q., Pang, J.S., Ralph, D.: Mathematical programs with equilibrum con-straints Cambridge University Press, Cambridge, (1996)

[17] Mainge, P.E.: Strong convergence of projected subgradient methods for nonsmooth and nonstrictly convex minimization, Set-Valued Analysis 16, 899-912 (2008) [18] Mainge, P.E.: Projected subgradient techniques and viscosity methods for optimiza-tion with variaoptimiza-tional inequality constraints European Journal of Operaoptimiza-tional Re-search 205, 501-506 (2010)

[19] Mainge, P.E., Moudafi, A.: Strong convergence of an iterative method for hierarchical fixed-point problems Pacific Journal of Optimization 3 (3), 529538 (2007)

[20] Moudafi, A.: Proximal methods for a class of bilevel monotone equilibrium problems Journal of Global Optimization 47, 287-292 (2010)

[21] Sabharwal, A., Potter, L.C.: Convexly constrained linear inverse problems: Iterative least-squares and regularization IEEE Trans Signal Process 46, 2345-2352 (1998) [22] Solodov, M.: An explicit descent method for bilevel convex optimization Journal of Convex Analysis 14, 227-237 (2007)

[23] Quoc, T.D., Muu, L.D., Nguyen, V.H.: Extragradient algorithms extended to equi-librium problems Optimization 57, 749-776 (2008)

[24] Trujillo-Corteza, R., Zlobecb, S.: Bilevel convex programming models Optimization

58, 1009-1028 (2009)

[25] Xu, M.H.: Viscosity methods for hierarchical fixed point approach to variational inequalities Taiwanese Journal of Mathematics 14 (2), 463-478 (2010)

[26] Xu, M.H., Li, M., Yang, C.C.: Neural networks for a class of bi-level variational inequalities Journal of Global Optimization 44, 535-552 (2009)

[27] Yao, Y., Marino, G., Muglia, L.: A modified Korpelevich´s method convergent to the minimum-norm solution of a variational inequality Optimization, 1-11, iFirst (2012)