Quoc Published online: 2 April 2009 © Springer Science+Business Media, LLC 2009 Abstract We make use of the Banach contraction mapping principle to prove the linear convergence of a regu
Trang 1DOI 10.1007/s10957-009-9529-0
Regularization Algorithms for Solving Monotone
Ky Fan Inequalities with Application
to a Nash-Cournot Equilibrium Model
L.D Muu · T.D Quoc
Published online: 2 April 2009
© Springer Science+Business Media, LLC 2009
Abstract We make use of the Banach contraction mapping principle to prove the
linear convergence of a regularization algorithm for strongly monotone Ky Fan in-equalities that satisfy a Lipschitz-type condition recently introduced by Mastroeni
We then modify the proposed algorithm to obtain a line search-free algorithm which does not require the Lipschitz-type condition We apply the proposed algorithms to implement inexact proximal methods for solving monotone (not necessarily strongly monotone) Ky Fan inequalities Applications to variational inequality and comple-mentarity problems are discussed As a consequence, a linearly convergent derivative-free algorithm without line search for strongly monotone nonlinear complementarity problem is obtained Application to a Nash-Cournot equilibrium model is discussed and some preliminary computational results are reported
Keywords Ky Fan inequality· Variational inequality · Complementarity problem · Linear convergence· Lipschitz property · Proximal point algorithm · Equilibria · Nash-Cournot model
1 Introduction
Let C be a nonempty closed convex set in a real Hilbert space H and f : C × C → R.
We consider the following problem:
(P) Find x∗∈ C such that f (x∗, y) ≥ 0, for all y ∈ C.
Communicated by F Giannessi.
L.D Muu ()
Institute of Mathematics, VAST, Hanoi, Vietnam
e-mail: ldmuu@math.ac.vn
T.D Quoc
Hanoi University of Science, Hanoi, Vietnam
Trang 2186 J Optim Theory Appl (2009) 142: 185–204
We will refer to this problem as the Ky Fan inequality due to his results in this field [1] Problem (P) is very general in the sense that it includes, as special cases, the optimization problem, the variational inequality, the saddle point problem, the Nash equilibrium problem in noncooperative games, the Kakutani fixed point problem and others (see for instance [2 9] and the references quoted therein) The interest of this problem is that it unifies all these particular problems in a convenient way More-over, many methods devoted to solving one of these problems can be extended, with
suitable modifications, to solving Problem (P) It is worth mentioning that when f is convex and subdifferentiable on C with respect to the second variable, then (P) can
be formulated as a generalized variational inequality of the form
Find x∗∈ C, z∗∈ ∂2 f (x∗, x∗)such thatz∗, y − x∗ ≥ 0, for all y ∈ C, where ∂f2 (x∗, x∗) denotes the subdifferential of f (x∗, ) at x∗.
In recent years, methods for solving Problem (P) have been studied extensively One of the most popular methods is the proximal point method This method was introduced first by Martinet [10] for variational inequalities and then was extended by Rockafellar [11] for finding the zero point of a maximal monotone operator Moudafi [6] and Konnov [12] further extended the proximal point method to Problem (P) with monotone and weakly monotone bifunctions respectively
Another solution-approach to Problem (P) is the auxiliary problem principle This principle was introduced first to optimization problems by Cohen [13] and then ex-tended to variational inequalities in [14] Recently, Mastroeni [4] further extended the auxiliary problem principle to Problem (P) involving strongly monotone bifunctions satisfying a certain Lipschitz-type condition Noor [8] used the auxiliary problem
principle to develop iterative algorithms for solving (P) where the bifunctions f were
supposed to be partially relaxed strongly monotone
Other solution methods well developed in mathematical programming and vari-ational inequalities such as the gap function, extragradient and bundle methods re-cently have been extended to Problem (P) [5,9,12,15]
In this paper, first we make use of the Banach contraction mapping principle to prove linear convergence of a regularization algorithm for strongly monotone Ky Fan inequalities that satisfy a Lipschitz-type condition introduced in [4] Then, we apply the algorithm to strongly monotone Lipschitzian variational inequalities As
a consequence, we obtain a new linearly convergent derivative-free algorithm for strongly monotone complementarity problems The obtained linear convergence rate allows the algorithm to be coupled with inexact proximal point methods for solving monotone (not necessarily strong) problem (P) satisfying the Lipschitz-type condi-tion introduced in [4] Finally, we propose a line-search free algorithm for the strong monotone problem (P) which does not require the Lipschitz-type condition as the algorithm presented in Sect.2
The rest of the paper is organized as follows In Sect.2, we describe an algorithm for a strongly monotone problem (P) and prove its linear convergence-rate This algo-rithm is then applied in Sect.3to strongly monotone variational inequalities and com-plementarity problems A new derivative-free linearly convergent algorithm without line search for strongly monotone complementarity problems is described at the end
of this section Section4is devoted to present an algorithm which does not require
Trang 3the above mentioned Lipschitz-type condition In Sect.5, we apply the algorithms obtained in the Sects.3and4to implement inexact proximal point methods for solv-ing monotone (not necessarily strong) Problem (P) We close the paper with some computational experiments and results for a Nash-Cournot equilibrium model
2 Linearly Convergent Algorithm
First of all, we recall the following well-known definitions on monotonicity that we need in the sequel
Definition 2.1 (See e.g [2]) Let f : C × C → R ∪ {+∞} The bifunction f is said to
be monotone on C if f (x, y) + f (y, x) ≤ 0, for all x, y ∈ C It is said to be strongly monotone on C with modulus τ > 0 if f (x, y) + f (y, x) ≤ −τ x − y 2, for all
x, y ∈ C.
Throughout the paper we suppose that the bifunction f satisfies the following
blanket assumption
Assumption A For each x ∈ C, the function f (x, ·) is proper, closed convex and subdifferentiable on C with respect to the second variable.
For each x ∈ C, we define the mapping S by taking
S(x):= argmin
y ∈C {ρf (x, y) + (1/2) y − x 2}, (1)
where ρ > 0 As usual, we refer to ρ as a regularization parameter Since the objective
function is strongly convex, problem (1) admits a unique solution Thus the mapping
Sis well defined and single valued
The following lemma can be found, for example, in [4] (see also [15])
Lemma 2.1 Let S be defined by (1 ) Then, x∗ is a solution to (P) if and only if
x∗= S(x∗).
Lemma2.1suggests an iterative algorithm for solving (P) by taking x k+1= S(x k )
It has been proved in [4] that, with suitable values of the regularization parameter ρ,
the sequence{x k}k≥0 converges strongly to the unique solution of (P) when f is
strongly monotone and satisfies the following Lipschitz-type condition introduced by Mastroeni in [4]
There exists constants L1 > 0 and L2 >0 such that
f (x, y) + f (y, z) ≥ f (x, z) − L1 x − y 2− L2 y − z 2,
Applying this inequality with x = z, we obtain
f (x, y) + f (y, x) ≥ −(L1 + L2 ) x − y 2, ∀x, y ∈ C.
Trang 4188 J Optim Theory Appl (2009) 142: 185–204
Thus, if in addition f is strongly monotone on C with modulus τ , then τ ≤ L1 + L2 For convenience of presentation, we refer to L1 and L2as the Lipschitz constants
for f
The following theorem shows that the sequence{x k}k≥0defined by x k+1= S(x k )
linearly converges to the unique solution of (P) under the same condition as in [4]
Theorem 2.1 Suppose that f is strongly monotone on C with modulus τ and satisfies
the Lipschitz-type condition (2 ) Then, for any starting point x0∈ C, the sequence {x k}k≥0defined by
x k+1:= argmin
y ∈C {ρf (x k , y) + (1/2) y − x k 2} (3)
satisfies
x k+1− x∗ 2≤ α x k − x∗ 2, ∀k ≥ 0, (4)
provided 0 < ρ ≤ 1/(2L2 ) , where x∗ is the unique solution of (P) and α :=
1− 2ρ(τ − L1 )
Proof For each k≥ 0, let
f k (x) := ρf (x k , x) + (1/2) x − x k 2.
Then, by the convexity of f (x k , ·), the function f k is strongly convex on C with
modulus 1, which implies
f k (x k+1) + (w k ) T (x − x k+1) + (1/2) x − x k+1 2≤ f k (x), ∀x ∈ C, (5)
where w k ∈ ∂f k (x k+1) Since x k+1is the solution of problem (3), (w k ) T (x −x k+1)≥
0 for every x ∈ C Thus, from (5), it follows that
f k (x k+1) + (1/2) x − x k+1 2≤ f k (x), ∀x ∈ C. (6) Applying (6) with x = x∗and using the definition of f
k, we obtain
x k+1− x∗ 2≤ 2ρ[f (x k , x∗) − f (x k , x k+1)]
+ x k − x∗ 2− x k+1− x k 2. (7)
Since f is strongly monotone on C with modulus τ ,
f (x k , x∗) ≤ −f (x∗, x k ) − τ x k − x∗ 2.
Substituting this inequality into (7), we have
x k+1− x∗ 2≤ (1 − 2ρτ) x k − x∗ 2
+ 2ρ[−f (x∗, x k ) − f (x k , x k+1) ] − x k+1− x k 2. (8)
Trang 5Now, applying the Lipschitz-type condition (2) with x = x∗, y = x k , and z = x k+1,
we obtain
−f (x k , x k+1) − f (x∗, x k ) ≤ −f (x∗, x k+1) + L1 x∗− x k 2+ L2 x k − x k+1 2
≤ L1 x∗− x k 2+ L2 x k − x k+1 2. (9) The latter inequality in (9) follows from f (x∗, x k+1) ≥ 0, since x∗ is the solution
of (P) Substituting into (8), we obtain
x k+1− x∗ 2≤ [1 − 2ρ(τ − L1 ) ] x k − x∗ 2− (1 − 2ρL2 ) x k+1− x k 2. (10)
By the assumption 0 < ρ ≤ 1/(2L2 ), it follows from (10) that
x k+1− x∗ 2≤1− 2ρ(τ − L1 )
x k − x∗ 2, (11)
The following corollary is immediate from Theorem2.1
Corollary 2.1 Let L1< τ and 0 < ρ ≤ 1/(2L2 ) Then,
x k+1− x∗ ≤ r x k − x∗ , ∀k ≥ 0,
where 0 < r:=√1− 2ρ(τ − L1 ) <1
Remark 2.1 Since τ ≤ L1 + L2 and 0 < ρ ≤ 1/(2L2 ), it is easy to see that
2ρ(τ − L1 ) < 1 Thus, r attains its minimal value at ρ = 1/(2L2 )
Based upon Theorem2.1and Corollary2.1, we can develop a linearly convergent
algorithm for solving problem (P) where f is τ -strongly monotone on C and
satis-fies (2) with positive constants L1 , L2 such that L1 < τ As usual, we call a point
x ∈ C an ε-solution to (P) if x − x∗ ≤ ε, where x∗is an exact solution of (P).
Algorithm A1 (Strongly Monotone Problem)
Initialization Choose a tolerance ε ≥ 0 and 0 < ρ ≤ 1/(2L2 ) Take x0∈ C.
Iteration k, k = 0, 1, Execute Steps 1 and 2 below:
Step 1 Compute x k+1by solving the strongly convex program
(Pk ) x k+1= argmin
y ∈C {ρf (x k , y) + (1/2) y − x k 2}.
Step 2 If x k+1− x k ≤ ε(1 − r)/r, with r :=√1− 2ρ(τ − L1 ), then terminate:
x k+1is an ε-solution to (P) Otherwise, increase k by 1 and go to iteration k.
Note that, by the contraction property
x k+1− x∗ ≤ r x k − x∗ , with r < 1,
Trang 6190 J Optim Theory Appl (2009) 142: 185–204
it is easy to see that
x k+1− x∗ ≤ r/(1 − r) x k+1− x k , ∀k ≥ 0.
Hence,
x k+1− x∗ ≤ r k+1/(1− r) x0− x1 , ∀k ≥ 0.
Thus, if
x k+1− x k ≤ ε(1 − r)/r or r k+1/(1− r) x0− x1 ≤ ε,
then indeed
x k+1− x∗ ≤ ε.
In this case, we can terminate the algorithm to obtain an ε-solution Clearly,
Algo-rithmA1terminates after a finite number of iterations when ε > 0.
Remark 2.2 This algorithm has been presented in [4], but its linear convergence was not proved there
3 Application to Variational Inequality and Complementarity Problems
Let C ⊆ H be a nonempty, closed, convex set as before, ϕ be a proper, closed, convex function on C, and let F : H → H be a multivalued mapping Suppose that
C ⊆ dom F := {x ∈ H : F (x) = ∅}.
Consider the following generalized (or multivalued) variational inequality:
(VIP) Find x∗∈ C, w∗∈ F (x∗) such that (w∗) T (y − x∗) ≥ 0, for all y ∈ C.
It is well known [3] that, when C is a closed convex cone, then (VIP) becomes the
following complementarity problem:
(CP) Find x∗∈ C, w∗∈ F (x∗) such that w∗∈ C∗, (w∗) T x∗= 0,
where
C∗:= {w | w T x ≥ 0, ∀x ∈ C}
is the polar cone of C.
We recall the following well known definitions (see e.g [3])
(i) The multivalued mapping F is said to be monotone on C if
(u − v) T (x − y) ≥ 0, ∀x, y ∈ C, ∀u ∈ F (x), ∀v ∈ F (y).
Trang 7(ii) F is said to be strongly monotone on C with modulus τ (shortly τ -strongly
monotone) if
(u − v) T (x − y) ≥ τ x − y 2, ∀x, y ∈ C, ∀u ∈ F (x), ∀v ∈ F (y) (iii) F is said to be Lipschitz on C with constant L (shortly L-Lipschitz) if
sup
u ∈F (x) v ∈F (y)inf u − v ≤ L x − y , ∀x, y ∈ C.
Define the bifunction f by taking
f (x, y):= sup
u ∈F (x) u
T (y − x) + ϕ(y) − ϕ(x). (12)
The lemma below follows immediately from Proposition 4.2 in [9]
Lemma 3.1 Let f be given by (12 ) The following statements hold:
(i) If F is τ -strongly monotone (resp monotone) on C, then f is τ -strongly
monotone (resp monotone) on C.
(ii) If F is Lipschitz on C with constant L > 0, then f satisfies the Lipschitz-type
condition (2 ); namely, for any δ > 0, we have
f (x, y) + f (y, z) ≥ f (x, z) − (L/(2δ)) x − y 2− ((Lδ)/2) y − z 2. (13)
Suppose that F (x) is closed and bounded and that f is defined by (12) Then, Problem (VIP) is equivalent to Problem (P) in the sense that their solution sets co-incide Lemma 3.1allows us to apply AlgorithmA1 to strongly monotone mixed variational inequalities
Remark 3.1 In order to apply AlgorithmA1 for strongly monotone variational
in-equality problems, it must hold that L1 < τ By Lemma3.1, L1 = L/(2δ) Hence,
L1< τ whenever δ > L/(2τ ).
Now, we apply AlgorithmA1to the complementarity Problem (CP) when C= Rn
+
and F is a single-valued and strongly monotone on C with modulus τ In this case,
Problem (CP) takes the form
Find x∗≥ 0 such that F (x∗) ≥ 0, F (x∗) T x∗= 0. (14) Note that, in this case, the subproblem
(Pk) x k+1= argmin
y ∈C {ρf (x k , y) + (1/2) x − x k 2} defined in AlgorithmA1takes the form
x k+1= argmin
y ∈C {ρF (x k ) T (y − x k ) + (1/2) y − x k 2},
Trang 8192 J Optim Theory Appl (2009) 142: 185–204 which in turns is
x k+1= PRn
+(x k − ρF (x k )),
where PRn
+ is the Euclidean projection of the point x k − ρF (x k )ontoRn
+ It is easy
to verify that, if y = (y1 , , y n ) T is the Euclidean projection of x = (x1 , , x n ) T
ontoRn
+, then for every i = 1, , n one has
y i = x i , if x i ≥ 0,
x i = 0, otherwise.
Suppose that F is single valued, τ -strongly monotone, and L-Lipschitz continuous
onRn
+ Then, AlgorithmA1applied to the complementarity problem (CP) collapses
into the following algorithm
Algorithm A2 (Strongly Monotone Complementarity Problem)
Initialization Fixed a tolerance ε ≥ 0 Choose δ and ρ such that δ > L/(2τ), 0 <
ρ ≤ 1/(Lδ) Take x0≥ 0
Iteration k, k = 0, 1, Execute Steps 1 and 2 below:
Step 1 Compute x k+1= (x k+1
1 , , x k+1
n ) T by taking
x k+1
i := x k
i , if ρF i (x k ) ≤ x k
i ,
x k+1:= 0, otherwise, where the subindex i stands for the ith coordinate of a vector.
Step 2 If x k+1− x k ≤ ε(1 − r)/r, with r :=√1− 2ρ(L/(2δ) − τ), then termi-nate: x k+1 is an ε-solution to (14) Otherwise, increase k by 1 and go to
iteration k.
The validity and linear convergence of AlgorithmA2are immediate from those of AlgorithmA1 AlgorithmA2is quite different from the derivative-free algorithm of Mangasarian and Solodov [16] In fact, our algorithm is based upon the contraction mapping approach and does not use a line search, whereas the algorithm in [16] is based upon a gap function using a line search technique defined by the derivative of
the cost mapping F
4 Avoiding the Lipschitz-Type Condition
In the previous section, we suppose that f satisfies the Lipschitz-type condition (2)
This assumption sometimes is not fulfilled; if it does, the constants L1 and L2are not
always easy to estimate In this section, we consider the case where the bifunction f
does not necessarily satisfy the Lipschitz-type condition (2)
In the following algorithm, we do not require the Lipschitz-type condition (2)
Trang 9Algorithm A3
Initialization Choose two sequences {σ k}k≥0⊂ (0, 1) and {ρ k}k≥0∈ (0, +∞) such
that
∞
k=0
ρ k σ k = ∞, ∞
k=0
σ k2< ∞,
and ρ k σ k ∈ (0, 1/(2τ)) for all k ≥ 0 Take x0∈ C.
Iteration k, k = 0, 1, Execute Steps 1 and 2 below:
Step 1 Find w k ∈ H such that
ρ k f (x k , y) + (w k ) T (y − x k ) ≥ 0, ∀y ∈ C, (15)
where ρ k >0 is a regularization parameter
(a) If w k = 0, then terminate: x kis the solution of (P)
(b) If w k= 0, go to Step 2
Step 2 Set z k+1= x k +σ k w k and x k+1= P C (z k+1) , where P
Cstands for the
Euclid-ean projection on C.
Remark 4.1 Note that the main subproblem in AlgorithmA3is problem (15) This problem can be solved, for example, as follows:
(i) Suppose that the convex program miny ∈C f (x k , y)admits a solution Let
m k:= − min
y ∈C f (x
k , y) < +∞.
Take w k ∈ H such that (w k ) T (y − x k ) ≥ ρ k m k , for all y ∈ C Then, it is easy to seethat w k is a solution to (15)
(ii) Since f (x, ·) is convex and subdifferentiable on C, we have
f (x k , y) − f (x k , x k ) ≥ (g k ) T (y − x k ), ∀y ∈ C, g k ∈ ∂2 f (x k , x k ).
Since f (x k , x k ) = 0, it follows that w k = −ρ k−1g ksatisfies the inequality
ρ k f (x k , y) + (w k )(y − x k ) ≥ 0, ∀y ∈ C.
Hence, w ksolves the subproblem (15)
Now, we are in position to prove convergence of AlgorithmA3
Theorem 4.1 Suppose that f is strongly monotone with modulus τ on C Let {x k}k≥0
be the sequence generated by Algorithm A3 Then, one has
x k+1− x∗ 2≤ (1 − 2τρ k σ k ) x k − x∗ 2+ σ2
k w k 2, ∀k ≥ 0, (16)
where x∗is the unique solution of (P) Moreover, if the sequence {w k}k≥0is bounded, then {x k } converges to the solution x∗of (P).
Trang 10194 J Optim Theory Appl (2009) 142: 185–204
Proof Let x∗be the unique solution of (P) Since x k+1= P C (z k+1), we have
x k+1− x∗ 2≤ z k+1− x∗ 2− z k+1− x k+1 2. (17) Substituting
z k+1= x k + σ k x k
in (17), we obtain
z k+1− x∗ 2= x k + σ k w k − x∗ 2
= x k − x∗ 2+ 2σ k (w k ) T (x k − x∗) + σ2
k w k 2. (18) Applying (15) with y = x∗, we obtain
ρ k f (x k , x∗) ≥ (w k ) T (x k − x∗). (19)
Since f is strongly monotone on C with modulus τ and since x∗is a solution to (P),
we have
ρ k f (x k , x∗) ≤ −ρ k τ x k − x∗ 2− ρ k f (x∗, x k ) ≤ −τρ k x k − x∗ 2. (20) From (18)–(20) it follows that
z k+1− x∗ 2≤ (1 − 2τρ k σ k ) x k − x∗ 2+ σ2
k w k 2. (21) Substituting (21) into (17), we obtain
x k+1− x∗ 2≤ (1 − 2τρ k σ k ) x k − x∗ 2+ σ2
k w k 2− z k+1− x k+1 2
≤ (1 − 2τρ k σ k ) x k − x∗ 2+ σ2
k w k 2,
which proves inequality (16)
To prove limk→∞x k = x∗, using the assumption of boundedness of the sequence
{w k}, from (16) we have
x k+1− x∗ 2≤ (1 − 2τρ k σ k ) x k − x∗ 2+ σ2
where M > 0 is a constant Let λ k = 2τρ k σ k; by the assumption on the sequences
{ρ k } and {σ k }, we have that λ k ∈ (0, 1), for all k ≥ 0, and ∞k=0λ k = ∞ On the other hand, since∞
k=1σ k2<+∞, it is easy to see from (22) that x k+1− x∗ → 0,
Note that, since AlgorithmsA3is not linearly convergent, we cannot use x k+1−
x k to check whether or not the iterate x k+1is an ε-solution as in Algorithm A1.
Instead, we may use the value of a gap function at the iterate to check its ε-solution.
The following two gap functions have been defined for Problem (P) (see e.g [5]):
g(x):= sup