On generalized derivatives, optimality conditions and uniqueness of solutions in nonsmooth optimization

The mth-order weak Clarke epiderivative, Lalitha and Arora 2008 for m = 1, of F at x0, y0 wrt u1, v1, ..., um−1, vm−1 has the value As being illustrated above, the variational sets are b

VIETNAM NATIONAL UNIVERSITY - HCMC

VIETNAM NATIONAL UNIVERSITY - HCMC

Referee 1: Assoc Prof Dr Nguyen Dinh Huy

Referee 2: Assoc Prof Dr Pham Hoang Quan

Referee 3: Dr Duong Dang Xuan Thanh

Independent Referee 1: Prof D.Sc Vu Ngoc Phat

Independent Referee 2: Assoc Prof Dr Do Van Luu

SCIENTIFIC SUPERVISOR:

Prof D.Sc Phan Quoc Khanh

Hochiminh City - 2012

I confirm that all the results of this thesis come from my work under the supervision ofProfessor Phan Quoc Khanh and helps of many my Professors and collaborations, espe-cially Professor Dinh The Luc They have never been published by other authors

Hochiminh City, 2012The author

Le Thanh Tung

i

et G´eom´etrie, Universit´e d’Avignon et de pays Vaucluse, especially Professor M.Volle, fortheir hospitality during my stay in Avignon I would like to address my thanks to mycolleagues from the seminar of the Section of Optimization and System Theory, headed

by Professor Phan Quoc Khanh, especially Nguyen Le Hoang Anh, who has collaboratedwith me in working on our two joint papers The last but not least thanks are devoted to

my teachers and colleagues from Department of Mathematics, College of Science, CanthoUniversity, my family and my friends who always encourage me during my research

ii

Nonsmooth analysis has been intensively developed for more than half century One of its

major purposes is applying in nonsmooth optimization Various generalized derivatives

have been introduced to replace the classical Fr´echet and Gˆateaux derivatives to meet

the continually increasing diversity of practical problems For comprehensive books, the

reader is referred to Clarke (1983), Aubin and Frankowska (1990), Rockafellar and Wets

(1998) and Mordukhovich (2006) We can observe a domination in use of the Clarke

derivative (1973), Aubin contingent derivative (1981) and Mordukhovich coderivative and

limiting subdifferential (1976) However, in particular problems, sometimes many other

generalized derivatives have advantages For instance, variational sets, proposed by Khanh

and Tuan (2008), are defined as follows Let X and Y be real normed spaces, F : X →

2Y, (x0, y0) ∈ grF and v1, , vm−1 ∈ Y The variational sets of type 1 and type 2 are

These subsets of the image space are larger than the images of the pre-image space

through many known generalized derivatives, which are shown as follows Let S ⊆ X and

x, u1, u2, , um−1 ∈ X, m ≥ 1 The mth-order contingent set of S at (x, u1, u2, , um−1) is

Based on these sets, some generalized derivatives were proposed as follows.

Let F : X → 2Y, (x0, y0) ∈ grF and (u1, v1), , (um−1, vm−1) ∈ X × Y The mth-order

contingent derivative, Aubin and Frankowska (1981), of F at (x0, y0) with respect to (wrt)

(u1, v1), , (um−1, vm−1) has the value at x ∈ X :

DmF (x0, y0, u1, v1, , um−1, vm−1)(x) = {y ∈ Y : (x, y) ∈ TgrFm (x0, y0, u1, v1, , um−1, vm−1)}

The mth-order adjacent derivative, Aubin and Frankowska (1981), of F at (x0, y0) wrt

(u1, v1), , (um−1, vm−1) has the following value at x ∈ X :

DbmF (x0, y0, u1, v1, , um−1, vm−1)(x) = {y ∈ Y : (x, y) ∈ TgrFbm(x0, y0, u1, v1, , um−1, vm−1)}

The mth-order Clarke derivative, Khanh and Tuan (2008), of F at (x0, y0) wrt (u1, v1), ,

(um−1, vm−1) has the value at x ∈ X :

Foreword v

The mth-order contingent epiderivative, Jahn and Rauh (1997) for m = 1 and Jahn et

al (2005) for m = 2, of F at (x0, y0) wrt (u1, v1), , (um−1, vm−1) is the single-valued

where, C ⊆ Y be a ordering cone and F+(x) = F (x) + C

The mth-order generalized contingent epiderivative, Li and Chen (2006), Chen and

Jahn (1998) for m = 1, Jahn et al (2005) for m = 2, of F at (x0, y0) wrt (u1, v1), , (um−1, vm−1)

kas the value

DgmF (x0, y0, u1, v1, , um−1, vm−1)(x)

= MinC{y ∈ Y : y ∈ DmF+(x0, y0, u1, v1, , um−1, vm−1)(x)}

Here, MinC{.} denotes the set of efficient points of the set {.} wrt C

The mth-order generalized Clarke epiderivative, Lalitha and Arora (2008) for m = 1,

of F at (x0, y0) wrt (u1, v1), , (um−1, vm−1) of x ∈ X ishas the value

DgcmF (x0, y0, u1, v1, , um−1, vm−1)(x)

= MinC{y ∈ Y : y ∈ DcmF+(x0, y0, u1, v1, , um−1, vm−1)(x)}

The mth-order weak contingent epiderivative, Chen et al (2009), of F at (x0, y0) wrt

(u1, v1), , (um−1, vm−1) has the value

DwmF (x0, y0, u1, v1, , um−1, vm−1)(x)

= WMinC{y ∈ Y : y ∈ DmF+(x0, y0, u1, v1, , um−1, vm−1)(x)}

Foreword vi

Here, WMinC{.} denotes the set of weak efficient points of the set {.} wrt C

The mth-order weak Clarke epiderivative, Lalitha and Arora (2008) for m = 1, of F at

(x0, y0) wrt (u1, v1), , (um−1, vm−1) has the value

As being illustrated above, the variational sets are bigger than the corresponding sets

defined by the mentioned derivatives and hence the resulting necessary conditions obtained

by separations are stronger than many known ones Of course, sufficient optimality

con-ditions based on separations of bigger sets may be weaker But using variational sets

we can establish sufficient conditions which have almost no gap with the corresponding

necessary ones The second advantage of the variational sets is that we can define these

sets of any order to get higher-order optimality conditions This feature is significant since

Foreword vii

many important and powerful generalized derivatives can be defined only for the first and

second orders and the higher-order optimality conditions available in the literature are

much fewer than the first and second-order ones The third strong point of the variational

sets is that almost no assumptions are needed to be imposed for their being well-defined

and nonempty and also for establishing optimality conditions

Another clear example is approximations, introduced by Jourani and Thibault (1993),

defined as follows A set Af(x0) ⊆ L(X, Y ) is said to be a first-order approximation of

f : X → Y at x0 ∈ X if there exists a neighborhood U of x0 such that, for all x ∈ U ,

f (x) − f (x0) ∈ Af(x0)(x − x0) + o(kx − x0k)

This kind of generalized derivatives contains a major part of known notions of derivatives,

as illustrated now If f is Fr´echet differentiable at x0, then {f0(x0)} is a first-order

approximation of f at x0 Let X = Rn, Y = Rm and f be a mapping of class C0,1; i.e.,

f is locally Lipschitz The Clarke generalized Jacobian, Clarke (1983), of f at x0 ∈ Rn,denoted by ∂Cf (x0), is defined by

∂Cf (x0) = {lim f0(xi) : xi → x0, f0(xi) exists}

Then, {∂Cf (x0)} is a first-order approximation of f at x0

Let f : Rn → Rm be continuous A closed subset ∂f (x0) ⊆ L(Rn, Rm) is called aapproximate Jacobian, Jeyakumar and Luc (1998), of f at x0 ∈ Rn

if, for each v ∈ Rmand u ∈ Rn,

Foreword viii

An approximate Jacobian ∂f (x0) is termed a Fr´echet approximate Jacobian, Luc (2001),

of f at x0 if there is a neighbourhood U of x0 such that, for each x ∈ U,

f (x) − f (x0) ∈ ∂f (x0)(x − x0) + o(kx − x0k)

It is obvious that any Fr´echet approximate Jacobian is a first-order approximation

If an approximate Jacobian ∂f (x0) is upper semicontinuous at x0, then clco∂f (x0) is a

first-order approximation

Furthermore, it is advantageous that even an infinitely discontinuous maps may have

approximations, as shown by the following example Let X = Y = R, x0 = 0,

In this thesis, three subjects of generalized derivatives and their applications in

non-smooth optimization are discussed The first one is calculus rules and their applications

for three generalized derivatives : higher-order variational sets in Chapter 1, higher-order

radial derivatives in Chapter 2 and approximations in Chapter 4

In Chapter 1, we establish elements of calculus for variational sets to ensure that they

can be used in practice Firstly, we establish the union rules and intersection rules for

two types of variational sets To get sum rules and Descartes product rules, we propose

a definition of proto-variational sets as follows Let F : X → 2Y, (x0, y0) ∈ grF and

v1, , vm−1 ∈ Y If the upper limit defining Vm(F, x0, y0, v1, , vm−1) is a full limit, i.e.,

the upper limit coincides with the lower limit, then this set is called a proto-variational

set of order m of type 1 of F at (x0, y0) If the similar coincidence occurs for Wm we

say that this set is a proto-variational set of order m of type 2 of F at (x0, y0) Then,

applying these definitions, sum rules and Descartes product rules are obtained Next, some

Foreword ix

composition rules including chain rules, composition with differentiable maps, composition

with linear continuous maps are established We pay attentions also on relations between

the established calculus rules and applications of some rules to get others For example,

our chain rules encompass sum rules as special cases as follows Investigate the sum

M + N of two multifunctions M, N : X → 2Y To express M + N as a composition,

define F : X → 2X×Y and G : X × Y → 2Y by, for I being the identity map on X and

We define other kinds of variational sets of G ◦ F with a significant role of intermediate

variable y as follows Let Z be a normed space, ((x, z), y) ∈ grC, u1, , um−1 ∈ Y and

w, w1, , wm−1 ∈ Z The mth-order y-variational set of the multimap G ◦ F at (x, z) is

Foreword x

Using the above definitions, some chain rules for G ◦ F using these variational sets are

established For (x, z) ∈ X × Y , setting

S(x, z) = M (x) ∩ (z − N (x))

Then, the resultant multimap C : X × Y → 2X×Y associated to these F and G is

C(x, z) = {x} × S(x, z)

To get the sum rules from chain rules, we define a kind of variational sets as follows

Given ((x, z), y) ∈ cl(grS) and v1, , vm−1 ∈ Y , the mth-order y-variational set of M + N

Then, Vm(M +yN, x, z, v1, , vm−1) = Vm(G ◦yF, x, z, v1, , vm−1) Hence, some sum

rules can be derived from the above chain rules Next, the calculus rules are also

es-tablished for other operators as the inner product, outer product, quotient, reciprocal,

maximum and minimum Many examples are given to illustrate properties of the above

calculus rules It turns out that the variational sets possess many fundamental and

com-prehensive calculus rules Although this construction is not comparable with objects in

the dual approach like Mordukhovich’s coderivatives in enjoying rich calculus, it may be

better in dealing with higher-order properties

Of course, significant applications should be those in other topics of nonlinear analysis

and optimization As such applications we provide a direct employment of calculus rules

of variational sets to investigate an explicit formula for a variational set of the solution

map to a parametrized variational inequality, defined as follows

Foreword xi

Let F : W × X → 2Z and N : X → 2Z be multimaps between normed spaces and K

be a subset of X Let

M (w, z) = {x ∈ K : z ∈ F (w, x) + N (x)},

where F : W × X → 2Z and N : X → 2Z are multimap between normed spaces and

K is a subset of X When K is convex, N (x) is the normal cone to K at x, and w is

a parameter, M is the solution map of a parametrized variational inequality Using our

sum rules of variational sets, an explicit formula for a variational set of the solution map

to a parametrized variational inequality is established Furthermore, chain rules and sum

and product rules are also used to prove optimality conditions for weak solutions of some

vector optimization problems Let Y be partially ordered by a closed pointed convex cone

C with nonempty interior, F : X → 2Y and G : X → 2X Consider

(P1) minF (x0) subject to x ∈ X and x0 ∈ G(x)

This problem can be restated as the following unconstrained problem: min(F ◦ G)(x)

By using our chain rules of variational sets, some necessary conditions for weak solutions

of vector optimization can be obtained while calculus rules of contingent epiderivatives in

Jahn and Khan (2002) cannot be applied

To illustrate sum rules, we consider the following problem

Foreword xii

(PC) min(F + sG)(x)

In a particular case, when Y = R and F is single-valued, (PC) is used to approximate(P2) in penalty methods We apply our sum rules and scalar product rules of variational

sets to get necessary conditions for a local weakly efficient pair of (PC) while calculus

rules of contingent epiderivatives in Jahn and Khan (2002) cannot be in use

In Chapter 2, a new definition of set-valued derivatives, the mth-order radial derivative,

is proposed as follows Let F : X → 2Y be a set-valued map and u ∈ X The mth-order

outer radial derivative of F at (x0, y0) ∈ grF is

DmRF (x0, y0)(u) = {v ∈ Y : ∃tn > 0 , ∃(un, vn) → (u, v), ∀n, y0 + tm

nvn ∈ F (x0 +

tnun)}

When m = 1, the above definition collapses to the normal radial derivative, proposed

by Taa (1997) The above definition is different from that of many known notions, for

in-stance, the well-known mth-order contingent derivative DmF (x0, y0, u1, v1, , um−1, vm−1)

and the higher-order radial derivatives, introduced by Anh and Khanh (2011) as follows,

DRmF (x0, y0, u1, v1, , um−1, vm−1)(u) = {v ∈ Y : ∃tn> 0, ∃(un, vn) → (u, v), ∀n,

y0 + tnv1 + + tm−1n vm−1 + tmnvn ∈ F (x0+ tnu1+ + tm−1n um−1+ tmnun)}

In this definition, the mth-order derivative continues to improve the approximating points

based on the given m − 1 lower-order directions (u1, v1), , (um−1, vm−1) with a mth-order

rate to get closer to the graph In our definition of the mth-order outer radial derivative,

the direction is not based on the given information of lower-order approximating

direc-tions, but also gives an approximation of mth-order rate Furthermore, the graph of our

derivative is not a corresponding tangent set of the graph of the map, because the rates

of change of the points under consideration in X and Y are different (tn and tmn)

We obtain main calculus rules for the mth-order radial derivative Firstly, we propose

the definition of proto-radial derivative to obtain sum rule and chain rule, defined as

Foreword xiii

follows

Let F : X → 2Y , (x0, y0) ∈ grF If DmRF (x0, y0)(u) = DmRF (x0, y0)(u) for any

u ∈ dom[DmRF (x0, y0)], then we call DmRF (x0, y0) a mth-order proto-radial derivative of F

at (x0, y0) Next, we present another kind of radial derivative of G ◦ F with a significant

role of intermediate variable y, to get some chain rules, as follows Let (x, z) ∈ gr(G ◦ F )

and y ∈ clR(x, z) The mth-order y-radial derivative of the multimap G ◦ F at (x, z) is

the multimap DmR(G ◦yF )(x, z) : X → 2Z given by

DmR(G ◦y F )(x, z)(u) = {w ∈ Z : ∃tn> 0, ∃(un, yn, wn) → (u, y, w),

Then, the sum rules are obtained from chain rules by using a kind of mth-order radial

derivative as follows Given (x, z) ∈ domS and y ∈ clS(x, z), the mth-order y-radial

derivative of M +y N at (x, z) is the multimap DmR(M +y N )(x, z) : X → 2Y given by

DmR(M +yN )(x, z)(u) = {w ∈ Y : ∃tn> 0, ∃(un, yn, wn) → (u, y, w),

∀n, yn∈ S(x + tnun, z + tmnwn)}

The calculus rules of mth-order radial derivative are applied to get the necessary

con-ditions for weak solutions of (P1) and (PC) while calculus rules of contingent epiderivatives

in Jahn and Khan (2002) cannot be applied

In Chapter 4, first product and quotient rules are established for first order

approxima-tions Next, many calculus rules including a sum rule, Descartes product rule, inner

prod-uct rule, prodprod-uct rule and quotient rule are established for second order approximations,

Foreword xiv

defined as follows A set (Af(x0), Bf(x0)) with Af(x0) ⊆ L(X, Y ), Bf(x0)L(X, X, Y ) is

called a second-order approximation, Allali and Amaroq (1997), of f : X → Y at x0 ∈ X

if

Af(x0) is a first-order approximation of f at x0, and,

f (x) − f (x0) ∈ Af(x0)(x − x0) + Bf(x0)(x − x0, x − x0) + o(kx − x0k2)

Then, Descartes product rule and quotient rule are applied to get optimality conditions for

multiobjective fractional programming The assumed m−calmness, used in establishing

some rules, defined as follows For m ∈ N, f : X → Y is said to be m−calm at x0 iffthere exists L > 0 and neighbourhood U of x0 such that, for all x ∈ U ,

kf (x) − f (x0)k ≤ Lkx − x0km

(1-calmness is called simply as calmness.) Of course, if f is m−calm at x0, then f is

continuous at x0, for any m ≥ 1

The second subject, discussed in the thesis, is optimality conditions for nonsmooth

optimization problems Optimality conditions for some types of solutions to special

vec-tor constrained minimization problems using calculus rules of higher-order variational sets

and higher-order radial derivatives have been obtained in Chapters 1 and 2 We also

inves-tigate in Chapter 2 the optimality conditions for a general set-valued vector optimization

problem with inequality constraints, defined as follows

(P) min F (x), subject to x ∈ S, G(x) ∩ (−D) 6= ∅,

where D ⊂ Y is a pointed closed convex cone with nonempty interior, and S ⊆ X, F :

X → 2Yand G : X → 2Z Using mth-order radial derivative, we establish necessary

conditions of Q-minimal solutions, Ha (2009), defined as follows Let Q ⊆ Y be an

arbitrary nonempty open cone, different from Y Let A := {x ∈ S : G(x) ∩ (−D) 6= ∅},

Foreword xv

x0 ∈ A and (x0, y0) ∈ grF We say that (x0, y0) is a Q-minimal point of (P) if

(F (A) − y0) ∩ (−Q) = ∅

Since various kinds of efficient solutions of (P) are in fact a Q- minimal solution with

Q being an appropriately chosen cone, we derive necessary conditions for other kinds of

efficient solutions of (P), for example, the weak efficient solution, Henig-properly efficient

solution, Borwein-properly efficient solution Then, mth-order radial derivative be used

to get sufficient conditions of (P) Here, none of convexity assumptions are required in

sufficient conditions An example is provided to illustrate that the second-order radial

derivative can be applied in checking necessary conditions for local weak solutions of

(P) while first-order radial derivatives cannot Since the convexity assumptions can be

avoid, some advantages of sufficient conditions by using higher-order radial derivatives

instead of first order radial derivatives, variational sets or contingent epiderivatives are

also illustrated by numerous examples

Chapter 3 is devoted to using first and second-order approximations as generalized

derivatives to establish necessary and sufficient optimality conditions for many kinds of

solutions to nonsmooth vector equilibrium problems with functional constraints Firstly,

using first approximations as generalized derivatives, we establish necessary conditions for

local weakly efficient solutions of nonsmooth vector equilibrium problems with functional

constraints, defined as follows If intC 6= ∅, a vector x0 ∈ Ω is said to be a local weak

solution of problem (EP), if there exists a neighbourhood U of x0 such that

F (x0, U ∩ Ω) 6⊆ −intC

y∈Ω

F (x, y) and Fx0 : X → Y defined

by Fx 0(y) = F (x0, y) for y ∈ X and Fx 0(x0) = 0 By using the weak feasible cone, we

remove the assumed boundedness of Ag(x0) and adding a case which ensure c∗ 6= 0 in the

Foreword xvi

necessary condition This removal is important, since a map with a bounded first-order

approximation at a point must be continuous (even calm) at this point Then, we obtain

sufficient conditions for the local Henig-proper solutions, local Benson-proper solutions,

defined as follows

Assume that C have a base B Let δ = inf{kbk| b ∈ B} For any 0 <  < δ, denote

C(B) = cone(B + BY) Vector x0 ∈ Ω is called a local strong Henig-proper solution to

(EP) if there exists a neighborhood U of x0 and some  ∈ (0, δ) such that

F (x0, U ∩ Ω) ∩ (−intC(B)) = ∅

A vector x0 ∈ Ω is determined as a local Benson-proper solution to (EP) if there exists

a neighborhood U of x0 such that

clcone(F (x0, U ∩ Ω) + C) ∩ (−C) = {0}

The generalized convexity like arcwise-connectedness and pseudoconvexity used in

sufficient conditions, are defined as follows A map f : X → Y is said to be pseudoconvex,

Aubin and Frankowska (1990), at x0 if

epif ⊆ (x0, f (x0)) + Tepif(x0, f (x0)),

where epif = {(x, y) ∈ X × Y : y ∈ f (x) + C} is the epigraph of f A subset S of X is

called arcwise-connected, Avriel (1976), at x0 if, for each x ∈ S, there exists a continuous

arc Lx0,x(t) defined on [0,1] such that Lx0,x(0) = x0, Lx0,x(1) = x and Lx0,x(λ) ∈ S for all

λ ∈ (0, 1) f is said to be C-arcwise-connected at x0 on S, where S is arcwise connected

at x0, if for all x ∈ S and all λ ∈ [0, 1],

(1 − λ)f (x0) + λf (x) ∈ f (Lx 0 ,x(λ)) + C

X is assumed to be a finite dimensional space but convex assumptions are not required

to get our sufficient conditions for local firm solutions order m of (EP), defined as follows

Foreword xvii

For m ≥ 1, a vector x0 ∈ Ω is said to be a local firm solution of order m of (EP) if

there exists a neighborhood U of x0 and γ > 0 such that, for all x ∈ U ∩ Ω \ {x0},

(F (x0, x) + C) ∩ BY(0, γkx − x0km) = ∅

Finally, employing second order approximations, we get second-order necessary and

sufficient conditions for local weak solutions and local firm solutions of order 2 to (EP)

If Fx 0 and g are first-order Fr´echet differentiable at x0, then second-order

approxi-mations (Fx0

0(x0), BFx0(x0)) and (g0(x0), Bg(x0)) of Fx0 and g are used to obtain secondorder optimality conditions If Fx0 and g are not first-order Fr´echet differentiable at x0,

then second-order approximations (AFx0(x0), BFx0(x0)) and (Ag(x0), Bg(x0)) of Fx 0 and g

are used to obtain second order optimality conditions In optimality conditions, we have

to impose the assumption that first and second order approximations are asymptotically

pointwise compact, defined as follows Let L(X, Y ) stand for space of the continuous

linear mappings from X into Y and B(X, X, Y ) denotes the space of continuous bilinear

mappings of X × X into Y A subset A ⊆ L(X, Y ) (B ⊆ L(X, X, Y )) is called

asymp-totically pointwise compact (shortly asympasymp-totically p-compact), Khanh and Tuan (2006),

if each bounded net {fα} ⊆ A (⊆ B, respectively) has a subnet {fβ} and f ∈ L(X, Y )

(f ∈ L(X, X, Y )) such that f = p− lim fβ; and, for each net {fα} ⊆ A (⊆ B, respectively)

with lim kfαk = ∞, the net {fα/kfαk} has a subnet which pointwise converges to some

f ∈ L(X, Y ) \ {0} (f ∈ L(X, X, Y ) \ {0})

Note that second-order optimality conditions for solutions of vector equilibrium

prob-lem have not been investigated before Many examples are given to illustrate that using

approximations in optimality conditions can be applied while using other derivatives such

as Gˆateaux derivative, Clarke approximate Jacobian or Fr´echet approximate Jacobian

cannot

Foreword xviii

In Chapter 4, we consider nonsmooth multiobjective fractional programming on normed

spaces, defined as follows,

the first-order approximation of ϕ Next, we establish sum rules and Descartes product

rules for asymptotical p-compact sets in L(X, Y ) and use them to find a sufficient

con-dition for the asymptotical p-compactness of ϕ Then, we establish first-order necessary

conditions for local weak solutions and first-order sufficient conditions for local firm

so-lutions of order 1 of (FP) In comparison, first-order necessary conditions are sharpened

by removing the assumed boundedness of first order approximations This removal is

important, since a map with a bounded first-order approximation at a point must be

continuous at this point For sufficient conditions, no convexity is needed Our results

can be applied even in infinite dimensional cases involving infinitely discontinuous maps

In similar ways, we construct second-order asymptotical p-compact approximations of ϕ

Then, we use them to establish second-order necessary conditions for local weak solutions

and second-order sufficient conditions for local firm solutions of order 2 to (FP) in two

cases :

Case 1 : Fx0 and g are first-order Fr´echet differentiable at x0,

Case 2 : Fx0 and g are not first-order Fr´echet differentiable at x0

In our second-order optimality conditions, 2-calmness assumptions on gi are used and

these assumptions are restrictive But we give an example showing that the 2-calmnesss

assumption cannot be replaced by the calmness assumption in establishing the quotient

rule of second-order approximations Furthermore, in applications, we can always rewrite

Foreword xix

the fractions involved in the problem, with new fi and gi, so that this new gi satisfies

these assumptions

The third topic of our thesis is uniqueness of solutions to nonsmooth equilibrium

problems and the obtained sufficient conditions are in terms of approximations, used as

derivatives Observing that, for equilibrium problems, there have not been contributions

to the uniqueness of solutions for vector problems, we consider this topic for a strong and

a weak vector equilibrium problems, defined in Chapter 5 as follows Let H ⊆ Rn benonempty, K ⊆ Rn nonempty, closed and convex, and C ⊆ Rl be a closed, convex andpointed cone with nonempty interior Let f : Rn→ Rm, g : Rn→ Rn, and ϕ : Rm× Rn →

Rl with the components (ϕ1(y, x), ϕ2(y, x), , ϕl(y, x)) Setting Ω := {x ∈ H : g(x) ∈

K}, the vector strong equilibrium problem (SEP) (weak equilibrium problem (WEP))

under our consideration is:

Find x0 ∈ Ω s.t., ∀x ∈ Ω,

ϕ(f (x0), g(x)) − ϕ(f (x0), g(x0)) ∈ C

(ϕ(f (x0), g(x)) − ϕ(f (x0), g(x0)) 6∈ −intC, respectively)

We say that a solution x0 of a problem is locally unique if there is a neighborhood

of x0 such that no other solutions are possible inside this neighborhood Our major

tool is the approximation notion, as a kind of generalized derivatives Since this notion

has an important advantage that even a map with infinite discontinuity at a point can

admit an approximation at this point, when applied to particular cases, our sufficient

conditions for the local uniqueness of solutions in terms of approximations improve recent

existing results Note also that, though this kind of generalized derivatives has been

employed in studies of some topics like metric regularity, optimality conditions, etc, it

is used for the first time in investigating the uniqueness of solutions in Chapter 5 As

Foreword xx

equilibrium problems encompass many optimization-related problems, we can derive for

them consequences from the results obtained In this chapter, we concretize our problems

to only two important particular cases : classical equilibrium problem and generalized

variational inequality, defined as follows Consider (SVP) and (WEP) If l = 1, C =

R+, n = m, H = K, f = g ≡ I, and if ϕ(x, x) = 0 for all x ∈ K, then the above problems

collapse to the classical equilibrium problem:

Find x0 ∈ K s.t., ∀x ∈ K, ϕ(x0, x) ≥ 0

If l = 1, C = R+ and ϕ(y, x) = hy, xi, two problems (SVP) and (WEP) come down tothe (scalar) generalized variational inequality of

(GVI): finding x0 ∈ Ω s.t., ∀x ∈ Ω, hf (x0), g(x) − g(x0)i ≥ 0

Using the result for the local uniqueness of solutions to (SVP) and (WEP), the

suffi-cient conditions for the local uniqueness of solutions of classical equilibrium problem and

(GVI) are derived

Many comparisons between our results and recent known ones, including for particular

cases, are provided in each chapter In Chapter 1, two types of variational sets, which

are bigger than the corresponding sets defined by the mentioned derivatives, are used to

get necessary conditions for local weak solution of (P1) and (PC) The resulting necessary

conditions obtained by separations are stronger than many known ones In Chapter 2,

mth-order radial derivatives are used to get necessary and sufficient conditions for

Q-minimal points of (P) These necessary conditions using mth-order radial derivatives can

be applied while radial derivatives or variational sets cannot in some case Moreover,

no convexity assumptions are required in these sufficient conditions of (P) In Chapters

3,4 and 5, the approximation notion, as a kind of generalized derivatives is used to get

optimality conditions for (VP) and (FP) and sufficient conditions for local uniqueness

of solutions for (SEP) and (WEP) These results have important advantages that even a

Foreword xxi

map with infinite discontinuity at a point can be applied, and, in some cases, these results

can be used in finite dimensional spaces Numerous examples are also given to illustrate

the above main results in each chapter

Observe that our thesis consists of five papers The first two papers have been

pub-lished in Nonlinear Analysis (2011) The third paper has been submitted to Journal of

Global Optimization The last two papers have been submitted to Journal of

Optimiza-tion Theory and ApplicaOptimiza-tions For paper 5, we have received positive referees’s reports

and submitted a revision A part of Chapter 1 is presented at 7th Workshop on

Optimiza-tion and Scientific Computing, Ba Vi, Ha Noi, April 22-24, 2009 Chapter 5 is presented

at CIMPA-UNESCO-VIETNAM SCHOOL Variational Inequalities and Related

Prob-lems, Ha Noi, May 10-21, 2010 Chapter 3 is talked at 8th Vietnam-Korea Workshop

Mathematical Optimization Theory and Applications, Da Lat, December 8-11, 2011 All

the results have been reported at the seminar on Optimization of Professor Phan Quoc

Khanh during the last three years

References

[Allali and Amaroq 1997] K Allali, T Amahroq, Second-Order Approximations and

Primal and Dual Necessary Optimality Conditions, Optimization 40 (1997), pp 229-246

[Anh and Khanh 2011] N.L.H Anh, P.Q Khanh, Optimality conditions in set-valued

optimization using radial sets and derivatives, submitted for publication

[Aubin 1981] J.-P Aubin, Contingent derivatives of set-valued maps and existence of

solutions to nonlinear inclusions and differential inclusions, Advances in Mathematics,

Supplementary studies, Ed Nachbin L., 160-232

[Aubin and Frankowska 1990] J.-P Aubin, H Frankowska, Set-Valued Analysis,

Birkh¨auser, Boston, 1990

[Clarke 1973] F.H Clarke, Necessary Condition for Nonsmooth Problem in Optimal

Control and Calculus of Variations, Ph.D Thesis, Univ of Washington

[Chen and Jahn 1998] G.Y Chen, J Jahn, Optimality conditions for set-valued

opti-mization problems, Math Methods Oper Res 48 (1998) 187-200

[Chen et al 2009] C.R Chen, S.J Li, K.L Teo, Higher order weak epiderivatives and

applications to duality and optimality conditions, Comput Math Appl 57 (2009)

1389-1399

[Ha 2009] T.X.D Ha, Optimality conditions for several types of efficient solutions of

set-valued optimization problems, Chap 21 in Nonlinear Analysis and Variational Problems

in: P Pardalos, Th M Rassis and A A Khan (Eds), 2009, pp 305-324

[Jahn and Khan 2002] J Jahn, A.A Khan, Some calculus rules for contingent

epi-derivatives, Optimization 52 (2002) 113-125

[Jahn and Rauh 1997] J Jahn, R Rauh, Contingent Epiderivatives and Set-Valued

Optimization, Math Meth Oper Res 46(1997), pp 193-211

[Jahn et al 2005] J Jahn, A.A Khan, P Zeilinger, Second-Order Optimality Conditions

in Set Optimization, J Optim Theory Appl 125 (2005) 331-347

[Jeyakumar and Luc 1998] V Jeyakumar, D T Luc, Approximate Jacobian matrices

for nonsmooth continuous maps and C1-Optimization, SIAM J Control Optim 36(1998),

pp 1815-1832

[Jourani and Thibault 1993] A Jourani, L Thibault, Approximations and metric

Foreword xxiii

regularity in mathematical programming in Banach spaces, Math Oper Res 18 (1993),

pp 390 - 400

[Khanh and Tuan 2008] P.Q Khanh, N.D Tuan, Variational sets of multivalued

map-pings and a unified study of optimality conditions, J Optim Theory Appl 139 (2008),

45-67

[Lalitha and Arora 2008] C.S Lalitha, R Arora, Weak Clarke epiderivative in

set-valued optimization, J Math Anal Appl 342 (2008) 704-714

[Li and Chen 2006] S.J Li, C.R Chen, Higher order optimality conditions for Henig

efficient solutions in set-valued optimization, J Math Anal Appl 323 (2006) 1184-1200

[Luc 2001] D T Luc, Fr´echet approximate Jacobian and local uniqueness of solutions

in variational inequalities, J Math Anal Appl 268(2002), pp 629-646

[Mordukhovich 1976] B S Mordukhovich , Maximum principle in problems of time

optimal control with nonsmooth constraints, J Appl Math Mech 40(1976), 960-969

[Mordukhovich 2006] B.S Mordukhovich, Variational Analysis and Generalized

Dif-ferentiation, Vol I-Basic Theory, Vol II-Applications, Springer, Berlin, 2006

[Rockafellar and Wet 1998] R.T Rockafellar, R J.-B Wets, Variational Analysis,

Springer, Berlin, 1998

[Taa 1997] A Taa, Set-valued derivatives of multifunctions and optimality conditions,

Num Funct Anal Optim 19 (1997), pp.121 -140

Chapter 2 Higher-order radial derivatives and optimality conditions in

Chapter 3 First and second-order optimality conditions using approximations

xxiv

3 First-order optimality conditions 46

Chapter 4 First and second order optimality condition for multiobjective

Chapter 1 Variational sets: calculus and applications to nonsmooth vector optimization

1

Contents lists available at ScienceDirectNonlinear Analysisjournal homepage: www.elsevier.com/locate/na

Variational sets: Calculus and applications to nonsmooth vector

optimization

Nguyen Le Hoang Anha,∗, Phan Quoc Khanhb, Le Thanh Tungc

aDepartment of Mathematics, University of Natural Sciences of Hochiminh City, 227 Nguyen Van Cu, District 5, Hochiminh City, Viet Nam

bDepartment of Mathematics, International University of Hochiminh City, Linh Trung, Thu Duc, Hochiminh City, Viet Nam

cDepartment of Mathematics, College of Science, Cantho University, Ninhkieu District, Cantho City, Viet Nam

© 2010 Elsevier Ltd All rights reserved.

1 Introduction

In nonsmooth optimization, many generalized derivatives have been introduced to replace the Fréchet and Gateaux derivatives which do not exist Each of them is adequate for some classes of problems, but not all In [ 1 , 2 ] we proposed two kinds of variational sets for mappings between normed spaces These subsets of the image space are larger than the images of the pre-image space through known generalized set-valued mappings Hence our necessary optimality conditions obtained by separation techniques are stronger than many known conditions using various generalized derivatives Of course, sufficient optimality conditions based on separations of bigger sets may be weaker But in [ 1 , 2 ], using variational sets we can establish sufficient conditions which have almost no gap with the corresponding necessary ones The second advantage of the variational sets is that we can define these sets of any order to get higher-order optimality conditions This feature is significant since many important and powerful generalized derivatives can be defined only for the first and second orders and the higher-order optimality conditions available in the literature are much fewer than the first and second-order ones The third strong point of the variational sets is that almost no assumptions are needed to be imposed for their being well-defined and nonempty and also for establishing optimality conditions Calculating them from the definition is only a computation of a Painlevé–Kuratowski limit However, in [ 1 , 2 ] no calculus rules for variational sets are provided.

In the present paper we establish elements of calculus for variational sets to ensure that they can be used in practice Most of the usual rules, from the sum and chain rules to various operations in analysis, are investigated It turns out that the

∗Corresponding author Tel.: +84 902971345.

E-mail addresses:nlhanh@math.hcmuns.edu.vn (N.L Hoang Anh), pqkhanh@hcmiu.edu.vn (P.Q Khanh), lttung@ctu.edu.vn (L.T Tung).

0362-546X/$ – see front matter © 2010 Elsevier Ltd All rights reserved.

doi:10.1016/j.na.2010.11.039

variational sets possess many fundamental and comprehensive calculus rules Although this construction is not comparable with objects in the dual approach like Mordukhovich’s coderivatives (see the excellent books [ 3 , 4 ]) in enjoying rich calculus,

it may be better in dealing with higher-order properties We pay attentions also on relations between the established calculus rules and applications of some rules to get others Of course, significant applications should be those in other topics

of nonlinear analysis and optimization As such applications we provide a direct employment of sum rules to establishing

an explicit formula for a variational set of the solution map to a parameterized variational inequality in terms of variational sets of the data Furthermore, chain rules and sum and product rules are also used to prove optimality conditions for weak solutions of some vector optimization problems.

The organization of the paper is as follows The rest of this section is devoted to recalling definitions needed in the sequel.

We present the two kinds of higher-order variational sets, including various equivalent formulations and simple properties

in Section 2 In the next Section 3 we explore comprehensive calculus rules for the variational sets We also try to illustrate

by example the unfortunate lack of expected rules We present in Section 4 direct applications of chain rules and sum and product rules obtained in Section 3 to considering stability and optimality conditions, as mentioned above.

Throughout the paper, if not otherwise specified, let X and Y be real normed spaces, C ⊆ Y a closed pointed convex cone with nonempty interior and F : X → 2Y For A ⊆ X , intA,clA (or A), bdA denote its interior, closure and boundary,¯respectively X∗is the dual space of X and B X stands for the closed unit ball in X For x0∈X,U(x0) is used for the set of all

A nonempty convex subset B of a convex cone C is called a base of C if C=coneB and 0̸∈clB.

For a set-valued map (known in the literature also as multimap or point-to-set map or multifunction or correspondence)

H:X→ 2Y , the domain, graph and epigraph of H are defined as

domH= {x∈X:H(x) ̸= ∅}, grH= { (x,y) ∈X×Y :y∈H(x)},

epiH= { (x,y) ∈X×Y:y∈H(x) +C}

The so-called profile mapping of H is H+defined by H+ (x) = H(x) +C The Painlevé–Kuratowski (sequential) outer (or

upper) limit is defined by

H is said to be compact at x0∈ cl (domH) if any sequence (x n,y n) ∈grH has a convergent subsequence as soon as x n→x0.

The closure of H is the multimap clH, whose graph is defined as the closure of grH Thus

clH(x0) = Limsup

x→H x0

H(x).

When clH(x0) =H(x0)we say that H is closed at x0.

For a subset A⊆X , the contingent cone of A at x0∈X is

Let (u1, v 1 ), , (u m− 1 , vm− 1 ) ∈X×Y The mth-order contingent derivative of H at(x0,y0) ∈grH with respect to (wrt)

(u1, v 1 ), , (u m− 1 , vm− 1 )is the mapping D m H(x0,y0,u1, v 1 , ,u m− 1 , vm− 1 ) with

grD m H(x0,y0,u1, v 1 , ,u m− 1 , vm− 1 ) =T grH m (x0,y0,u1, v 1 , ,u m− 1 , vm− 1 ).

The mth-order adjacent derivative of H at (x0,y0) wrt (u1, v 1 ), , (u m− 1 , vm− 1 ) is the set-valued mapping

D bm H(x0,y0,u1, v 1 , ,u m− 1 , vm− 1 ) defined by the following

grD bm H(x0,y0,u1, v 1 , ,u m− 1 , vm− 1 ) =T grH bm(x0,y0,u1, v 1 , ,u m− 1 , vm− 1 ).

2 Variational sets

In the sequel, if not otherwise stated, let X and Y be real normed spaces, F :X→ 2Y, (x0,y0) ∈grF andv 1 , , vm− 1 ∈Y

Definition 2.1 (See [ 1 ]) The variational sets of type 1 are defined as follows:

A set-valued mapping H :X → 2Y between two linear spaces is said to be star-shaped at x0 ∈S on the star-shaped at x0subset S⊆domH if, for all x∈S andα ∈ [ 0 , 1 ] ,

Proposition 2.3 (i) If F is star-shaped at x0, then

V1(F,x0,y0) =W1(F,x0,y0).

(ii) If we assume more that F is locally convex at(x0,y0)then these variational sets are convex.

Proof (i) Because we always have V m(F i,x0,y0, v 1 , , vm− 1 ) ⊆W m(F i,x0,y0, v 1 , , vm− 1 )for all m, we need to check only the reverse containment for m= 1 Let vbelong to the right-hand side, i.e there are x n→F x0, vn→ v, y n∈F(x n) and

h n > 0 such that vn = h n(y n−y0) It is clear that one can choose a sequence t n → 0 +such that t n h n → 0 + Then, for n large so that t n h n< 1,

y0+t nvn∈F(x0) +t n h n(F(x n) −F(x0)) ⊆F(x0+t n h n(x n−x0)) :=F(x n).

This means v ∈V1 (F,x0,y0)

(ii) Assume that vi ∈ W1(F,x0,y0), i.e there are x i,n

F

→ x0, vi,n → vi,y i,n ∈ F(x i,n) and h i,n > 0 such that

vi,n=h i,n(y i,n−y0)for i= 1 , 2 Then we see that

v 1 ,n+ v 2 ,n= (h1,n+h2,n)[(h1,n y1,n+h2,n y2,n)(h1,n+h2,n) − 1 −y0]

lies in cone + (F(x n) −y0)for x n = (h1,n x1,n+h2,n x2,n)(h1,n+h2,n) − 1, for all n, by the assumed convexity This means that

the limit v 1 + v 2belongs to W1 (F,x0,y0) 

Proposition 2.4 (See [ 1 ]) Let x0∈S⊆domF and y0∈F(x0) Assume that

(i) S is star-shaped at x0and F is C -star-shaped at x0on S; or

(ii) F is pseudoconvex at(x0,y0).

Then,∀x∈S,F(x) −y0⊆V1 (F+ ,x0,y0).

Therefore the following notion used later is a natural modification.

Definition 2.3 F : X → 2Y is said to be pseudoconvex of type 1 at (x0,y0) ∈ grF if, for all x ∈ domF,F(x) −y0 ⊆

V1 (F,x0,y0) ; and to be pseudoconvex of type 2 at (x0,y0)if, for all x∈domF,F(x) −y0⊆W1 (F,x0,y0)

3 Calculus of variational sets

3.1 Algebraic and set operations

As in Section 2, let X and Y be real normed spaces andv 1 , , vm− 1 ∈Y

Proposition 3.1 (Union Rule) Let F i:X→ 2Y,i= 1 , ,k, (x0,y0) ∈ k

i= 1grF i and I(x0,y0) = {i| (x0,y0) ∈grF i} Then (i) V m(k

Proposition 3.2 (Intersection Rule) Let F i:X→ 2Y,i= 1 , ,n and(x0,y0) ∈ n

i= 1grF i Then (i) V m(n

i= 1F i,x0,y0, v 1 , , vm− 1 ) ⊆ n

i= 1V m(F i,x0,y0, v 1 , , vm− 1 ); (ii) W m(n

V1(F1∩F2, 0 , 0 ) = R − ,W1(F1∩F2, 0 , 0 ) = R

The following definition is needed for some further developments.

Definition 3.1 Let F :X→ 2Y, (x0,y0) ∈grF andv 1 , , vm− 1 ∈Y If the upper limit defining V m(F,x0,y0, v 1 , , vm− 1 )

is a full limit, i.e the upper limit coincides with the lower limit, then this set is called a proto-variational set of order m of type 1 of F at(x0,y0)

If the similar coincidence occurs for W m we say that this set is a proto-variational set of order m of type 2 of F at(x0,y0)

Proposition 3.3 (Sum Rule for V m ) Let F i :X → 2Y,x0 ∈domF1

int k

i= 2domF i,y i∈F i(x0)andvi, 1 , , vi,m− 1 ∈Y for

i= 1 , ,k If F i,i= 2 , ,k have proto-variational sets V m(F i,x0,y0, vi, 1 , , vi,m− 1 ), respectively, then

Proof Considervi∈V m(F i,x0,y i, vi, 1 , , vi,m− 1 ),i= 1 , ,k One finds sequences t n→ 0 + ,x n→F1 x0and y1,n∈F1(x n) such that

Since V m(F i,x0,y i, vi, 1 , , vi,m− 1 ), i= 2 , ,k, are proto-variational sets and x0 ∈intdomF i , there are y i,n∈F i(x n), i=

2 , ,k, for large n such that

We cannot reduce the condition x0∈domF1

int k

i= 2domF i to x0∈ k

i= 1domF ias illustrated by

Example 3.3 Let X=Y =R,x0=y1=y2=0 and F1,F2:X→ 2Ybe defined by

Furthermore, the following example explains, unfortunately, that W mdoes not satisfy the rule similar to Proposition 3.3

even for m= 1 However, here a reverse containment is true for W1 It is interesting that this reverse containment holds

for W1 in a general case as shown in Proposition 3.4 below.

Example 3.4 Let X = Y = R,x0= 0 ,y1= 1 ,y2= −1 and F1,F2:X→ 2Ybe defined by

Proof For the sake of simplicity we discuss only the case k= 2 (the same is for general k) Let y∈W1 (F1+F2,x0,y1+

Since F1and F2are compact, there exist two subsequences (the subscripts of the second one are taken among those of the

first), denoted by the same notation y i,n , which converge to y i , respectively, for i= 1 ,2 Consequently, h nalso tends to some

nonnegative number h and we have in the limit

y= 1

h[(y1−y1) + (y2−y2)].

Observing that y i,n−y i ∈ F i(x n) −y i for all n, which means y i−y i ∈ W1(F i,x0,y i), and W1(F i,x0,y i) is a cone, the last equality completes the proof 

Unfortunately, the similar rule is not true for V1as indicated by the example below, which says also that the variationality assumed in Proposition 3.3 cannot be dropped.

proto-Example 3.5 Let X=Y=R,x0= 0 ,y1= 0 ,y2=1 and F1,F2:X→ 2Ybe defined by

has a proto variational set of type 1 at (x0,y0)

The following result can be validated similarly as Proposition 3.3

Proposition 3.5 (Descartes Product) Let F i:X i→ 2Y i,x i∈domF i,y i∈F i(x i)andvi, 1 , , vi,m− 1 ∈Y i for i= 1 , ,k Then

The following example says that even for m= 1 the counterpart of Proposition 3.5(ii) for W1is not true.

Example 3.6 Let X=Y=R,F1,F2:X→ 2Ybe defined by

W1(F1, 0 , 0 ) = R + , W1(F2, 0 , 1 ) = R − ,

W1(F1×F2, ( 0 , 0 ), ( 0 , 1 )) = ( R + × { 0 } ) ∪ ({ 0 } ×R−) ∪ {(y, −y) :y≥ 0 }

Hence, W1 (F1×F2, ( 0 , 0 ), ( 0 , 1 ))is strictly included in W1 (F1, 0 , 0 ) ×W1 (F2, 0 , 1 )

Moreover, assertion (ii) is not a necessary condition even with m=1 for the equality to hold for V1or W1 as shown by the next result.

Proposition 3.6 (Descartes Product for V1) Let F i:X i → 2Y i be star-shaped at x i,x i∈domF i and y i∈F i(x i)for i= 1 , ,k Then

Proof First, for V1 we have to check only the inclusion ⊆ Let (z1, ,z k) ∈ ∏k

i= 1V1(F i,x i,y i) Then one has sequences

The following example explains that the star-shape condition cannot be dispensed within the preceding statement.

Example 3.7 Let X=Y =R,F1,F2:X→ 2Ybe defined by

(ii) If additionally F has a proto-variational set of order m of type 1 at(x0,y0), then

D m G(y0,z0,u1, v 1 , ,u m− 1 , vm− 1 )(V m(F,x0,y0,u1, ,u m− 1 )) ⊆V m(G◦F,x0,z0, v 1 , , vm− 1 ).

(iii) If F is l.s.c at(x0,y0)then V m(GF,x0,z0, v 1 , , vm− 1 ) ⊆V m(G,y0,z0, v 1 , , vm− 1 ).

Proof (i) Let z∈D bm G(y0,z0,u1, v 1 , ,u m− 1 , vm− 1 )(V m(F,x0,y0,u1, ,u m− 1 )) There exists v ∈V m(F,x0,y0,u1, ,

u m− 1 )such that z∈D b(m)G(y0,z0,u1, v 1 , ,u m− 1 , vm− 1 )(v) Hence, for v, there exist t n→ 0 + ,x n→F x0and vn→ v such that

(ii) Let z∈D m G(y0,z0,u1, v 1 , ,u m− 1 , vm− 1 )(V m(F,x0,y0,u1, ,u m− 1 )) Then there exists v ∈V m(F,x0,y0,u1, ,

u m− 1 )such that z∈D m G(y0,z0,u1, v 1 , ,u m− 1 , vm− 1 )(v) Since V m(F,x0,y0,u1, ,u m− 1 )is a proto-variational set of F

of order m of type 1 at(x0,y0), for all sequences t n→ 0 +and x n→F x0, there exists a sequence vn→ v such that

y0+t n u1+ · · · +t n m−1u m− 1 +t n mvn∈F(x n)

as z∈D m G(y0,z0,u1, v 1 , ,u m− 1 , vm− 1 )(v), there exists t n→ 0+and (vn,z n) → (v,z) satisfying

z0+t nvn+ · · · +t n m−1vm− 1 +t n m z n∈G(y0+t n u1+ · · · +t n m−1+t n mvn).

The rest of the proof is the same as for (i).

(iii) Let w ∈V m(GF,x0,z0, v 1 , , vm− 1 ) Then there exist sequences t n→ 0 + , x n GF→x0and wn → w such that, for

We also use the corresponding subsequences a s=a n s and b s=b n s In virtue of the assumed star-shapedness one has

(ii) It is analogous to the proof of (iii) of Proposition 3.7

(iii) Let v ∈ W1(G,y0,z0) Then, there exist h n > 0 , vn → vand y n ∈ domG ⊆ ImF with y n → y0 such that

vn∈h n(G(y n) −z0) By the lower semicontinuity of F−1, there exists x n∈domF such that y n∈F(x n)for all n and x n→x0 Therefore,

vn∈h n(G◦F(x n) −z0),

i.e v ∈W1 (G◦F,x0,y0) 

Now we consider a special case where G=g is a differentiable single-valued map, which is important in practice and

we have formulas similar to the classical rule We discuss various situations, with increasing regularity properties towards

the case of g being linear (inPropositions 3.11 and 3.12 ).

Proposition 3.9 (Composition with Differentiable Map) Let F:X→ 2Y, (x0,y0) ∈grF and g:Y →Z be differentiable at y0 Then

Proof Letv ∈ V1(F,x0,y0)and sequences t n → 0+,x n →F x0and vn → vsatisfy y0 +t nvn ∈ F(x n)for all n Then

g(y0+t nvn) ∈ (g◦F)(x n) On the other hand,

Hence g′ (y0)v ∈V1 (g◦F,x0,g(y0)) Since the latter object is a closed cone, we arrive at the required inclusion.

Now assume that g′′ (y0) exists Let v 2 ∈ V2 (F,x0,y0, v 1 ),t n → 0 + ,x n →F x0 and v2n → v 2 be such that, for all

2g

′′ (y0)(v 1 , v 1 ) +g′(y0)v2n+ ϑ(t n)

] Therefore,

1

2g

′′ (y0)(v 1 , v 1 ) +g′(y0)v 2 ∈V2(g◦F,x0,z0,g′(y0)v 1 ) 

The inclusion in Proposition 3.9 (i) becomes equality under lower semicontinuity and calmness assumptions as follows.

Proposition 3.10 (Equality in Composition with Differentiable Map) Let Y be finite dimensional, F :X → 2Y, (x0,y0) ∈ grF and g:Y →Z Assume that

(i) F is l.s.c at(x0,y0);

(ii) g is differentiable at y0;

(iii) the map g−1: (g◦F)(x0) → 2F(x0 )defined by z →g− 1 (z) ∩F(x0)satisfies the calmness property: for some l>0 and all

z in a neighborhood of g(y0),

d(y0,g−1(z) ∩F(x0)) ≤ ‖z−g(y0)‖.

Then

cl (g′(y0)V1(F,x0,y0)) =V1(g◦F,x0,g(y0)).

Proof We need to prove only g′ (y0)V1 (F,x0,y0) ⊇V1 (g◦F,x0,g(y0)) For y∈V1 (g◦F,x0,g(y0)) , there exist sequences

t n→ 0 + , x n g→◦F x0, vn→y such that g(y0) +t nvn∈g◦F(x n)for all n By the calmness assumption, for large n,

d(y0,g−1(z n) ∩F(x0)) ≤l‖z n−f(y0)‖.

Hence, for ϵ >0, there is y n∈g− 1 (z n) ∩F(x0)such that, for u n:= 1

t n(y n−y0) ,

‖u n‖ ≤ (l+ ϵ)‖vn‖

Therefore, we have a subsequence, denoted also by u n , which converges to some u This results in u ∈V1 (F,x0,y0) , since

by the lower semicontinuity of F one has, for large n,

Proposition 3.11 (Composition with Linear Continuous Map) Let F :X→ 2Y,x∈domF and g∈L(Y,Z) Then

(i) for any m∈N there holds

cly∈g−1 (z)∩F(x)g(V m(F,x,y, v 1 , , vm− 1 )) ⊆V m(g◦F,x,z,g(v 1 ), ,g(vm− 1 )).

If additionally F is pseudoconvex of type 1 at(x0,y0) ∈grF , then one has equality for m=1;

(ii) for all m∈N one has

cly∈g−1 (z)∩F(x)g(W m(F,x,y, v 1 , , vm− 1 )) ⊆W m(g◦F,x,z,g(v 1 ), ,g(vm− 1 )).

If additionally F is pseudoconvex of type 2 at(x0,y0) ∈grF , then one has equality for m=1.

Proof (i) For each y∈g− 1 (z) ∩F(x) we have

If F is pseudoconvex of type 1 at(x0,y0)and x n∈domF , for y∈V1(g◦F,x0,g(y0)), there exist t n→ 0+, x n→F x0and

(ii) The assertion for W m can be checked by Theorem 4.26 of [ 5] as for V mbut we give a simple direct proof Let

y∈W m(F,x0,y0, v 1 , , vm− 1 )and x→F x0, t n→ 0+and y n→y with

In the case where Y is finite dimensional, for m = 1 we can obtain the equality in the conclusion of the preceding proposition under a condition on ker (g)(the null space of g) instead of the pseudoconvexity assumption We need the following definition of the horizon upper limit of F:X→Y in [5 ]

t n(y n−y) If { vn} is bounded then one can assume that vntends to some v , which satisfies v ∈V1 (F,x,y) and

g(v) =u as required So it remains to check this boundedness Suppose‖ vn‖ → ∞ and set vn= vn

‖ vn‖ which is assumed to have a limit vwith norm one Then g(v) = 0 Furthermore v ∈ limsup∞

x′→F x,t→ 0+

1

t(F(x′ ) −y) , which is impossible 

For the following special case, equality holds for m= 1 without any assumption.

Corollary 3.13 Let F:X→ 2Y, (x0,y0) ∈grF andλ ∈R.

(i) λV m(F,x0,y0, v 1 , , vm− 1 ) ⊆V m(λF,x0, λy0, λv 1 , , λvm− 1 ) The equality always holds for m=1.

(ii) λW m(x0,y0, v 1 , , vm− 1 ) ⊆W m(λF,x0, λy0, λv 1 , , λvm− 1 ) The equality always holds for m=1.

Remark 3.1 For scaling only the directionsv 1 , , vm− 1 we easily demonstrate by definition the following rule (This is

not for direct calculus on F , but relevant toCorollary 3.13 )

Scaling the Directions: Let F:X→ 2Y, (x0,y0) ∈grF, λ > 0 and v 1 , , vm− 1 ∈Y Then

(i) V m(F,x0,y0, λv 1 , , λm− 1 vm− 1 ) = λm V m(F,x0,y0, v 1 , , vm− 1 ) ;

(ii) W m(F,x0,y0, v 1 , , λm− 2 vm− 1 ) = λm− 1W m(F,x0,y0, v 1 , , vm− 1 )

Let us return to the general multimaps Suggested by a referee that general chain rules may often encompass sum rules as

special cases, we now investigate the sum M+N of two multifunctions M,N:X→ 2Y To express M+N as a composition, define F:X→ 2X×Y and G:X×Y → 2Y by, for I being the identity map on X and(x,y) ∈X×Y ,

Then clearly M+N=G◦F However, the rule given inProposition 3.7 , though simple and relatively direct, is not suitable

for dealing with these F and G, since the intermediate space (Y there and X×Y here) is little involved Inspired by [8

we develop another composition rule as follows Let general multimaps F :X → 2Y and G :Y → 2Zbe considered The

so-called resultant multimap C:X×Z→ 2Yis defined by