Real Analysis with Economic Applications - Chapter G pptx

In particular, we prove here the linear algebraic formulations of theHahn-Banach Theorems on the extension of a linear functional and the separation ofconvex sets, along with the Krein-R

Chapter G

Convexity

One major reason why linear spaces are so important for geometric analysis is thatthey allow us to define the notion of “line segment” in algebraic terms Among otherthings, this enables one to formulate, purely algebraically, the notion of “convex set”which figures majorly in a variety of branches of higher mathematics.1 Immediatelyrelevant for economic theory is the indispensable role played by convex sets in op-timization theory At least for economists, this alone is enough of a motivation fortaking on a comprehensive study of convex sets and related concepts

We begin the chapter with a fairly detailed discussion of convex sets and cones.Our emphasis is, again, on the infinite dimensional side of the picture In particu-lar, we consider several examples that are couched within infinite dimensional linearspaces After all, one of our main objectives here is to provide some help for the novice

to get over the sensation of shock that the strange behavior of infinite dimensionalspaces may invoke at first We also introduce the partially ordered linear spaces, anddiscuss the important role played by convex cones thereof

Among the topics that are likely to be new to the reader are the algebraic rior and algebraic closure of subsets of a linear space These notions are developedrelatively leisurely, for they are essential for the treatment of the high points of thechapter, namely, the fundamental extension and separation theorems in an arbitrarylinear space In particular, we prove here the linear algebraic formulations of theHahn-Banach Theorems on the extension of a linear functional and the separation ofconvex sets, along with the Krein-Rutman Theorem on the extension of positive lin-ear functionals We then turn to Euclidean convex analysis, and deduce Minkowski’sSeparating and Supporting Hyperplane Theorems — these are among the most widelyused theorems in economic theory — as easy corollaries of our general results As afinal order of business, we discuss the problem of best approximation from a convexset in a Euclidean space In particular, we introduce the orthogonal projection op-erator, and use it to obtain the Euclidean version of the Krein-Rutman Theorem.Several economic applications of the main theorems established in this chapter areconsidered in Chapter H.2

inte-1 See Klee (1971) for a concise, yet instructive, introduction to the theory of convex sets at large.

2 There are many excellent textbooks on finite dimensional convex analysis Rockafellar (1970) is

a classic in the field, but of course, there are more recent expositions In particular, Hiriart-Urruty and Lemaréchal (2000) provide a very nice introduction, and Borwein and Lewis (2000) take one to the next level.

Elementary treatments of infinite dimensional convex analysis are harder to find Certainly the linear algebraic treatment I present here is not commonly adopted in textbooks on mathematical analysis One major exception is the first chapter of the excellent text by Holmes (1975) However,

if this is your first serious encounter with infinite dimensional linear spaces, I would suspect that it would be wiser to go to this reference only after completing the present chapter.

1 Convex Sets

1.1 Basic Definitions and Examples

You are familiar with the way one defines a convex set in the context of Euclideanspaces — we have already used this concept a few times in this book This definitioncarries over to the case of arbitrary linear spaces without modification

Dhilqlwlrq For any 0 < λ < 1, a subset S of a linear space X is said to be λ-convexif

λx + (1− λ)y ∈ S for any x, y ∈ S,

The interpretation of these concepts are straightforward For any two distinctvectors x and y in the linear space X, we think of the set {λx+ (1 − λ)y ∈ X : λ ∈ R}

as the line through x and y From this point of view, the set {λx + (1 − λ)y ∈ X :

0 ≤ λ ≤ 1} corresponds to the line segment between x and y.3 Consequently,

a subset S of X is midpoint convex iff it contains the midpoint of the line segmentbetween any two of its constituent vectors It is convex iff it contains the entire linesegment between any two of its elements (Figure 1)

∗ ∗ ∗ ∗ FIGURE G.1 ABOUT HERE ∗ ∗ ∗ ∗

As for examples, note that Q is a midpoint convex subset of R which is not convex.Indeed, a nonempty subset of R is convex iff it is an interval In any linear space X,all singleton sets, line segments and lines, along with ∅, are convex sets Any linearsubspace, affine manifold, hyperplane or half space in X is also convex For instance,for any 0 < λ < 1, a hyperplane H in a linear space X is a λ-convex set, because, byCorollary F.4, H = {x ∈ X : L(x) = α} for some nonzero L ∈ L(X, R) and α ∈ R,and thus, for any x, y ∈ H, we have

L(λx + (1− λ)y) = λL(x) + (1 − λ)L(y) = λα + (1 − λ)α = α,

that is, λx + (1 − λ)y ∈ H

3 Observe that it is the linear structure of a linear space that lets us “talk about” these objects

— we could not do so, for instance, in an arbitrary metric space.

As an immediate illustration of the appeal of convex sets — we will of course seemany more examples later — let us note that the affine hull of a convex set can ingeneral be expressed much more easily than that of a nonconvex set Indeed, for anyconvex subset S of a linear space, we simply have

aff (S) = {λx + (1 − λ)y : λ ∈ R and x, y ∈ S}

(Geometric interpretation?) The proof requires an easy induction argument — weleave that to you But we note that convexity is essential for this observation Forinstance, the set S := {0, e1, e2

} in R2 does not satisfy the equation above

Exercise 1 Determine which of the following sets are necessarily convex (in the linear spaces they naturally live):

(a) a nonempty connected subset of R2;

(b) {(t, y) ∈ R2 : f (t)≥ y}where f is a concave self-map onR;

(c){(t, y) ∈ R2 : sin t≥ y};

(d ) an open neighborhood of a given point inR4;

(e) the set of all semicontinuous functions on[0, 1];

(f ) {f ∈ B[0, 1] : f ≥ 0};

(g){f ∈ C[0, 1] : f(1) = 0}

Exercise 2 Show that the intersection of any collection of λ-convex subsets of a linear space isλ-convex,0 < λ < 1

Exercise 3 Show that if A and B are λ-convex subsets of a linear space, so is

αA + B for any α∈ R and0 < λ < 1

Exercise 4.HProve: A convex setSis contained in one of the open halfspaces induced

by a hyperplaneH in a linear space iffS∩ H = ∅

Let S be any set in a linear space X, and let S be the class of all convex subsets

of X that contain S We have S = ∅ — after all, X ∈ S Then, by Exercise 2, W

S is

a convex set in X which, obviously, contains S Clearly, this set is the smallest (that

is, ⊇-minimum) subset of X that contains S — it is called the convex hull of S, anddenoted by co(S) (Needless to say, S = co(S) iff S is convex; see Figure 1.) Thisnotion proves useful when one needs the convexity of the set one is working with,even though that set is not known to be convex We will encounter many situations

of this sort later

It is worth noting that the definition of co(S) uses vectors that lie outside S(because the members of S may well contain such vectors) For this reason, this def-inition can be viewed as an external one We may also characterize co(S) internally,that is, by using only the vectors in S Indeed, co(S) is none other than the set of allconvex combinations of finitely many members of S That is,

co(S) =

mSi=1

mSi=1

λi = 1

, m = 1, 2,

This shows that the way we define the convex hull of a set here is in full accord withhow we defined this concept in Section E.5.3 to state Carathéodory’s Theorem inthe context of Euclidean spaces Moreover, for any x, y ∈ X, we see that co({x, y})

— which is denoted simply as co{x, y} henceforth — is nothing but the line segmentbetween x and y, and we have

co(S) =V

{co(T ) : T ∈ P(S)},where P(S) is the class of all nonempty finite subsets of S

Exercise 5 Prove (1).

Exercise 6 Show that, for any subset S of a linear spaceX and x∈ X,

co(S + x) = co(S) + x

Exercise 7 Prove or disprove: IfXandY are linear spaces, andA⊆ X andB ⊆ Y,

then co(A × B) = co(A)× co(B), where the underlying linear space is X × Y

an affine map; recall Exercise F.32.)

All algebraic properties of concave functions that you are familiar with are valid

in this more general setup as well For example, if T is a nonempty convex subset of

a linear space, then ϕ ∈ RT

is concave iff {(x, a) ∈ T × R : ϕ(x) ≥ a} is a convexsubset of T × R Moreover, if ψ ∈ RT is also concave, then so is ϕ + ψ However, theset of all concave real functions on T is not an affine manifold of RT (Why?)

The next exercise introduces the notion of convex correspondence, which plays animportant role in optimization theory In fact, we have already used convex corre-spondences when studying the theory of dynamic programming (recall PropositionE.7) We will come back to them in due course.

Exercise 8.H (Convex Correspondences) LetX and Y be two linear spaces, and S a nonempty convex subset ofX We say that Γ : S ⇒ Y is a convex correspondence

ifGr(Γ) is a convex subset ofX× Y, that is,

λΓ(x) + (1− λ)Γ(x ) ⊆ Γ(λx + (1 − λ)x ) for any x, x ∈ Sand 0≤ λ ≤ 1

(a) Let T be a convex subset ofY, take any L ∈ L(X, Y ), and define Γ : S ⇒ Y

by Γ(x) := L(x) + T.Show that Γis a convex correspondence.

(b) Let f ∈ RS be a convex function Show that Γ : S ⇒ R, defined by Γ(x) :={a ∈ R : a ≥ f(x)},is a convex correspondence.

(c) Show that the budget correspondence (Example E.2) is a convex-valued spondence which is not convex.

corre-(d ) Every linear correspondence Γ : X ⇒ Y is convex (Exercise F.31) Is the verse true?

con-(e) (Deutsch-Singer) Prove: IfΓ : X ⇒ Y is a convex correspondence with|Γ(x0)| =

1for somex0 ∈ X,then there exists an affine mapf ∈ YX such that{f(x)} = Γ(x)

λx∈ C for any x ∈ C and λ ≥ 0

If C is, in addition, closed under addition, that is, C + C ⊆ C, i.e

x + y ∈ C for any x, y ∈ C,then it is called a convex cone We say that a cone C in X is pointed if C ∩ −C ={0},4 generating if span(C) = X, and nontrivial if C = {0}

Geometrically speaking, a cone is a set that contains all rays that start from theorigin and pass through another member of the set (Figure 2) In turn, a convex

4 Reminder −C := (−1)C, that is, −C = {−x : x ∈ C}, and C − C := C + −C, that is,

C − C = {x − y : x, y ∈ C}.

cone is none other than a cone which is also convex set (Why?) In a manner ofspeaking, the concept of convex cone lies in between that of a convex set and that of

a linear subspace This point will become clear as we develop the theory of convexsets further

Remark 1 Recall that the dimension of a set in a linear space X is the dimension ofits affine hull (Remark F.1) But since any cone C in X contains the origin 0 (yes?),

we have aff (C) = span(C) (Why?) Thus dim(C) = dim(span(C)), so a cone C in afinite dimensional linear space X is generating iff dim(C) = dim(X)

∗ ∗ ∗ ∗ FIGURE G.2 ABOUT HERE ∗ ∗ ∗ ∗E{dpsoh 1 [1]Any linear subspace C of a linear space X is a convex cone In thiscase C ∩ −C = C, so C is a pointed cone iff C = {0} Moreover, C is generating iff

C = X

[2] The smallest (i.e ⊇-minimum) convex cone in a linear space X is the triviallinear subspace {0} while the largest (i.e ⊇-maximum) convex cone is X itself.Moreover, if C is a convex cone in this space, so is −C

[3] For any given n ∈ N, the set Rn+ is a pointed generating convex cone in Rn —

we have dim(Rn

+) = n On the other hand, Rn

++ is not a cone in Rn since it does notcontain the origin 0 Indeed, Rn

++∪{0} is a pointed generating convex cone in Rn.Anexample of a convex cone in Rn, n ≥ 2, which is not pointed is {x ∈ Rn : x1 ≥ 0}, andthat of one which is not generating is {x ∈ Rn : x1 = 0} (What are the dimensions

of these cones?)

[4] If X and Y are two linear spaces, and L ∈ L(X, Y ), then {x ∈ X : L(x) ≥ 0}

is a (pointed) convex cone in X This observation allows us to find many convexcones For instance, {(xm) ∈ R∞ : x4 ≥ 0}, or {f ∈ B[0, 1] : f(1

2) ≥ 0}, or{f ∈ C[0, 1] : U1

0 f (t)dt ≥ 0} are pointed convex cones by this very token All ofthese cones are infinite dimensional and generating

[5] Both {f ∈ C1[0, 1] : f ≥ 0} and C := {f ∈ C1[0, 1] : f ≥ 0} are convex cones

in C1[0, 1] The former one is pointed, but the latter is not — all constant functionsbelong to C ∩ −C

Let S be any nonempty set in a linear space X It is easily checked that S is

a convex cone in X iff S

x∈T λ(x)x ∈ S for any nonempty finite subset T of S and

λ ∈ RT+ (Verify!) It follows that the smallest convex cone that contains S — theconical hull of S — exists and equals the set of all positive linear combinations

of finitely many members of S (Why?) Denoting this convex cone by cone(S),therefore, we may write

cone(S) =

Sx∈Tλ(x)x : T ∈ P(S) and λ ∈ RT+

where, as usual, P(S) stands for the class of all nonempty finite subsets of S (Byconvention, we let cone(∅) = {0}.) For instance, for any n ∈ N,

cone(Rn++) = Rn++∪ {0} and cone({e1, , en}) = Rn+,where {e1, , en} is the standard basis for Rn

Exercise 9.H Let c0 be the linear space of all real sequences all but finitely many terms of which are zero Prove:

(a) C :={(xm)∈ c0 : xm > 0for somem}is not a cone in c0;

(b) C∪ {0} is an infinite dimensional cone which is not midpoint convex;

(c)cone(C) = c0

Exercise 10 Show that ifC is a cone in a linear spaceX, thenspan(C) = C − C

Thus, C is generating iffX = C − C

Exercise 11 LetS be any nonempty set in a given linear space Show that

if a given cone has a base The following exercise provides two criteria for this to bethe case

∗ ∗ ∗ ∗ FIGURE G.3 ABOUT HERE ∗ ∗ ∗ ∗

∗ Exercise 13.H LetCbe a nontrivial convex cone in a linear spaceX, andB a convex subset of X.Prove that the following statements are equivalent.

(i) B is a base for C;

B = {0} is trivially dismissed as a potential candidate for a base, of course.)

1.3 Ordered Linear Spaces

Let (X, ) be a preordered set (Section A.1.4) If X is a linear space, and iscompatible with the addition and scalar multiplication operations on X in the sensethat

for any x, y, z ∈ X and λ ≥ 0, then we say that (X, ) is a preordered linearspace, and refer to as a vector preorder More precisely, a preordered linearspace is a list (X, +, ·, ) where (X, +, ·) is a linear space, (X, ) is a preordered set,and these two mathematical systems are connected via the compatibility requirement(2) Of course, (X, +, ·, ) is unnecessarily mouthful We denote this system simply

as X to simplify the notation, and write X for the associated preorder on X If

X is a preordered linear space and X is a partial order, then we say that X is apartially ordered linear space, and refer to X simply as a vector order.There is an intimate relation between convex cones and vector (pre)orders First

of all, by using the convex cones of a linear space X, we can obtain various vectorpreorders on X, and thus make X a preordered linear space in a variety of ways Tosee this, take any nonempty subset C of X, and define the binary relation C on X

as follows:

x C y if and only if x∈ y + C

It is easy to see that C is a vector preorder on X iff C is a convex cone — in thiscase the preorder C is called the preorder induced by C Conversely, every vectorpreorder on X arises in this way Indeed, if is such a preorder, then X+( ) :={x ∈ X : x 0} is a convex cone — called the positive cone induced by — and

we have = X + ( ) Moreover, is a partial order on X iff X+( )is pointed.6The upshot is that there is a one-to-one correspondence between convex cones in

a linear space X and vector preorders on X The idea is rather transparent in R Forany two real numbers a and b, the statement a ≥ b is equivalent to a − b ≥ 0, so since

a = b + (a− b), another way of saying a ≥ b is to say a ∈ b + R+.That is, the naturalorder ≥ of R is simply the vector order on R induced by the convex cone R+ Putdifferently, ≥ = R +

For any n ∈ N, exactly the same situation ensues in Rn

, that is, x ≥ y iff x ∈

y + Rn+ Of course, we would have obtained different ways of ordering the real vectors if we used a convex cone in Rn

n-distinct from Rn

+.For instance, if we designated

C :={x ∈ R2 : x1 > 0 or x1 = 0 and x2 ≥ 0} as our “positive cone” in R2, then wewould be ordering the vectors in R2 lexicographically, that is, C = lex (ExampleB.1)

The smoke must be clearing now A preorder on a linear space X tells us whichvectors in X are “positive” in X — a vector x in X is deemed “positive” by iff

6 The following two extreme situations may be worth noting here We have = X × X iff

X+( ) = X (the case of complete indifference), and = {(x, x) : x ∈ X} iff X + ( ) = {0} (the case of full non-comparability).

x 0 Thus, if is a vector preorder, then

x y if and only if x = y +a positive vector in X

In this case, therefore, all there is to know about can be learned from the positivecone induced by This simple observation brings the geometric analysis of convexcones to the fore of order-theoretic investigations

Notation As noted above, we denote the preorder associated with a given ordered linear space X by X In turn, the positive cone induced by X is denoted

pre-by X+,that is,

X+ :={x ∈ X : x X 0}

In the same vein, the strictly positive cone of X is defined as

X++ :={x ∈ X : x X 0},where X is the asymmetric part of X (Exercise A.7).7 Clearly, if X is a partiallyordered linear space, then X++ = X+\{0}

Warning The strictly positive cone of Rn (with respect to the partial order ≥)equals Rn

++ iff n = 1 (Why?)

E{dpsoh 2 [1] Let X be a preordered linear space, and Y a linear subspace of X

A natural way of making Y a preordered linear space is to designate X+∩ Y as Y+.The induced vector preorder Y on Y is then X ∩ (Y × Y ) When Y is endowedwith this vector preorder, we say that it is a preordered linear subspace of X

[2]For any nonempty set T, recall that the natural partial order ≥ on RT is definedpointwise, that is, f ≥ g iff f(t) ≥ g(t) for all t ∈ T The positive cone in RT is thusthe class of all nonnegative-valued maps on T : (RT)+ := RT

+ Unless otherwise isstated explicitly, all of the function (and hence sequence) spaces that we consider inthis text (e.g C[0, 1], C1[0, 1], p, 1≤ p ≤ ∞, etc.) are ordered in this canonical way.That is, for any given nonempty set T and linear subspace X of RT, X is thought

of as a preordered linear subspace of X — we have X+ := RT+∩ X In this case, wedenote the induced vector order on X simply by ≥ (instead of X)

[3] Let X and Y be two preordered linear spaces A natural way of making thelinear product space X × Y a preordered linear space is to endow it with the productvector preorder That is, we define X×Y as

(x, y) X×Y (x , y ) if and only if x X x and y Y y

7 Strictly speaking, I’m abusing the terminology here in that X ++ is not a cone, for it does not contain 0 (Of course, X ++ ∪ {0} is a convex cone.)

for all x, x ∈ X and y, y ∈ Y The natural positive cone of X ×Y is thus (X ×Y )+:=

X+× Y+ Endowed with this order structure, X × Y is called a preordered linearproduct of X and Y

Exercise 14.H Let X be a preordered linear space Show that X+ is a generating cone inX iff, for anyx∈ X, there exists a y∈ X such thaty X xand y X −x

Exercise 15 (Peressini) LetX be a preordered linear space.

(a) Show that ifXcontains a vectorx∗ such that, for allx∈ X,there exists aλ > 0

withλx∗ X x, thenX+ must be generating.

(b) Use part (a) to conclude that the positive cones of any of the following partially ordered linear spaces are generating: p (with 1≤ p ≤ ∞),the linear space cof all convergent real sequences,B(T ) (for any nonempty setT).

f (x)∈ f(x ) + Y+ whenever x∈ x + X+,and strictly increasing iff it is increasing and

f (x)∈ f(x ) + Y++ whenever x∈ x + X++

for any x, x ∈ S It is obvious that these definitions reduce to the usual definitions

of increasing and strictly increasing functions, respectively, in the case where X is aEuclidean space and Y = R

E{dpsoh 3 [1] For any two preordered linear spaces X and Y, a linear ator L ∈ L(X, Y ) is increasing iff L(X+) ⊆ Y+ — such an L is called a positivelinear operator Similarly, L ∈ L(X, Y ) is strictly increasing iff L(X+)⊆ Y+ andL(X++)⊆ Y++— such an L is called a strictly positive linear operator Of course,when Y = R, we instead talk of positive linear functionals and strictly positive linearfunctionals, respectively For instance, the zero functional on X is a positive linearfunctional which is not strictly positive

oper-[2] Take any m, n ∈ N, and let A := [aij]m×n ∈ Rm×n Recall that L : Rn

→ Rm,defined by L(x) := Ax, is a linear operator (Example F.6) This operator is positive

iff aij ≥ 0 for all i and j, and strictly positive iff (ai1, , ain) > 0for each i = 1, , m

[3] Each of the maps (xm) → x1, (xm) → (x2, x3, ) and (xm) → S∞ 1

2 ixi arepositive linear functionals on ∞ The first two of these are not strictly positive, butthe third one is

[4]The linear operator D : C1[0, 1]→ C[0, 1] defined by D(f) := f is not positive.(Why?)

Exercise 16 (a) Define L ∈ RC[0,1] by L(f ) := f (0) Is L a positive linear tional? A strictly positive one?

func-(b) Define the self-mapL on C[0, 1] by L(f )(x) :=Ux

0 f (t)dt for all0≤ x ≤ 1 Is

La positive linear operator? A strictly positive one?

Exercise 17 Prove: IfKandLare two nonzero positive linear functionals on a given partially ordered linear space, thenK− Lis a positive linear functional iff K = L

Exercise 18 LetX andY be two partially ordered linear spaces, and denote the set

of all positive linear operators fromX intoY byL+(X, Y ) Show thatL+(X, Y )is

a convex cone inL(X, Y ) Moreover, ifX+ is generating, thenL+(X, Y )is pointed Exercise 19 LetX be a preordered linear space, andI an interval We say that the correspondence Γ : I ⇒ X is strictly increasing if for any a, b ∈ I with a > b, and anyx∈ Γ(a),there exists ay∈ Γ(b)such thatx X y.Show that, for any such

Γ,there exists a unique strictly increasing surjection f : Γ(I)→ I with Γ = f−1

1.4 Algebraic and Relative Interior of a Set

The generality of the treatment set aside, the basic concepts covered so far in thischapter are probably not new to you Starting from this point on, however, it is likelythat we will be entering a new territory The idea is this We are interested in talkingabout open and closed convex sets, the boundary of a convex set, and so on We can

do this in Rn easily by using the metric structure of Rn.At the moment, however, all

we have is a linear space, so Rn example is not readily helpful Instead, we notice thatopenness of convex sets in a Euclidean space can alternatively be described by usingonly the notion of line segments For instance, one way of proving that (0, 1) is anopen interval in R is to observe that, for any a ∈ (0, 1), some part of the line segmentbetween a and any other point in R (not including the endpoint a) is contained in(0, 1) (That is, for any a ∈ (0, 1) and any real number b, there is a c in co{a, b}\{a}such that co{a, c} ⊆ (0, 1).) In this sense all points of (0, 1) are in the “interior” of(0, 1), and hence we may think of (0, 1) as open Clearly, we could not do the samefor (0, 1] The line segment between 1 and 2, say, intersects (0, 1] only at the endpoint1

These considerations become trivial, of course, if the term “open” is understoodrelative to the usual metric on R The point is that, in this discussion, we did notuse a metric at all, everything was algebraic So, perhaps there is a way of defining

an openness notion, at least for convex sets, which would be identical to the usualnotion of openness in Euclidean spaces but that would not require us to use a distancefunction This would let us talk about open convex sets in an arbitrary linear space

in a geometrically meaningful way

We now develop this idea formally.

Dhilqlwlrq Let S be a subset of a linear space X A vector x in S is called analgebraic interior point of S (in X) if, for any y ∈ X, there exists an αy > 0 suchthat

(1− α)x + αy ∈ S for all 0 ≤ α ≤ αy.The set of all algebraic interior points of S in X is called the algebraic interior of

S (in X) and is denoted by al-intX(S) (Note al-intX(∅) = ∅.) If S ⊆ al-intX(S), wesay that S is algebraically open in X

Geometrically speaking, x ∈ al-intX(S) means that one may move linearly from

xtowards any direction in the linear space X without leaving the set S immediately.(See Figure 4 and Exercise 26 below.) While intuitive (is it?), this definition iscompletely algebraic — it is not based on a distance function — so it makes sense inthe setting of an arbitrary linear space As we shall see shortly, however, there arestrong links between the algebraic interior and the usual interior of a convex set in avariety of interesting cases

∗ ∗ ∗ ∗ FIGURE G.4 ABOUT HERE ∗ ∗ ∗ ∗Whether or not a given set is algebraically open depends crucially on relative

to which space one asks the question.8 Indeed, if Y is a linear subspace of X, and

S ⊆ Y ⊆ X, then al-intY(S)and al-intX(S)may well be quite different For instance,{0} ⊆ R is algebraically open in the trivial linear space {0}, but it is not in R.Similarly, if S := {(t, t) : 0 ≤ t ≤ 1} and Y := {(t, t) : t ∈ R}, then al-intY(S) ={(t, t) : 0 < t < 1} while al-intR 2(S) =∅

Now, the natural habitat of a convex set is its affine hull In particular, if S is

a subset of a linear space X with 0 ∈ S, then all algebraic properties of S can bestudied within span(S) — we do not, in effect, need the vectors in X\span(S) for thispurpose Nothing changes, of course, if we drop the assumption 0 ∈ S here, exceptthat the natural residence of S then becomes aff (S) Thus, the algebraic interior of asubset of a linear space relative to its affine hull is of primary importance In convexanalysis, it is referred to as the relative interior of that set

Dhilqlwlrq Let S be any subset of a linear space X A vector x in S is called arelative interior point of S if, for any y ∈ aff (S), there exists an αy > 0 such that

(1− α)x + αy ∈ S for all 0 ≤ α ≤ αy.The set of all relative interior points of S is called the relative interior of S and isdenoted by ri(S) (Note ri(∅) = ∅.) If S ⊆ ri(S), we say that S is relatively open

8 Go back and read the same sentence again, but this time omit the word “algebraically.” Sounds familiar?

Thus, a relative interior point of a set S is a vector x in S such that, when movingfrom x towards any direction in aff (S), we do not immediately leave S For example,

we have ri((0, 1)), but 1 /∈ ri((0, 1]) But note that S = ri(S) does not imply that S

is algebraically open Indeed, {0} = ri({0}), because aff {0} = {0}, but {0} is notalgebraically open in R, for al-intR({0}) = ∅ Similarly, if S := {(t, t) : 0 ≤ t ≤ 1},then ri(S) = {(t, t) : 0 < t < 1} while al-intR 2(S) =∅

Before considering less trivial examples, let us clarify the relation between thealgebraic and relative interior of an arbitrary subset S of a linear space X:

al-intX(S) =

ri(S), if aff (S) = X

(Since S ⊆ X, we have aff (S) ⊆ X, so this equation covers all contingencies.)

Here is why If aff (S) = X, we trivially have al-intX(S) = ri(S), so considerinstead the case aff (S) ⊂ X Take any x∗ ∈ S, and let Y := span(S − x∗).9 Since

aff (S) is a proper subset of X, Y is a proper linear subspace of X Pick any w ∈ X\Y,and notice that (1 − α)z + αw /∈ Y for any z ∈ Y and 0 < α ≤ 1 (Otherwise wewould contradict w /∈ Y, wouldn’t we? See Figure 5.) But then

(1− α)(z + x∗) + α(w + x∗) /∈ Y + x∗ = aff (S)for any z ∈ Y and 0 < α ≤ 1 Letting y := w+x∗,therefore, we find (1−α)x+αy /∈ Yfor any x ∈ S and 0 < α ≤ 1 Conclusion: al-intX(S) = ∅

It is worth putting on record the following immediate consequence of (3)

Observation 1.For any subset S of a linear space X, we have al-intX(S) =∅ if, andonly if, aff (S) = X and ri(S) = ∅

∗ ∗ ∗ ∗ FIGURE G.5 ABOUT HERE ∗ ∗ ∗ ∗Let us now move on to examples Our first set of examples explores the connectionbetween the notions of algebraic and metric openness in the context of Euclideanspaces Please note that in Examples 4-6 we simplify our notation by writing al-int(S) for al-intRn(S), and int(S) for intRn(S)

E{dpsoh 4 Given any n ∈ N, consider any nonempty subset S of Rn

Question: How does al-int(S) relate to int(S)?

Answer: al-int(S) is surely larger than int(S), but the converse well, it depends!For, if x ∈ int(S), then there exists an ε > 0 such that Nε,R n(x) ⊆ S, while, forany (fixed) y ∈ Rn,a straightforward continuity argument shows that there exists an

αy > 0such that d2((1− α)x + αy, x) < ε for all 0 ≤ α ≤ αy.Thus

(1− α)x + αy ∈ Nε,R n(x)⊆ S, 0≤ α ≤ αy,

9 Reminder aff (S) = span(S − x) + x for any x ∈ S (Section F.1.4).

and we may conclude that x ∈ al-int(S) Conclusion:

int(S) ⊆ al-int(S) for any set S in Rn (4)

By the way, combining this observation with (3), we find, for any subset S of Rn,

int(S)

⊆ ri(S), if dim(S) = n

=∅, if dim(S) < n .This answer is only partial, however To complete the picture, we need to findout whether we can strengthen ⊆ to = in (4) In fact, we can’t Consider the set

S :={(θ, r) : 0 ≤ θ ≤ 2π and 0 ≤ r ≤ 1 −2πθ }which is expressed in polar coordinates (Figure 6) It is easily checked that 0 /∈intR2(S) but 0 ∈ ri(S) = al-int(S) Thus: The interior of a subset of a Euclideanspace may be a proper subset of its relative and/or algebraic interior

∗ ∗ ∗ ∗ FIGURE G.6 ABOUT HERE ∗ ∗ ∗ ∗E{dpsoh 5 We now show that the anomaly depicted in the second part of Example

4 was in fact due to the nonconvexity of the set in question That is, we wish to showthat we can in fact strengthen ⊆ to = in (4), provided that S is convex

To see this, take any nonempty convex subset S of Rn,and let x ∈ al-int(S) Then

we can move from x in any direction without leaving S immediately In particular,consider the vectors yi := x + ei, i = 1, , n, and yi := x− ei, i = n + 1, , 2n,where

ei is the ith unit vector in Rn Since x is an algebraic interior point of S in Rn, foreach i ∈ {1, , n} there exists an αi > 0such that

x + αei = (1− α)x + αyi ∈ S, 0≤ α ≤ αi,and similarly, for each i ∈ {n + 1, , 2n} there exists an αi > 0 such that

x− αei ∈ S, 0≤ α ≤ αi.Letting α∗ := min{α1, , α2n}, therefore, we find that x + αei

∈ S for all i = 1, , nand α ∈ [−α∗, α∗].But since S is convex, we have

T :=co

x + αei :−α∗ ≤ α ≤ α∗, i = 1, , n 

⊆ S(Figure 7) Of course, x ∈ int(T ) and int(T ) ⊆ int(S), so x ∈ int(S), as we sought.Conclusion:

int(S) = al-int(S) for any convex set S in Rn (5)This is what we’ve meant by “ well, it depends!” in our answer to the questionposed at the beginning of Example 4

∗ ∗ ∗ ∗ FIGURE G.7 ABOUT HERE ∗ ∗ ∗ ∗The following is an immediate consequence of (3) and (5).

Observation 2.Given any n ∈ N and a convex subset S of Rn we have

int(S) =

ri(S), if dim(S) = n

∅, if dim(S) < n .Moreover, S is open in Rn

if, and only if, it is algebraically open in Rn

E{dpsoh 6 Take any n ∈ N and let {e1, , en

} stand for the standard basis for Rn,

as usual Consider the set

S :=co

{0, e1, , en}

=

 nSi=1

λiei : λi ≥ 0, i = 1, , n and

nSi=1

λi ≤ 1

It is easy to see that the algebraic interior of S in Rn is not empty, and thus equalsri(S) (See Figure 8.) For instance, the “center” of S is in the algebraic interior of S

in Rn, that is,

x :=

nSi=1

1 n+1ei ∈ al-int(S)

Indeed, for any y ∈ Rn

and 0 ≤ α ≤ 1, we have(1− α)x + αy =

nSi=1

θi(α)ei where θi(α) := 1−αn+1 + αyi, i = 1, , n

It is readily verified that there exists an α∗ > 0 small enough to guarantee that

θi(α) ≥ 0 for each i and Sn

θi(α) ≤ 1 whenever 0 < α ≤ α∗ This means that(1− α)x + αy ∈ S.10 Keeping in mind that any finite dimensional linear space isisomorphic to Rn for some n (Corollary F.3), we may then conclude: In a finitedimensional linear space, the convex hull of any basis and the origin has nonemptyalgebraic interior (Why exactly?)

∗ ∗ ∗ ∗ FIGURE G.8 ABOUT HERE ∗ ∗ ∗ ∗

As we have observed earlier, if a subset S of a linear space X has a nonemptyalgebraic interior in X, then the affine hull of S must equal the entire space (Ob-servation 1) We shall see shortly (in Examples 7 and 8) that the converse of thisstatement is not true, even for convex sets However, as you are asked to prove inthe next exercise, all goes well in the finite-dimensional case

10 Thanks to (5), it is enough to observe that

x ∈ R n ++ : S n

xi< 1

is open in R n to conclude that al-int(S) = ∅.

Exercise 20 H Let S be a nonempty convex subset of a finite dimensional linear spaceX Prove:

(a) ri(S) = ∅; (b)al-intX(S) =∅ iff aff (S) = X

The following exercise notes that the relative interior of a convex set in a Euclidean space Rn is none other than its interior in its affine hull (when the latter is viewed

E{dpsoh 7 It is obvious that Rn

++ is algebraically open in Rn This might temptone to think that the set R∞

++ of all positive real sequences is also algebraically open

in R∞ This is, however, false Take any (xm)∈ R∞++ and consider the real sequence(ym) := (−x1,−2x2,−3x3, ) Clearly, for any real number α,

(1− α)(xm) + α(ym) = ((1− 2α)x1, (1− 3α)x2, (1− 4α)x3, )

But it is obvious that, for any fixed α > 0, we cannot have (1 − (m + 1)α) > 0 for all

m∈ N It follows that (1 − α)(xm) + α(ym) /∈ R∞

++ for any α > 0 Conclusion:al-intR∞(R∞++) = ∅

How about the relative interior of R∞

++? Surely, that must be nonempty (Afterall, we have seen in Exercise 20 that the relative interior of a nonempty set in Rn hasalways a nonempty relative interior.) But no, this is not the case Indeed, we have

aff (R∞++) = R∞, so Observation 1 tells us that ri(R∞++) = ∅ as well!11 Conclusion:The relative interior of a nonempty convex subset of an infinite dimensional linearspace may well be empty

11 How do I know that aff (R ∞

++ ) = R ∞ ? Because I see that span(R ∞

++ − 1) = R ∞ , where 1 := (1, 1, ), so aff (R ∞

++ ) = R ∞ + 1 = R ∞ Okay, why is span(R ∞

++ − 1) = R ∞ true? Well, take any (x m ) ∈ R ∞ , and define I := {i : x i > 0} and J := N\I If (y m ) and (z m ) are the sequences defined by

++ , this shows that (x m ) ∈ span(R∞++ − 1).

E{dpsoh 8 In Example 6 we have seen that, in a finite dimensional linear space,the convex hull of any basis and the origin has nonempty algebraic interior We nowwish to show that this observation is not true in any infinite dimensional linear space!

So pick any such space X and take any basis A for X (Theorem F.1) Now considerthe set

S :=co(A∪{0}) =

 kSi=1

λixi : k ∈ N, (xi, λi)∈ A × R+, i = 1, , k and

kSi=1

λi ≤ 1

We claim that al-intX(S) = ∅ (You should be somewhat intrigued at this point.After all, we seem to know next to nothing about X.) To see this, take any k ∈ Nand (x1, λ1), , (xk, λk)∈ A × [0, 1] with Sk

λi = 1, and let x := Sk

λixi ∈ S Nowhere is what is funny about infinite dimensional spaces, they possess infinitely manybasis vectors, so, no matter how large k is, we can find an xk+1

∈ A\{x1, , xk

}.(Of course, we could do no such thing (for k > n) in Example 6.) Choose, then,

y := 12x−32xk+1, and observe that, for any α > 0,

(1− α)x + αy =

kSi=1

In any infinite dimensional linear space, there exists a nonempty convex set S withri(S) = ∅

The second diagram of Figure 8 may help clarify what is going on here Consider

a vector like x in that figure This point is expressed by using 0 and e1 which donot constitute a basis for R2.But then we can find a vector, say e2, which is linearlyindependent of e1, and use the line segment through e2 and x to show that x cannot

be in the algebraic interior of co({0, e1, e2}) (We couldn’t do such a thing for thevector x in the first diagram since x was expressed there by using both e1 and e2.)The upshot is that all points in co(A ∪ {0}) in the present example behave just likethe vector x in the second diagram, precisely because in an infinite dimensional spaceevery vector is expressed as a linear combination of finitely many (hence not all) basisvectors with nonzero coefficients This is the gist of infinite dimensionality

An important insight that stems from Example 8 is that, in an infinite dimensionallinear space, algebraically open sets, which have nonempty algebraic interiors bynecessity, are “really, really large.” You should carry this intuition with you at alltimes After all, Example 8 shows that even if we have a convex set S so large that

aff (S) is actually the entire mother space, we are still not guaranteed the existence

of a single algebraic interior point This is, of course, in stark contrast with the finitedimensional case

Exercise 23 Show thatal-intp( p+) =∅ for all 1≤ p < ∞.

Exercise 24 Prove:

al-int∞( ∞+) ={(xm)∈ ∞+ : inf{xm : m∈ N} > 0}

Exercise 25.H Compute the algebraic interior of the positive cone ofC[0, 1]

E{dpsoh 9 For any nonzero linear functional L on a linear space X, consider theopen half space S := {x ∈ X : L(x) > 0} We claim that S is algebraically open in X.Indeed, let x ∈ S and pick any y ∈ X Since L((1 − α)x + αy) = (1 − α)L(x) + αL(y),

by choosing

αy =

+ 1

2, if L(y) ≥ 0L(x)

L(x)−L(y), if L(y) < 0 ,

we find (1 − α)x + αy ∈ S for all 0 ≤ α < αy Thus: Any open halfspace in X isalgebraically open in X Since a halfspace cannot be empty, therefore: The algebraicinterior of an halfspace is not empty, and equals its relative interior

Exercise 26 H For any subsetS of a linear spaceX,show that x∈ al-intX(S)iff, for each y ∈ X, there exists an εy > 0 such that x + εy ∈ S for all ε ∈ R with

|ε| ≤ εy

Exercise 27 H Let S be a convex subset of a linear spaceX Prove:

(a) Ifx∈ al-intX(S),then(1− α)x + αy ∈ al-intX(S) for any(α, y)∈ [0, 1) × S;

(b) Ifal-intX(S) =∅, then aff (S) = X = aff (al-intX(S))

Exercise 28 For any algebraically open subset A of a linear space X, show that

A + λB is algebraically open in X for any (λ, B)∈ R × 2X

The following exercise shows how algebraic interior points of a convex set may be relevant in optimization theory It says that if a concave real map attains its minimum

on the algebraic interior of its domain, then it must be constant.

Exercise 29 LetS be a convex subset of a linear space X Where x∗ ∈ al-intX(S)

and f ∈ RS is a concave function, prove:

(a) For everyx∈ S,there exists an (α, y)∈ (0, 1) × S withx∗ = (1− α)x + αy;

(b) Ifx∗ ∈ arg min{f(x) : x ∈ S},thenf must be a constant function.

1.5 Algebraic Closure of a Set

We now turn to the algebraic formulation of a notion of “closedness” for subsets of alinear space

Dhilqlwlrq Let S be a subset of a linear space X A vector x in X is called analgebraic closure point of S (in X) if, for some y ∈ S, we have

(1− α)x + αy ∈ S for all 0 < α ≤ 1

The set of all algebraic closure points of S (in X) is called the algebraic closure

of S (in X), and is denoted by al-clX(S) (Note al-clX(∅) = ∅.) If al-clX(S) ⊆ S,

we say that S is algebraically closed in X Finally, the set al-clX(S)\al-intX(S) iscalled the algebraic boundary of S (in X), and is denoted by al-bdX(S)

Geometrically speaking, if x is an algebraic closure point of a set S (in a linearspace), we understand that this vector is “reachable” from S in the sense that for atleast one vector y in S the entire line segment between x and y, possibly excludingthe endpoint x, stays within S.12 The algebraic boundary points of S are similarlyinterpreted

Warning By contrast to the notion of algebraic interior, it is not important relative

to which linear space we consider the algebraic closure operator For any subset S

of a linear space X, all that matters for the determination of al-clX(S) is the affinehull of S.13 Consequently, from now on, we will denote al-clX(S) simply by al-cl(S),and if S is algebraically closed in some linear space, then we will simply say that it

is algebraically closed Since the algebraic boundary of a set depends on its algebraicinterior, however, “X” in the notation al-bdX(S) is not superfluous For instance,

if S = {(t, 0) ∈ R2 : 0 < t < 1}, then al-bdR×{0}(S) = {(0, 0), (1, 0)} while

al-bdR2(S) = [0, 1]× {0} (Notice that the algebraic closure of S is [0, 1] × {0} whether

we consider S as a subset of R × {0} or R2.)

The following is a reflection of Examples 4 and 5 (Throughout the followingexample, we denote the closure of S in Rn simply by cl(S).)

E{dpsoh 10 Given any n ∈ N, consider any nonempty subset S of Rn

Question: How does al-cl(S) relate to cl(S)?

Answer: cl(S) is surely larger than al-cl(S), but the converse well, it depends!(Sounds familiar?) Indeed, if x ∈ al-cl(S), then there exists a y ∈ S such that

The converse is false, however For instance, Q is an algebraically closed set in Rwhich is not closed (Think about it!) Yet, the converse holds, provided that S isconvex Given the analysis of the previous subsection, this should not be terribly

12 For this reason, some authors refer to the algebraic closure points of S as the points that are linearly accessible from S, and denote the algebraic closure of S by lin(S).

13 That is, for any linear spaces X and Y that contain aff (S), we have x ∈ al-cl X (S) iff x ∈

al-clY(S) (Why?)

surprising to you The proof is a little subtle, however It is based on the followingresult of Euclidean analysis.

Claim If S is convex and X := aff (S), then

(1− λ)cl(S) + λintX(S)⊆ S for any 0 < λ ≤ 1.14Proof of Claim Take any x ∈ cl(S) and y ∈ intX(S), and fix an arbitrary

1−λ

(intX(S)− y) is an open subset of X that contains

0 (Why?) Thus S + λ

1−λ

(intX(S)− y) is an open subset of X that contains

clX(S) = cl(S) (Why?) So,

x∈ S + λ

1−λ

(intX(S)− y) ⊆ S + λ

1−λ

(S− y),and it follows that (1 − α)x + αy ∈ S

Let us now go back to the task at hand Assume that S is convex and x ∈ cl(S) Ofcourse, intX(S) = ∅ (because intX(S) = ri(S) and the relative interior of a nonemptyconvex set in Rn has always a nonempty relative interior — recall Exercises 20 and21) Then pick any y ∈ intX(S),and use Claim 1 to conclude that (1 − λ)x + λy ∈ Sfor any 0 < λ ≤ 1 This shows that x ∈ al-cl(S) Conclusion:

cl(S) = al-cl(S) for any convex set S in Rn.(Compare with (5).)

We have seen earlier that the notions of openness and algebraic openness coincidefor convex subsets of any Euclidean space By Example 10, we now know that thesame is true for the notions of closedness and algebraic closedness as well

Observation 3 Given any n ∈ N, a convex subset S of Rn

is closed in Rn if, andonly if, it is algebraically closed

Exercise 30.H LetS be a subset of a linear space X

(a) Show that if S is algebraically open in X, then X\S is algebraically closed in

X,but not conversely.

(b) Prove: Provided thatS is a convex set, it is algebraically open inX iff X\S is algebraically closed inX

14 In fact more is true Since (1 − λ)cl(S) + λint X (S) is an open subset of X (why?), this result entails that (1 − λ)cl(S) + λint X (S) ⊆ int X (S) for any 0 < λ ≤ 1.

Exercise 31 Let X be a linear space and L ∈ L(X, R) Prove or disprove: For any α ∈ R, either of the closed halfspaces induced by the hyperplane L−1(α) is algebraically closed.

Exercise 32 Show that every linear subspace of a linear space is algebraically closed Conclude that any linear subspace of a Euclidean space is closed.

1.6 Finitely Generated Cones

In convex analysis one is frequently confronted with the problem of determiningwhether or not a given conical (or convex) hull of a set is algebraically closed Thisissue is usually settled within the specific context under study; there are only fewgeneral results that ascertain the algebraic closedness of such sets Indeed, the conicalhulls of even very well behaved sets may fail to be algebraically closed For instance,

in R2, consider the closed ball around (1, 0) with radius 1, that is, the set S of all

The situation is quite satisfactory for conical hulls of finite sets, however Indeed,

it turns out that, while compactness of a set in a Euclidean space may not be enough

to guarantee that its conical hull is closed, its finiteness is In fact, we will provehere the stronger result that the conical hull of a finite set in any linear space isalgebraically closed

Let us first agree on some terminology

Dhilqlwlrq Let X be a linear space and C a convex cone in X We say that C isgenerated byS ⊆ X if C = cone(S) If C is generated by a finite subset of X, then

it is called finitely generated Finally, we say that C is basic if it is generated by

a linearly independent subset of X

The main result we wish to prove here is the following

Theorem 1 Every finitely generated convex cone in a linear space X is algebraicallyclosed

Since a convex subset of a Euclidean space is closed iff it is algebraically closed(Observation 3), the following fact is an immediate consequence of Theorem 1

Corollary 1 For any n ∈ N, every finitely generated convex cone in Rn is a closedset

The rest of this section is devoted to proving Theorem 1 While this is quite aninnocent looking claim, we will have to put some work to prove it It is actually a

good idea to stick around for this proof, because we shall divide the argument intothree steps, and each of these steps contains a result that is of interest in its ownright For instance, how about the following?

Lemma 1 Every basic convex cone in a linear space X is algebraically closed

Proof Let S be a linearly independent subset of X, and C := cone(S) If S = ∅,then C = {0} so the claim is trivial Assume then S = ∅, and take any x ∈ al-cl(C)

We wish to show that x ∈ C If x = 0, there is nothing to prove, so let x = 0 Ofcourse, x ∈ span(S) =: Y.15 So, since S is a basis for Y, there exist a unique nonemptyfinite subset A of S and a λ : A → R\{0} such that x =S

y∈Aλ(y)y(Corollary F.2)

We wish to complete the proof by showing that λ(A) ⊆ R+ — this would mean x ∈ C.Since x ∈ al-cl(C), there exists a z ∈ C such that xm := (1 − m1)x + m1z ∈ C foreach m = 1, 2, If z = 0, all is trivial, so we let z = 0 Then, there exist a uniquenonempty finite subset B of S and a γ : B → R\{0} such that z = S

y∈Bγ(y)y,and since z ∈ C, we have γ(B) ⊆ R+ Now each xm can be expressed as a linearcombination of finitely many vectors in S with nonzero coefficients in only one way(Corollary F.2) Moreover, we have, for each m,

xm = 

1−m1

 Sy∈A

λ(y)y + m1 S

y∈Bγ(y)y

1−m1

 Sy∈A\B

y + m1 Sy∈B\Aγ(y)y

with the convention that any of the terms on the right hand side equals 0 if it is

a sum over the empty set So, since each xm

∈ C, we must have λ(y) ≥ 0 for all

y∈ A\B and 

1− m1

λ(y) + m1γ(y)≥ 0 for all y ∈ A ∩ B and m ∈ N Thus, letting

m→ ∞ shows that λ(y) ≥ 0 for all y ∈ A, so we conclude that λ(A) ⊆ R+

Lemma 2 Let S be a subset of a linear space X, and I(S) the class of all linearlyindependent subsets of S in X Then,

cone(S) =V

{cone(T ) : T ∈ I(S)}

Proof.16

Since cone(∅) = {0}, the claim is trivial if S = ∅ — we thus assume

S = ∅ Moreover, we only need to prove the ⊆ part of the claimed equation Takeany x ∈ cone(S), and observe that if x = 0, we are done trivially We thus assume

15 First verify the claim by using the definition of the algebraic closure operator Then reflect on why I started the sentence by saying “Of course ” (Hint the span of a set = the span of the algebraic closure of that set.)

16 The proof is so similar to that of Carathéodory’s Theorem that I have contemplated quite a bit about leaving it as an exercise I decided at the end not to do so, but you might want to have a quick look at Example F.4 first, and then write down your own proof.

x = 0 Clearly, there exists a nonempty finite subset T of S such that x ∈ cone(T ).(Right?) Let Tx be a ⊇-minimal such T Then there exists a λ ∈ RT

We may assume that α(y) > 0 for some y ∈ Tx (for otherwise we would work with

−α) Then A := {y ∈ Tx : α(y) > 0} = ∅, and we may define

θ := minq

λ(y) α(y) : y ∈ Ar

and C := {y ∈ Tx : λ(y)− θα(y) = 0}

Clearly, θ > 0 and C = ∅ Moreover,

x = Sy∈T x

λ(y)y− θ S

y∈T x

α(y)y = S

y∈T x \C(λ(y)− θα(y)) y

But by the choice of θ, we have λ(y) − θα(y) ≥ 0 for all y ∈ Tx, so x ∈ cone(Tx\C).Since C = ∅, this contradicts the ⊇-minimality of Tx,and the proof is complete

Combining Lemmas 1 and 2, we see that every finitely generated convex cone in

a linear space equals the union of finitely many algebraically closed convex cones.Theorem 1 thus readily follows from the following fact

Lemma 3 Let A be a nonempty finite collection of convex subsets of a linear space

∈ V

A such that (1 − α)x + αy1

∈ VAfor all 0 < α ≤ 1 Suppose y1 ∈ A1, where A1 ∈ A If (1 − α)x + αy1 ∈ A1 for all

0 < α ≤ 1, then x ∈ al-cl(A1) and we are done So suppose instead that there is avector y2 ∈ A/ 1 on the line segment between x and y1 which belongs to some othermember, say A2,of A Clearly, no vector on the line segment between x and y2 maybelong to A1 (Since A1 is convex, and y2 is on the line segment between any suchvector and y1, we would otherwise have y2

∈ A.) Moreover, if (1 − α)x + αy2

∈ A2 forall 0 < α ≤ 1, then x ∈ al-cl(A2)and we are done Suppose this is not the case Thenthere is a vector y3 ∈ A/ 1∪A2 on the line segment between x and y2.(Again, no vector

on the line segment between x and y3 may belong to A1∪ A2.) If (1 − α)x + αy3

∈ A3for all 0 < α ≤ 1, then x ∈ al-cl(A3) and we are done Otherwise, there exists avector y4 ∈ A/ 1∪ A2∪ A3 on the line segment between x and y3 Since A is finite, bycontinuing this way we are bound to find a member B of A that contains a vector ysuch that (1 − α)x + αy ∈ B for every 0 < α ≤ 1

Exercise 33.H Prove: IfAis a nonempty collection of convex subsets of a linear space

This section contains the main results of this chapter We first study the problem ofextending a given linear functional, which is defined on a subspace of a linear spaceand which satisfies a certain property, to the entire space in a way that preservesthat property The spirit of this problem is analogous to the continuous extensionproblem we studied in Section D.7 However, unlike that problem, it is intimatelylinked to the geometric problem of separating a convex set from a point in its exterior

by means of a hyperplane This connection is crucial, as the latter problem arises inmany economic and optimization-theoretic applications

In what follows, we first study these linear extension and separation problems

in the context of an arbitrary linear space The important, but substantially morespecial, setting of Euclidean spaces is taken up in Section 3

2.1 Extension of Linear Functionals

To get a sense of the basic linear extension problem that we wish to study here, take aEuclidean space Rn

with n > 1, and observe that Y := Rn−1×{0} is a linear subspace

of Rn (which is obviously isomorphic (and homeomorphic) to Rn−1) Suppose that

we are given two linear functionals L1 and L2 on Rn and a linear functional L on Ysuch that

L1(x)≤ L2(x) for all x ∈ Rn+and

L1(x)≤ L(x) ≤ L2(x) for all x ∈ Y ∩ Rn+.The question is this: Can we find a linear functional L∗ : Rn

→ R such that L∗(x) =L(x) for all x ∈ Y (that is, L∗|Y = L) and L1 ≤ L∗ ≤ L2 on Rn

+ This is a simpleinstance of the basic linear extension problem that we tackle below Before moving

on to the general discussion, however, let us observe that our little problem here can

be answered very easily, thanks to the finite dimensionality of Rn We know thatthere exists a vector αj ∈ Rn such that Lj(x) =Sn

αjixi for all x ∈ Rn and j = 1, 2(Example F.6) Since L1 ≤ L2 on Rn

+,it is plain that we have α1

n ≤ α2

n On the otherhand, there also exists a vector α ∈ Rn−1 such that L(x) =Sn−1

αixi for all x ∈ Y.Therefore, for any α1n ≤ αn≤ α2n,the map L∗ : Rn→ R defined by L∗(x) :=Sn

αixi

satisfies all of the properties we seek — any such map is a linear functional on Rn thatextends L and satisfies L1 ≤ L∗ ≤ L2 on Rn

+.This is a typical example which shows that linear extension problems are oftensettled relatively easily in finite dimensional linear spaces, because we have a com-plete characterization of the linear functionals defined on any such space While thesituation is quite different in the infinite dimensional case, there are still powerful ex-tension theorems that apply to arbitrary linear spaces The first result of this sectionwill provide an excellent case in point

We begin by introducing some preliminary concepts

Dhilqlwlrq Let X be a linear space We say that ϕ ∈ RX is positively neousif

homoge-ϕ(αx) = αϕ(x) for all x ∈ X and α > 0,and subadditive if

ϕ(x + y)≤ ϕ(x) + ϕ(y) for all x, y ∈ X

−ϕ(x − y) ≤ ϕ(x) − ϕ(y) ≤ ϕ(y − x) for all x, y∈ X

The following is a fundamental principle of linear analysis It is difficult to phasize its importance

overem-The Hahn-Banach Extension overem-Theorem 1.17 Let X be a linear space and ϕ apositively homogeneous and subadditive real map on X If L is a linear functional

on a linear subspace Y of X such that L(y) ≤ ϕ(y) for all y ∈ Y, then there exists

an L∗ ∈ L(X, R) such that

L∗(y) = L(y) for all y ∈ Y and L∗(x)≤ ϕ(x) for all x ∈ X

17 There are a variety of results that go with the name of “the” Hahn-Banach Extension Theorem —

I present two such results in this book We owe these results to the 1927 contribution of Hans Hahn and the monumental 1932 work of Stefan Banach, who is justly viewed by many as the “father” of functional analysis Indeed, Banach’s book (which is essentially his 1920 dissertation) brought this topic to the fore of mathematical analysis, and subsequent work established it as one of the major fields in mathematics at large.

Proof Take a linear subspace Y of X, and fix an L ∈ L(Y, R) with L ≤ ϕ|Y If

Y = X there is nothing to prove, so assume that there exists a vector z in X\Y Wewill first show that L can be extended to a linear functional K defined on the linearsubspace Z := span(Y ∪ {z}) such that K(x) ≤ ϕ(x) for all x ∈ Z For any x ∈ Z,there exist a unique yx ∈ Y and λx ∈ R such that x = yx+ λxz (Why unique?) So

K is a linear extension of L to Z iff

where α := K(z) All we need to do is to choose a value for α which would ensurethat K(x) ≤ ϕ(x) for all x ∈ Z Clearly, such an α exists iff

L(y) + λα≤ ϕ(y + λz) for all y ∈ Y and λ ∈ R\{0} (8)

On the other hand, given the positive homogeneity of ϕ and linearity of L, this isequivalent to say that, for any y ∈ Y,

L1

λy+ α≤ ϕ1

Put differently, (8) holds iff L(u) + α ≤ ϕ(u + z) and L(v) − α ≤ ϕ(v − z) for all

u, v∈ Y (Yes?) Our objective is thus to pick an α ∈ R such that

L(v)− ϕ(v − z) ≤ α ≤ ϕ(u + z) − L(u) for all u, v ∈ Y (9)Clearly, we can do this iff the sup of the left-hand-side of this expression (over all

v ∈ Y ) is smaller than the inf of its right-hand-side That is, (9) holds for some

α∈ R iff L(u) + L(v) ≤ ϕ(u + z) + ϕ(v − z) for all u, v ∈ Y But then we’re just fine,because, given that L is linear, ϕ is subadditive, and L ≤ ϕ|Y,we have

L(u) + L(v) = L(u + v)≤ ϕ(u + v) ≤ ϕ(u + z) + ϕ(v − z) for all u, v ∈ Y

So, tracing our steps back, we see that there exists a K ∈ L(Z, R) with K|Y = Land

K ≤ ϕ|Z

If you recall how we proved Theorem F.1 by Zorn’s Lemma, you know what’scoming next.18

Let L stand for the set of all K ∈ L(W, R) with K|Y = L and

K ≤ ϕ|W, for some linear subspace W of X with Y ⊂ W By what we just proved,

L = ∅ Define next the binary relation on L as

K1 K2 if and only if K1 is an extension of K2

18 The use of Zorn’s Lemma (and thus the Axiom of Choice) cannot be avoided in the proof of the Hahn-Banach Extension Theorem 1 The same goes for all major theorems that are proved in this section However, given my “applied” slant here, I will not worry about proving these claims here.

It is easily verified that (L, ) is a poset and that any loset in (L, ) has an upperbound (which is a linear functional defined on the union of all the domains of themembers of the family) Then, by Zorn’s Lemma, there must exist a maximal element

L∗ in (L, ) L∗must be defined on the entire X, otherwise we could extend it further

by using the argument outlined in the previous paragraph, which would contradictthe maximality of L∗ Since L∗ ∈ L, we are done

Corollary 2 Every linear functional defined on a linear subspace of a linear space

X can be extended to a linear functional on X

Exercise 36 Prove Corollary 2.

Even though this may not be readily evident to you, the Hahn-Banach ExtensionTheorem 1 is of great importance for linear analysis Rest assured that this pointwill become abundantly clear as we develop the theory further As for an immediateremedy, we offer the following example which illustrates how one may need this result

in answering even some of the very basic questions about linear functionals

E{dpsoh 11 (Existence of Linear Functionals) We have seen plenty of examples

of linear functionals in the previous chapter, but did not worry about the followingfundamental question:

Is it true that in any nontrivial linear space, there exists a nonzero linearfunctional ?

Think about it, this is not all that trivial — the arbitrariness of the linear space makes

it impossible to write down a formula for the sought linear functional Fortunately,this does not hamper our ability to prove the existence of such a functional, and theHahn-Banach Extension Theorem 1 provides a very easy way of doing this

Let X be a nontrivial linear space, and pick any x ∈ X\{0} Let Y := {λx : λ ∈R} and define L ∈ RY by L(λx) := λ It is easy to see that L ∈ L(Y, R) (Yes?)That’s it! Since Y is a linear subspace of X, we now simply apply Corollary 2 toextend L to a linear functional defined on X, thereby establishing the existence of anonzero linear functional on X

Observe that this is not a constructive finding in the sense that it doesn’t provide

us with a formula for the linear functional we seek All is rather based on the Banach Extension Theorem 1 (and hence, indirectly, on the Axiom of Choice) This

Hahn-is a prototypical, if trivial, example of what thHahn-is theorem can do for you.19

19 The use of the Hahn-Banach Extension Theorem 1 is actually an overkill here — I only wished

to give a general feeling for what sort of things can be done with this result Indeed, the existence of nonzero linear functionals can be proved without invoking Corollary 2 (while the use of the Axiom

∗ ∗ ∗ ∗ FIGURE G. 7 ABOUT HERE ∗ ∗ ∗ ∗The following is an immediate... single algebraic interior point This is, of course, in stark contrast with the finitedimensional case

Exercise... ∈ al-cl X (S) iff x ∈

al-clY(S) (Why?)

surprising