fixed point algorithms for inverse problems in science and engineering bauschke, burachik, combettes, elser, luke wolkowicz 2011 06 01 Cấu trúc dữ liệu và giải thuật

1 Chebyshev Sets, Klee Sets, and Chebyshev Centers with Respect to Bregman Distances: Recent Results and Open Problems.. Chapter 1Chebyshev Sets, Klee Sets, and Chebyshev Centers with Re

Fixed-Point Algorithms for Inverse Problems

in Science and Engineering

For further volumes:

http://www.springer.com/series/7393

Springer Optimization and Its Applications

J Birge (University of Chicago)

C.A Floudas (Princeton University)

F Giannessi (University of Pisa)

H.D Sherali (Virginia Polytechnic and State University)

T Terlaky (McMaster University)

Y Ye (Stanford University)

Aims and Scope

Optimization has been expanding in all directions at an astonishing rate ing the last few decades New algorithmic and theoretical techniques have been developed, the diffusion into other disciplines has proceeded at a rapid pace, and our knowledge of all aspects of the field has grown even more profound At the same time, one of the most striking trends in optimization

dur-is the constantly increasing emphasdur-is on the interddur-isciplinary nature of the field Optimization has been a basic tool in all areas of applied mathematics, engineering, medicine, economics and other sciences.

The series Springer Optimization and Its Applications publishes

under-graduate and under-graduate textbooks, monographs and state-of-the-art tory works that focus on algorithms for solving optimization problems and also study applications involving such problems Some of the topics cov- ered include nonlinear optimization (convex and nonconvex), network flow problems, stochastic optimization, optimal control, discrete optimization, multi-objective programming, description of software packages, approxima- tion techniques and heuristic approaches.

exposi-Heinz H Bauschke • Regina S Burachik

Editors

Fixed-Point Algorithms for Inverse Problems in Science and Engineering

ABC

Heinz H Bauschke

Department of Mathematics and Statistics

University of British Columbia

School of Mathematics & Statistics

Division of Information Technology

Engineering & the Environment

University of South Australia

Mawson Lakes Campus

Universit´e Pierre et Marie Curie

Laboratoire Jacques-Louis Lions

Cornell UniversityClark Hall14853–2501 Ithaca, New YorkUSA

ve10@cornell.edu

D Russell LukeInstitut f¨ur Numerische und AngewandteMathematik

Universit¨at G¨ottingenLotzestr 16-18, 37073 G¨ottingenGermany

r.luke@math.uni-goettingen.de

Henry WolkowiczDepartment of Combinatorics

& OptimizationFaculty of MathematicsUniversity of WaterlooWaterloo, OntarioCanada

hwolkowicz@uwaterloo.ca

ISSN 1931-6828

ISBN 978-1-4419-9568-1 e-ISBN 978-1-4419-9569-8

DOI 10.1007/978-1-4419-9569-8

Springer New York Dordrecht Heidelberg London

Library of Congress Control Number: 2011928237

c

 Springer Science+Business Media, LLC 2011

All rights reserved This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York,

NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis Use in connection with any form of information storage and retrieval, electronic adaptation, computer software,

or by similar or dissimilar methodology now known or hereafter developed is forbidden.

The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject

to proprietary rights.

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com)

This book brings together 18 carefully refereed research and review papers in thebroad areas of optimization and functional analysis, with a particular emphasis ontopics related to fixed-point algorithms The volume is a compendium of topics

presented at the Interdisciplinary Workshop on Fixed-Point Algorithms for Inverse Problems in Science and Engineering, held at the Banff International Research Sta-

tion for Mathematical Innovation and Discovery (BIRS), on November 1–6, 2009.Forty experts from around the world were invited Participants came from Australia,Austria, Brazil, Bulgaria, Canada, France, Germany, Israel, Japan, New Zealand,Poland, Spain, and the United States

Most papers in this volume grew out of talks delivered at this workshop, althoughsome contributions are from experts who were unable to attend We believe that thereader will find this to be a valuable state-of-the-art account on emerging directionsrelated to first-order fixed-point algorithms

The editors thank BIRS and their sponsors – Natural Sciences and EngineeringResearch Council of Canada (NSERC), US National Science Foundation (NSF),Alberta Science Research Station (ASRA), and Mexico’s National Council for Sci-ence and Technology (CONACYT) – for their financial support in hosting theworkshop, and Wynne Fong, Brent Kearney, and Brenda Williams for their help

in the preparation and realization of the workshop We are grateful to Dr MasonMacklem for his valuable help in the preparation of this volume Finally, we thankthe dedicated referees who contributed significantly to the quality of this volumethrough their instructive and insightful reviews

December 2010

v

1 Chebyshev Sets, Klee Sets, and Chebyshev Centers

with Respect to Bregman Distances: Recent Results

and Open Problems 1Heinz H Bauschke, Mason S Macklem, and Xianfu Wang

2 Self-Dual Smooth Approximations of Convex Functions

via the Proximal Average 23

Heinz H Bauschke, Sarah M Moffat, and Xianfu Wang

3 A Linearly Convergent Algorithm for Solving a Class

of Nonconvex/Affine Feasibility Problems 33

Amir Beck and Marc Teboulle

4 The Newton Bracketing Method for Convex Minimization:

Convergence Analysis 49

Adi Ben-Israel and Yuri Levin

5 Entropic Regularization of the0Function 65

Jonathan M Borwein and D Russell Luke

6 The Douglas–Rachford Algorithm in the Absence

of Convexity 93

Jonathan M Borwein and Brailey Sims

7 A Comparison of Some Recent Regularity Conditions

for Fenchel Duality 111

Radu Ioan Bot¸ and Ern¨o Robert Csetnek

8 Non-Local Functionals for Imaging 131

J´erˆome Boulanger, Peter Elbau, Carsten Pontow,

and Otmar Scherzer

vii

viii Contents

9 Opial-Type Theorems and the Common Fixed Point

Problem 155

Andrzej Cegielski and Yair Censor

10 Proximal Splitting Methods in Signal Processing 185

Patrick L Combettes and Jean-Christophe Pesquet

11 Arbitrarily Slow Convergence of Sequences of Linear

Operators: A Survey 213

Frank Deutsch and Hein Hundal

12 Graph-Matrix Calculus for Computational Convex

Analysis 243

Bryan Gardiner and Yves Lucet

13 Identifying Active Manifolds in Regularization Problems 261

W.L Hare

14 Approximation Methods for Nonexpansive Type Mappings

in Hadamard Manifolds 273

Genaro L´opez and Victoria Mart´ın-M´arquez

15 Existence and Approximation of Fixed Points of Bregman

Firmly Nonexpansive Mappings in Reflexive Banach

Spaces 301

Simeon Reich and Shoham Sabach

16 Regularization Procedures for Monotone Operators:

Recent Advances 317

J.P Revalski

17 Minimizing the Moreau Envelope of Nonsmooth

Convex Functions over the Fixed Point Set of Certain

Quasi-Nonexpansive Mappings 345

Isao Yamada, Masahiro Yukawa, and Masao Yamagishi

18 The Br´ezis-Browder Theorem Revisited and Properties

of Fitzpatrick Functions of Order n 391

Liangjin Yao

Heinz H Bauschke Department of Mathematics, Irving K Barber School,

University of British Columbia, Kelowna, B.C V1V 1V7, Canada,

heinz.bauschke@ubc.ca

Amir Beck Department of Industrial Engineering, Technion, Israel Institute

of Technology, Haifa 32000, Israel,becka@ie.technion.ac.il

Adi Ben-Israel RUTCOR – Rutgers Center for Operations Research, Rutgers

University, 640 Bartholomew Road, Piscataway, NJ 08854-8003, USA,

adi.benisrael@gmail.com

Jonathan M Borwein CARMA, School of Mathematical and Physical Sciences,

University of Newcastle, NSW 2308, Australia,

jonathan.borwein@newcastle.edu.au

Radu Ioan Bot¸ Faculty of Mathematics, Chemnitz University of Technology,

09107 Chemnitz, Germany,radu.bot@mathematik.tu-chemnitz.de

J´erˆome Boulanger Johann Radon Institute for Computational and Applied

Mathematics, Austrian Academy of Sciences, Altenbergerstraße 69, 4040 Linz,Austria,jerome.boulanger@ricam.oeaw.ac.at

Andrzej Cegielski Faculty of Mathematics, Computer Science and Econometrics,

University of Zielona G´ora, ul Szafrana 4a, 65-514 Zielona G´ora, Poland,

a.cegielski@wmie.uz.zgora.pl

Yair Censor Department of Mathematics, University of Haifa, Mt Carmel, Haifa

31905, Israel,yair@math.haifa.ac.il

Patrick L Combettes UPMC Universit´e Paris 06, Laboratoire Jacques-Louis

Lions – UMR CNRS 7598, 75005 Paris, France,plc@math.jussieu.fr

Ern¨o Robert Csetnek Faculty of Mathematics, Chemnitz University of

Technology, 09107 Chemnitz, Germany,

robert.csetnek@mathematik.tu-chemnitz.de

Frank Deutsch Department of Mathematics, Pennsylvania State University,

University Park, PA 16802, USA,deutsch@math.psu.edu

ix

x Contributors

Peter Elbau Johann Radon Institute for Computational and Applied Mathematics,

Austrian Academy of Sciences, Altenbergerstraße 69, 4040 Linz, Austria,

peter.elbau@ricam.oeaw.ac.at

Bryan Gardiner Computer Science, I K Barber School, University of British

Columbia Okanagan, Kelowna, B.C V1V 1V7, Canada,

khumba@interchange.ubc.ca

W L Hare Department of Mathematics and Statistics, UBC Okanagan Campus,

Kelowna, B.C V1V 1V7, Canada,warren.hare@ubc.ca

Hein Hundal 146 Cedar Ridge Drive, Port Matilda, PA 16870, USA,

hundalhh@yahoo.com

Yuri Levin School of Business, Queen’s University, 143 Union Street, Kingston,

ON K7L 3N6, Canada,ylevin@business.queensu.ca

Genaro L´opez Department of Mathematical Analysis, University of Seville,

41012 Seville, Spain,glopez@us.es

Yves Lucet Computer Science, I K Barber School, University of British

Columbia Okanagan, Kelowna, B.C V1V 1V7, Canada,yves.lucet@ubc.ca

D Russell Luke Institut f¨ur Numerische und Angewandte Mathematik

Universit¨at G¨ottingen, Lotzestr 16-18, 37073 G¨ottingen, Germany

r.luke@math.uni-goettingen.de

Mason S Macklem Department of Mathematics, Irving K Barber School,

University of British Columbia, Kelowna, B.C V1V 1V7, Canada,

mason.macklem@ubc.ca

Victoria Mart´ın-M´arquez Department of Mathematical Analysis, University of

Seville, 41012 Seville, Spain,victoriam@us.es

Sarah M Moffat Department of Mathematics, Irving K Barber School,

University of British Columbia, Kelowna, B.C V1V 1V7, Canada,

sarah.moffat@ubc.ca

J.-C Pesquet Laboratoire d’Informatique Gaspard Monge, UMR CNRS 8049,

Universit´e Paris-Est, 77454 Marne la Vall´ee Cedex 2, France,

jean-christophe.pesquet@univ-paris-est.fr

Carsten Pontow Department of Mathematics, University Innsbruck, Technikerstr.

21a, 6020 Innsbruck, Austria,Carsten.Pontow@uibk.ac.at

Simeon Reich Department of Mathematics, The Technion – Israel Institute of

Technology, 32000 Haifa, Israel,sreich@tx.technion.ac.il

J.P Revalski Institute of Mathematics and Informatics, Bulgarian Academy of

Sciences, Acad G Bonchev Street, block 8, 1113 Sofia, Bulgaria,

revalski@math.bas.bg

Shoham Sabach Department of Mathematics, The Technion – Israel Institute of

Technology, 32000 Haifa, Israel,ssabach@tx.technion.ac.il

Contributors xi

Otmar Scherzer Computational Science Center, University Vienna, Nordbergstr.

15, 1090 Vienna, Austria, and Johann Radon Institute for Computational andApplied Mathematics, Austrian Academy of Sciences, Altenbergerstraße 69,

4040 Linz, Austria,otmar.scherzer@univie.ac.at

Brailey Sims CARMA, School of Mathematical and Physical Sciences, University

of Newcastle, NSW 2308, Australia,brailey.sims@newcastle.edu.au

Marc Teboulle School of Mathematical Sciences, Tel Aviv University, Tel Aviv

69978, Israel,teboulle@post.tau.ac.il

Xianfu Wang Department of Mathematics, Irving K Barber School, University

of British Columbia, Kelowna, B.C V1V 1V7, Canada,shawn.wang@ubc.ca

Isao Yamada Department of Communications and Integrated Systems, Tokyo

Institute of Technology, S3-60, Tokyo, 152-8550 Japan,isao@sp.ss.titech.ac.jp

Masao Yamagishi Department of Communications and Integrated Systems,

Tokyo Institute of Technology, S3-60, Tokyo, 152-8550 Japan,

myamagi@sp.ss.titech.ac.jp

Liangjin Yao Department of Mathematics, Irving K Barber School, University of

British Columbia, Kelowna, B.C V1V 1V7, Canada,ljinyao@interchange.ubc.ca

Masahiro Yukawa Department of Electrical and Electronic Engineering,

Niigata University, 8050 Ikarashi Nino-cho, Nishi-ku, Niigata, 950-2181 Japan,

yukawa@eng.niigata-u.ac.jp

Chapter 1

Chebyshev Sets, Klee Sets, and Chebyshev

Centers with Respect to Bregman Distances: Recent Results and Open Problems

Heinz H Bauschke, Mason S Macklem, and Xianfu Wang

Abstract In Euclidean spaces, the geometric notions of nearest-points map,

farthest-points map, Chebyshev set, Klee set, and Chebyshev center are well knownand well understood Since early works going back to the 1930s, tremendous theo-retical progress has been made, mostly by extending classical results from Euclideanspace to Banach space settings In all these results, the distance between points isinduced by some underlying norm Recently, these notions have been revisitedfrom a different viewpoint in which the discrepancy between points is measured

by Bregman distances induced by Legendre functions The associated frameworkcovers the well known Kullback–Leibler divergence and the Itakura–Saito distance

In this survey, we review known results and we present new results on Klee setsand Chebyshev centers with respect to Bregman distances Examples are providedand connections to recent work on Chebyshev functions are made We also identifyseveral intriguing open problems

Keywords Bregman distance · Chebyshev center · Chebyshev function

· Chebyshev point of a function · Chebyshev set · Convex function · Farthest

point · Fenchel conjugate · Itakura–Saito distance · Klee set · Klee function

· Kullback–Leibler divergence · Legendre function · Nearest point · Projection

AMS 2010 Subject Classification: Primary 41A65; Secondary 28D05, 41A50,

H.H Bauschke et al (eds.), Fixed-Point Algorithms for Inverse Problems in Science

and Engineering, Springer Optimization and Its Applications 49,

DOI 10.1007/978-1-4419-9569-8 1, cSpringer Science+Business Media, LLC 2011

1

2 H.H Bauschke et al.

1.1 Introduction

1.1.1 Legendre Functions and Bregman Distances

Throughout, we assume that

X= Rnis the standard Euclidean space with inner product·,·, (1.1)

with induced norm·: x →x,x, and with metric (x,y) → x−y In addition,

it is assumed that

f : X → ]−∞,+∞] is a convex function of Legendre type, (1.2)

also referred to as a Legendre function We assume the reader is familiar with basicresults and standard notation from Convex Analysis; see, e.g., [33,34,40] In par-

ticular, f ∗ denotes the Fenchel conjugate of f , and int dom f is the interior of the domain of f For a subset C of X , C stands for the closure of C, convC for the con- vex hull of C, andιC for the indicator function of C, i.e.,ιC (x) = 0, if x ∈ C and

Note that U= Rninss(i), whereas U= Rn

++in(ii)and(iii).

Further examples of Legendre functions can be found in, e.g., [2,5,12,33]

1 Here and elsewhere, inequalities between vectors in Rnare interpreted coordinate-wise.

1 Chebyshev Sets, Klee Sets, and Chebyshev Centers 3

Fact 1.2 (Rockafellar) (See [33, Theorem 26.5].) The gradient map ∇f is a continuous bijection between int dom f and int dom f ∗, with continuous inverse map(∇f)−1=∇f ∗ Furthermore, f ∗is also a convex function of Legendre type

Given x ∈ U and C ⊆ U, it will be convenient to write

ancy between points in U

Definition 1.3 (Bregman distance) (See [13,15,16].) The Bregman distance with respect to f , written D f or simply D, is the function

D : X ×X → [0,+∞] : (x,y) →



f (x) − f (y) − ∇f (y),x − y, if y ∈ U;

Fact 1.4 (See [2, Proposition 3.2(i) and Theorem 3.7(iv) and (v)].) Let x and y be

in U Then the following hold:

(i) D f (x,y) = f (x) + f ∗ (y ∗ ) − y ∗ ,x = D f ∗ (y ∗ ,x ∗)

(ii) D f (x,y) = 0 ⇔ x = y ⇔ x ∗ = y ∗ ⇔ D f ∗ (x ∗ ,y ∗) = 0

Example 1.5 The Bregman distances corresponding to the Legendre functions of

Example1.1between two points x and y in X are as follows:

4 H.H Bauschke et al.

From now on, we assume that C is a subset of X such that

The power set (the set of all subsets) of C is denoted by 2 C

We are now in a position to introduce the various geometric notions

1.1.2 Nearest Distance, Nearest Points, and Chebyshev Sets

Definition 1.6 (Bregman nearest-distance function and nearest-points map).

The left Bregman nearest-distance function with respect to C is

Definition 1.7 (Chebyshev sets) The set C is a left Chebyshev set with respect

to the Bregman distance, or simply← −

D-Chebyshev, if for every y ∈ U, ← − P C (y) is a singleton Similarly, the set C is a right Chebyshev set with respect to the Bregman

distance, or simply− →

D-Chebyshev, if for every x ∈ U, − → P C (x) is a singleton.

Remark 1.8 (Classical Bunt-Motzkin result) Assume that f is the halved energy as

in Example1.1(i) Since the halved Euclidean distance squared (see Example1.5(i))

is symmetric, the left and right (Bregman) nearest distances coincide, as do the

corresponding nearest-point maps Furthermore, the set C is Chebyshev if and only

2 This operator, which has turned out to be quite useful in Optimization and which has found many applications (for a recent one, see [32]), is often referred to as the Bregman projection.

1 Chebyshev Sets, Klee Sets, and Chebyshev Centers 5

if for every z ∈ X, the metric3projection P C (z) is a singleton It is well known that

if C is convex, then C is Chebyshev In the mid-1930s, Bunt [14] and Motzkin [28]showed independently that the following converse holds:

For other works in this direction, see, e.g., [1,9 11,17,22,24,25,35–37] It is stillunknown whether or not (1.13) holds in general Hilbert spaces We review corre-sponding results for the present Bregman setting in Sect.1.3

1.1.3 Farthest Distance, Farthest Points, and Klee Sets

Definition 1.9 (Bregman farthest-distance function and farthest-points map).

The left Bregman farthest-distance function with respect to C is

D-Klee, if for every x ∈ U, − → Q C (x) is a singleton.

Remark 1.11 (Classical Klee result) Assume again that f is the halved energy as

in Example1.1(i) Then the left and right (Bregman) farthest-distance functions

3 The metric projection is the nearest-points map with respect to the Euclidean distance.

6 H.H Bauschke et al.

coincide, as do the corresponding farthest-points maps Furthermore, the set C is Klee if and only if for every z ∈ X, the metric farthest-points map Q C (z) is a singleton It is obvious that if C is a singleton, then C is Klee In 1961, Klee [27]showed the following converse:

See, e.g., also [1,11,17,23–25,29,39] Once again, it is still unknown whether or not(1.18) remains true in general Hilbert spaces The present Bregman-distance setting

is reviewed in Sect.1.4

1.1.4 Chebyshev Radius and Chebyshev Center

Definition 1.12 (Chebyshev radius and Chebyshev center) The left ← −

byshev radius of C is

Remark 1.13 (Classical Garkavi-Klee result) Again, assume that f is the halved

energy as in Example1.1(i)so that the left and right (Bregman) farthest-distancefunctions coincide, as do the corresponding farthest-points maps Furthermore, as-

sume that C is bounded In the 1960s, Garkavi [19] and Klee [26] proved that theChebyshev center is a singleton, say{z}, which is characterized by

See also [30,31] and Sect.1.5 In passing, we note that Chebyshev centers are alsoutilized in Fixed Point Theory; see, e.g., [20, Chap 4]

1 Chebyshev Sets, Klee Sets, and Chebyshev Centers 7

1.1.5 Goal of the Paper

The aim of this survey is threefold First, we review recent results ing Chebyshev sets, Klee sets, and Chebyshev centers with respect to Bregmandistances Second, we provide some new results and examples on Klee sets andChebyshev centers Third, we formulate various tantalizing open problems on thesenotions as well as on the related concepts of Chebyshev functions

concern-1.1.6 Organization of the Paper

The remainder of the paper is organized as follows In Sect.1.2, we record auxiliaryresults which will make the derivation of the main results more structured Cheby-shev sets and corresponding open problems are discussed in Sect.1.3 In Sect.1.4,

we review results and open problems for Klee sets, and we also present a new sult (Theorem1.27) concerning left Klee sets Chebyshev centers are considered

re-in Sect.1.5, where we also provide a characterization of left Chebyshev centers(Theorem1.31) Chebyshev centers are illustrated by two examples in Sect.1.6 Re-cent related results on variations of Chebyshev sets and Klee sets are considered inSect.1.7 Along our journey, we pose several questions that we list collectively inthe final Sect.1.8

P f ∗ ,C ∗

U ∗=∇f ◦ − → P f ,C ◦∇f ∗ and − →

P f ∗ ,C ∗

U ∗ =∇f ◦ ← P − f ,C ◦∇f ∗ Proof This follows from Fact 1.2, Fact 1.4(i), and Definition 1.6 (See also

Q f ∗ ,C ∗

U ∗=∇f ◦ − → Q f ,C ◦∇f ∗ and − →

Q f ∗ ,C ∗

U ∗ =∇f ◦ ← Q − f ,C ◦∇f ∗

Z f ,C=∇f ∗ − →

Z f ∗ ,C ∗and− →

Z f ∗ ,C ∗ =∇f ← −

Z f ,C The remaining identi-ties follow similarly

(iv)and(v): Clear from(ii)and(iii)and Fact1.2 The following two results play a key role for studying the single-valuedness of

Lemma 1.17 Let V and W be nonempty open subsets of X , and let T : V → W

be a homeomorphism, i.e., T is a bijection and both T and T −1 are continuous Furthermore, let G be a residual4subset of V Then T (G) is a residual subset of W Proof As G is residual, there exist sequence of dense open subsets (O k)k ∈N of V such that G ⊇ k ∈N O k Then T (G) ⊇ T ( k ∈N O k) = k ∈N T (O k ) Since T : V → W

is a homeomorphism and each O k is dense in V , we see that each T (O k) is open and

dense in W Therefore, k ∈N T (O k ) is a dense Gδ subset in W 

Lemma 1.18 Let V be a nonempty open subset of X , and let T : V → R n be locally Lipschitz Furthermore, let S be a subset of V that has Lebesgue measure zero Then,

T (S) has Lebesgue measure zero as well.

4 Also known as “second category”.

1 Chebyshev Sets, Klee Sets, and Chebyshev Centers 9

Proof Denote the closed unit ball in X by B For every y ∈ V, let r(y) > 0 be such that T is Lipschitz continuous with constant c (y) on the open ball O(y) centered at

y of radius r(y) In this proof, we denote the Lebesgue measure byλ Let K be a compact subset of X To show that T (S) has Lebesgue measure zero, it suffices to

show thatλ(T (K ∩ S)) = 0 because

K ⊆

m

j=1

We now proceed using a technique implicit in the proof of [21, Corollary 1] Set

c = max{c1,c2, ,c m } Givenε> 0, there exists an open subset G of X such that

G ⊇ K ∩ S andλ(G) <ε For each y ∈ K ∩ S, let Q(y) be an open cubic interval centered at y of semi-edge length s (y) > 0 such that

whereχQ k stands for the characteristic function of Q kand where the constantθonly

depends on the dimension of X Thus,

Now set d = (c √ n) nso thatλ(Q ∗

k ) ≤ dλ (Q k) Then, using (1.29) and (1.31), wesee that

10 H.H Bauschke et al.Alternatively, one may argue as follows starting from (1.28) We have K ∩ S ⊆

Since T is Lipschitz on each O (y j ) with constant c(y j) and sinceλ(O(y j )∩S) = 0,

we apply [18, Proposition 262D, page 286] and conclude thatλ(T (O(y j )∩S)) = 0.

D-Chebyshev sets) (See [6, Theorem 4.7].) Suppose that f is

super-coercive5and that C is ← −

D-Chebyshev Then C is convex.

Fact 1.20 (− →

D-Chebyshev sets) (See [6, Theorem 7.3].) Suppose that dom f = X, that C ∗ ⊆ U ∗ , and that C is − →

D-Chebyshev Then C ∗is convex

It is not known whether or not Fact1.19and1.20are the best possible results.For instance, is the assumption on supercoercivity in Fact1.19really necessarily?

Similarly, do we really require full domain of f in Fact1.20?

Example 1.21 (See [6, Example 7.5].) Suppose that X= R2, that f is the negative

entropy (see Example1.1(ii)), and that

right-Fact 1.22 (See [4, Lemma 3.5].) Suppose that f is the negative entropy (see

Example1.1(ii)) and that C is convex Then C is − →

1 Chebyshev Sets, Klee Sets, and Chebyshev Centers 11Fact1.22raises two intriguing questions Apart from the case of quadratic func-

tions, are there instances of f , where f has full domain and where every closed convex subset of U is − →

D-Chebyshev? Because of Fact 1.20, an affirmative swer to this question would imply that ∇f is a (quite surprising) nonaffine yet convexity-preserving transformation Combining Example1.21and Fact1.22, we

an-deduce that – when working with the negative entropy – if C is convex, then C is

We also note that C is “nearly ← −

D-Chebyshev” in the following sense.

Fact 1.23 (See [6, Corollary 5.6].) Suppose that f is supercoercive, that f is twice continuously differentiable, and that for every y ∈ U, ∇2f (y) is positive definite.

Then,← −

P Cis almost everywhere and generically6single-valued on U

It would be interesting to see whether or not supercoercivity is essential inFact 1.23 By duality, we obtain the following result on the single-valuedness

Proof By Lemma1.14(ii),− →

D-Klee Then C is a singleton.

Fact1.25immediately raises the question whether or not supercoercivity is really

an essential hypothesis Fortunately, thanks to Fact1.26, which was recently provedfor general Legendre functions without any further assumptions, we are now able topresent a new result which removes the supercoercivity assumption in Fact1.25

6That is, the set S of points y ∈ U where ← P − C (y) is not a singleton is very small both in measure theory (S has measure 0) and in category theory (S is meager/first category).

12 H.H Bauschke et al.

Theorem 1.27 (← −

D-Klee sets revisited) Suppose that C is bounded and that C is

← −

D-Klee Then C is a singleton.

Proof On the one hand, since C is compact, Fact1.2implies that C ∗is compact

On the other hand, by Lemma1.15(iii), the set C ∗ is − →

D f ∗-Klee Altogether, wededuce from Fact1.26(applied to f ∗ and C ∗ ) that C ∗ is a singleton Therefore, C is

Q C is almost everywhere and generically single-valued on U

Again, it would be interesting to see whether or not supercoercivity is essential

in Fact1.28 Similarly to the proof of Corollary1.24, we obtain the following result

is almost everywhere and generically single-valued on U

1.5 Chebyshev Centers: Uniqueness and Characterization

D-Chebyshev centers) Suppose that C is bounded Then the left

Chebyshev center with respect to C is a singleton, say ← −

Z C = {y}, and y is terized by

1 Chebyshev Sets, Klee Sets, and Chebyshev Centers 13

= {∇f ∗ (y ∗ )} = {y} is a singleton Moreover, using

Lemma1.15(ii), we see that the characterization (1.39) becomes

−

→

F C= sup

is convex (as the supremum of convex functions) and then to apply the

Ioffe-Tihomirov theorem (see, e.g., [40, Theorem 2.4.18]) for the subdifferential ofthe supremum of convex function In contrast,← −

F C = supx ∈C D(x,·) is generally not convex (For more on separate and joint convexity of D, see [3].)

1.6 Chebyshev Centers: Two Examples

1.6.1 Diagonal-Symmetric Line Segments in the Strictly

14 H.H Bauschke et al.

Theorem 1.33 Suppose that f is any of the functions considered in Example 1.1 Then the left Chebyshev center is the midpoint of C, i.e., ← −

Z C = {c1/2 }.

Proof By Theorem1.31, we write← −

Z C = {y}, where y = (y1,y2) ∈ U In view

of (1.37) and Fact 1.4(ii), we obtain that ← −

Q C(y) contains at least two elements.

On the other hand, since← −

Q C (y) consists of the maximizers of the convex function

D(·,y) over the compact set C, [33, Corollary 32.3.2] implies that← −

On the other hand, a symmetry argument identical to the proof of [8, Proposition 5.1]

and the uniqueness of its Chebyshev center show that y must lie on the diagonal,

i.e., that

The result now follows because the only point satisfying both (1.47) and (1.48) is

Remark 1.34 Theorem1.33is in stark contrast with [8, Sect 5], where we

investi-gated the right Chebyshev center in this setting Indeed, there we found that the

right Chebyshev center does depend on the underlying Legendre function used(see [8, Examples 5.2, 5.3, and 5.5]) Furthermore, for each Legendre function f

considered in Example1.1, we obtain the following formula

Indeed, since for every y∈ U, the function D(·,y) is convex; the points where the

supremum is achieved is a subset of the extreme points of C, i.e., of {c0,c1} fore, it suffices to compare D(c0,y) and D(c1,y).

There-1.6.2 Intervals of Real Numbers

Theorem 1.35 Suppose that X = R and that C = [a,b] ⊂ U, where a = b Denote the right and left Chebyshev centers by x and y, respectively Then7

1 Chebyshev Sets, Klee Sets, and Chebyshev Centers 15

Proof Analogously to the derivation of (1.46), it must hold that

← −

This implies that y satisfies D (a,y) = D(b,y) In turn, using Fact1.4(i), this last

equation is equivalent to D f ∗ (y ∗ ,a ∗ ) = D f ∗ (y ∗ ,b ∗ ) ⇔ f ∗ (y ∗ ) + f (a) − y ∗ a =

f ∗ (y ∗ ) + f (b) − y ∗ b ⇔ f (b) − f (a) = y ∗ (b − a) ⇔ y ∗ = ( f (b) − f (a))/(b − a),

Example 1.36 Suppose that X = R andC = [a,b], where 0 < a < b < +∞ In each of

the following items, suppose that f is as in the corresponding item of Example1.1

Denote the corresponding right and left Chebyshev centers by x and y, respectively.

Then the following hold:

1.7 Generalizations and Variants

Chebyshev set and Klee set problems can be generalized to problems involvingfunctions

Throughout this section,

g : X → ]−∞,+∞] is lower semicontinuous and proper (1.54)For convenience, we also set

q=1

Definition 1.37 (i) The function g is prox-bounded if there existsλ> 0 such that

eλg ≡ −∞ The supremum of the set of all suchλ is the thresholdλgof the

prox-boundedness for g.

(ii) The constantμgis defined to be the infimum of allμ> 0 such that g −μ−1 q is bounded below on X ; equivalently,φμg(0) < +∞

Fact 1.38 (See [34, Examples 5.23 and 10.32].) Suppose that g is prox-bounded

with thresholdλg, and letλ∈ ]0,λ g [ Then, Pλg is everywhere upper semicontinuous and locally bounded on X , and eλg is locally Lipschitz on X

Fact 1.39 (See [38, Proposition 4.3].) Suppose that μ>μg Then, Qμg is upper semicontinuous and locally bounded on X , andφμg is locally Lipschitz on X

Definition 1.40 (i) We say that g isλ-Chebyshev if Pλg is single-valued on X (ii) We say that g isμ-Klee if Qμg is single-valued on X

Facts1.41and1.43below concern Chebyshev functions and Klee functions; see[38] for proofs

Fact 1.41 (Single-valued proximal mappings) Suppose that g is prox-bounded

with thresholdλg, and letλ ∈ ]0,λ g[ Then the following are equivalent

(i) eλg is continuously differentiable on X

(ii) g isλ-Chebyshev, i.e., Pλg is single-valued everywhere.

(iii) g+λ−1 q is essentially strictly convex.

1 Chebyshev Sets, Klee Sets, and Chebyshev Centers 17

If any of these conditions holds, then

∇(g +λ−1 q) ∗= Pλg ◦ (λId). (1.60)

Corollary 1.42 The function g is convex if and only ifλg= +∞and Pλg is

single-valued on X for everyλ > 0.

Fact 1.43 (Single-valued farthest mappings) Suppose thatμ>μg Then the lowing are equivalent

fol-(i) φμg is (continuously) differentiable on X

(ii) g isμ-Klee, i.e., Qμg is single-valued everywhere.

(iii) g −μ−1 q is essentially strictly convex.

If any of these conditions holds, then

∇(g −μ−1 q) ∗

Corollary 1.44 Suppose that g has bounded domain Then dom g is a singleton if

and only if for allμ> 0, the farthest operator Qμg is single-valued on X

Definition 1.45 (Chebyshev points) The set ofμ-Chebyshev points of g is

argminφμg

If argminφμg is a singleton, then we denote its unique element by pμ and we refer

to pμ as theμ-Chebyshev point of g

The following result is new

Theorem 1.46 (Chebyshev point of a function) Suppose thatμ>μg Then, the set ofμ-Chebyshev points is a singleton, and theμ-Chebyshev point is character-

is finite Hence,φμg is strictly convex and supercoercive; thus,φμg has a unique

minimizer Furthermore, we have

∂φμg(y) = μ1



18 H.H Bauschke et al.

by the Ioffe–Tikhomirov Theorem [40, Theorem 2.4.18] Therefore,

We now provide three examples to illustrate the Chebyshev point of functions

Example 1.47 Suppose that g = q Thenμg= 1 and forμ> 1, we have

φμg : y → sup

x

1

Hence, theμ-Chebyshev point of g is pμ= 0

Example 1.48 Suppose that g=ι[a,b] , where a < b Thenμg= 0 and forμ> 0, we

have

φμg : y → sup

x

1

ex-1 Chebyshev Sets, Klee Sets, and Chebyshev Centers 19

1.8 List of Open Problems

Problem 1 Is the assumption that f be supercoercive in Fact1.19really essential?

Problem 2 Are the assumptions that f have full domain and that C ∗ ⊆ U ∗ in

Fact1.20really essential?

Problem 3 Does there exist a Legendre function f with full domain such that f is

not quadratic yet every nonempty closed convex subset of X is − →

D-Chebyshev? In

view of Fact1.19, the gradient operator∇f of such a function would be nonaffine

and it would preserve convexity

Problem 4 Is it possible to characterize the class of− →

D-Chebyshev subsets of the strictly positive orthant when f is the negative entropy? Fact1.22and Example1.21

imply that this class contains not only all closed convex but also some nonconvexsubsets

Problem 5 Is the assumption that f be supercoercive in Fact1.23really essential?

Problem 6 Is the assumption that f be supercoercive in Fact1.28really essential?

Problem 7 For the Chebyshev functions and Klee functions, we have used the

halved Euclidean distance What are characterizations of f and Chebyshev point

of f when one uses the Bregman distances?

20 H.H Bauschke et al.

Problem 8 How do the results on Chebyshev functions and Klee functions extend

to Hilbert spaces or even general Banach spaces?

1.9 Conclusion

Chebyshev sets, Klee sets, and Chebyshev centers are well known notions in sical Euclidean geometry These notions have been studied traditionally also in aninfinite-dimensional setting or with respect to metric distances induced by differentnorms Recently, a new framework was provided by measuring the discrepancy be-tween points differently, namely by Bregman distances, and new results have beenobtained that generalize the classical results formulated in Euclidean spaces Theseresults are fairly well understood for Klee sets and Chebyshev centers with respect

clas-to Bregman distances; however, the situation is much less clear for Chebyshev sets.The current state-of-the-art is reviewed in this paper and several new results havebeen presented The authors hope that the list of open problems (in Sect.1.8) willentice the reader to make further progress on this fascinating topic

Acknowledgements The authors thank two referees for their careful reading and pertinent

com-ments Heinz Bauschke was partially supported by the Natural Sciences and Engineering Research Council of Canada and by the Canada Research Chair Program Xianfu Wang was partially sup- ported by the Natural Sciences and Engineering Research Council of Canada.

References

1 Asplund, E.: Sets with unique farthest points Israel J Math 5, 201–209 (1967)

2 Bauschke, H.H., Borwein, J.M.: Legendre functions and the method of random Bregman

pro-jections J Convex Anal 4, 27–67 (1997)

3 Bauschke, H.H., Borwein, J.M.: Joint and separate convexity of the Bregman distance In:

D Butnariu, Y Censor, S Reich (ed.) Inherently Parallel Algorithms in Feasibility and mization and their Applications (Haifa 2000), pp 23–36 Elsevier (2001)

Opti-4 Bauschke, H.H., Noll, D.: The method of forward projections J Nonlin Convex Anal 3,

191–205 (2002)

5 Bauschke, H.H., Borwein, J.M, Combettes, P.L.: Essential smoothness, essential strict

convex-ity, and Legendre functions in Banach spaces Commun Contemp Math 3, 615–647 (2001)

6 Bauschke, H.H., Wang, X., Ye, J., Yuan, X.: Bregman distances and Chebyshev sets.

J Approx Theory 159, 3–25 (2009)

7 Bauschke, H.H., Wang, X., Ye, J., Yuan, X.: Bregman distances and Klee sets J Approx.

Theory 158, 170–183 (2009)

8 Bauschke, H.H., Macklem, M.S., Sewell, J.B., Wang, X.: Klee sets and Chebyshev centers for

the right Bregman distance J Approx Theory 162, 1225–1244 (2010)

9 Berens, H., Westphal, U.: Kodissipative metrische Projektionen in normierten linearen R¨aumen In: P L Butzer and B Sz.-Nagy (eds.) Linear Spaces and Approximation, vol 40,

pp 119–130, Birkh¨auser (1980)

10 Borwein, J.M.: Proximity and Chebyshev sets Optim Lett 1, 21–32 (2007)

11 Borwein, J.M, Lewis, A.S.: Convex Analysis and Nonlinear Optimization, 2nd edn Springer (2006)

1 Chebyshev Sets, Klee Sets, and Chebyshev Centers 21

12 Borwein, J.M., Vanderwerff, J.: Convex Functions: Constructions, Characterizations and terexamples Cambridge University Press (2010)

Coun-13 Bregman, L.M.: The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming U.S.S.R Comp Math Math

16 Censor, Y., Zenios, S.A.: Parallel Optimization Oxford University Press (1997)

17 Deutsch, F.: Best Approximation in Inner Product Spaces Springer (2001)

18 Fremlin, D.H.: Measure Theory, vol 2 Broad Foundations, 2nd edn Torres Fremlin, Colchester (2010)

19 Garkavi, A.L.: On the ˇCebyˇsev center and convex hull of a set Usp Mat Nauk 19, 139–145

22 Hiriart-Urruty, J.-B.: Ensembles de Tchebychev vs ensembles convexes: l’etat de la situation

vu via l’analyse convexe non lisse Ann Sci Math Qu´ebec 22, 47–62 (1998)

23 Hiriart-Urruty, J.-B.: La conjecture des points les plus ´eloign´es revisit´ee Ann Sci Math.

Qu´ebec 29, 197–214 (2005)

24 Hiriart-Urruty, J.-B.: Potpourri of conjectures and open questions in nonlinear analysis and

optimization SIAM Rev 49, 255–273 (2007)

25 Hiriart-Urruty, J.-B., Lemar´echal, C.: Convex Analysis and Minimization Algorithms II Springer (1996)

26 Klee, V.: Circumspheres and inner products Math Scand 8, 363–370 (1960)

27 Klee, V.: Convexity of Chebyshev sets Math Ann 142, 292–304 (1960/1961)

28 Motzkin, T.: Sur quelques propri´et´es caract´eristiques des ensembles convexes Atti.

Accad Naz Lincei, Rend., VI Ser 21, 562–567 (1935)

29 Motzkin, T.S., Straus, E.G., Valentine, F.A.: The number of farthest points Pac J Math 3,

221–232 (1953)

30 Nielsen, F., Nock, R.: On the smallest enclosing information disk Inform Process Lett 105,

93–97 (2008)

31 Nock, R., Nielsen, F.: Fitting the smallest enclosing Bregman ball In: J Gama, R Camacho,

P Brazdil, A Jorge and L Torgo (eds.) Machine Learning: 16th European Conference on Machine Learning (Porto 2005), pp 649–656, Springer Lecture Notes in Computer Science vol 3720 (2005)

32 Reich, S., Sabach, S.: Two strong convergence theorems for Bregman strongly nonexpansive

operators in reflexive Banach spaces Nonlinear Anal 73, 122–135 (2010)

33 Rockafellar, R.T.: Convex Analysis Princeton University Press, Princeton (1970)

34 Rockafellar, R.T., Wets, R J.-B.: Variational Analysis Springer, New York (1998)

35 Singer, I.: Best Approximation in Normed Linear Spaces by Elements of Linear Subspaces Springer (1970)

36 Singer, I.: The Theory of Best Approximation and Functional Analysis Conference Board

of the Mathematical Sciences Regional Conference Series in Applied Mathematics, No 13 Society for Industrial and Applied Mathematics (1974)

37 Vlasov, L.P.: Approximate properties of sets in normed linear spaces Russian Math Surv 28,

Chapter 2

Self-Dual Smooth Approximations of Convex Functions via the Proximal Average

Heinz H Bauschke, Sarah M Moffat, and Xianfu Wang

Abstract The proximal average of two convex functions has proven to be a useful

tool in convex analysis In this note, we express the Goebel self-dual smoothingoperator in terms of the proximal average, which allows us to give a different proof

of self duality We also provide a novel self-dual smoothing operator Both operatorsare illustrated by smoothing the norm

Keywords Approximation · Convex function · Fenchel conjugate · Goebel

smoothing operator· Moreau envelope · Proximal average

AMS 2010 Subject Classification: Primary 26B25; Secondary 26B05, 65D10,

90C25

2.1 Introduction

Let X be the standard Euclidean spaceRn, with inner product·,· and induced

norm ·  It will be convenient to set

q = 1

Now let f : X → ]−∞,+∞] be convex, lower semicontinuous, and proper Sincemany convex functions are nonsmooth, it is natural to ask: How can one approximate

f with a smooth function?

The most famous and very useful answer to this question is provided by the

Moreau envelope [15,17], which, forλ> 0, is defined by1

1 The symbol “” denotes infimal convolution: ( f1 f2)(x) = inf y

f1(y) + f2(x − y) H.H Bauschke ( )

Department of Mathematics, Irving K Barber School, University of British Columbia, Kelowna, B.C V1V 1V7, Canada

e-mail: heinz.bauschke@ubc.ca

H.H Bauschke et al (eds.), Fixed-Point Algorithms for Inverse Problems in Science

and Engineering, Springer Optimization and Its Applications 49,

DOI 10.1007/978-1-4419-9569-8 2, cSpringer Science+Business Media, LLC 2011

23

24 H.H Bauschke et al.

It is well known that eλ f is smooth (i.e., continuously differentiable) and that

limλ→0+eλ f = f point-wise; see, e.g., [17, Theorems 1.25 and 2.26] cally, other approaches to smoothing are Ghomi’s integral convolution method [9],Seeger’s ball rolling technique [18], and Teboulle’s entropic proximal map-pings [19]

Parentheti-Let us now consider the norm, which is nonsmooth at the origin

Example 2.1 (Moreau envelope of the norm) Let λ ∈ ]0,1[, set f =  · , and denote the closed unit ball by C Then, for x and x ∗ in X , we have2

2 Self-Dual Smooth Approximations of Convex Functions via the Proximal Average 25

By [17, Example 11.26(b) on page 495], we obtain

= f ∗+λq =ιC+λq Alternatively, one may use [7, Example 2.16],

which provides the proximal mapping of f , and then use the proximal mapping

calculus to obtain these results Finally, a referee pointed out that (2.11) can also bederived by reducing the computation of the Moreau envelope to

26 H.H Bauschke et al.and proved that



Gλ f∗

that is, Fenchel conjugation and Goebel smoothing commute! For applications of

the Goebel smoothing operator, see [11]

The purpose of this note is twofold First, we present a different tion of the Goebel smoothing operator which allows us to prove self-duality usingthe Fenchel conjugation formula for the proximal average Second, the proximalaverage is also utilized to obtain a novel smoothing operator Both smoothing oper-ators are computed explicitly for the norm The formulas derived show that the newsmoothing operator is distinct from the one provided by Goebel

representa-For f1and f2, two functions from X to ]−∞,+∞] that are convex, lower tinuous and proper, and for two strictly positive convex coefficients (λ1+λ2= 1),

semicon-the proximal average is defined by

pav( f1, f2;λ1,λ2) =λ1( f1+ q)∗+λ2( f2+ q)∗∗ − q. (2.21)The proximal average, which is actually a convex function, has been a useful tool forconstructing primal-dual symmetric antiderivatives [4] and for extending monotoneoperators [2]; see also [3,5,6,11,12] for further information and applications One

of the key properties is the Fenchel conjugation formula

pav( f1, f2;λ1,λ2)∗ = pav( f ∗

1, f ∗

2;λ1,λ2); (2.22)see [3, Theorem 6.1], [5, Theorem 4.3], or [6, Theorem 5.1]

We use standard convex analysis calculus and notation as, e.g., in [16,17,21]

In Sect.2.2, we consider the Goebel smoothing operator from the proximal-averageview point The new smoothing operator is presented in Sect.2.3

2.2 The Goebel Smoothing Operator

Definition 2.2 (Goebel smoothing operator) Let f : X → ]−∞,+∞] be vex, lower semicontinuous and proper, and letλ∈ ]0,1[ Then the Goebel smoothing operator [11] is defined by

con-Gλ f = (1 −λ2)eλ f+λq. (2.23)Note that (2.23) and standard properties of the Moreau envelope imply thatpoint-wise

lim

and that each Gλ f is smooth.

2 Self-Dual Smooth Approximations of Convex Functions via the Proximal Average 27Our first main result provides two alternative descriptions of the Goebelsmoothing operator The first description, item (i) in Theorem2.3, shows a pleasingreformulation in terms of the proximal average The second description, item (ii) inTheorem2.3, is less appealing but has the advantage of providing a different proof

of the self-duality, item (iii), observed by Goebel.

Theorem 2.3 Let f : X → ]−∞,+∞] be convex, lower semicontinuous and proper, and letλ ∈ ]0,1[ Then the following hold.3

H.H Bauschke et al (eds.), Fixed- Point Algorithms for Inverse Problems in Science< /small>

and Engineering, Springer Optimization and Its Applications 49,... referees for their careful reading and pertinent

com-ments Heinz Bauschke was partially supported by the Natural Sciences and Engineering Research Council of Canada and. .. Borwein, J.M.: Joint and separate convexity of the Bregman distance In:

D Butnariu, Y Censor, S Reich (ed.) Inherently Parallel Algorithms in Feasibility and mization and