Duality for Nonconvex Approximation and Optimization (CMS Books in Mathematics)

Q>

Canadian Mathematical Society Societe mathematique du Canada Editors-in-Chief Redacteurs-en-chef J. Borwein K.Dilcher

Advisory Board Comite consultatif P. Borwein R. Kane S. Shen

CMS Books in Mathematics Ouvrages ofe mathematiques de la SMC 1

HERMAN/KUCERA/SIMSA

2

ARNOLD

3

BORWEIN/LEWIS

Convex Analysis and Nonlinear Optimization, 2nd Ed.

4

LEVIN/LUBINSKY

Orthogonal Polynomials for Exponential Weights

5 KANE

Equations and Inequalities

Abelian Groups and Representations of Finite Partially Ordered Sets

Reflection Groups and Invariant Theory

6

PHILLIPS

TWO Millennia

7

DEUTSCH

8

FABIAN ET AL.

of Mathematics

Best Approximation in Inner Product Spaces Functional Analysis and Infinite-Dimensional Geometry

9 KRI^EK/LUCA/SOMER

17 Lectures on Fermat Numbers

Computational Excursions in Analysis and Number Theory

10

BORWEIN

11

REED/SALES

(Editors)

Recent Advances in Algorithms and Combinatorics

12 HERMAN/KUCERA/SIMSA 13

NAZARETH

14

PHILLIPS

15

BEN-ISRAEL/GREVILLE

Counting and Configurations

Differentiable Optimization and Equation Solving Interpolation and Approximation by Polynomials Generalized Inverses, 2nd Ed.

16 ZHAO Dynamical Systems in Population Biology 17 GoPFERT ET AL. 18

Variational Methods in Partially Ordered Spaces

AKIVIS/GOLDBERG

Differential Geometry of Varieties with Degenerate

Gauss Maps 19

MIKHALEV/SHPILRAIN/YU

20 BORWEIN/ZHU 21

Techniquesof Variational Analysis

VAN BRUMMELEN/KINYON

22 LuccHETTi

Combinatorial Methods

Mathematics and the Historian's Craft

Convexity and Well-Posed Problems

23

NICULESCU/PERSSON

Convex Functions and Their Applications

24

SINGER

25

HIGGINSON/PIMM/SINCLAIR

Duality for Nonconvex Approximation and Optimization Mathematics and the Aesthetic

Ivan Singer

Duality for Nonconvex Approximation and Optimization With 17 Figures

^ Sprimger

Ivan Singer Simion Stoilow Institute of Mathematics 014700 Bucharest Romania Editors-in-Chief Redacteurs-en-chef Jonathan Borwein Karl Dilcher Department of Mathematics and Statistics Dalhousie University Halifax, Nova Scotia B3H 3J5 Canada cbs-editors @ cms.math.ca

Mathematics Subject Classification: 46N10,49N15,90C26,90C48 Library of Congress Cataloging-in-PubHcation Data Singer, Ivan. DuaHty for nonconvex approximation and optimization / Ivan Singer. p. cm. — (CMS books in mathematics; 24) ISBN-13: 978-0-387-28394-4 (alk. paper) ISBN-10: 0-387-28394-3 (alk. paper) ISBN-10: 0-387-28395-1 (e-book) 1. Convex functions. 2. Convex sets. 3. Duality theory (Mathematics) 4. Approximation theory. 5. Convex domains. 6. Convexity spaces. I. Title. II. Series. QA640.S56 2005 515'.8-dc22

2005051742

Printed on acid-free paper. © 2006 Springer Science+Business Media, Inc. All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, Inc., 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed in the United States of America. 9 8 7 6 5 4 3 2 1 springeronline.com

To the memory of my wonderful wife, Crina

Contents

List of Figures Preface

xi xiii

1

Preliminaries 1.1 Some preliminaries from convex analysis 1.2 Some preliminaries from abstract convex analysis 1.3 Duality for best approximation by elements of convex sets 1.4 Duality for convex and quasi-convex infimization 1.4.1 Unperturbational theory 1.4.2 Perturbational theory

1 1 27 39 46 47 71

2

Worst Approximation 2.1 The deviation of a set from an element 2.2 Characterizations and existence of farthest points

85 86 93

3

Duality for Quasi-convex Supremization 3.1 Some hyperplane theorems of surrogate duality 3.2 Unconstrained surrogate dual problems for quasi-convex supremization 3.3 Constrained surrogate dual problems for quasi-convex supremization 3.4 Lagrangian duality for convex supremization 3.4.1 Unperturbational theory 3.4.2 Perturbational theory

101 103 108 121 127 127 129

viii

Contents 3.5 Duality for quasi-convex supremization over structured primal constraint sets

131

4

Optimal Solutions for Quasi-convex Maximization 4.1 Maximum points of quasi-convex functions 4.2 Maximum points of continuous convex functions 4.3 Some basic subdifferential characterizations of maximum points

137 137 144 149

5

Reverse Convex Best Approximation 5.1 The distance to the complement of a convex set 5.2 Characterizations and existence of elements of best approximation in complements of convex sets

153 154

Unperturbational Duality for Reverse Convex Infimization 6.1 Some hyperplane theorems of surrogate duality 6.2 Unconstrained surrogate dual problems for reverse convex infimization 6.3 Constrained surrogate dual problems for reverse convex infimization 6.4 Unperturbational Lagrangian duality for reverse convex infimization 6.5 Duality for infimization over structured primal reverse convex .constraint sets 6.5.1 Systems 6.5.2 Inequality constraints

169 171 175

6

7

8

9

Optimal Solutions for Reverse Convex Infimization 7.1 Minimum points of functions on reverse convex subsets of locally convex spaces 7.2 Subdifferential characterizations of minimum points of functions on reverse convex sets Duality for D.C. Optimization Problems 8.1 Unperturbational duality for unconstrained d.c. infimization 8.2 Minimum points of d.c. functions 8.3 Duality for d.c. infimization with a d.c. inequality constraint 8.4 Duality for d.c. infimization with finitely many d.c. inequality constraints 8.5 Perturbational theory 8.6 Duality for optimization problems involving maximum operators 8.6.1 Duality via conjugations of type Lau 8.6.2 Duality via Fenchel conjugations

161

184 189 190 190 198 203 203 209 213 213 221 225 232 244 247 248 252

Duality for Optimization in the Framework of Abstract Convexity . . . 259 9.1 Additional preliminaries from abstract convex analysis 259 9.2 Surrogate duality for abstract quasi-convex supremization, using polarities AG : 2^ ^ 2^ and AG : 2^ ^ 2^^^ 267

Contents 9.3 Constrained surrogate duality for abstract quasi-convex supremization, using families of subsets of X 9.4 Surrogate duality for abstract reverse convex infimization, using polarities AG : 2^ ^ 2 ^ and A c : 2^ -> 2^^^ 9.5 Constrained surrogate duality for abstract reverse convex infimization, using families of subsets of X 9.6 Duality for unconstrained abstract d.c. infimization 10 Notes and Remarks

ix

270 271 273 275 279

References

329

Index

347

List of Figures

1.1

3

1.2

12

1.3 1.4 1.5

40 44 49

2.1 2.2 2.3 2.4 2.5 2.6 2.7

86 88 90 91 92 95 96

5.1 5.2 5.3 5.4 5.5

153 156 157 159 163

Preface

In this monograph we present some approaches to duaUty in nonconvex approximation in normed Hnear spaces and to duaUty in nonconvex global optimization in locally convex spaces. At the first stage of development of approximation theory in normed linear spaces, the "best approximation" of an element by linear subspaces, and more generally, by convex sets (i.e., the minimization of the distance of an element to a convex set) was studied. Later, the following two main classes of nonconvex approximation problems were considered: "worst approximation," i.e., the maximization of the distance of an element to an arbitrary set, and "reverse convex best approximation," i.e., the minimization of the distance of an element to the complement of a convex set. These may be called "anticonvex" problems (following Penot [175], who has used this term in the more general context of optimization theory). The first results on duality for these problems were obtained in the papers [73], [74]. In optimization theory in locally convex spaces, first linear optimization problems, and more generally, convex optimization problems, i.e., the minimization of a convex function on a convex set (clearly, best approximation of an element by the elements of a convex set belongs to this class of problems) were studied. Later, the duality results obtained in this direction were extended to duality results for nonconvex problems, based on generalizations of convexity and of the methods of convex analysis by Elster and Nehse [60], Balder [13], Lindberg [134], Dolecki and Kurcyusz [48], Dolecki [47], and others. Independently, some classes of nonconvex optimization problems of a different type were studied, which Hiriart-Urruty [102] called "convex-anticonvex" problems (and we shall also adopt this terminology), since they have the following specific structure. They are minimization problems in which convexity is present twice.

xiv

Preface

in the constraint set and/or in the objective function, but once in the reverse way; namely, these are "convex maximization," i.e., maximization of a convex function on a convex set (or equivalently, minimization of a concave function on a convex set), "reverse convex minimization," i.e., minimization of a convex function on the complement of a convex set, and "d.c. optimization," i.e., optimization problems involving differences of convex functions. Of course, the latter also encompasses convex optimization problems as a particular case. The first results on duality for these problems were obtained in the papers [215], [218], [220], [217], and the paper [280]ofToland. For some time, approximation theory and optimization theory have developed independently, in parallel. In the 1960s it was observed that optimization, i.e., the minimization or maximization of a function, contains approximation as a particular case. Indeed, approximation is the minimization or maximization of a particular function on a normed linear space X, namely, the function f{y) = \\xo-y\\

(yeX).

Thus, in the 1970s there appeared naturally the idea of studying them together in this spirit, as reflected for example by the titles of the monographs of Laurent, Approximation et optimisation {191 Qi) [129], Holmes, A Course on Optimization and Best Approximation (1972) [106], Krabs, Optimierung und Approximation (1975) [122], Hettich and Zencke, Numerische Methoden der Approximation und semi-infiniter Optimierung (1982) [96], Glashoff and Gustaffson, Linear Approximation and Optimization (1983) [84], and Jongen, Jonker, and Twilt, Nonlinear Optimization in IR^. I. Morse Theory, Chebyshev Approximation (1986) [114]. The same point of view also appeared in parts of other monographs on optimization theory. On the other hand, going in the opposite direction, Cheney and Goldstein [32] have extended a result on the existence of best approximations to a result on the existence of optimal solutions of minimization problems. Starting with [212], [213], there was suggested and systematically carried out a program of work in this direction, namely, to show that many methods and results of approximation theory are so strong that they can be generalized to yield new methods and results in optimization theory. Subsequently, others also adopted this latter point of view (e.g., Wriedt [295], Berdyshev [17]). In the present monograph we shall study these two theories and their interactions, going from approximation to optimization and vice versa. It has long been known that duality is a powerful tool in the study of approximation and optimization problems. For problems of approximation in a normed linear space X, namely, of minimization or maximization of the distance to a given subset of X, "duality" means simply their study with the aid of the elements of the conjugate space X*. In a general setting, "duality theory" in optimization means the simultaneous study of a pair of optimization problems, related in some way, namely, the initial problem, called the "primal problem," of minimization or maximization of a function on a subset of a locally convex space X, and the "dual problem" of minimization or maximization of a function on a subset of a locally convex space W, with the aim of obtaining more information on the primal problem (on its "optimal value," on its "optimal solutions," etc.). In general (with the exceptions of Sec-

Preface

xv

tions 9.3 and 9.5), W is a set of functions on X, or alternatively, W is an arbitrary set, but paired with X with the aid of a function on the Cartesian product X x W called a "coupling function." In fact, although the latter is apparently more general, it turns out that these two methods are equivalent. We shall avoid the use of the term "duality" in other senses (so instead of "dual space" we shall use "conjugate" space; instead of "duality" between families of subsets we shall use "polarity"; etc.). The monographs devoted to approximation theory in normed linear spaces ([210], [211]) and those containing some chapters or sections on approximation in such spaces (e.g., Akhiezer [1], Cheney [31], Tikhomirov [277]) treat duality mainly for the case of best approximation by convex sets or special classes of convex sets (Unear subspaces, cones) or do not consider duality at all (Deutsch [41], devoted to best approximation in inner product spaces; Braess [25]). Also, the monographs on approximation and optimization, mentioned above, of Laurent, Krabs, and others consider duality mainly for convex sets and functions, or like the one of Jongen, Jonker, and Twilt, do not consider duality at all. Furthermore, most of the existing monographs on optimization theory or convex analysis and optimization treat duality mainly for the convex and quasi-convex cases (e.g., Stoer and Witzgall [262], Auslender [11], loffe and Tikhomirov [111], Ekeland and Temam [54], Elster, Reinhardt, Schauble, and Donath [61], Barbu and Precupanu [14], Pshenichnyi [182], Ponstein [180], Hettich andZencke [96], Glashoff and Gustaffson [84], Ekeland and Tumbull [55], Hiriart-Urruty and Lemarechal [104], Golshtein and Tretyakov [91], Borwein and Lewis [21]) or include some brief parts on nonconvex duality, especially on d.c. duality (e.g., Konno, Thach, and Tuy [120], Strekalovsky [267], Rubinov [193], Pallaschke and Rolewicz [169], Rockafellar and Wets [187], Tuy [284]). A section of the recent monograph of Rubinov and Yang, Lagrange-Type Functions in Constrained Nonconvex Optimization [201], presents the general theory of Lagrange-type functions and duality, developed mainly by the authors [201, Ch. 3, Section 3.2]. The monographs devoted especially to duality in optimization theory, by Golshtein, The Theory of Duality in Mathematical Programming and Its Applications (in Russian, 1971) [90], Rockafellar, Conjugate Duality and Optimization (1974) [185], and Walk, Theory of Duality in Mathematical Programming (1989) [293], treat only duality for convex optimization and some nonconvex generalizations of it. The monograph of Gao, Duality Principles in Nonconvex Systems: Theory, Methods and Applications (2000) [76], addressed to those working in applied mathematics, physics, mechanics, and engineering, presents a brief combination of Rockafellar's perturbational duality theory for convex problems and Auchmuty's [10] extended Lagrange duality theory as part of Gao's larger original theory of duality, which aims to encompass "duality in natural phenomena." Finally, the recent monograph of Goh and Yang, Duality in Optimization and Variational Inequalities (2001) [89], contains a short chapter on a nonconvex duality theory due to the authors (Goh and Yang [88]) for the classical mathematical programming problem in R". However, there is no monograph devoted to duality for nonconvex approximation and optimization problems.

xvi

Preface

There are detailed surveys on some of the approaches to nonconvex duaHty, described above. Thus, for the nonconvex duaHty results based on generalizations of convexity and generalizations of the methods of convex analysis see Martinez-Legaz ([143], [140]), and for the nonconvex duality results based on various Lagrangetype functions see the respective chapters of the monographs of Goh and Yang [89] and Rubinov and Yang [201]. Therefore, these approaches will be presented here more briefly, mainly in Chapters 1 and 10. The present monograph is devoted to the study of duality for the anticonvex approximation and convex-anticonvex optimization problems, in the above-mentioned senses. Note that these include a very broad class of nonconvex problems. For example, as we shall see in Chapter 8, the infimization of a lower semicontinuous function over a closed subset of a Hilbert space can be easily reformulated as the problem of infimization of a continuous linear function subject to a d.c. constraint, or alternatively, of a convex function subject to a reverse convex constraint. We shall concentrate here only on duality, so we shall not consider, for example, characterizations of primal optimal solutions involving only the primal constraint set and the primal objective function (with the exception of those, such as Remarks 4.1 and 7.1, that are used to prove duality results). We shall study duality only for global approximation and optimization, but some results for the local case will be also mentioned briefly in the Notes and Remarks. We shall not consider here duality for multiobjective optimization. In order to limit the size of this monograph, quadratic optimization and differentiable optimization will not be considered here; also, algorithms are not given here (for the latter, see for example the survey article Tuy [283] and the monographs Konno, Thach, and Tuy [120] and Tuy [284]). Being the first of this kind in the literature, the present monograph is based entirely on articles in mathematical journals. Some unpublished results and some new proofs are also given. Let us describe, briefly, the contents of the chapters of the book. In Chapter 1, after some preliminaries from convex analysis and abstract convex analysis, we give some results on duality for best approximation by elements of convex sets in normed linear spaces, and on duality for the infimization of convex and quasi-convex functions on convex sets in locally convex spaces. These will serve as a basis of comparison with the nonconvex duality results of the subsequent chapters and with the methods of obtaining them. In Chapter 2 we consider the deviation 5(G, JCQ) of a set G from an element XQ in a normed linear space X, i.e., the supremum of the distances ||g — XQ\\ = dist(g, XQ), over all g € G. We give duality formulas for 8(G, XQ) and characterizations of the elements go ^ G for which the above supremum is attained, i.e., of the so-called elements of worst approximation (or farthest points). Chapter 3 is devoted to the more general problem of quasi-convex supremization sup / ( G ) , where G is a set in a locally convex space X and f: X ^^ R is a. quasi-convex function. We introduce and study both unconstrained and constrained surrogate dual problems, as well as unperturbational and perturbational Lagrangian dual problems for quasi-convex supremization. Also, we consider surrogate duality for the case that the primal constraint set G is expressed with the aid of a "system."

Preface

xvii

In Chapter 4 we present various characterizations of the optimal solutions for quasi-convex supremization problems sup/(G), i.e., of the elements go e G such that/(go) = max/(G). In Chapter 5 we study the best approximation dist(xo, CG) by the complement CG = X\G of a convex set G in a normed linear space X, i.e., the infimum of the distances ||jco — z|| = dist(xo, z), over all z e CG. We give duality formulas for dist(jco, CG), and characterizations of the elements zo ^ CG for which the above infimum is attained. Chapter 6 is devoted to the more general problem of reverse convex infimization i n f / ( C G ) , where G is a convex set in a locally convex space X and f: X -^ R is a function. We introduce and study both unconstrained and constrained surrogate dual problems, and unperturbational Lagrangian dual problems for reverse convex infimization. Also, we consider surrogate duality for the case that the primal constraint set G is expressed with the aid of a system or with the aid of inequahties. In Chapter 7 we present various characterizations of the optimal solutions for reverse convex infimization problems i n f / ( C G ) (where G is a convex subset of a locally convex space X), i.e., of the elements zo ^ CG such that f(zo) = min / ( C G ) . Chapter 8 is devoted to "d.c. optimization," i.e., to optimization problems involving differences of convex functions. We first give duality results for the unconstrained infimization of the difference f — h of two functions on a locally convex space X, the first of them being arbitrary and the second one convex and lower semicontinuous. Next we give some characterizations for optimal solutions of such problems. We also study duality for the infimization of the difference f — h, where / , h are convex functions, on a constraint set defined by an inequality l{x) — k{x) < 0 or l(x) — k(x) < 0, where /, /: are convex functions, or on a constraint set defined by finitely many such inequalities. Furthermore, we present some results of perturbational Lagrangian duality for d.c. infimization. Finally, we present some duality results for the unconstrained problem of infimization of the pointwise maximum of two functions / and —h on a. locally convex space X, the first of them being arbitrary and the second one quasi-convex (or more particularly, convex) and lower semicontinuous (it turns out that this is, essentially, a d.c. problem). The framework of abstract convexity, which encompasses various generalizations of convex sets and convex functions, permits us to study optimization of more general functions on more general sets. In Chapter 9 we present briefly some duality results for such optimization problems. The concluding Chapter 10 contains some comments, bibliographical references, and additional results for each of the preceding chapters. We hope that this book will interest a large circle of readers, including those who want to use it for research or as a reference book, or for a graduate course, or for independent study (to this end, we have given detailed proofs of the results and several illustrations). I would like to express my profound gratitude to my long-time friend J.-E. Martinez-Legaz for his support of the project of this book and his generous help in its materialization. He has patiently and carefully read several versions of the whole manuscript, making valuable suggestions for corrections, improvements and

xviii

Preface

additions. Furthermore, I thank A.M. Rubinov for his stimulating interest and encouragement and for helpful comments on some parts. Also, I thank C. Zalinescu for prompt answers to some questions. I am grateful for the excellent working conditions ensured by the Simion Stoilow Institute of Mathematics of the Romanian Academy during the writing and preparation for print of the manuscript. Finally, I wish to thank J.M. Borwein for accepting to publish this book in his prestigious series and for his invaluable strong support in various stages of its production. Last, but not least, my thanks are due to Springer and the Canadian Mathematical Society, for their efforts and care in the production process. Bucharest, Romania October 2005

Ivan Singer

Duality for Nonconvex Approximation and Optimization

1 Preliminaries

1.1 Some preliminaries from convex analysis In this section we recall some basic definitions and results about convex analysis in the framework of normed linear spaces, in which we shall study the approximation problems, and in the more general framework of locally convex spaces in which we shall study the optimization problems. A (real) linear space is a set X in which there are defined two "vector operations," namely, an internal binary operation, called "addition," which associates, to each pair of elements x,y e X an element x -\- y e X, and an external binary operation, called "multiplication by a scalar," which associates, to each pair (a, x) consisting of a real number a e R and an element x € X, an element ax e X, with these operations satisfying the following conditions, for all x, j , z € X and a,b e R: (1)

X -\-y = y -{-X,

(2) x + (y-\-z) =

(x-\-y)-\-z,

(3) x-\-y=x-i-z=>y

= z,

(4)

a(x + y) =ax

(5)

(a + b)x = ax -\- bx,

(6)

a(bx) = (ab)x,

(7)

-^ay,

lx=x.

From these "axioms" one deduces easily that there exists a unique element 0 e X such that jc + 0 = 0 + jc = 0 for all jc e X. The "opposite element" of any

2

1. Preliminaries

jc G X is defined by -x := (-l)jc, and "subtraction" of elements is defined by X - y := X + (-y) (x, y e X). For example, the set R"^ of all ordered n-tuples of real numbers x = ( x i , . . . , x„), where I < n < +00, with componentwise vector operations X + y = (xi + y\, ...,Xn-\-yn).

ax = (axu ...,axn),

(1.1)

where a e /?, is a linear space. A normed linear space is a linear space X in which to each element x e X there is associated a real number \\x\\, called the "norm" of x, satisfying the following conditions, for all jc, j e X and a e R: (8)

||0|| =Oand||jc|| > 0 for each jc 7^ 0,

(9)

\\x + y\\oo Xn — x. A complete normed linear space is also called a Banach space. Here are some important examples of Banach spaces, to which we shall refer later: (i) The space /^, i.e., the linear space /?" endowed with the norm \\x\\ = max \xi\.

(1.2)

(ii) The space /", i.e., the linear space R^ endowed with the norm n

lkll=2]|x,|.

(1.3)

i= \

(iii) The "Euclidean space" l^, i.e., the linear space R" endowed with the norm

11.^11 -

£kp. N

(1.4)

When w = 2 or n = 3, it is easy to visualize the "unit ball" Bx = {xeX\

\\x\\ 0. Consequently, we have 0^ = 0 . D Remark 1.4. If for a function O G X* the hyperplane H defined by (1.62) quasisupports the ball B(x, r), then, necessarily, \\^\\ = 1, since by (1.60) and Lemma 1.5 we have

.^dist(.,//):=l^«^I^^^l±^ = - ^ . Il^ll ll^ll Definition 1.2. We shall say that a closed half-space V (respectively, an open halfspace U) quasi-supports a subset C of a locally convex space X if the hyperplane bd V (respectively, bd U) quasi-supports the set C. Lemma 1.7. A closed half-space V quasi-supports a set C if and only if it has one (and only one) of the forms Vi={yeX\sup(C)},

(1.63)

or V2 = {ye X\(y) < supO(C)},

(1.64)

1.1 Some preliminaries from convex analysis

17

and an open half-space U quasi-supports a set C if and only if it has one (and only one) of the forms Ux=[y

e X\ 0 ( y ) > sup(D(C)},

(1.65)

U2 = [y e X\ 0 ( j ) < sup 0 ( C ) } ,

(1.66)

or

where ^ e Z*\{0}, sup 0 ( C ) e R. Also, conversely, every closed half-space V of the form (1.63) or (1.64), and every open half-space U of the form (1.65) or (1.66), where O G Z*\{0}, sup 0 ( C ) e R, quasi-supports the set C. Proof This follows from Definition 1.2 and Corollary 1.1, since for y = Vj or V = V2 of (1.63) or (1.64), respectively, and U = UiorU = U2 of (1.65) or (1.66), D respectively, we have bdV =bdU = {y e X\ ^(y) = sup 0 ( C ) } . Corollary 1.3. Every closed (respectively, open) half-space V (respectively, U) quasi-supporting C and not containing C (respectively, int C j can be written in the form (1.63) (respectively, (1.65)), where O G X*\{0}, sup 0. Then A^.(5/Uo)(/); ^o) = ^^>odsW)(xo).

(1.132)

We recall that if Z is a linear space, a function f: X -^ R is said to be quasiconvex if fii^xi + (1 - i^)x2) < max{/(xi), /(X2)}

(xi, X2 G X, 0 < z^ < 1);

(1.133)

it is well known and easy to see that this happens if and only if all level sets Sdif) (d G R) of (1.22) are convex, or equivalently, all level sets Ad(f) of (1.23) are convex. Clearly, every convex function is quasi-convex, but the converse is not true. A function / : X -> R is said to be quasi-concave if the function — / i s quasiconvex. For any function f:X -> /? on a linear space X we shall denote by /q the quasi-convex hull of / , that is, the greatest quasi-convex minorant of / (i.e., the greatest quasi-convex function majorized by / ) . When X is a locally convex space, a function / : X ^- R is quasi-convex and lower semicontinuous if and only if all level sets Sdif) {d G R) are closed and convex. For any function / : X -^ /? on a locally convex space X we shall denote by /q the lower semicontinuous quasi-convex hull of / , i.e., the greatest lower semicontinuous quasi-convex minorant of / . We recall that for any function f: X ^^ R we have (e.g., by (1.153) below, applied to the polarity A = A^^ of (1.189) below), mf

/q(^) =

d ==

deR

xeco Sdif)

= sup sup deR OGX*

inf

deR

xeco Adif)

inf /(_y) = sup y^^

inf

f(y)

(x G X).

(1.134)

ex* y ^ ^

^{x)>d^(y)>d 0(v)>OU)-l When X is a locally convex space, a function / : X ^- R is said to be evenly quasi-convex if all level sets Sd(f) (d G R) of (1.22) are evenly convex. For any function / : X -^ /? we shall denote by /eq the evenly quasi-convex hull of / , i.e., the greatest evenly quasi-convex minorant of / . We recall that (e.g., by (1.153) below, applied to the polarity A = A ^^ of (1.191) below) for any function / : X -^ R we have fea(x) = ^

inf

deR xeecoSdif)

= sup sup deR

d=

inf

deR xeecoAdif)

inf f{y)=

'^^ (x)>d ^(y)>d

d sup

inf

eX*

>'^^ cD(v)>4>(^)

f(y)

(x e X).

(1.135)

1.2 Some preliminaries from abstract convex analysis

27

A function / : X -> /^ on a locally convex space X is said to be evenly quasicoaffine if all level sets S^if) {d e R) of (1.22) are evenly coaffine. For any function f:X -^ /? we shall denote by /qca the evenly quasi-coaffine hull of / , i.e., the greatest evenly quasi-coaffine minorant of / . We recall that (e.g., by (1.153) below, applied to the polarity A = A^^ of (1.193) below) for any function / : X -> /? we have /qca(^)=sup sup

iuf / ( > ' ) = sup

(x)=d J? we shall denote by /q(A'A) the A' A-quasi-convex hull (i.e., the greatest A'A-quasi-convex minorant) of / . We have (see, e.g., [254, p. 301, formulas (8.265) and (8.262)]) /q(A'A)(-^)=

inf xe^HSAf))

d=

sup

inf

f{y)

(x € X).

(1.153)

^eCA'({«;})

In the sequel we shall be interested in polarities for the case that X is a locally convex space and W = X*\{0} or W = (X*\{0}) x R. For the first case, let G be a subset of X. We mention now some special polarities A' = AQ : 2^ -^ 2^*^^^^ (/ = 1, 2, 3, 4), depending on G. (1) Let us first consider the polarity A = A^ : 2^ -^ 2^*^^^^ defined by AJ^(C)

:= {O G X*\{0}|O(c) < supO(G) (c e C)}

(C c X).

(1.154)

For this polarity we have, by (1.150), (A'^YiW)

= {xeX\

0(jc) < supcI>(G)}

(cD e X*\{0}).

(1.155)

30

1. Preliminaries

Lemma 1.10. (a) For any set G the polarity A = A^^ satisfies A|^j({g}) = 0 ( ^ e G ) ,

(1.156)

C A ^ ( G ) = {CD € X*\{0}| 3^ G G, O(^) = sup cD(G)}.

(1.157)

(b) The set G is (A^)^A^-c6>/2V^x if and only iffor each jc G CG there exists ^ = ^^e X*\{0} such that 0(g) < supO(G) < a)(x)

(geG).

(1.158)

Hence, ifG is (AQYA^-convex, then it is evenly convex. (c) A function f'• X -^ R is (A^jY A^-quasi-convex if and only if for each d e R andx e CSd(f) there exists O = ^d,x ^ ^*\{0} such that R is (A^)'A^-quasi-convex if and only if for each d e R mdx e CSd(f) there exists O = O^,^ e X*\{0} satisfying (1.152) for A = A^, that is, (1.165). Hence, if this condition is satisfied, then each level set Sd(f) (d e R) is closed and convex, that is, / is lower semicontinuous and quasiD convex. (3) Let us consider now the polarity A = A^ : 2^ ^ 2^*^^^^ defined by A^(C) := {O G Z*\{0}| supO(G) ^ 0 ( 0 }

(C c X).

(1.166)

For the polarity A = A^ we have (AlnW)

= {xeX\ cD(x) # supc|>(G)}

(O G X * \ { 0 } ) .

(1.167)

Lemma 1.12. (a) For any set G the polarity A = A^ satisfies AU{g})

=0

igeG),

Ai;(C) c Alio

(1.168)

(C c X).

AliG) = A^iG), (Aj;)'({4))) c (Alyi{}),

that is, for A = A^ we have (1.149) with C = G, w = ^, which yields the first assertion. Furthermore, by (1.171) and (1.170), we have G C (Aj,)^Aj,(G) c (AlYA'^iG)

= (AlYA^iG),

(1.174)

whence the second assertion follows. (c) A function f:X -^ R is (A^)'A^-quasi-convex if and only if for each d e R andx e CSd(f) there exists O = O^,^ e X*\{0} satisfying (1.152) for A = A^, u; = O, that is, (1.173). Hence, by the definition of evenly quasi-coaffine functions, we obtain the second statement. D Corollary 1.5. If a set G ^ X is {A^)' A]j-convex and A^^(G) = X*\{0},

(1.175)

G = [x eX\ 0(x) < sup 0(G) (O G Z * \ { 0 } ) } .

(1.176)

then we have

Proof. By (1.166), we have (1.175) if and only if for each O G X * \ { 0 } , we have O(^) < sup 0(G) {g G G), that is, if and only if G c {jc G Xl 0(jc) < sup(|>(G) (O G A:*\{0})}.

(1.177)

If also G is (A^)^A^-convex, then by Lemma 1.12 (b), we have the opposite inclusion as well, and hence the equality (1.176). D Remark 1.12. By (1.176), the set G is an intersection of open half-spaces; i.e., it is evenly convex. Corollary 1.6. For a set G c X, let us consider the following statements: 1°. G is (A^QYA^Q-convex and A^(G) = X*\{0}.

(1.178)

(Aj;)\X*\{0}) = G.

(1.179)

2°. We have

3°. We have (1.116). 4°. G is (AJ.)'AJ^-CO«V^JC, and we have (1.178). Then r =>2° d}(x)

(x G X, O € X*, J G /?),

(1.212)

then, by (1.202), r ^ ^ ^ O , J ) = SUp{-X{,eX|0(v)>d}(^) + xeX

=

sup (-f(y))= yeX d

-inf veX 4>(v)>J

-fix)}

f{y)

(O eX*,d

e R)

(1.213)

1.2 Some preliminaries from abstract convex analysis

37

which is (modulo an inessential additive term + J ) the so-called quasi-conjugate of / , in the sense of Greenberg and Pierskalla [95], which plays an important role in duaUty for quasi-convex optimization; then, since -X{yeX\ix^iy)>fid}

= -X{yeX\iy)>d}

(M > 0),

we have jlw = w for all w = (O, d) e W = X* x R, /x > 0, so the mapping If ^- 25 is not one-to-one. Nevertheless, for any coupling function (p: X xW -> R we have the implication wu W2 eW,wi=W2=^

f^'^\wx)

= f^'^\w2),

(1.214)

since sup l(p(x, wi) f -fix)} xeX

= sup {W2(x) f

= sup {Si (x) f -fix)} xeX

-fix)}

xeX

= sup {(pix, W2) + xeX

-fix)}. ^

Hence, one can uniquely define a conjugation f e R /*(S):=/^(^)(u;)

y/

-> / * e /? by

iweW).

(1.215)

(c) For W = X* X R,it is convenient to denote the quasi-conjugate (1.213) of / , in the sense of Greenberg and Pierskalla, mentioned above, by / J . The second quasi-conjugate of / is the function (/J)^ : X -> R defined [95] by (fJ)dM

= -inf / J W

U € X),

(1.216)

(y)>d

and the normalized second quasi-conjugate of / is the function f^^iX defined [95] by

fyy = sup if J)',.

-> R

(1.217)

deR

It is well known and easy to see that for any function f. X -^ R wc have

r = sup/J, f>f'^>f'\

(1.218)

deR

where /*, /** are the Fenchel conjugates (1.95), (1.97). Corresponding to (1.100), we have} r'=/eq

(/e^""),

(1.219)

with /eq being the evenly quasi-convex hull (1.135) of / . (d) There are also other "conjugates" of a similar form, useful for duality in convex and quasi-convex optimization, that are particular cases of the Fenchel-Moreau

38

1. Preliminaries

conjugates /^^*^^ (for suitable coupling functions (p), for example, the "pseudoconjugates" defined by / ; ( 0 ) = -inf/(x)

(O eX\d

G R),

(1.220)

e /?),

(1.221)

xeX ^{y)=d

and the "semiconjugates" defined by /;(cD)=

- i n f f{x)

(CD eX\d

xeX ^(y)>d-\

for which one introduces the second conjugates (/J^)J,(/j)j and the normahzed second conjugates / ^ ^ , /^^, similarly to (1.216) and (1.217) respectively (mutatis mutandis). We have r«-/q

ifeR''},

(1.222)

with /q being the lower semicontinuous quasi-convex hull (1.134) of / . Let us return now to the more general case in which X and W are two arbitrary sets. For any polarity A : 2^ —> 2^ the conjugation of type Lau associated with A is the mapping L(A): R -^ R defined by f^^^\w)

:=

- i n f fix)

( / e^^^we

W).

(1.223)

xeCA'({u;})

One can show (see [254, p. 279, Theorem 8.14]) that the mapping c = L(A): —X —w R ^ R satisfies (1.203), (1.204) (i.e., it is a conjugation), and that the mappmg X \Y c((p): R -^ R defined by (1.202) is a conjugation of type Lau if and only if cp takes only the values 0 or —oo, i.e., if and only if (p = — Xc» for some subset C of X xW. If X and W are two sets, C is a subset of X, and A: 2^ ^ 2^ is a polarity, then for the "representation function" pc'. X -> {—oo, -hoo} defined by

{

—00 if X e C, P^

(1-224)

+00 if jc G LC, we have (Pc)^^^^=/OA(C).

(1.225)

For any polarity A: 2^ ^ 2 ^ , the dual of L(A): R -^ R is the mapping w

Y

L(Ay: R -^ R defined by g^(^)'(x):=

- i n f g{w) weW xe{^A'({w})

(geR'^^xeX).

(1.226)

1.3 Duality for best approximation by elements of convex sets

39

The dual L(A)^ of L(A) is again a conjugation of type Lau (namely, L(A)^ = L(AO, with A^ of (1.143)), and we have L{Ay = (L(A)O' = L(A). For any / : X ^ /?, the function (/^(A))/^(A)' : X-> R is denoted by /^(^)^(^)'. By (1.226) and (1.223), we have /'^^^^^^^'=/q(A'A)

(/e^""),

(1.227)

with /q(A'A) of (1.153). In particular, for f = pc of (1.224) (where C c X is any set), we have (PC)^^^^^^^^'=PA'A(C).

(1.228)

For the polarities A = A^^ of (1.189), A = A^^ ^f (1.191), and A = A^^ ^f (1.193) we have, by (1.223), /^^^"\ O(xo) - supO(G)

(O € X*, ||0|| = 1),

(1.239)

and there exists OQ G X* such that IIOoll = 1,

(1.240)

dist(xo, G) = Oo(xo) - supOo(G).

(1.241)

Proo/ We have (1.239), since lUo - g\\ > (xo -g)>

O(xo) - sup cD(G)

(CD e X\ ||0|| = 1).

Furthermore, since XQ G CG, we have dist(jco, G) > 0. Let A:={y e X\ \\XQ - y\\ < dist(jco, G)} = int^(jco, dist(xo, G)). (1.242) Then A is a nonempty open convex set, and G (1 A = &. Hence, by the separation theorem, there exists OQ G X * \ { 0 } such that sup(Do(G) < inf Oo(A);

(1.243)

we may assume without loss of generality (dividing by ||Ooll, if necessary) that ||a>o|| = 1. We have Oo(xo)-supcDo(G)>0;

(1.244)

indeed, otherwise, from (1.243) we would obtain Oo(jco) < inf Oo(A), in contradiction to XQ G A. Let us consider the hyperplane Ho := {y G X\ (G)(.vo)

indeed, for any O G Z* with ||4>|| = 1, supcI)(G) > O(xo), we have sup in (1.260) is obvious from (1.239). On the other hand, if Oo G X* is as in 4° of Theorem 1.12, then ||Oo|| = 1, OQ e A^(G; go) and we have dist(xo, G) = \\xo - goII = ^o(-^o - go) = ^o(-^o) - sup Oo(G), whence (1.260), with the max attained at 4>o.

•

1.4 Duality for convex and quasi-convex infimization In this section we shall present some duality results and some methods of obtaining them, for infimization of convex and quasi-convex functions on convex sets in locally convex spaces (we consider also the latter, since for the validity of some results only the quasi-convexity of functions is needed, i.e., the convexity of their level sets, rather than their convexity, i.e., the convexity of their epigraphs). These will serve as a basis of comparison with the nonconvex duality results of Chapters 3, 4, 6, and 7 and with the methods of obtaining them. Although the infimization of quasi-convex functions is, actually, nonconvex infimization, we shall include it in this section (rather than devoting to it a separate chapter), since quasi-convexity belongs to the field of "generalized convexity." In the first part we shall present some elements of the theory of "unperturbational dual problems," i.e., of dual problems defined without using perturbations. In the second part we shall present some elements of the theory of "perturbational dual problems," i.e., of dual problems defined with the aid of perturbations of the primal problem, and we shall show that various unperturbational dual problems can be deduced from the perturbational theory, in a unified way, for suitable choices of particular perturbations.

1.4 Duality for convex and quasi-convex infimization

47

1.4.1 Unperturbational theory Assume that we are given a constrained "primal" problem, called a "problem of infimization," (P)

a = inf/(G),

(1.261)

where the "constraint set" G is a subset of a locally convex space X and f: X ^^ R is a function, called "the objective function." When G = X, problem (P) is called "unconstrained." Any go e G for which the inf in (1.261) is attained, i.e., such that /(go) = inf/(G),

(1.262)

is called a (global) "optimal solution" of problem (P). The set of all optimal solutions will be denoted by Scif), that is, Scif)

'= {go e G\ f(go) = inf/(G)};

(1.263)

naturally, one can also write min instead of inf in (1.262) and (1.263). If G is a convex set and / is a convex (respectively a quasi-convex) function, then (P) of (1.261) is called a problem of convex (respectively, quasi-convex) infimization. As has been observed above, best approximation may be regarded as a particular case of infimization, by taking Z to be a normed linear space, xo ^ X, and / : X -> R the convex function fiy):=\\xo-y\\

(yeX);

(1.264)

indeed, then inf/(G) = dist(jco, G),

(1.265)

and the optimal solutions go ^ ^ of problem (P), for this case, are the elements of best approximation of XQ by G. Therefore, it is natural that many results on infimization can be applied to the particular case of best approximation. Moreover, in the converse direction, although the extension from the particular function / of (1.264) to a function / : X ^^ P on a locally convex space Z is a rather big step, it turns out that many results and methods of the theory of best approximation can be extended to results on the infimization of functions. For example, since for the function (1.264) we have Sdif) = [yeX\

\\xo - y\\ G X*, such as formula (1.238) in Theorem 1.11. In the theory of duality for the more general case of infimization of convex functions, instead of such equalities there appears a "dual problem" of supremization of a "dual objective function" on a "dual constraint set," and the infimum occurring in the primal problem (P) of (1.261) is not necessarily equal to the supremum in the dual problem; they are equal only under some conditions on the primal variables G and / , but those are satisfied by the particular function (1.264) on a normed linear space X (for example, / of (1.264) is convex and continuous). This explains why in Section 1.3 on best approximation by convex sets no "dual problem" has appeared explicitly. There is also another essential difference between the duality theories for best approximation by convex sets and for convex infimization. Namely, while the results of Section 1.3 have been proved by arguments that work directly in X and X* (e.g., separation theorems for convex subsets of X), problem {P) of (1.261) requires one to find the "lowest point" of the graph (or, equivalently, of the epigraph) of the restriction / | G of the convex function / to the convex set G c X (see Figure 1.5, in which X = R^, endowed with its natural topology); therefore, duality results for (1.261) are often obtained by applying separation theorems for the epigraph (1.21) of / , which is a subset of X x /?, by functions in (X x /?)* = X* x R (thus, instead of working with functions on X, one works with functions "one floor higher"); this is made possible by some useful connections between / and epi / (for example, it is well known that / is a convex function if and only if epi / is a convex set). There are two main types of dual problems to any (convex or nonconvex) constrained primal optimization (infimization or supremization) problem: "Lagrangian dual problems" and "surrogate dual problems." Roughly speaking, a Lagrangian dual problem, say, to (1.261), is an optimization problem whose objective function is defined by replacing the primal constraint set G by the whole space X, at the price of adding a "penalty term" to the objective function / (in order to compensate the "violation of the constraints"), and a surrogate dual problem to (1.261) is an optimization problem whose objective function is defined with the aid of the same objective function / , but replacing the constraint set G by a family of "surrogate constraint sets" (usually related in some way to G). We have the following basic theorem of Lagrangian duality. Theorem 1.13. Let X be a locally convex space, G a convex subset of X, and f: X ^ R a proper convex function that is continuous at some point ofGH dom /

1.4 Duality for convex and quasi-convex infimization

49

//

Figure 1.5. {i.e., finite and continuous at some point ofG). Then inf/(G) = max inf [f{y) - 0 ( j ) + inf cD(G)}.

(1.268)

4>GX* yeX

Proofi Clearly, for any G and / we have inf/(G) > inf {/(g) - (D(g) + inf cD(G)} geG

> i n f { / ( j ) - < I . ( y ) + inf sup inf [f{y) - —oo. Let M := {(g, ri)eXxR\geG,r]
0) and taking d = f(x) in (1.272), and O o : = - i v i / (6X*\{0}),

(1.273)

we obtain Oo(x) - fix) < cDo(g) - inf/(G)

(X e d o m / g e G),

(1.274)

(x G d o m / g G G).

(1.275)

whence inf/(G) < fix)-

cDo(x) + cDo(g)

Consequently, inf/(G) < fix) - (Do(x) + infcDo(G)

(x e X),

which together with (1.269), proves (1.268) (with the max attained for O = OQ)- D Remark 1.18. (a) This proof illustrates the "epigraphic methods" of proofs in convex analysis, mentioned above. Later we shall also give another proof of Theorem 1.13, deducing it as a particular case of more general results. (b) Any condition involving the primal constraint set that ensures that (weak or strong) duality holds, e.g., the condition of Theorem 1.13 that / should be continuous at some point of G fl dom / , is called a constraint qualification. There are also other constraint qualifications that ensure the validity of the strong duality formula (1.268), e.g., the following condition discovered by Attouch and Brezis ([7, Corollary 23]): X is a Banach space, G is a closed convex subset ofX,f\X^^Risa lower semicontinuous proper convex function, fl«JU^>o/x(G —dom/) = X (in particular, the latter equality is satisfied when dom f = X). For some other constraint quaUfications see, e.g., Hiriart-Urruty and Lemarechal [104]. (c) In particular, when X is a normed linear space and / is the function (1.264), formula (1.268) means that dist(jco, G) = max inf {||xo-JC|| - CD(JC) + inf 0 ( G ) } . OGX*

(1.276)

xeX

Theorem 1.13 suggests that the left-hand and right-hand sides of the duality formula (1.268) be split into two optimization "problems," namely, the initial primal problem (P) of (1.261) and the "dual problem"

1.4 Duality for convex and quasi-convex infimization (D)

p = supA.(X*) = sup X(cD),

51

(1.277)

where A(cD) = inf {f(y) - cl>(^) + inf 0(G)}

(cD e X*);

(1.278)

yeX

this is a Lagrangian dual problem in the sense mentioned above, with the penalty terms 7T^(y) := -(D(y) + inf 0(G)

(y e X);

(1.279)

in other words, the Lagrangian dual problem (1.277), (1.278) "penaHzes," via the term (1.279) added to the primal objective function / (with lower addition), the fact that the initial constraint set G is replaced by the whole space X. The "dual constraint set" is the whole conjugate space X* (so (D) is an unconstrained supremization problem), and the "dual objective function" is X: X* -^ R of (1.278). The numbers a and fi are called the (optimal) values of problems (P) and (D) respectively. With the above notations for the Lagrangian dual problem (1.277), (1.278), the "duality inequality" (1.269) can be written as a>p.

(1.280)

For any primal-dual pair {(P), (D)} of optimization problems, when a = ^ (that is, when the values of (P) and (D) coincide), one says that weak duality holds, or that there is no duality gap', when a > p, one says that there is a duality gap. If we have a = fi and the dual problem (D) has an optimal solution, that is, if the value of (D) is attained for some OQ e X*, then one says that strong duality holds (see, e.g.. Theorems 1.11 and 1.13). Besides the use of constraint qualifications, another method of getting rid of a possible duality gap of a primal-dual pair [(P), (D)} of optimization problems is to replace the dual problem (D) by a new dual problem (DO

)6' = supV(X*),

(1.281)

for which a = fi' (possibly without assuming any constraint qualification). For the case of Lagrangian dual problems, one way of doing this is that of replacing the Lagrangian (1.287) by an "augmented Lagrangian" L': X x X* -> /?. To this end, a useful tool is provided by abstract convex analysis (for some details, see, e.g., [254, Section 0.8a]). Let us return now to the Lagrangian duality result (1.268). Applying formula (1.268) to a hyperplane G = H = [y e X\ (Do(jc) = JQ}, where OQ e X*\{0}, do e R, and observing that for any O G X* we have inf(H) = |''^o instead of max^^/? in the right-hand side of (1.283), by taking, instead of the hyperplane H = {x e X\ ^oM = do}, the closed half-space D:={x

eX\^oM>do},

(1.284)

where OQ and do are as above (so // = bd D, the boundary of D) and assuming that / is a proper convex function that is continuous at a point of D D dom / . Indeed, we have, for any 4> G X*,

inf OCD) = I "^0 ^ ^

[ -cx)

*^rVr'*^'

(1-285)

if Jy? > 0, O = r]^o,

whence by (1.268) (with G = D), we obtain inf

fix) = max inf {f(y)-r]o(y) + r]do}.

xeX ^o(x)>do

(1.286)

ri>0 yeX

The following is a useful tool for the study of the Lagrangian dual problem (1.277) to (P) of (1.261): the function L: X x X* -^^ defined by L(jc, O) := f(x) - (t>(x) + inf 0(G) (x e X, O e X*), (1.287) is called the Lagrangian function, or simply the Lagrangian, associated with the primal-dual pair {(P), (/))}, or with the dual problem (D). Thus, by (1.278), (1.287), and (1.277), X(0) = inf L{y, cD)

(O e X*),

(1.288)

yeX

P= sup inf L(j,(D);

(1.289)

therefore, conversely, (D) of (1.277) may be called the dual problem associated with the Lagrangian function (1.287). Duality results for inf / ( G ) , such as Theorem 1.13, can be used to derive characterizations of optimal solutions of convex optimization problems, e.g., the following one, due to Pshenichnyi and Rockafellar (see, e.g., [106, p. 30]): Theorem 1.14. Let X be a locally convex space, G a convex subset of X, and f: X -^ R a convex function that is continuous at some point of G. Then for an element go ^ G the following statements are equivalent: r. go e Scif) (i.e., /(go) = min/(G)).

1.4 Duality for convex and quasi-convex infimization

53

2°. There exists o e X* such that 4>o e Sfigo),

(1.290)

ct)o(go) = min cI>o(G).

(1.291)

Proof. If figo) = min/(G), then by Theorem 1.13, there exists o e X* such that f(go) = inf {f{y) - o{x} + info(G), which, since go e G, yields (1.291). Furthermore, by (1.293), we have figo) - Oo(^o) < figo) + sup (-Oo)(G) = f(go) - inf a)o(G) < / ( x ) - Oo(x)

(X G X),

so (1.290) holds. Conversely, assume 2°. Then by (1.290), we have f(go) - ^o(go) = inf {fix) - cI>o(x)}, xeX

which, together with (1.291), yields (1.292). Hence, by Theorem 1.13 and go ^ G, we obtain /(go) = min / ( G ) . D Remark 1.20. (a) Let us also mention a more classical proof of Theorem 1.14, based on the fact that 2° is equivalent to -A^(G;go)na/(go)7^0.

(1.294)

By the definition (1.122) of XG. 1° can also be written in the form ( / + XG)igQ) = m i n ( / + XG)iG), and, by the definition (1.110) of the subdifferential, this equality holds if and only if 0 e 9 ( / -J- XG)(^O)- But since dom XG = G, by Theorem 1.5 and formula (1.124) we have dif + XG)igo) = a/(go) + dxGigo) = dfigo) + NiG; go), so 1° is equivalent to 0 e 9/(go) + A^(G; go), that is, to (1.294). (b) In the particular case that X is a normed linear space and / is the function (1.264), from Theorem 1.14 one obtains again Theorem 1.12 on the characterization of the elements of best approximation by using the subdifferential formula (1.267). In the case that optimal solutions exist. Theorem 1.14 permits the following sharpening of the basic Lagrangian duality formula (1.268):

54

1. Preliminaries

Corollary 1.11. Let X be a locally convex space, G a convex subset of X, and f: X ^ R a proper convex function that is continuous at some point ofGD dom / . If problem (P) has a solution, say go, then min/(G) = /(go) =

max

inf {f(x) + (x) + - s u p O ( G ) } .

(1.295)

^eNiG;go)xeX

Proof The inequality > in (1.295) with max replaced by sup, is obvious. On the other hand, if OQ e X* is as in Theorem 1.14, then - O Q e N(G; go), and we have inf/(G) = /(go) = inf {f(x) - o(x) + inf Oo(G)}, xeX

whence (1.295), with the max attained at O = —OQ.

•

We have the following simultaneous characterization of primal and dual solutions, and strong Lagrangian duality: Proposition 1.2. Let X be a locally convex space, G a convex subset of X, go G G, f: X -> R a function, and Oo ^ ^*- The following statements are equivalent: r. We have (1.292). 2°. go is a solution of problem (P) (of (1.261)), 4>o is a solution of the dual problem (D) (of (1.277)), and we have strong duality (i.e., a = fi, with fi being attained). Proof If 1° holds, then, by the duality inequality (1.280) we have a = min / ( G ) < /(go) = inf {f(x) - o(x) + inf c|>o(G)}

o, and T = {d e R\d > Jo}. (b) Problem (1.298) is equivalent to problem (P) of (1.261). Indeed, given X, Z, w, r , and / as above, the programming problem (1.298) is nothing but the optimization problem (1.261) with G = {x e X \ u{x) e T} := u~\T). Conversely, every optimization problem (1.261) can be written in the form of a programming problem (1.298), for example, by taking Z = X, M = /x, the identity operator in X (i.e., M(X) = X for all jc E X), and T = G. Naturally, a constraint set G of an optimization problem (1.261) can be written in different ways in the form (1.298), which give rise to different dual problems. Note that the advantage of (1.296), (1.298), is that one can also use the properties of T and u. (c) One can combine problems (1.261) and (1.298), considering the infimization problem (P)

Qf= inf / ( x ) ,

(1.299)

xeG u(x)eT

where X, Z,u,T and / are as above, and G is a subset of X, called an "abstract constraint set." However, we shall not pursue this direction. Let us give now some results of unperturbational Lagrangian duality for problems with structured primal constraint sets in the sense of (1.296), which will be used later. Some other unperturbational Lagrangian duality theorems will be deduced in Section 1.4.2 from more general results on perturbational duality. The following result holds for arbitrary extended-real valued functions. Proposition 1.3. Let X be a linear space, and let fj\,... functions. Then

Jrn'- X -^ R be m -\- \ m

inf f{y) > inf 0' = l,...,m)

{i = \,...,m)

f(y) > sup inf Ifiy)

^Yriiliiy)],

(1.300)

56

1. Preliminaries

with the upper addition + of (1.84) and the uppermultiplication x of (1.92), which we have denoted simply by x, that is,

here, as well as throughout the sequel, Y^=\ "^eans upper addition and R+ — [0, +oo). Proof. By (1.301), for any y e X we have fiy) + X[xex\i,M^) > yeX li(y) p. Hence, if a = —oo, then fi = —oo, and therefore, for any r] e R^, \ ^ 1 Of = - o o < inf /(y)4-y]^7///(j)

a

(z e R"").

(1.318)

'= '

(i6/„)

We claim that rj'^ e /?™. Indeed, let j e /„ and define a sequence {z"} Q R'" by in

if/=7,

(J 319)

then, from (1.312) and (1.319) we have li(yo) < z" (i e /„), whence, by (1.312), (1.316), (1.317), and (1.319), + oo > fiyo) - viO) >

inf f{y)-v(0)

= v{z")-v{0)>-nii°

yeX 1 /,.\

(n = l , 2 , . . . ) , •'

n

and therefore r]^j > 0, proving the claim. Let us prove now that m

a<mf\f(y) yeX

+ Tri%(y)\.

I

.^j

(1.320) J

Let J e X. If there exists j e Im such that lj{y) = +00 or if f{y) = +00, then, by the rules (1.84) and (1.92) for + and x, we have f(y) + Yl?=\ ^%iy) = + ^ Assume now that //(j) < +00 (/ e Im) and f(y) < +00, and let J:={i

eUli{y)

=-00}.

(1.321)

Define a sequence {y"} c R^ by

Then //(y) < yf (/ G /^; AZ = 1, 2 , . . . ) , whence, by (1.318) and (1.322), m

a < f(y) + J2 nhl = f(y) + E ''°''(>') - « E''°-

^1-323)

If there exists j e J such that ry^ > 0, then, passing to the limit in (1.323) for n -> +00, we obtain a contradiction with of > — (X). Therefore, rj^ = 0 for all / G J, and hence, by (1.92) and (1.323), we have (1.320). Consequently, m

P R be m -\- \ finite valued lower semicontinuous convex fiinctions. Then f(y) = sup inf f(y) + T r]My)\.

inf yeK

nm yeK I

^

(1.326)

J

(i=h...,m)

Proofi For each y e X with // (j) < 0 (/ = 1 , . . . , m) we have f

f(y) =sup\

^

1

f(y)^TriMy)\.

whence inf yeK //(>')(G)

However, the answer is negative, even when G is a closed convex set and / is a finite continuous convex function on a finite-dimensional space X, as shown by the following example: In X = R, let G = {x eX\ -2<x f(x)=x (xeX), (Do(jc)=jc (xeX).

(—O)(jco), we can write (1.330) in the equivalent form

1.4 Duality for convex and quasi-convex infimization inf/(G) =

max

inf

4>eX*\{0} 0(jco)(>')=infO(G)

f{y).

65

(1.352)

Nov^ formula (1.352) and Corollary 1.10 (with OQ = O, JQ = inf 0(G)) imply inf/(G) = =

max

inf

/(y)

sup inf/(G), each g e G such that d > f{g) > inf/(G) and each O e X*\{0} we have g e A j ( / ) nC(A^)^({4>}) (note: equivalendy, one can observe directly that we always have inf/(G)>^^:=

sup inf/(C(A^^)^({cD})) eX*\{0}

=

sup

inf

fix),

(1.360)

because G ^ [x e X\ (x) < supcI>(G)}). Second, clearly, condition 2° above is satisfied if and only if for each d e R,d < inf/(G), there exists Oj e Z*\{0} such that

AAf) n C(A^^)^({0^}) = AAf) n{xex\

Ax) < sup AG)} = 0. (1.361)

Hence, the assertion on condition 2° follows. The proof for condition 3° is similar. D Remark 1.27. (a) The assumption G^ 7^ 0 implies that G ^ X (since otherwise, for each G Z * \ { 0 } we would have sup 0(G) = sup 0(G) = sup 0 ( Z ) = +CXD,

1.4 Duality for convex and quasi-convex infimization

67

so G^ = 0), and if G is convex, then the converse is also true (take any x ^ G, and apply the strict separation theorem). (b) Geometrically, formula (1.355) means that inf/(G) = sup i n f / ( V | ^ ) ,

(1.362)

where V^^ is as in (1.30), with d = sup 0(G), i.e., the smallest closed half-space determined by O containing G; note that if O G G^, then V^^ ^ X. (c) If G^ 7^ 0 and (1.355) holds, then we also have inf/(G) =

sup supcD(G)<J

inf /(jc),

(1.363)

^ ^-

or, geometrically. inf/(G) = sup inf/((V),

(1.364)

VeV GCV

where V denotes the set of all closed half-spaces in X. Indeed, by (1.362) we have the inequality < in (1.364), and on the other hand, for any set V with G c y we have inf/(G) > inf/(V). We have the following result of strong duality. Theorem 1.19. Let X be a locally convex space, G a subset ofX with G^ ^ 0, and f: X ^ R a function. The following statements are equivalent: 1°. We have inf/(G) = max

inf

/(y).

(1.365)

cD(>')<supcD(G)

2°. There exists O^ € X*\{0} such that supcD«(G) - o o , then, by (1.365), there exists O^ e G^ such that a = inf/(G) =

inf

f{y),

(1.367)

0„(>0<supO«(G)

whence by Chapter 3, Lemma 3.4 (a) (with $^ = {y G X| ^^(y) < sup «(G)} and d = Of), we obtain (1.366).

68

1. Preliminaries

Conversely, assume now 2°. If o: = —oo (so (1.366) is vacuously satisfied for any ^^ e X*\{0}), then by (1.360), we have (1.365). If of > - o o , then by 2° and Chapter 3, Lemma 3.4(a), we have inf

f(y)>a

=

mff(G),

yeX (G) < inf 0 ( A J / ) ) < (x)

(x € A , ( / ) ) ,

where the last inequahty holds by Lemma 1.8. Hence, by Theorem 1.19, we have (1.365). D We also have the following result on surrogate duality formulas having inf^ez,cD(j)<supO(G) f(y) in the right-hand side (half-space theorems): Proposition 1.4. Let X be a locally convex space, G a subset of X, f: X -^ R a function satisfying (1.328), and XQ an element of X satisfying (1.329). The following statements are equivalent: \\ We have {1355). 2°. We have inf/(G) =

sup

inf

f{y)

(1.368)

supO(G)(G)

Also, a similar equivalence holds for the corresponding strong duality equalities (i.e., for (1.365) and sup replaced by max in (1.368)j. Proof By (1.329), we have sup

inf

f(y) < f(xo) < inf/(G),

cI>(jco)<sup')<supcD(G).

and hence if 1° holds, then we have 2°. On the other hand, by G sup ^

Qb

inf

/(_y) >

yeX cD(j)<sup0(G).

sup

inf

69

/(>'),

^ ^b yeX , u p n G ) < ^ { x o ) ^(^)<sup0(G)-

and hence if 2° holds, then we have 1°. The proof for the corresponding strong duality equalities is similar.

D

Remark 1.28. Besides the close connection between Lagrangian and surrogate du-> R WQ ality theorems, mentioned in Remark 1.26(a), for any function f:X have the following obvious relation between Lagrangian dual values ^Lagr (see, e.g., (1.268)) and surrogate dual valuesfi^unof type (1.355): inf/(G) > y^suiT := = sup

sup oex*\{0} inf

inf f(y) y^^ cD(v)<sup4)(G) f{y) > sup inf {/(g) + 0(g) + - sup 0(G)}

,(x) G CD,(G)} - 0.

(1.381)

Proof The proof is similar to the above proof of Theorem 1.19, using now the set Q = [yeX\ PT(0) = sup v,ez*{^(0) + Lix, ^ ) } = sup L(jc, vl/) (jc G X),

(1.401)

vPeZ*

and hence by the inequality inf sup > sup inf and (1.400), we obtain the "duality inequality"

74

1. Preliminaries a = inf 0(Z) > inf sup L(x, ^) > sup inf L(x, ^) =

fi.

(1.402)

Actually, one is interested in obtaining conditions for "weak duality," i.e., the equality a = inf sup L(x, ^) = sup inf L{x, ^) =

fi,

(1.403)

or strong duality, i.e., (1.403), with the second sup of (1.403) attained for some ^0 ^ ^*; in general, it is convenient to use, to this end, some minimax theorems, such as Theorems 1.8, 1.9. If in addition Px(0) = p**(0) (x e X) (e.g., if for each x e Z the partial function px of (1.397) is proper, convex, and lower semicontinuous), then, similarly to (1.401), there follows (P(x) = PAO) = PT(0) = sup vi;ez*{^(0) + L(x, vl/)} = sup L(jc,vl/) (x G Z ) ,

(1.404)

vl/eZ*

and thus in this case, a = inf sup L(jc, ^ ) .

(1.405)

Remark 1.30. (a) If px = /?** (x e X) (i.e., if for each JC e Z the partial function Px of (1.397) is proper, convex, and lower semicontinuous), then for all jc e X and z e Z WQ have p(x, z) = Pxiz) = PT(Z) = sup {vl/(z) - /.^(vl/)} VI>GZ*

= sup {vI/(z) + L(x,vI/)},

(1.406)

^eZ*

which expresses p with the aid of L. (b) It is well known and easy to show (see, e.g., Ekeland and Temam [54, Ch. Ill, Lemma 2.1 and Remark 2.1]) that if X and Z are linear spaces and f: X ^^ R is convex, then so is the "(optimal) value function" (called also "marginal function") v: Z -^ R (where v stands for "value") defined by (1.389); also, by (1.387) and (1.389), we have a = v(0). There are many duality results involving the value function v. For example, note that by (1.392) and (1.389), X(^) = inf inf {p{x, z) - ^(z)} = inf {inf p(x, z) - ^(z)} xeX zeZ

zeZ

= inf {viz) - ^(z)} = -i;*(^)

xeX

(^ e Z*).

(1.407)

zeZ

Also, by (1.394) and (1.407) we have P = sup A.(^) = sup {vI/(0) - i;*(^)} = i;**(0); ^GZ*

vi/eZ*

hence weak duality a = ^ holds if and only if v{0) = i;**(0).

(1.408)

1.4 Duality for convex and quasi-convex infimization

75

By (1.401) and (1.399), we have 0(jc) > L(jc, vl/) > A,(vl/)

(x eX,^

e Z*).

(1.409)

A pair (XQ, ^O) G X X Z* is called a saddle point of L if L(x, ^o) > ^(^0, ^o) > L(xo, vl/) When pAO) = p^i^),

(jc G X, ^ G Z*).

(1.410)

by (1.404) and (1.399) condition (1.410) is equivalent to 0(xo) = L(xo,^o) = ^(^o).

(1-411)

Theorem 1.23. If (1.405) holds, then for a pair (XQ, ^O) e X X Z* the following statements are equivalent: 1°. xo G X is a solution of the primal problem (P) of (1.386), ^o G Z* is a solution of the dual problem (D) of (1.391), and we have min0(X) = maxX(Z*).

(1.412)

2°. (xo, ^o) is a saddle point of the Lagrangian L. Proof See, e.g., [185, Theorem 2] or [54, Ch. Ill, Proposition 3.1].

D

One can show (see, e.g., [185]) that this duality theory is symmetric; i.e., one can embed the dual problem (D) of (1.391) into a family of perturbed problems that generates, as the dual problem to (D), the initial problem (P). The above scheme encompasses as particular cases many known unperturbational dual problems to convex infimization problems. For example, given a linear system (X, Z, u) (see Definition 1.3 (b)), let / : X -> /? and h\ Z ^^ /? be two convex functions, and let us consider the primal infimization problem {P)

oc=mf{f{x)^h{u{x))].

(1.413)

xeX

In what follows, for simplicity, when dealing with the composition of two functions, we shall omit the symbol of composition o between them; thus, instead of {h ou){x) and ( ^ o w)(jc) we shall write hu{x) and ^w(x), respectively. For problem (P) of (1.413), let (t) = f-\-hu{=

f + {hou))

(1.414)

and let us define the perturbation function p: X x Z -^ Rby p(x, z) := fix) + h(u(x) -z)

{xeX,ze

Z),

(1.415)

which satisfies (1.387). Then, by (1.398), (1.397), (1.415), (1.86), and (1.91), we have

76

1. Preliminaries - L ( x , vl/) = p;(vl/) = sup {^(z) - p(x, z)] zeZ

= sup {-^(u(x)

- z) + ^(u(x))

- fix) + -h(u(x)

- z)}

zeZ

= -fix)

+ ^iuix))

+ sup [-^iuix)

-z) + -hiuix)

- z)}

zeZ

= -fix)

+ ^iuix))

-h h\-^)

(jc G X, vl/ G Z*),

(1.416)

and hence by (1.399) and (1.391), the dual objective function k and the dual problem (D)are A.(vl/) = inf L(x, ^) = inf [fix) - ^uix) + -/i*(-vl/)} xeX

(D)

xeX

=-/*(VI/M) f-/z*(-4') (^ G Z*), ^ - sup {-/*(VI/M) + -/z*(-vl/)}.

(1.417) (1.418)

Given a convex system (X, Z, u) (see Definition 1.3 (c)) and / , h, (P), 0, and p as above, the dual objective function A. and the dual problem (D) are still (1.417) and (1.418) respectively, but with ^u e R instead of ^w G X* (because we apply X

Fenchel-Moreau conjugation (1.206) to W = R , instead ofW = X*). Remark 1.31. For any (not necessarily linear) mapping u: X ^^ Z and any ^ G Z*, ^ o w = vi/w is denoted by M * ( ^ ) , where u*: Z* -> X* is the "adjoint" of w, so M* is defined by w*(^)(jc) := ^(M(JC))

ix eX,^

e Z*);

(1.419)

however, in the present chapter we shall not use this notation in order to avoid confusion with the Fenchel and Fenchel-Moreau conjugate functions. From the above we obtain the following result, whose part (a) is a classical theorem of Fenchel-Rockafellar: Theorem 1.24. (a) (See [183].) Let (X, Z, u) be a linear system and / : X -> R, h: Z ^^ R two convex functions for which there exists an element XQ G d o m / such that h is finite and continuous at M(JCO). Then inf {fix) + hiuix))} = max { - / * ( ^ M ) -f - / z * ( - ^ ) } . xeX

(1.420)

vi/ez*

(b) Let iX, Z,u) be a convex system, where Z = iZ, R. Proof (a) Let (X, Z, w) be a linear system. By /(JCQ) < +00, /z(w(xo)) < +00, we have inf;cGX {fix) 4- hiuix))} < +00. Furthermore, since u is linear and / , h are

1.4 Duality for convex and quasi-convex infimization

77

convex, 0 of (1.414) and p of (1.415) (which satisfies (1.387)) are convex as well. Finally, since h is continuous at w(xo), the function Pxo' z^

p(xo, z) = f(xo) 4- hiu(xo) - z)

(1.421)

is finite and continuous at z = 0. Hence, by Proposition 1.6 and (1.417), we obtain (1.420). (b) Let (X, Z, u) be a convex system. Then infj^ex {fM + h(u(x))} < -\~oo (as in part (a)). Let us observe now that for any increasing convex function h: X ^^ R, the function hu is convex; indeed, since u is convex and h is increasing, for any x\,X2 ^ X and 0 < c < 1 we have h(u(cxi + (1 — c)x2)) < h(cu(xi) -f- (1 — c)u(x2)), which, since h is convex, is < ch(u(x\)) + (1 - c)h(u(x2)). Also, 0 of (1.414) is convex (since so are its summands), whence so is p of (1.415), and as in part (a), the function (1.421) is finite and continuous at z = 0. Hence, by Proposition 1.6 and (1.417), we obtain (1.420). D Remark 1.32. In the particular case that Z = X is a locally convex space and u = Ix, the identity operator in X (i.e., u(x) — x for all x e X), Theorem 1.24 (a) yields the following classical result (see, e.g., [183, 185]) on the problem of the infimization of the (upper) sum / -j- /z of two convex functions, that is, the problem (P)

a= inf {fix)+

hix)}:

(1.422)

xeX

If X is a locally convex space and f,h\ X -> R are two convex functions for which there exists an element XQ e dom / such that h isfiniteand continuous at XQ, then inf {fix) + hix)} = max {-/*(^) + -h\-^)}. jceX

(1.423)

^eX*

However, this result does not imply directly Theorem 1.24 (a) above when applied to the convex functions f: X -^ R and hu\ X ^> R, since its assumption is that hu is continuous at XQ, while in Theorem 1.24 (a) it is assumed only that h is continuous at w(xo). Note that formula (1.423) is symmetric in / and /z, since max {-/z*(vl/) + -/*(-vI/)} = max {-/*(^) -h

-h\-^)}.

Hence instead of assuming in the above result that h isfiniteand continuous at some xo € dom / , we may assume that f is finite and continuous at some XQ G dom/i. Let us give now an application to the primal "programming" problem (1.298), in the particular case that Z = (Z, R be a quasi-convex function that is upper semicontinuous along lines {i.e., for every X\,X2 G /?", ^/{r]) := r]X\ + (1 — r])x2 is an upper semicontinuous function of r] for rj e [0, 1]). If inf xeR" f{x) is finite, then we have (1.460) with u{x)J ^(.V)>^

with L^ 1 = I supOo(G) - o(xo)|, so (2.44) is not satisfied.

(2.50)

98

2. Worst Approximation

Theorem 2.6. Let X be a normed linear space, XQ e X, and G a subset of X such that G / {^o}- For an element go e G, the following statements are equivalent: WgoeTcixo). 2°. There exists OQ G X* such that (2.51)

'i'oigo --^o) = 1,

1

llgo--xoll =

1

HI

11^0

(2.52)

r

1

max

=

3geG ,^(g)>«I>Uo)+l

II CD

(2.53)

IfG is weakly compact, these statements are equivalent to: 3°. There exists ^'^ e X* satisfying (2.51), (2.52), and max

k>n II

eX* ""

IIO

-.

(2.54)

supOUo)+l

Proof r => 2°. We shall show that if % e X* satisfies (2.51)-(2.53), then 4>o := -[T^^o satisfies (2.36), (2.42), and (2.44). Indeed, (2.36) is obvious. Also, by (2.53), (2-33), (2.10), ^'^ ^ 0, and (2.51), we have 1

1

I sup CD(G) - cD(xo)| |supcD;j(G) - ^^-n n— Wioi llll \\%\\ ^ %igo) - o(xo) ^ 1 (2.55)

=

1 ^ 0 II

ll^oll

which yields (2.44). Finally, by (2.55) and go ^ G, we obtain ^ ,

,

%(So - xo)

I sup %(G) - %{xo)\

1 ^ 0 II

11^0 I

> sup Oo(G) - cDo(xo) > cDo(go - -^o), whence (2.42). 2° =^ 1°. We shall show that if G 7^ {xo} and CDQ e X* satisfies (2.36), (2.42), and (2.44), then sup cDo(G) ^ cDo(xo) and cD^^ := ,^^^^^G)- /?, in this chapter we shall give duality results for the primal supremization problem (n

=

(PGJ)

« ' = < / = sup / ( G ) .

(3.1)

Any go e G for which the sup in (3.1) is attained, i.e., such that /(go) = sup/(G),

(3.2)

is called an optimal solution of problem (P^); these will be studied in Chapter 4. The set of all optimal solutions will be denoted by Mcif), that is, Mcif)

:= {go e G\ figo) = sup/(G)};

(3.3)

naturally, one can also write max instead of sup in (3.2) and (3.3). If / is a quasiconvex function, then (P^) of (3.1) is called a problem of quasi-convex supremization. Taking / ' := —/, which is a quasi-concave function, one can also write (3.1) as the infimization problem -a' = - s u p / ( G ) = i n f / ( G ) ;

(3.4)

thus, quasi-convex supremization is equivalent to quasi-concave infimization. However, here we shall consider only quasi-convex supremization. In contrast to the cases of convex and quasi-convex infimization (see Chapter 1, Section 1.4), it will turn out that for quasi-convex supremization the theory of surrogate duality is more developed (see Sections 3.1-3.3) than the theory of Lagrangian duality (see Section 3.4).

102

3. Duality for Quasi-convex Supremization

Our starting point for the study of surrogate duality will be the observation that worst approximation may be regarded as a particular case of supremization, by taking X to be a normed linear space, XQ e X, and / : X ^- /? the convex function (1.264); indeed, then sup/(G) = 5(G,xo),

(3.5)

the deviation (2.1) of G from XQ, and, for this case, the optimal solutions go ^ G of problem (P^) are the elements of worst approximation of xo by G. Although the extension from the particular function / of (1.264) to a function f: X -> R on a locally convex space X is a rather big step, it turns out that similarly to the case of passing from best approximation by convex sets to convex infimization, many results and methods of the theory of worst approximation can be extended to results on the supremization of functions. Similarly to the fact that formula (1.249) on the distance to a convex set extends to the surrogate duality formula (1.330) on quasi-convex infimization, it is natural to expect that formula (2.11) on the deviation will extend, under certain assumptions on G and / , to a formula like sup/(G) =

sup

inf

f{y),

(3.6)

^' '(G)

obtained formally by replacing in (2.11) the function / of (1.264) by a function / on a locally convex space X\ this will be achieved in Section 3.1. Next, corresponding to formula (1.355) on infimization, one would like to replace the hyperplanes [y e X\ 0(y) = sup 0(G)} of (3.6) by other sets, e.g., closed half-spaces. Therefore, in Section 3.2, we shall consider "unconstrained surrogate dual problems" to problem (P^) of (3.1), defined as supremization problems of the form ^ ' = supA.^(X*\{0}),

(3.7)

where X*\{0} is the dual set (unconstrained), and X^ = A.^ ^: X*\{0} -> P is a function (the dual objective function, depending on G and / ) of the form A^(cD) = inf /(^G.o)

(^ € X*\{0}),

(3.8)

with {^G,O}OGX*\{0} being a family of subsets of X related in some way to G. The right-hand side of (3.6) is indeed of the form (3.7), with k^ of the form (3.8), where the surrogate constraint sets ^G,4> are the hyperplanes ^G,^ = [yeX\

0 ( j ) = sup cD(G)}

(O G Z*\{0}).

(3.9)

Problem (3.7), with X^ of (3.8), is an unperturbational dual problem to ( P ^ , since it is defined directly, without using the method of embedding first (P^) into a family of perturbed primal problems, and it is a surrogate dual problem to (P^), since it replaces the primal constraint set G of (3.1) by a family of "surrogate constraint

3.1 Some hyperplane theorems of surrogate duality

103

sets" ^G,cD c X (O e X*\{0}) (while it keeps the primal objective function / unchanged). Next, more generally, in view of further applications, given an arbitrary set Z, a subset G of X and a function / : X ^ /?, for the supremization problem (P^) of (3.1) we shall consider in Section 3.3 a "surrogate dual problem" of the form ^ ^ = ^ ^ ^ = supA(W),

(3.10)

where W = WQ ^ is a. set (the dual constraint set) and A = A^ y^: W ^^ /? is the function (the dual objective function) defined by k^GjM = inf fiQcw)

(w e W),

(3.11)

with {^G,w}wew being a family of subsets of X related in some way to G. Then, taking Z to be a locally convex space, W = X*\{0}, and A = A^ of (3.11), problem (3.10) reduces to problem (3.7), (3.8). Furthermore, taking X to be a locally convex space, W c Z*\{0} or W c (X*\{0}) x R, and X = X' of (3.11), we shall obtain some useful unconstrained and "constrained" surrogate dual problems to problem (P^) of (3.1). Actually, instead of {^G,w}wew, we shall find it more convenient to use the equivalent language of polarities A: 2^ -^ 2^ (this will be explained in Section 3.2). In Section 3.4 we shall deal with Lagrangian dual problems to problem (P^) of (3.1). Finally, the general dual problem (3.10) will permit us to study (unconstrained and constrained) surrogate duality for more structured primal supremization problems (i.e., in which the primal constraint set G is expressed in more structured ways), by considering suitable dual constraint sets W and dual objective functions A = r ^ ^ : W - ^ ^ as in (3.11) (see Section 3.5).

3.1 Some hyperplane theorems of surrogate duality In this section we shall give some hyperplane theorems of surrogate duality, generalizing the (equivalent) geometric forms (2.11), (2.13) of Chapter 2, Theorem 2.1. Let us first give a lemma, in a somewhat more general form than needed in the sequel. For a linear space X, we shall denote by X* the set of all linear (not necessarily continuous) functions O: X -> /?. Lemma 3.1. Let X be a linear space, O e X*\{0}, and f: X ^ function, and let co(d):=

inf f(y)

(d e R).

R a convex (3.12)

yeX

If (D{d) > - o o {d e /?), then CO is finite and convex, and hence continuous on R.

(3.13)

104

3. Duality for Quasi-convex Supremization

Proof. Since O / 0, we have [y e X\ ^(y) = J} ^ 0, so (jo(d) < +oo (d e R), whence by (3.13), a)(R) c^ R. Lctdud2 e R,0 < IX < 1, and £ > 0. Then by (3.12) and co(R) c R, there exist y[,y2e X with cD(jj) = d\, ^(y!^) = d2, such that fiyD < co(di) -\-s (i = 1,2). But then, since O is Unear and / is convex, we obtain a)(ixdi + (1 - /x)^2) =

inf

f{y)

yeX 0(>')=M+(1-Ai)^2

< / ( / x j ; + (1 - /x)y^) < nf(y[) + (1 - /x)/(j2) < iio)(di) + (1 - /x)co(d2) + £, which, since 0 were arbitrary, proves that co is convex on R. Hence, by a well-known property of finite convex functions (see, e.g., [104], Chapter I, Theorem 3.1.1), a> is continuous on R. D Now we can prove the following theorem: Theorem 3.1. Let X be a locally convex space, with conjugate space X*, and G a subset of X. (3) If f: X ^^ R is a lower semicontinuous quasi-conv ex function, then sup/(G)
- 0 0

(O G Z*\{0}, d e R),

(3.15)

yeX ^(y)=d

or G is weakly compact and f: X -> R is an arbitrary function, then sup/(G) > sup inf f(y).

(3.16)

^' ^4)(y)=sup R is a lower semicontinuous quasi-convex function, then we have the equality (3.6). Proof (a) Let / : Z -^ /? be a lower semicontinuous quasi-convex function, and assume, a contrario, that sup/(G) >

sup

inf

f(y).

(3.17)

f(y).

(3.18)

0 such that /(go)-^>

sup oexnio}

inf y^^

3.1 Some hypeq^lane theorems of surrogate duality

105

Hence, /(go)-^>

inf

f{y)

(OGX*),

(3.19)

yeX 4>(>')=sup4)(G)

and thus for any O e Z*\{0} there exists y = y^ e X with 0(3;) = sup 0(G), /(go) - e > f(y).

(3.20)

Case r . /(go) < +00. Let ^/(.o)-.(/) := {y ^ ^1 f(y) < f(8o) - e},

(3.21)

Then, by (3.20), Sf(gQ)-s(f) 7^ 0. Furthermore, since / is a lower semicontinuous quasi-convex function, Sf^g^^sif) is a closed convex set; also, since /(go) < +C)0, we have go ^ 5'/(gQ)_e(/). Hence, by the strict separation theorem, there exists Oo G X*\{0} such that Oo(go) > sup cDo(5/(,,)_,(/)).

(3.22)

We claim that inf

f(y) > /(go) - e;

(3.23)

^o(>')=supo(G)

indeed, otherwise, there would exist yo ^ X with Oo(jo) = supOo(G) such that f(yo) < /(go) - £ (so, yo e 5/(^,)_,(/)), whence by (3.22), Oo(jo) = sup Oo(go) > supOo(S/(^,)_^(/)) > ^o(yo). which is impossible. But (3.23) contradicts (3.18). Thus (3.17) cannot hold, which proves (3.14) for case 1°. Case 2°. /(go) = +00. Let d e Rbe any number such that d >

sup ^'

inf

f(y).

(3.24)

(y)=sup^(G)

Then by (3.24), SAf) 7^ 0, and by /(go) = +cx), we have go ^ SAf). so the above argument of case 1° yields (3.23) with /(go) - £ replaced by d, which contradicts (3.24). This proves (3.14) for case 2°. (b) Let G c X be a (nonempty) bounded set (hence supO(G) e R for all O G X*), and f: X ^ R a convex function satisfying (3.15). Then by Lemma 3.1, CO of (3.12) is continuous on R, for all O G X*\{0}. Assume now, a contrario, that sup/(G)
(G)

Indeed, the inequality < in (3.30) is obvious. On the other hand, for each ^ e X*\{0} and for OQ = 0 we have inf yeX ^{y)=sup(G)

/(^)>inf/(X)=

inf

f(y),

yeX a)o(v)=supOo(G)

whence we obtain the inequality > in (3.30). However, formula (3.6) has the advantage that it is a "hyperplane theorem" (by 3.29)), while for OQ = 0 we have {y e X\ ^o(y) = sup Oo(G)} = X, which is not a hyperplane. (c) The assumption of boundedness of G cannot be omitted in parts (b) and (c) of Theorem 3.1. Indeed, for example, if X = G = R and f(y) = 1 for all y e X, then the right-hand sides of (3.16) and (3.6) are -\-oo (since supO(G) = -hoo, so {y eX\

cD(j)

= sup 0(G)}

= 0, for all O G X * \ { 0 } ) , but sup / ( G )

= 1.

3.1 Some hyperplane theorems of surrogate duality

107

In the particular case G = [XQ] (a singleton, hence weakly compact), from Theorem 3.1 we obtain the following corollary: Corollary 3.1. Let X be a locally convex space, XQ e X, and f: X ^^ R a lower semicontinuous quasi-convex function. Then f(xo)=

sup

inf

f(y).

(3.31)

Remark 3.2. Formula (3.31) admits the geometric interpretation fixo) = sup i n f / ( / / ) ,

(3.32)

Hen xoeH

where H denotes the family of all hyperplanes in X. The sup in (3.31) need not be attained, even when / is finite, as shown by the following example: Example 3.1. Let 5 be a nonreflexive Banach space, let X = B*, endowed with the weak* topology a{B*, B), and let f(y) = \\y\\

(yeX).

(3.33)

Then Z is a locally convex space and / is a finite lower semicontinuous convex function on X (but it is not continuous at any XQ e X). Hence, by Corollary 3.1, we have (3.31), which, since X* = (B*, a(B*, B))* can be identified with B, means that ||xol|=

sup

inf

||x||=

sup disi(0,Hb,,,)

(xo e B'),

(3.34)

where H,,,, = {xeB''\xib)=xo(b)},

(3.35)

Thus, for each b e B, Hbx^ is a hyperplane in B*, and therefore, by Lemma 1.5, dist(0, Hb,xo) = l-^o(^)l/ ll^ll- Consequently, (3.34) becomes the well-known formula lko||=

sup 1 ^ ^ beB\{0}

(xoeB^).

(3.36)

ll^ll

But since B is nonreflexive, by a well-known theorem of R.C. James (see, e.g., [40], p. 63, part (7)), there exists XQ e B* for which the sup in (3.36), and hence in (3.34), is not attained. For the next hyperplane theorem we need some preparation.

108

3. Duality for Quasi-convex Supremization

Lemma 3.2. Let X be a locally convex space, ^ e X*\{0} and f: X -> R a function such that the (possibly empty) sets ^r = [ 2^ a polarity, and a e R. The following statements are equivalent: 1°. We have a>^l

= sup inf /(CA^({it;})).

(3.50)

2°. We have Ad{f) n ZA\{W})

7^ 0

{w eW,deR,d>a).

(3.51)

7^ 0

{w eW,deR,d>oi).

(3.52)

3°. We have Sdif) n ZA\[W])

3.2 Unconstrained surrogate dual problems for quasi-convex supremization

111

Proof, r =4^ 2°. If r holds, then for each w; G W and J G /?, J > a, we have d > inf f(CA\{w})), whence by Lemma 3.4(a), we obtain (3.51). The implication 2° =^ 3° is obvious. 3° =» 1°. If 3° holds, then by Lemma 3.4 (b), we have X'^(w) = inf f(ZA\{w}))

a),

whence ^^ = sup A,^(W) < infj>(^ d — ot.

•

Proposition 3.2. Let X, W be two sets, f: X ^ ^ a function, A: 2^ -> 2^ a polarity, and a G R. The following statements are equivalent: r. We have a 2° is obvious. 2° => 1°. Ifd and wj are as in 2°, then by Lemma 3.4(a), we have X^iWd) = mf

f(CA\{w,}))>d.

whence ^^ = sup A,^(W) > sup^^^ d = a.

•

Combining Propositions 3.1 and 3.2, we obtain the following result: Theorem 3.3. Let X, W be two sets, f: X ^^ R a function, A: 2^ -> 2 ^ a polarity, and a G R. The following statements are equivalent: 1°. We have a = Pl=

sup inf f(CA\[w}))\

(3.56)

weW

2°. We have (3.51) and for each d e R, d < a, there exists Wd e W satisfying (3.54). 3°. We have (3.52) and for each d e R, d < a, there exists Wd ^ W satisfying (3.55).

112

3. Duality for Quasi-convex Supremization

Now we shall give, for the case when a = a^ of (3.1), some convenient sufficient conditions in order that a' > yS^ or a' < ^^ or a' = yS^- ^^ ^^is end, let us first prove a lemma: Lemma 3.5. Let X and W be two sets, A: 2^ ^ following statements are equivalent: 1°. We have

2^ a polarity and XQ e X. The

H[xo]) = 0.

(3.57)

2°. We have xo ^ U^ew^\M).

(3.58)

Proof. If (3.58) does not hold, i.e., if there exists WQ ^ W such that XQ e A'({ifo}), then A({jco}) ^ AA'({u;o}) ^ w^o, so (3.57) does not hold. Conversely, if we do not have (3.57), i.e., if there exists WQ e A({jco}), then A\{wo}) 5 A'A({jco}) 3 JCo, so (3.58) does not hold. D Theorem 3.4. Let X and W be two sets, f: X ^ R a function, and AG : 2^ -)• 2 ^ (G C X) a family of polarities such that for any G C. X we have A{,}(lg}) = id

(geG).

(3.59)

inf/(CA^c({u;})) < supinf/(C A;^J({U;}))

(W

e W).

(3.60)

geG

Then, given G C. X, sup/(G)>)Sl^.

(3.61)

Moreover, if we have (3.59), (3.60) and f is A^^ Ac-quasi-convex, then sup/(G) = ^ l ^ .

(3.62)

Proof By (3.59) and Lemma 3.5, we have g e C Aj^}({w;}) (g e G,w e W), whence inf/(CA;^J({W;})) < f{g) (g eG,w e W). Therefore, by (3.60), inf/(CA^^({u;})) < supinf/(CA;^j({u;})) < sup/(G)

(w e W),

geG

and hence by (3.42), we obtain (3.61). Furthermore, if also / is Aj^Ac-quasiconvex, then by (3.111) below (applied to A = AG), we have sup / ( G ) =

sup

inf f(CA'a({w})) < sup inf f{CA'^({w})) = ^ 1 ^ ,

weCAciG)

whence by (3.61), we obtain (3.62).

ujeW

D

3.2 Unconstrained surrogate dual problems for quasi-convex supremization

113

Theorem 3.5. Let X and W be two sets, G a subset of X, f: X -> R a function, and A: 2^ -> 2 ^ a polarity. The following statements are equivalent, where a, = a'= sup f(G): r. We have

2°. We have

3°. We have

4°. We have

{a=)supfiG)sup Q^Q

inf

fix)

(O G X*\{0}).

(3.106)

xeX

but we cannot obtain conditions for the opposite inequality and the equality in Theorem 3.10 by applying Theorem 3.4 to A = A^, because of (1.162).

3.3 Constrained surrogate dual problems for quasi-convex supremization

121

3.3 Constrained surrogate dual problems for quasi-convex supremization In this section we shall consider "constrained surrogate dual problems" to problem (P^) of (3.1), defined as supremization problems of the form ^^ = sup X^(Wlj), where the dual constraint set WQ is a proper subset either of an arbitrary set W, or of (X*\{0}) X R, or of X*\{0}, depending on G, and the dual objective function is (3.11). For the families {^G,( of the subsequent results, which hold for arbitrary functions f:X -^ R, but only the equality parts. Lemma 3.7. For any family of sets {A/}/^/ and any function f: have

U/e/ A/ —> R, we

inf inf/(Ay) = inf/(U/,/A,),

(3.107)

iel

sup sup/(A/) = sup/(U/e/A/).

(3.108)

iel

Proof The inequality > in (3.107) is obvious. Conversely, for each /x > inf/(U/e/A/) there exists a^ e U/^/A/, whence a^ e A/^ for some /^ ^ L such that M > /(«/x) > inf/(A/^) > infinf/(A/), whence, since /x > inf/(U/^/A/) was arbitrary, we obtain (3.107). This formula implies (3.108), since -sup/(U,e/A,) = i n f ( - / ) ( U , e / A , ) = infinf(-/)(A,) iel

= inf ( - s u p / ( A / ) ) = - s u p sup/(A/). i^^

D

iel

The following general duality theorem will be applied to various special polarities A: 2^ ^ 2^x*\m^R and A: 2^ ^ 2^*\{0}^ Theorem 3.11. Let X be a set, W ^ ^^, A: 2^ -> 2^ a polarity, f \ X -^ ~R a A' A-quasi-convex function, and G C. X. Then sup / ( G ) = - i n f f^^^\CA(G)).

(3.109)

Proof Let us first observe that by (1.139) we have U,,C;(CA({^}))

= C(n,,aA({g})) =

CA(G).

(3.110)

Hence, since f: X ^^ Ris A^ A-quasi-convex, by (1.153), (1.144), Lemma 3.7, (3.110), and (1.223), we obtain

122

3. Duality for Quasi-convex Supremization sup / ( G ) = sup /q(A'A)(g) = sup g^G

geG

sup

sup

iuf

f(CA\{w}))

we{lA({g})

inf f(CA'{{w}))

u;eU,,G(CA({g}))

sup

i-f^^^\w))

= - i n f f^^^\CA(G)).

n

u;eCA(G)

Remark 3.9. (a) By (1.223) and (1.144), one can also write (3.109) in the form sup / ( G ) =

sup

inf/(CA'({W;})) =

.eCA(G)

sup -'^^^^^^

inf

/(JC),

(3.111)

u^etlix})

which expresses sup / ( G ) as a *'sup inf," similarly to the preceding duality formulas. (b) Theorem 3.11 gives explicidy the reladon between the constraint sets, and the reladon between the objective funcdons, of the primal problem (P^) and the dual problem. Indeed, by Theorem 3.11, if X is a set, V^ c ;^^, A: 2^ ^ 2 ^ is a polarity, f e R , and G c X, then the supremizadon problem (Z)A)

y^A = sup AA(CA(G)),

(3.112)

where AA(M;)

= -f^^^\w)

= inf f(CA\{w}))

(w e C A ( G ) ) ,

(3.113)

might be called the "(A-)dual problem" to (P') (of (3.1)), while the set C A ( G ) and the function X^ of (3.113) might be called the "(A-)dual constraint set" and the "(A-)dual objective function," respectively. However, it will be more convenient to consider, instead of (DA), the infimizadon problem (DA)

h

= inf ( - A A ( C A ( G ) ) ) = inf / ^ ^ ^ ^ ( C A ( G ) ) = -y^A,

(3.114)

with AA of (3.113), as the (A-)dual problem to (P^) (of (3.1)), since then we will obtain a symmetric duality between abstract quasi-convex supremization problems and infimization problems with an abstract reverse convex constraint set (see Chapter 6, Remark 6.15 (b)). (c) Formulas (3.112)-(3.114) are surrogate dual problems, with "surrogate constraint sets" CA'({U;}) (W G C A ( G ) ) , instead of the inidal constraint set G of (3.1). Note that each A\{w}) (w e W) is A'A-convex (since A'AA'({M;}) = A\{w})), so each CA^({W;}) in (3.113) is a reverse A'A-convex constraint set. Let us first apply Theorem 3.11 to the special polarities A^^A^^,A^^:2^-> 2(x*\{0})xR ^^^ ^01^ ^02. 2X ^ 2^*\{0} of Section 1.2. (1) For the polarity A = A^^ of (1.189), we obtain the following corollary of Theorem 3.11: Corollary 3.9. Let Xbea locally convex space, / : X ^^ R a lower semicontinuous quasi-convex function, and G C X. Then

3.3 Constrained surrogate dual problems for quasi-convex supremization sup / ( G ) -

sup

inf

f(y).

123 (3.115)

(j)e(x*\mxR y^^^^ sup ^{G)>d

^(v)>^

Proof. For the polarity A = A^^ of (1.189) we have (1.190), so / is (A^^YA^^quasi-convex if and only if it is lower semicontinuous and quasi-convex. Hence, applyingfomiula(3.111) to A — A'^ we obtain (3.115). D Corollary 3.10. Let X be a locally convex space, f: X ^^ R a lower semicontinuous quasi-convex function, and G c. X. Then sup / ( G ) =

sup inf f(U),

(3.116)

where U denotes the family of all open half-spaces in X. Proof The open half-spaces in X are the sets of the form U^^d = [y^X\^{y)>d]^

(3.117)

where (cD, d) e (X*\{0}) x R, and sup d if and only if G n ^o,^ / 0Hence, (3.115) is equivalent to (3.116). D Remark 3.10. Formula (3.116) is another instance of the reduction principle: it reduces the computation of sup / ( G ) to the computation of inf f(U), for all U e U

with una

^0,

(2) For the polarity A = A^^ of (1.191), we obtain the following corollary of Theorem 3.11, which should be compared with Corollary 3.6: Corollary 3.11. Let X be a locally convex space, f: X -> R an evenly quasiconvex function, and G ^ X. Then sup / ( G ) =

sup

inf

fiy)=

(O,J)6(X*\{0})x/? y\^ 3geG,(g)>d ^ ( > ) > ^

sup

inf

(ct),g)eX*xG

-v^^ ^(>')>^(g)

f{y),

(3.118)

and if G is weakly compact, then sup / ( G ) =

sup

inf

f{y)=

sup cD(G)>^

^(>)>^

sup

inf

f(y).

(3.119)

cD(>')>supcD(G)

Proof For the polarity A = A^^ ^f (1.191) we have (1.192), so / is (A^2y^i2_ quasi-convex if and only if it is evenly quasi-convex. Hence, applying formula (3.111) to A = A^^, we obtain the first equality of (3.118). The second equality of (3.118) always holds, since sup

inf f(y) — sup

(cI>,J)e(X*\{0})x/? y^^ 3geG,cI>(g)>^ ^(>')>^

sup

OeX"^ {g4)^GxR ^{g)>d

= sup sup sup

inf

f{y)

J^^ ^ '^iy)>d

inf f{y) =

eX* geG deR >'^^ (g)>d iy)>d

sup

inf

f(y).

(^>,?)eX*xG ^^ >;^^^ ^ ^ ^(y)>^(8)

124

3. Duality for Quasi-convex Supremization

When G is weakly compact, the first equality of (3.119) follows from the first equality of (3.118), since sup (G) is attained for each O G X* (see Lemma 1.3). The second equality of (3.119) always holds, since sup

inf f(y) = sup

sup d ^(y^^^

sup

inf f(y) = sup

supO(G)>JO(j)>J

Corollary 3.12. Let X be a locally convex space, f:X convex function, and G c. X. Then sup / ( G ) =

sup

inf

fiy)-

•

0(>')>sup R an evenly quasi-

inf / ( V ) ,

(3.120)

VeV

where V denotes the family of all closed half-spaces in X. Proof The proof is similar to that of Corollary 3.10, using the fact that the closed half-spaces in X are the sets of the form V^^d = [x

> J},

GX|0(JC)

(3.121)

where ( inf /(X\{0}) = /(O),

whence, using that / is a lower semicontinuous quasi-convex function and (1.198), sup / ( G ) = sup / ( G \ { 0 } ) = sup / q ( G \ { 0 } ) = sup /q((A0iyA0i)(G\{0}).

126

3. Duality for Quasi-convex Supremization

Hence, by Theorem 3.11 applied to /q((Aoi)'AO') and G\{0}, and by (1.227) and ^L(A)L(AyL(A)^y.L(A)^^g obtain

sup / ( G ) = -inf/^^^"^(CAO^(G\{0})),

(3.127)

and thus by 0(0) = 0, (1.223) and (1.196), it follows that sup / ( G ) = -

inf

(-

OGX*\{0}\ supO(G)>l

inf

fix))=

.veC(AOi )'({(!>})

/

sup

inf

OGX*\{0} supO(G)>l

"i^,^ , '^^•'^^^

f(x).

D

Remark 3.13. The assumption G 7^ {0} cannot be omitted in Theorem 3.12. Indeed, for G = {0} we have sup / ( G ) = /(O), but {O e X*\{0}| supO(G) > 1} = 0, so the right-hand side of (3.126) is —00. Theorem 3.13. Let X be a locally convex space, f an evenly quasi-convex function satisfying (1.195), andG £X, G 7^ {0}. Then sup / ( G ) =

sup

inf

€X*\{0}

^ ^ l

fix).

(3.128)

Proof The proof is similar to that of Theorem 3.12, using now (1.201) and (1.199). D Remark 3.14. (a) Similarly to Remark 3.13, the assumption G 7^ {0} cannot be omitted in Theorem 3.13. (b) As an application to approximation, let us note that Theorem 3.13 yields again Theorem 2.3. Indeed, we may assume that JCQ = 0 and G 7^ {0}. Then, by Theorem 3.13 appHed to the function f(y) = \\y\\

(yeX),

(3.129)

which satisfies (1.195), we have sup llgll = geG

sup

dist(0, {yeX\

0 ( j ) > 1}),

(3.130)

cDeX*\{0} 3geG,{g)>\

whence by Corollary 1.4, we obtain (2.33) for XQ = 0. One can obtain duality theorems for sup / ( G ) for many other classes of functions f: X ^^ Rby choosing suitable polarities A such that / is A^ A-quasi-convex and applying Theorem 3.11. Indeed, let us give here an example of such a result. We recall that a set G c X is called R-evenly convex if it is the intersection of a family of open half-spaces whose closures do not contain 0, and a function / : X -> /? is called R-evenly quasi-convex if all Sdif) (d e R) are /^-evenly convex. Corollary 3.15. Let X be a locally convex space, f: X ^^ R an R-evenly quasiconvex function, and G C. X. Then sup / ( G ) =

sup eX* 3geG,(g)>-\

inf xeX ^(-^)>-l

fix).

(3.131)

3.4 Lagrangian duality for convex supremization

127

Proof. From the general form of open half-spaces, it follows that a set G c X is /^-evenly convex if and only if it is the intersection of a family of sets of the form [x e X\ 0(jc) < - 1 } , where O e X*. Hence, if we define a polarity A^"^: 2^ -^ 2^*\{0} b y

A^\C)

= {CD G X*\{0}| 0(c) < - 1 (c G C)}

(C c X),

(3.132)

then G is (A'^)'A^'^-convex if and only if it is /?-evenly convex, so / : X -> /? is (A^^)^A^'*-quasi-convex if and only if it is /?-evenly quasi-convex. Hence, formula (3.111) yields the result. D

3.4 Lagrangian duality for convex supremization 3.4,1 Unperturbational theory Theorem 3.14. Let X be a locally convex space, / : X ^- R a function, and G a subset of X. Then s u p / ( G ) > sup inf{/(j)-cD(y)-hsupO(G)}.

(3.133)

Moreover, if f is a proper lower semicontinuous convex function, then sup / ( G ) = sup inf [f{y) - (D(y) -h sup cD(G)}.

(3.134)

Proof Since G is nonempty, we have sup 0(G) > —oo (O e X*). Let O € X* and J G /?, J < sup 0(G). Then there exists ^' = g'^j e G such that O(g0 > ^. Consequently, sup / ( G ) > /(g^) > fig') - cD(g^) + J > inf {/(j) - 0(};) + J}, whence, since O e X* and J < sup 0(G) were arbitrary, we obtain (3.133). On the other hand, if / is a proper lower semicontinuous convex function, then by (1.99), we have fig) = sup {inf[/(j) - 0 ( j ) ] + ig)} < sup{inf[/(>;)-(D(y)] + supcD(G)}

ig e G).

Hence by (3.133) and (3.135), we obtain (3.134).

(3.135) D

Remark 3.15. (a) If (3.16) holds, then for any O e X*\{0} we have sup/(G) >

inf

fiy)>

yeX a)(>0=sup(G)

>

inf

inf

fiy)

yeX 4>(v)>sup(G)

{/(^)-(D(^)} + supa)(G)

yeX sup(G)

> inf [f(y)-^(y)} yeX

+sup ^{G),

(3.136)

128

3. Duality for Quasi-convex Supremization

whence, by Remark 3.1(b), sup/(G) > sup

inf

f(y)>

sup

(t>{y)=sup(G)

inf

f(y)

(p(y)>sup(G)

> s u p [ i n f [ / ( ^ ) - 0 ( > ; ) ] + supO(G)j,

(3.137)

which implies some relations between Lagrangian duality (3.134) and hyperplane and half-space theorems of surrogate duality (3.6) and (3.77) (for example, in this case the Lagrangian duality equality (3.134) implies the surrogate duality equalities (3.6) and 0.11)). (b) If G is bounded, then by the "substitution method" described in Remark 1.26 (a), combining Theorem 3.1 (c) and formula (1.283), with 4>o, d^ replaced by O and sup 0(G) respectively (which is a Lagrangian duality formula for the infimum of / on a hyperplane), we obtain sup/(G) =

sup

maxinf {/(>;)-6>;)+^supcD(G)},

(3.138)

from which one can deduce again the equality (3.134); however, the above direct method of proof is simpler. (c) In the particular case that X is a normed linear space and / is the finite continuous convex function (3.123), from (3.134) we obtain the following formula of Lagrangian duality for the deviation of a set G from JCQI sup \\g - xoW = g^G

sup

eX*\{0}

inf {||xo - y\\ - i n f { | | x o - j | | + ^ ( x o - y ) - c D ( x o ) + s u p 0 ( G ) } geG

y^^

> -0(jco) + supO(G)

(O eZMlcDII = 1),

whence s u p | | g - x o | | > sup {-cD(xo) + supcD(G)}. geG

(3.140)

eX*

\m=\ In order to prove the opposite inequality, let g e G and e > 0. Choose O^ e X* with II ^ i = 1 such that \g - XQ) > \\g - XQW - s. Then sup {-0(xo) + supcD(G)} >^\g-XQ)

>

\\g-xo\\-£,

ex* l|0|| = l

whence, since geG

and s > 0 were arbitrary, we obtain sup {-O(xo) + sup 0(G)} > sup \\g - xoll ^eX*

geG

\\n=\ which, together with (3.140), yields the equality (2.14).

3.4 Lagrangian duality for convex supremization

129

3A.2 Perturbational theory In this section we shall develop a perturbational theory of Lagrangian duality for convex supremization, by suitably modifying the one for quasi-convex infimization (See Chapter 1, Section 1.4.2). Assume that we are given a constrained primal supremization problem a = sup/(G),

(3.141)

where G is a subset of a locally convex space X and f: X -^ /? is a function. Clearly, sup/(G) = sup7(X),

(3.142)

where / : X -^ /? is the function defined by fix) ifxeG, —oo if jc e CG. Thus, problem (3.141) and the primal problem (P)

a = supf(X)

(3.143)

have the same value. Moreover, if / | G # — oo, which we shall assume in the sequel without any special mention, then problems (P) and (P) have the same optimal solutions; indeed, i^go ^ G, /(go) = sup / ( G ) , then f(gol = /(go) + ^Xcigo) = sup/(G) = sup/(X)^and conversely, if jco G X and f(xo) = s u p / ( Z ) , then = sup/(G) > - o o , whence xo e G /(•^o) t -XG(XO) = 7(xo) = supf(X) and f(xo) = sup / ( G ) . Therefore, we shall assume from the beginning that we are given an unconstrained primal supremization problem (P)

Qf = sup0(X),

(3.144)

and then, taking in particular 0 = f-\-—XG and a suitable permutation p, the duality theory for (P) of (3.144) will yield a duality theory for (P) of (3.141). We shall define a dual problem to the primal supremization problem (P) of (3.144) by embedding it into a family of "perturbed" supremization problems, as follows. Let Z be a locally convex space (called a set of "perturbations" or of "parameters"), and p: X x Z -> /? a function (called a "perturbation function") such that /?(jc,O)=0(jc)

(JCGX),

(3.145)

so (P) of (3.144) is nothing other than a = supp(x,0); xeX thus, (P) is embedded into the family of supremization problems (P)

(3.146)

130

3. Duality for Quasi-convex Supremization (P,)

v(z) := SUP/7U, z)

(z e Z).

(3.147)

Let us define the Lagrangian dual problem associated with the perturbation function p as the unconstrained supremization problem (D)

^:=supA(Z*),

(3.148)

where X: Z* ^^ Ris the dual objective function defined by A(vl/) := sup {inf {p(x. z) - ^(z)}} xeX

(^ e Z*).

(3.149)

e Z*)

(3.150)

2^2

The function L: X x Z* ^^ R defined by L(x, vl/) := inf {p(x, z) - ^(z)}

(x eX,^

zeZ

is called the Lagrangian function, or simply the Lagrangian, associated with p\ note that this is the same as (1.396). Thus, considering the partial functions pAz) := p{x, z) (xeX^ze Z), (3.151) we have L(x, vl/) = M{pAz)

- ^(z)} = -p:m

(X € X, vl/ G Z*).

(3.152)

zeZ

By (3.145), (3.151), and (3.152), Hx) = PAO) > PT(0) = sup v,ez*{^(0) + L(x, vl/)} = sup L(jc,vl/) (x eX).

(3.153)

Furthermore, by (3.149) and (3.150), A(vl/) = supL(x, vl/)

(vl/ 6 Z*),

(3.154)

and hence by (3.148), P = sup supL(x, vl/). ^eZ*

(3.155)

xeX

Thus, by (3.153) and (3.155), a = sup0(X) > sup sup L(jc, ^) = j6.

(3.156)

.VGX vi/eZ*

If in addition, /7;,(0) = / ^ f W ' then (t>M = Px(0) = PT(0) = sup vi;ez*{^(0) + L(x, vj/)} = sup L(jc, ^ ) (jc G Z),

(3.157)

VI/GZ*

and thus in this case. a = sup0(X) = sup sup L(x, ^) = JCGXVI/GZ*

fi.

(3.158)

3.5 Duality for quasi-convex supremization over structured primal constraint sets

131

Remark 3.16. For the constrained primal supremization problem (3.141), let Z = X, 0 = / + — XG- Then the perturbation function p: X x X -^ R defined by p{x, z) := fix + z) +

-XGU)

fix + z) —oo

if.eG, if jc ^ G

satisfies (3.145), and the perturbational dual (3.148) yields the unperturbational dual of Section 3.4.1. Indeed, by (3.152) and (3.159), for any x G Z and vj/ G Z* we have L(x, vl/) = inf {p(x, z) - ^(z)}

(3.160)

zeZ

= inf {fix + z) - vl/(z) + vi/(;c) - vl/(x)} +

-xcix)

zeX

= inf {fix') - *(x')} + ^ix) + -XG(X)},

(3.161)

x'eX

whence, by (3.155) we obtain ^ = sup sup Lix, vi/) = sup inf {fix') - ^ix') + X G ( ^ ) } ' ^eX*xeX

vi>eX*-^'^^

which is nothing other than the right-hand side of (3.134).

3.5 Duality for quasi-convex supremization over structured primal constraint sets The primal constraint set G considered in the preceding sections of this chapter has been an arbitrary subset of a locally convex space X. Now we shall study some more structured ways of expressing the primal constraint sets G c X. In the present section we shall consider one of these ways, namely that of systems, and (surrogate and Lagrangian) duality for supremization in systems. We recall (see Chapter 1) that a system is a triple (X, Z, u), consisting of two sets X, Z and a mapping u: X -^ Z. Given a system (X, Z, u), a subset T of Z (called "target set"), and a function / : X —> /?, we shall consider the primal supremization problem ^ "^K-HT)./ = ^^P f^^^-

(3.162)

xeX u(x)eT

Remark 3.17. (a) If M(X) n 7 = 0, then w ' ^ r ) = {x e X|w(x) e T} = 0, whence a = sup0 = — oo. Therefore, in the sequel we shall assume, without any special mention, that w(X)nr#0.

(3.163)

132

3. Duality for Quasi-convex Supremization

(b) Problem (3.162) is equivalent to problem (P^) of (3.1). Indeed, given a system (X, Z, u) and T, f as above, problem (3.162) is nothing other than (3.1) with G = {x eX\ u{x) eT} = U-\T) (7^ 0).

(3.164)

Conversely, every problem (3.1) can be written in the form (3.162), by taking Z = X, u = Ix, the identity operator in X (i.e., u{x) = x for all x e X), and T = G. However, in the study of the "mathematical programming problem" (3.162) one can also use the properties of T and u. Now we shall assume that (X, Z, u) is a system in which X and Z are locally convex spaces, with conjugate spaces X* and Z*, 7 is a subset of Z, and f: X ^^ R is a function. There are several natural ways to introduce unconstrained dual problems to (3.162), which generalize the dual problems of the preceding sections. (l)Let W := M*(Z*)\{0} = [^u\ vl/ e Z*}\{0} ( c X*\{0}),

(3.165)

where w* is the adjoint operator of u (that is, w*(^)(jc) = ^u(x) for all x G X, ^ € Z*) and let A^,,^^^: 2^ -^ 2"*^^*^^^^^ be the polarity defined by ^ i - U r ) ( 0 := {w*(^) e w*(Z*)| w*(vl/)(c) < supuH^^)(u-\T)) = Ai_,(^)(C) n (w*(Z*)\{0})

(c e C)}

(C c X),

(3.166)

where A^_,^y,^: 2^ -> 2^*\^^^ is the polarity (1.154) (with G = u'^T)). since ^M(w-Hr)) = {^u(x)\ u(x) eT} =

VI/(M(X) DT)

(^ e Z*),

Note that (3.167)

we have, for any set C c X, Ai_,^y,^(C) = {^u\ ^ 6 Z*, ^(u(c)) < supvI/(M(X) n 7) (c e C)}.

(3.168)

Clearly, for the particular case X = Z, w = Ix (the identity operator on X), W = X*\{0} and T = G, the polarity A^_,^y,^ of (3.168) reduces to A]J of (1.154). In the converse direction, given any (X, Z, u) and T as above, by (3.166) we have Al_,^^^(C) c A ; ; _ , ( ^ / C ) ( C C X). For the polarity A = ^l-un ^^ (3.168), the dual objective function (3.41) and the dual value (3.42) respectively, become X'-,

(^M) =

^.-1(7-)

y6|,

inf

fix)

( ^ G Z*, ^u # 0),

(3.169)

xeX ^u(x)>sup^(u(X)r]T)

= sup l^^O

inf

fix).

(3.170)

^uix)>supyl'(u(X)nT)

Hence, by Remark 3.4 (b) (with W of (3.165)), we obtain the following generalization of Theorem 3.7 (c):

3.5 Duality for quasi-convex supremization over structured primal constraint sets

133

Theorem 3.15. Let (X, Z, u) be a system in which X and Z are locally convex spaces, let T be a subset of Z, and let f: X -> R be a function. If we have inf

f(x) < sup

inf

fix),

(3.171)

then for each d < ^^Pxex,u(x)eT /(-^) ^here exists ^ = ^ j e Z* with ^^w / 0 such that ^u(y)

< sup^{u(X)

n D

(ye SAf))

(3.172)

fix)'

(3.173)

if and only if sup fix) = sup

One can define, similarly, polarities

[^u 7^ 0| vl/ € Z*, sup ^uiO

inf

A;^.,^^,^

: 2^ -> 2"*^^*^^^^^ (/ = 2, 3, 4) by

< sup vl/(M(X) n 7)}

[^u / 0| vl/ e Z*, sup VI/(M(X) n

D

^ vi/f^(C)}

{vI/M ^ 0 | v l / e Z*,^w(C) c vi/(M(X)nr)}

(C c X),

(C c X),

(C c X),

(3.174) (3.175) (3.176)

and one can obtain for them results corresponding to those of Section 3.2. (2) Instead of A^,,^^,^: 2^ -> 2"*^^*^^^^^ of (3.168), let us consider the polarity A^V : 2^ -^ 2^*\^0J defined by A ' ! r ( 0 := [^ e Z*\{0}| ^uic) < s u p ^ ( 7 ) (c e C)}

(C c X);

(3.177)

thus, the only difference between (3.168) and (3.177) is that sup^iuiX) (1 T) is replaced by s u p ^ ( r ) . For A = A^'^^, the dual objective function (3.41) and the dual value (3.42) become A^2.^(vI/)= /S;„ =

mf sup

fix) inf

(vj/€ Z*\{0}), fix).

(3.178) (3.179)

^^^ \i^/ vl/M(A-)>supvl/(r)

Again, AJ^ of (1.154) is the particular case X = Z,u = Ix (the identity operator on X) and 7 = G, of the polarity A^'y^ of (3.177), but the converse direction no longer works, since we have only sup ^ ( M ( X ) DT) < sup ^iT), whence ^i, ,

Z, the family of polarities A|^_, depends on subsets u~\T) of X, while the family of polarities /s}Jj depends on subsets T of Z, so one needs some care when generahzing the expression A{g}({g}) of (3.59) to the family A^^^^. In order to obtain duaHty theorems using the polarities A^J^^ of (3.177), let us first give the following generalization of Theorem 3.4: Theorem3.16. Let (Z, Z,u) be a system (so X and Z are two sets andu: X -^ Z is a mapping), T a subset ofZ (satisfying (3.163)), W a set, f: X ^^ R a function, and Auj: 2^ -> 2^ (T c Z) a family of polarities such that AuAu(x)}({x}) = ^ (xeu-\T)), i n f / ( C A 1 ^ ( { I . } ) ) < sup i n f / ( C A ; ^,(,)j({u;}))

(3.181) (3.182)

(w e W).

xeX u(x)eT

Then, given T ^ Z, we have sup f(x) xeX u(x)eT

> ^l

^ (= sup inf / ( C A ; T({W}))). "' weW

(3.183)

Moreover, if we have (3.181), (3.182) and f is (AujY^uj-^l^^si-convex,

then

sup f{x)^fil^.

(3.184)

xeX u(x)eT

Proof By (3.181) and Lemma 3.5 applied to A = Au,{u(x)}^ we have ^ e ^KAU(X)}({^})

U

e u-'(T),

w e W),

whence inf/(CA;,

j,(,)j({u;})) < f(x)

(X e U-\T)^

W

e W).

Therefore, by (3.182), inf/(CA:,^({w;}))
2^^*\^^ /5 the polarity (1.189)) A'^(%,o)(/))^A^\G).

(4.4)

Proo/ By (1.190), / is lower semicontinuous quasi-convex if and only of it is (A^^)'A^^-quasi-convex, so the result follows from Theorem 4.1 applied io W = (Z*\{0}) X /?andA = A^^ D Corollary 4.2. Let X be a locally convex space, f: X -^ R an evenly quasi-convex function, and G c. X. For an element go e G, the following statements are equivalent: r . / ( g o ) = max/(G). 2°. We have (where A^^. 2^ -^ 2^^*^^^^^''^ is the polarity (1.191)) ^''(Sfiso)(f))

^ A^2(G).

(4.5)

Proof By (1.192), / is evenly quasi-convex if and only if it is (A ^^)^ A ^^-quasiconvex, so the result follows from Theorem 4.1 applied to W = (Z*\{0}) x R and A = Ai2. n Corollary 4.3. Let X be a locally convex space, / : X -^ R a lower semicontinuous quasi-convex function, and G C. X. For an element go e G, /(O) < f(go), the following statements are equivalent: 1°./(go) = max/(G). 2°. We have {where A^^: 2^ -> 2^*\^^^ is the polarity (1.196))

Proof 1° =^ 2°, by Remark 4.1 and since A^^ is antitone. 2° ^ 1°. By /(O) < /(go), we have 0 G Sf^g^){f). Hence if (4.6) holds, then since (A^^)^ is antitone, we obtain, by (1.197) and since / is a lower semicontinuous quasi-convex function, G c {A^'yA^\G)

c (A«^)^A«^(5/(,,)(/)) = co5/(,,)(/) = 5^(,o)(/).

•

Remark 4.2. Condition (4.6) can be also written as Sf(gQ){fy c G°, an inclusion between the usual polar sets (1.82).

4.1 Maximum points of quasi-convex functions

139

Corollary 4.4. Let X be a locally convex space, f: X -^ R an evenly quasi-convex function and G C Z. For an element go e G with /(O) < f(go), the following statements are equivalent: 1°./(go) = max/(G). 2°. We have {where A^^: 2^ -> 2^*\/ The proof is similar to the above proof of Corollary 4.3, using now (1.200). D Now we shall give some subdifferential characterizations of maximum points. To this end, let us first introduce the following class of abstract quasi-convex functions: Definition 4.1. Let X be a locally convex space. We shall say that a function f:X -> /? is strongly evenly quasi-convex if all Aj(/) {d G R) of (1.23) are evenly convex. Remark 4.3. (a) Every strongly evenly quasi-convex function f: X -^ Ris evenly quasi-convex, since Sdif) = n^>^A^(/) (d e R) and since the family of all evenly convex sets is closed for intersections. (b) Every upper semicontinuous quasi-convex function / : X -> R is strongly evenly quasi-convex (since each A^(/) is open and convex, and hence evenly convex). Proposition 4.1. Let X be a locally convex space, f: X -^ R a strongly evenly quasi-convex function, and XQ e X such that /(JCQ) G R. Then a^(^")/(xo)/0,

(4.8)

where A^^: 2^ -> 2^^*^^^^^''^ is the polarity (1.19\). Proof Since / is strongly evenly quasi-convex, the set Af(xQ){f) is evenly convex. Hence since XQ ^ Af(xQ)(f), there exists OQ G X * \ { 0 } such that if)).

(4.9)

Therefore, we have f(x) > f{xo) for allx e X with o(x) > Oo(xo), whence by (1.223) applied to IV = (X*\{0}) x R, fixo)^

min

/ ( x ) = -/^ R an upper semicontinuous quasi-convex function, and G C X. For an element go ^ G with /(go) ^ R, the following statements are equivalent: r . / ( g o ) = max/(G). 2°. We have 0 7^ a^^^"V(^o) ^ [(X*\{0}) x R]\A]^(G), and each (O, J) e 9^^"^ V(go) ^"^' «« optimal solution of the dual problem (D^n) (of (3.114)/or A = A^^X i.e., /^(^"HcD,^) = min /^^^"^([(X*\{0}) X R]\A'\G))

((O,

t/) € a^^^"V(go)).

(4.10)

3°. r/z^r^ exists (Oo, ^0) ^ 9^^^ ^/(go) ^^^^ '•5' «« optimal solution of the dual problem (D^n), i.e., such that /^(^">(Oo, Jo) = min /^^^"^([(X*\{0}) x

R]\A'\G)).

(4.11)

Proo/ Observe that since / is strongly evenly quasi-convex (by Remark 4.3 (b)), and /(go) e /?, we have a^^^"V(go) 7^ 0 (by Proposition 4.1). 1° ^ 2°. If 1° holds, then for any polarity A: 2^ ^ 2(^*\ 2(x*\{0})xR ^^^Yi that every upper semicontinuous quasi-convex function is A^Aquasi-convex,wehave,by(Oo, Jo) e a^^^^/(go) ^ [(X*\{0})x/?]\A(G), (1.232), (4.11) (for A), Theorem 3.11, and go e G, /(go) = -/^^^\4>o,Jo) = - m i n /^^^^([(X*\{0}) X R)]\A(G)) = max / ( G ) .

D

Proposition 4.2. L^r X be a locally convex space, f: X -^ R a strongly evenly quasi-convex function, and XQ e X such that /(O) < f(xo) < +00.

(4.13)

a^'^^'/Uo) ^ 0,

(4.14)

Then

where A"^ w the polarity (1.199).

4.1 Maximum points of quasi-convex functions

141

Proof. By the above proof of Proposition 4.1, there exists OQ ^ ^*\{0} satisfying (4.9), whence f{x) > /(XQ) for all x G X with O Q U ) > OO(JCO). But by (4.13) we have 0 G A/(;CQ)(/), whence by (4.9), 0 = Oo(0) < Oo(xo). Consequently, /(xo)=

min

f{x)=

min

/(x) = - / ^ ^ ^ " ^ ( - - ^ c D o ) ,

andthus^cDoGa^(^">/(xo).

•

Theorem 4.3. Let X be a locally convex space, f an upper semicontinuous quasiconvex function satisfying (1.195), and G C. X such that /(O) < sup / ( G ) . For an element go ^ G with /(go) ^ ^, the following statements are equivalent: 1°./(go) = max/(G). 2°. We have 9:^ ^^^^''^figo) ^ ( X * \ { 0 ] ) \ A 0 2 ( G ) , and each O G a^^^''V(^o) />y an optimal solution of the dual problem (D^oz) (of (3.114) for A = A^^), i.e., /^(A")() = min /^/teo)). (4.15)

3°. There exists OQ G 9^^^ ^ f(go) that is an optimal solution of the dual problem (D^oi), i.e., such that /^(^")(cI>o) = min /^^^"H(X*\{0})\A«2(G)).

(4.16)

Proof If 1° holds, then by our assumptions, /(O) < sup / ( G ) = /(go)- Hence since / is strongly evenly quasi-convex (by Remark 4.3 (b)) and /(go) G R, we have 9^^^ ^f(go) 7^ 0 (by Proposition 4.2). The remainder of the proof is similar to that of the above proof of Theorem 4.2, replacing (O, d), (4>o, do) G (X*\{0}) x R, and A^^ by O, Oo G X*\{0}, and A^^ respectively, and using Theorem 3.13. D Remark 4.4. (a) If X is a locally convex space, A^^: 2^ -> 2^*^^^^ is the polarity (1.199), / G ^ ^ , and xo e X with /(xo) G /?, then by (1.232) applied to A = A^^ we have d'^^^''^f(xo) = { 1, /(xo) = - / ^ ^ ^ ' ' ^ O o ) } .

(4.17)

(b) In the particular case when X = R'\ Thach ([272], Definition 2.2 and the remarks made after it) has introduced a similar subdifferential, namely 9^/(xo) = {Oo G X*\{0}| cDo(xo) = 1, f(xo) = -/""(Oo)},

(4.18)

where / ^ is the "quasi-conjugate" of / defined [272] by /^(A«^)(0) = - i n f ,ex f(x) 0(.v)>i - s u p f(X) thus in fact.

if O G X*\{0}, (4.19) if 0 = 0;

142

4. Optimal Solutions for Quasi-convex Maximization d"f(xo)

= {Oo e X*\{0}| c|>o(xo) = 1, f(xo) = -/''^^''^(Oo)}.

(4.20)

For this subdifferential, Thach has proved some results corresponding to Theorems 4.2 and 4.3 above (see [272], Theorems 2.6, 6.1, and Corollary 6.1 ii)). If X is a locally convex space, then clearly, for any function / : X ^ /? we have d" fixo) ^ 9^^"^^ V(-^o)- Let us observe that if f: X -^ R is "strictly increasing along segments starting from 0" (i.e., for each x e X\{0} and 0 < y] < 1 we have f{r]x) < f{x)\ then a^^^''V(^o) = d^ f(xo). Indeed, if OQ e a^(^''V(^o) and Oo(jco) > 1, then 0 < ^ ^ < 1, whence by (1.232) (for A = A^^), we obtain / ( ^ , ,-^0 I < /(-^o) =

min f(x) < f (

xo ) ,

which is impossible. Therefore, Oo(jco) = 1, so OQ G 9 ^ / ( X O ) (since by OQ 7^ 0 we have / ^ ( O Q ) = /^^"^ H^o)). which proves our assertion. For example, if f: X -> R is ''strongly quasi-convex" (i.e., for each x,y e X with x ^ y and each 0 < ri < I we have f(r]x + (1 — r])y) < max {/(x), f(y)}) and if f(0) = min f(X) (in particular, if f satisfies (1.195)), then f is strictly increasing along segments from 0; indeed, for any x e X and 0 < ^ < 1 we have firjx) = f(r]x + (1 - rj)0) < max {f(x), /(O)} = / ( x ) . Also, the function / of (3.129) on a normed linear space X is strictly increasing along segments from 0. Hence, in these cases, 9^^^ V(-^o) = 9^/(jco). Another such case is given in (c) below. (c) In the particular case that X is a normed linear space, Jo e X, and / is the function f(y) = \\xo-y\\

(yeX),

(4.21)

for each XQ e X\{xo} we have a^(^°V(xo) - ( o o € X*| /(xo) = {cDo e X*l cDo(xo) > 1, llxo - xoll = dist(xo, {x e X\ cDo(x) > 1))) = Uo e X*\ o(xo) > 1, llxo - xoll = I

^ - J ^ l ll«I>oll I

Now let 4>o e 3^*^ ^f(xo). If o(xo) > 1, then we obtain 1 - Oo(Jo) < ^o(-^o - Xo) < ||4>o|| llxo - Xoll = 1 - ^o(^o),

4.1 Maximum points of quasi-convex functions

143

which is impossible. Thus Oo(xo) = 1, which proves (4.22). Note that in this case Proposition 4.2 asserts that if 11^0 II < 11^0--^0II,

(4.23)

then 9^^^ VUo) # 0- This can also be seen directly, as follows: since ||Jo —-^oll i^ 0 (by (4.23)), there exists O^^ G X * \ { 0 } such that ^ ; ( x o - y o ) = l|Ooll 11^0-^0 II (by a corollary of the Hahn-Banach theorem). We claim that o(^o) < 0, then by (4.24) and (4.23) we obtain ll^oll 11^0-xoll = o ; j ( x o - x o ) < ^'^{-x^)

which is impossible. Thus, have Oo(xo) = 1 and

OQ(JCO)

(4.24) OQ(XO)

> 0. Indeed, if

< ||0;)|| llxoll < \\'o\\ 11^0-^oll,

> 0. Hence by (4.24), for

= o ^ ^ o ^^

OQ

1 - ^0(^0) ^ ^o(xo) - ^0(^0) ^ IIOp 111^0-^0II ^ ll^oll ll^oll^oUo) ll^oll^o(^o)

_~ '

that is, OQ G 9^^^ V(-^o) of (4.22), which proves our assertion. (d) As an application to approximation, let us give now another proof of Theorem 2.6, using Theorem 4.3. Let us denote XQ of Theorem 2.6 by x, and assume first that X = 0. Note that for JCQ = 0 and XQ = go 7^ 0, (4.22) becomes ^'^^^''Vteo) = 1^0 e X*| cDo(go) = 1, ll^oll

l^oll

Then by Theorem 4.3, equivalence 1° ;eX|cD;(y)> 1}) l^ol = /^(A"^)(0;,)= =

min 3geG,\

min

f'^^''\)

[-dist(0,{yeX|l))]-

min

( - ^ ) ,

3geG,^(g)>l

i.e., the equivalence 1° s u p / ( G ) = /(go)

(ye Ho).

(d) Using normal cones (see Section 1.3, formula (1.123)), condition 2° of Theorem 4.4 can be written as N{S; go) ^ {0}.

(4.33)

Note also that for any Oo e X*\{0} satisfying (4.25) we have cI>o(go) = maxcDo(G);

(4.34)

^ ( 5 ; go) ^ ^ ( G ; go).

(4.35)

that is, using normal cones,

Indeed, if N(S; go) — {0}, this is obvious; on the other hand, if N(S; go) / {0}, then from go G G c 5 and (4.25) it follows that ^o(go) < supOo(G) < supc|>o(5) = Oo(go).

146

4. Optimal Solutions for Quasi-convex Maximization

Let us consider now the particular case when G = Theorem 4.4(a) yields the following corollary:

{JCQ},

a singleton. In this case,

Corollary 4.5. Let X be a locally convex space and f: X ^^ R an upper semicontinuous convex function. Then for each XQ e X satisfying inf/(X) < /(jco) < +00

(4.36)

there exists ^o ^ A'*\{0} such that Oo(xo) =

max

^o{y).

(4.37)

/(y)o(-^o)} that supports the level set S/^XQ) = {y e X\ f(y) < /(JCQ)} at XQ. Moreover, from Remark 4.5(c) above it follows that for any such hyperplane HQ we have XQ G HQ and f(xo) = min / ( / / Q ) . (b) The assumption of upper semicontinuity is not necessary in Corollary 4.5, as shown by the following example: Let X be a normed linear space endowed with the weak topology a (X, Z*) and let / be the function f(y) = \\y\\

(yeX)

(4.38)

(i.e., (3.123) with XQ = 0). Then X is a locally convex space and / is a finite lower semicontinuous function on X that is not upper semicontinuous at any jo ^ ^, and (4.36) is equivalent to XQ 7^ 0. Also, by a corollary of the Hahn-Banach theorem, for each XQ e X\{0} there exists 4^0 € X*\{0} (X* is the same both for the weak and for the norm topology on X) such that ^o(^o) = ll^oll lUoll =

max ^oiy) = \eX ILvblkoll

max

^oiy)-

veX /(>0o ^ X*\{0} such that f(xo)=

min

f(y).

(4.39)

yeX oiy)=oixo)

Hence we have (3.31) with the sup being attained. Proof If xo satisfies (4.36), then (4.39) follows from the last part of Remark 4.6(a) above. On the other hand, if /(JCQ) = min/(X), then for each OQ e X*\{0} we have

4.2 Maximum points of continuous convex functions /(xo) >

inf

f{y)

147

> inf/(X) = /(XQ),

yeX

whence (4.39). Hence the last statement follows since the inequahty > in (3.31) always holds. D Remark 4.7. (a) Geometrically, Corollary 4.6 means that if f: X -^ R is an upper semicontinuous convex function, then for each JCQ G X there exists a hyperplane HQ with XQ G HQ such that f(xo)=mmf(Ho).

(4.40)

(b) If X is a normed linear space and / is the function (4.38), then condition (4.39) is equivalent, by Lemma 1.5, to II

II

•

lUoll =

II II

mm

l^o(-^o)l

IIJII =

y^^

,

11^0 II

^o(>')=^oUo)

and it is a well-known corollary of the Hahn-Banach theorem that such a function Oo e X*\{0} exists. In the case that / is also continuous, we have the following theorem: Theorem 4.5. Let X be a locally convex space, f: X ^^ R a continuous convex function, and G a subset ofX satisfying (1.73). For an element go e G the following statements are equivalent: P . / ( g o ) = max/(G). 2°. There exists OQ G X * \ { 0 } satisfying (4.34) and inf

/(>;)=

yeX ;) = inf/(//o) = /(go) = m a x / ( G ) =

sup

inf

f(y).

2° =^ I M f 4)0 is as in 2°, then by (4.34), inf

fiy) < /(go).

yeX Oo(>0=supo(G)

Hence by Theorem 3.1 and (4.41), we obtain sup/(G) =

sup

inf

fiy)=

(v)=supO(G)

which, together with go e G, yields 1°.

inf

/(j) 0, since go ^ C,x e C); clearly, x ebd C.Let us consider the open ball O(go,ro) = {y e ^\ Wgo — y\\ < ro). Since C is convex and O(go, ro) is open and convex, and since O(go, ro) n C = 0, by Chapter 1, Theorem 1.1 we can separate C and O(go, ro); i.e., there exists OQ e X*\{0} such that y := sup cDo(C) < mf y - y = 0, so OQ ^ N(G; x). Thus, (4.46) does not hold. D Theorem 4.7. Let X be a normed linear space, f: X -> R a continuous convex function, G a subset of X and go e G such that the level set Sf(g(^)(f) = {x e ^\ fM S f(go)} isproximinal and Mf{X)

< /(go) < +00.

(4.49)

The following statements are equivalent: 1°./(go) = max/(G). 2°. We have df(x) c N(G; X)

(X e X, f(x) = /(go)).

(4.50)

Proof By the first part of Remark 4.1, condition 1° is equivalent to G C Sf(gQ)(f), which, in turn, by Lemma 4.1 applied to G and the proximinal convex set C = Sf(gQ)(f), is equivalent to A^(%,o)(/); ^) ^ ^(G; X)

(X e bd 5/(,,)(/)).

(4.51)

But by (4.49) and since / is a continuous convex function, bd 5/(,„)(/) = 5/(,„)(/)\/l/(,„,(/) ^{xeX\

fix) = /(go)}

(4.52)

(see Remark 1.7). Hence by Theorem 1.6, we have 7V(5/(,,)(/); X) = ^ ( 5 / ( , , ) ( / ) ; x) = U,>or]df{x)

(x e bd 5/(,,)(/)).

Consequently, (4.51) is equivalent to (4.50).

D

Replacing in (4.49) inf/(X) by inf/(G), one can replace in (4.50) the extended normal cones A^, considered for all x G X with f(x) = /(go), by usual normal cones A^, considered only for elements g e G with / ( g ) = /(go). Namely, we have the following theorem: Theorem 4.8. Let X be a locally convex space, / : X -> R a lower semicontinuous convex function, and G a subset of X, such that G C int dom / . For an element go G G satisfying inf/(G) < /(go),

(4.53)

the following statements are equivalent: r . / ( g o ) = max/(G). 2°. We have df(g) c N(G; g)

(g e G, f(g) = /(go))-

(4.54)

4.3 Some basic subdifferential characterizations of maximum points

151

Proof. 1° => 2°. Assume 1° and let g e G, f{g) = /(^o), ^o e df{g). Then ^o(j) - Oo(g) < / ( y ) - / ( g ) = / ( j ) - / ( g o ) < 0

(jG % , , ) ( / ) ) .

(4.55)

But by 1°, G c 5/(,,)(/), and hence by (4.55), 1°. Assume that 1° does not hold, so there exists g' e G such that /(go) < fig'). Note that by (4.53), there exists g" e G such that f{g") < f(go) < f(g'). Let g^ := ng'^ (\ - r])g''

(0 0, by [184], Theorem 23.1 we h a v e / ( g ; g-gO =

f\g\

t{g" - g)) = tfig; g" - g) < t{f{g") - f(g)) < 0, whence 0 < / ( g ; g^ -

8) + f(g', g-g') < f\g\ g ' - g ) . Hence since/Xg; g'-g) = maxci>e3/(g) O ( g ' - g ) (by (1.121)), it follows that there exists c|>o ^ 9/(g) such that Oo(g^ - g) > 0, i.e., such that Oo ^ A^(G; g). Thus, 2° does not hold. D Remark 4.10. (a) The assumptions (4.49) and (4.53) in Theorems 4.7 and 4.8 cannot be removed. Indeed, for example, if / : X -> R is differentiable and has a unique minimum on X at some go G intG, then conditions (4.50) and (4.54) are satisfied, since {x G X\ f(x) = /(go)} = {go} and a/(go) = {0} c A^(/; go), but /(go)=min/(X)^max/(G). (b) In Theorem 4.8 one cannot replace (4.53) by the weaker assumption (4.49), as shown by the following example: Let X = R^ with the Euclidean norm, f(x\,X2) = (1 — xi) + jc| (so / is convex and differentiable), and G = {0} x [ - 1 , +1]. Then for go = (0, 0) G G we have (4.49) and (4.54), but not /(go) = max/(G). Indeed, if g = (0, g2) G G, / ( g ) = /(go) = 1, thengi = 0, 1+gf = 1, whence g = 0, and 9/(0) = {V/(0)} = {(-1,0)} c A^(G; 0). On the other hand, /(go) # max/(G) (since /(go) = /(O, 0) = 1 < g^ + 1 = /(O, g2) for all (0, g2) G G, g2 # 0 ) . One can also see directly that (4.50) does not hold either: for x = (1, 1) (^ G) we have f{x) = 1 = /(go) and 9/(1, 1) = {V/(l, 1)} = { ( - 1 , 2)} (the gradient of / at (1, 1)), but ( - 1 , 2) ^ iV(G; (1, 1)), since for (0, 1) G G we have ( - 1 , 2)(0, 1) = 2 ^ (-1,2)(1,1) = 1. By introducing a parameter s, namely, by considering the ^-subdifferentials 9e/(go) (^ > 0) instead of the subdifferentials df(g) (g G G, / ( g ) = /(go)), and the ^-normal sets Ns(G; go) instead of the normal cones N(G; g) (g G G, / ( g ) = /(go)), one can transform the purely local conditions of (4.54) into global conditions. Indeed, we have the following: Theorem 4.9. Let Xbea locally convex space, / : X ^^ Ra proper lower semicontinuous convex function, and G a subset ofX. For an element go G G, the following statements are equivalent:

152

4. Optimal Solutions for Quasi-convex Maximization r . / t e o ) = max/(G). 2°. We have Ssf(go)£Ns(G;go)

{e>0),

(4.57)

Proof, r =^ 2°. Assume T and let £ > 0, OQ e a^/Cgo). Then 0 > fig) - f(go) > oa./(go) ^ n^^oNeiG; go) = A^(^; go),

(4.58)

so (4.57) holds also for s = 0. Assume now that 1° does not hold, that is, using (3.134), /(go) < sup / ( G ) = sup inf {fix) - cD(x) + sup 0(G)}.

(4.59)

Then there exists 4>o G X* such tl^at /(go) < inf {/(x) - Oo(x) + supcDo(G)}, jceX

whence sup /(go) - inf {/(x) - cDo(^)} > o(go).

(4.60)

xeX

By (4.59) and (4.60), we have /(go) € /?. Let 8 := sup[oix) - fix)] - (cDo(go) - /(go)).

(4.61)

xeX

Then by (4.61), we have e >0 and sup{(|>o(x) - fix)} = Oo(go) - /(go) + £, xeX

SO cI)o G dsfigo). On the other hand, by the first inequahty in (4.60), sup cI>o(G) > /(go) - inf {fix) - ^oix)} = Oo(go) + 6:, JCGX

so o ^ A'.CG; ^o).

•

5 Reverse Convex Best Approximation

The study of reverse convex best approximation, that is, of best approximation by complements of convex sets, is motivated, among others, by its connections with the famous unsolved problem whether in a Hilbert space every Chebyshev set (i.e., such that each x e X has a unique element of best approximation in the set) is necessarily convex. Namely, it is known (see the Notes and Remarks to Section 5.2) that if a Hilbert space X contains a Chebyshev set that is not convex, then X also contains a Chebyshev set that is the complement CG of an open bounded convex subset G (7^ 0) of X. Geometrically, if G is a convex set with intG 7^ 0, and XQ e intG, the problem of finding dist(jco, CG) amounts to finding the greatest radius of an open ball with center XQ contained in G (see Figure 5.1); clearly, when xo G bd G, no such open ball exists, and we have dist (JCQ, CG) = 0.

Figure 5.1. We shall be concerned with the following two main problems: (1) Find convenient formulas for dist(jco, CG).

154

5. Reverse Convex Best Approximation

(2) Give characterizations of elements of (reverse convex) best approximation (i.e., necessary and sufficient conditions in order that an element zo e X satisfy zo e 7^CG(-^O), that is, zo ^ CG and ||xo - Zoll = dist(xo, CG).) We shall obtain duality results, using the elements O of the conjugate space X*. Remark 5.1. If XQ e bdG, and hence in particular, if G is any subset of X with int G = 0 and XQ G G, then dist(xo,CG) = 0.

(5.1)

Indeed, if XQ G bdG, then every ball with center XQ intersects CG, whence dist(xo,CG) = 0. This applies, in particular, if intG = 0 (hence G c bdG) and XQ £ G. Therefore, it is natural that in most of the subsequent results we shall assume that int G ^ &.

5.1 The distance to the complement of a convex set The following theorem gives an explicit formula for the distance to the complement of a convex set. Theorem 5.1. Let X be a normed linear space, G a convex subset ofX with int G 7^ 0, andxo e G. Then dist (xo, CG) = inf {sup cD(G) - O(jco)}.

(5.2)

\\n=\ Proof Let us first assume that XQ = 0, so formula (5.2) becomes dist(0,CG)= inf supO(G).

(5.3)

\\n=\ Since int G 7^ 0, for each z e CG there exists O^ e X* with ||0J| = 1 such that sup 0^(G) < 0^(z) (by the separation theorem). Hence Ikll > ^z(z) > supcI>,(G) > inf supO(G) inf sup ^ ( G ) .

(5.4)

On the other hand, since 0 € G, for each O G X* with ||c|>|| = 1 we have supO(G) > ^(0) > Oand CG^{X

e X\ (D(jc) > sup 0(G)},

(5.5)

5.1 The distance to the complement of a convex set

155

whence by Corollary 1.4, dist(0, CG) < dist(0, {jc G X| supcD(G)}) = supcD(G), which, together with (5.4), yields (5.3). Assume now that XQ e G is arbitrary. Then, since z ^ G if and only if z — XQ ^ G — xo, we have dist(jco, CG) = inf ||jco - z|| = dist(0, C(G - JCQ)),

(5.6)

zeCG

where G — XQ is a convex set containing 0, with int (G — XQ) ^ 0. Hence by (5.3), dist(0, C(G - jco)) = inf sup 0 ( G - XQ) = inf {sup a>(G) - O(xo)}, IIOII-l

(5.7)

||ct>|| = l

which, together with (5.6), yields (5.2).

D

Remark 5.2. (a) If int G = 0, the expression infyon^i{sup 0(G) - O(xo)} may have any value d >0. Indeed, for example, if X == C([0, 1]) and G = the (convex) set of all algebraic polynomials of norm < d, then int G = 0, so dist (0, CG) = 0, but G = {jc G X| ||jc|| < d] (by the classical theorem of Weierstrass on the uniform approximation of continuous functions by polynomials), and hence supO(G) = supcD(G) = d\\^\\ for all O e X*, so inf|,o||=i{supO(G) - 0(0)} = d. (b) If G is open, then by Lemma 1.8, CG 5 {JC G X\ 0(jc) > sup cD(G)},

(5.8)

and hence in particular, CG 2 {JC G X\ 0(jc) = sup 0(G)}. (c) By Lemma 1.5, Theorem 5.1 admits the following geometric interpretation: if G is a convex subset ofX such that int G 7^ 0, and ifxo G G, then dist (xo, CG) = inf dist(xo, //cD,supcD(G)) = eX*

^

11^11 = 1

inf

inf

eX*\{0}

yeX

||xo - yh

(5.9)

cD(j)=supO(G)

where // 1 was arbitrary, we obtain, using also (5.22), that inf supcD(G)== inf sup 0(G). \m=\

(5.23)

iicDii>i

Furthermore, since G ^ X and G is convex, by the strict separation theorem there exists Oo € Z*\{0} such that sup Oo(G) < +oo.LetOo G X*with||Ool| = 1, sup Oo(G) < -\-OQ be arbitrary and let /x > 1. Then ||/xOol| > 1 and inf supO(G) < sup(/xOo)(G) = /xsupOo(G),

OeX* ||0||>1

whence, using that ^ > 1 and Oo G X* with ||Oo|| = 1 were arbitrary, we obtain inf supO(G)
1

IIO||=l

(5.24)

By (5.23) and (5.24), it follows that inf s u p O ( G ) = inf supO(G)
1

||cD|| = l

which yields (5.21). Assume now that XQ G G is arbitrary. Then G — JCQ is a convex subset of Z, containing 0, and hence, applying (5.21) to G — JCQ and using that sup 0 ( G — XQ) = sup 0(G) - 0(xo), we obtain (5.20). D Remark 5.4. Combining Theorem 5.1 and Proposition 5.1, one obtains further expressions of dist (jco, CG).

5.1 The distance to the complement of a convex set

159

One can replace in (5.9) hyperplanes by other sets, such as quasi-supporting closed or open half-spaces (see Figures 5.4 (a) and (b)). Indeed, we have the following theorem: Theorem 5.2. Let X be a normed linear space, G a convex subset ofX such that int G 7^ 0, and XQ e G. Then dist(xo, CG) = inf

inf

\\y - XQ\\ = inf

(G) 0. Then the function O' = ^O e Z*\{0} satisfies supO^(G) < 1. Also, J-O(xo) ^

l-cD-(xo) ^ l-cD^(xo)

ll^ll

^lioni

lio^i

'

which, by the last part of Remark 5.3(b), yields the inequalities > in (5.29), and hence the equalities. • Remark 5.5. By Corollary 1.4, one can also write the first equality of (5.29) in the following geometric form: dist (Jo, CG) =

inf

dist

(XQ,

{y

G X|CD(J)

> 1}),

(5.31)

OGG°\{0}

where G° is the (usual) polar (1.82) (with C = G) of G. In the case dim X < +oo, one can obtain more complete results. Indeed, let us first prove the following proposition: Proposition 5.2. If G is a convex subset of a finite-dimensional normed linear space X, and XQ G G, then dist (jco, CG) = dist (XQ, CG).

(5.32)

Proof Since G c G, we have CG ^ CG, whence dist(xo, CG) < dist(jco, CG).

(5.33)

Assume now that the inequality (5.33) is strict, so there exists ^ > 0 such that dist(jco, CG) 4- 2^ < dist(jco, CG).

(5.34)

Choosez G CG such that II jco - z | | < dist (JCQ, CG) 4-6:. Then z G G (since if Z G CG, then dist(xo, CG) < jjjco — zjj < dist(jco, C G ) + ^ , which contradicts (5.34)). Hence, z G G n CG c bd G = bd G, where the last equality holds by dim X < -\-OQ and the convexity of G. Consequently, there exists j G CG such that \\z — y\\ < £• Then by (5.34), we obtain Iko - y\\ < \\xo - z\\ + \\z - y\\ < dist(xo, CG) ^le

< dist(xo, CG),

in contradiction to j G CG.

D

Remark 5.6. (a) The assumption dim X < -hoo cannot be omitted in Proposition 5.2, as shown by Remark 5.2(a) above. (b) One can also give the following alternative proof of Proposition 5.2: Since int G = int G (by dim X < +oo and the convexity of G), we have, by (1.52) and Lemma 1.2, dist(jco, CG) = dist(jco, CG) = dist(jco, C(int G)) = dist (JCQ, C(int G)) = dist(jco, CG) = dist(jco, CG).

5.2 Elements of best approximation in complements of convex sets

161

Proposition 5.3. Let dim X < +oo, G a convex subset ofX, and XQ e G. Then we have (5.2). If in addition, 0 G G, then we have also (5.29). Proof. For the first part, by Theorem 5.1 and Remark 5.1, we have to prove that if int G = 0, then the right hand side of (5.2) is 0. Since dim X < -foo and G is a convex set with int G = 0, G is contained in some hyperplane {x e X\ 0, we obtain 0
o ll/xOoll

..1 n mf — = 0.

n U

||^olU>OM

sup4)(G)

Oo(zo - -^o) > sup o(xo) > inf {sup 0(G) - O(xo)} ||cD|| = l

= dist(xo,CG) = \\xo - zoh whence (5.36) and (5.37). Thus, T =^ 2°. _ Assume now 2°. Then by (5.35), we have zo ^ bd G c G, whence o(zo) < sup 00(G). Therefore, by (5.37), (5.36), int G / 0, and Theorem 5.1, we get lUo - ^oll = Oo(zo - ^0) < supOo(G) - Oo(xo) = inf {supO(G) - cD(xo)} = dist(xo, CG), whence (5.38) (since zo e CG)). Thus, 2° => 3°. Furthermore, assume 3°. Then by (5.38), (5.36), int G # 0, and Theorem 5.1, we have \\xo - ZoW = supOo(G) - cDo(xo) = inf {supcD(G) - O(xo)} = dist(xo, CG),

\\n=\ so Zo ^ ^CG(-^O). Thus, 3° => 1°, which proves the equivalence of 1°, 2°, and 3°. Finally, if we have 2°, then by (5.37), |10o|| > 1 (since otherwise Oo(zo —-^o) S ll^oll Iko - ^oll < Iko - -^oll), so l i ^ < 1. Hence by (5.36), ^^P^(^) - ^(^0) = ^

11^0 II

11^0 II

i^f {supcD(G) - (xo)}

11^0 II if^^i^*^ < inf {supO(G)-0(jco)}, 11^11 = 1

and therefore s u p - ^ ( G ) - T^(xo)

11^0 II

= inf {supO(G) - (xo)},

11^0 II ^^^fj^

which shows that (5.36) is satisfied also for Oo replaced by TTI^, i.e., that in 2° we may assume (5.39). Consequently, in 3°, too, we may assume (5.39) (because in the above proof of the implication 2° =^ 3° we have used the same Oo). •

5.2 Elements of best approximation in complements of convex sets

163

Remark 5.8. When G is a bounded convex set, the equivalence 1° 3° of Theorem 5.4 admits the following geometric inteq^retation:/or an element zo ^ CG we have zo ^ 7^CG(-^O) if and only if there exists ^o ^ ^* "^ith ||Oo|| = 1 such that the quasi-support hyperplane //o,supci>o(G)) =

inf dist(xo, / / ) ,

(5.41)

HeHc

lUo - ^oll = dist(xo, //ci>o,supci>o(G)); or, equivalently (by Corollary \.l),for an element zo G CG we have if and only if there exists a hyperplane HQ e He satisfying dist(xo, Ho) = inf dist(A:o, / / ) ,

(5.42) ZQ G PCG(-^O)

(5.43)

HeHc

(5.44)

Iko-zoll =dist(xo,//o) (see Figure 5.5). Indeed, by XQ e G, we have ||4>o|| = 1 and Lemma 1.5, dist (xo, //cDo,supo(G)) =

I^O(-^O)

OO(JCO)

< sup o(^). and hence by

- sup Oo(G)| = sup cDo(G) - Oo(xo). (5.45)

•^0

J

m Hn

Figure 5.5.

Remark 5.9. (a) When G is unbounded, there exists (by the uniform boundedness principle) OQ G X* such that supcI)o(G) = +oo, so then //(i>o,supOo(G) = 0; hence in this case, (5.41) does not hold (since its left-hand side is +oc, while its right-hand side is finite). (b) The above proof of the implication 2° => 3° shows that for each pair zo, OQ as in 2° of Theorem 5.4, we have Oo(zo) = supa>o(G).

(5.46)

(c) When G is a bounded convex set, (5.46) is equivalent to zo G //o(G)This, together with (5.42), gives that Zo G 'P//o(G)(-^o).

(5.47)

164

5. Reverse Convex Best Approximation

Now we shall give some examples in the plane R^ endowed with the Euclidean norm IkII : - V|xiP + |x2p

{X = (XUX2) e R ' ) ,

(5.48)

showing that various parts of 2° and 3° above cannot be omitted. Example 5.1. Let X = R^, with the norm (5.48), G = {y e R^\ \\y\\ < 1}, XQ = 0. Then for the element zo = (2, 0) e int CG and the function OQ e (R^)* defined by Oo(x)=x,

(x = ixuX2)eR^),

(5.49)

we have \\^o\\ = 1, supO(G) = ||0|| (O e (/?^)*), whence supOo(G) cI>o(-x:o) = infci>eXM|ci>iNi{sup4)(G) - (xo)] = 1, and ^oizo - XQ) = ||xo - Zoll = 2. Thus, (5.39), (5.36), and (5.37) hold, but (5.35), (5.38), and 1° are not satisfied. Example 5.2. Let X = R^, with the norm (5.48), G = {y e R^\ \yi \ < 2, |J2I < 1}, xo = 0. Then for the element zo = (2, 0) e bd CG and the function ^0 of (5.49) we have supcI>o(G^) - ^o(-^o) = 2, inf(|>GXM|0||=i{supcI>(G) - ^{XQ)} = dist (xo, CG) = 1 (by Theorem 5.1), and Oo(zo - XQ) = 2 = \\xo - Zoll • Thus, (5.35), (5.39), (5.37), and (5.38) hold, but (5.36) and 1° are not satisfied. Example 5.3. Let X = R^, with the norm (5.48), G = {y e R^\ max ( | ji |, | J2I) < l},xo = 0. Then for the element zo = (1,1) e bd CG and the function OQ of (5.49) we have supo(G) - c|)o(jco) = 1, infci>eXM|0||=i{supO(G) - c|)(xo)} = dist (xo, CG) = 1, Oo(zo - -^o) = 1 and \\xo - zo\\ = v ^ . Thus, (5.35)-(5.36) hold, but (5.37), (5.38), and 1° are not satisfied. Let us give now a characterization of best approximations in CG for the case 0 G G, using the distance formula of Theorem 5.3. Theorem 5.5. Let G be an open convex subset of X containing 0, and let XQ G G. For an element ZQ G CG, the following statements are equivalent: lMko-zoll=dist(xo,CG). 2°. There exists % G X*\{0} such that \\x,-z,\\='-^^^,

i ^ ^ ^ = WA " ""'

(5.50)

inf

CDGX*\{0} ^{g) 2 ^ . In Section 6.4 we shall deal with unperturbational Lagrangian dual problems to problem (PO of (6.1). Finally, the general dual problem (6.7) will permit us to study (unconstrained and constrained) surrogate duality for more structured primal reverse convex infimization problems (i.e., in which the primal constraint set G is expressed in more structured ways), by considering suitable dual constraint sets W and dual objective functions X = X[jj : W -> R asin (6.8) (see Section 6.5). Remark 6.1. This chapter is devoted to unperturbational duality results only, since until the present there exists no perturbational duality theory for reverse convex infimization corresponding to those for convex infimization (see Chapter 1, Section 1.4.2) and convex supremization (see Chapter 3, Section 3.4.2). Similar to (1.383), we have i n f / ( C G ) = inf/(X), where / = / + XCG' ^^t the theory of Chapter 1 cannot be apphed directly to this function / , since for a convex set G, in general XCG is not convex. Another attempt could be to note that inf / ( C G ) = inf ( / + XCG)(^) = inf ( / +

- (-XCG))(^).

(6.9)

and h ^ c e to develog^a perturbational theory for infimization problems inf/(X), with / of the form / = / -j—h. In Chapter 8 we shall present a perturbational duality theory for such problems, but only when h is convex, so that theory cannot be applied to h = —XCG' where G is convex, i.e., to reverse convex infimization, since in general — XCG is not convex (however, note that it is quasi-convex when G is convex, since 5^(—XCG) = either G or X, for allJ € /?).

6.1 Some hyperplane theorems of surrogate duaUty Let us start with a generalization of Chapter 5, Remark 5.1. Remark 6.2. If G is a subset of a locally convex space X with intG = 0, and f: X ^ R is an upper semicontinuous function, then inf/(CG) = inf/(X).

(6.10)

172

6. Unperturbational Duality for Reverse Convex Infimization

Indeed, then by Lemmas 1.1 and 1.2, we have i n f / ( C G ) = i n f / ( C G ) = inf/(C(intG)) =

inff(X).

We have the following hyperplane theorem of surrogate duality, generalizing the (equivalent) geometric form (5.9) of Chapter 5, Theorem 5.1. Theorem 6.1. Let X be a locally convex space, G a convex subset of X, and f: X ^ R a function. (a) If f is upper semicontinuous, then inf/(CG)
(G)

(b) If f is quasi-convex, int G 7^ 0, and inf/(G)

inf

inf

OGXniO}

yeX (I)(v)=supO(G)

f{y).

(6.13)

(c) If f is upper semicontinuous and quasi-convex, intG ^ 0, and if (6.12) holds, then (6.3) holds. Proof If G = Z, then both sides of (6.11), (6.13), and (6.3) are +00 (since inf 0 = +00). Thus, we may assume that G ^ X. (a) If intG := 0, then (6.11) holds by Remark 6.2. If intG ^ 0, let O € X*\{0} and H '.= [y e X\ cD(j) = sup 0(G)}.

(6.14)

If sup ^(G) = +CXD, then // = 0, whence i n f / ( C G ) < infvGX. 2^, so the two languages (6.8), (6.7) and (6.25), (6.26), are equivalent ways of expressing the dual objective function A.^ and the dual value )S^. In the sequel we shall choose the language (6.25), (6.26), since this will allow us, by using (1.140), to express the results, e.g., on the relations between the primal and dual problems, in a more concise way. Thus in particular, in this section we shall consider unconstrained surrogate dual problems (6.4) to (P^), with the dual objective function being of the form A^^(cD) = inf f(CA\{})) =

inf

f(x)

( e X*\{0}),

(6.27)

xeX CDGCA(U})

where A = AG : 2^ ^ 2^*\^^^ is a polarity (depending on G). Then by (6.4) and (6.27), the dual value (i.e., the value of the dual problem) will be

176

6. Unperturbational Duality for Reverse Convex Infimization P\=

inf

inf/(CA\{^})) =

inf

inf

f{x).

(6.28)

cDeCA({jc})

(b) If there exists WQ e W such that CA'({W;O}) = 0, then by (6.25), we have k\{wQ) = inf 0 = +CXD. Consequently, by (6.26), )S^ = inf inf f{CA\{w})),

(6.29)

weG'

where G' := {w; G W| CA^({U;}) # 0).

(6.30)

(c) We have P'^ = inf

inf

fix) = inf

inf

f(x).

(6.31)

weGA({x})

Indeed, (6.31) follows from (6.26) and inf

f(x) = H-oo.

(6.32)

Jc€(C(dom/))nCA'({u;})

(d) In the particular case of Theorems 6.1 and 6.2, we have W = Z*\{0}, and by (6.6) and (3.39), the surrogate constraint sets are CA^({(I>}) = [y e X\ (^(y) = sup cD(G)}

(cD e X*\{0}),

(6.33)

where A = AG : 2^ ^ 2^*^^^^ is the polarity A^ of (1.166), and the dual objective function is r^(c|>)=

inf

f(y)

(cD6X*\{0}).

(6.34)

(jc)>sup4)(G)

P\, =

inf

inf fiddly(W))

=

inf

inf

f(x).

(6.71)

4)(jc)>supcD(G)

Remark 6.9. (a) For O G X * \ { 0 } such that supO(G) == +oc, we have {x e X\^{x) > supO(G)} = 0, whence inf^ex,o(jc)>sup0(G)/(-^) = +oo. Therefore, the inf (Dex*\{0} in (6.71) can be replaced by inf^^G''. where G^ is the "barrier cone" (1.347) of G. A similar remark is valid also for some of the subsequent results, but for simplicity, in the sequel we shall use only inf(i>ex*\{0}(b) By (6.31) applied to A = A^, we have ^;, = ^G

inf

inf

fix),

(6.72)

eX*\{0} xedomf 4>U)>sup0(G)

(c) By Proposition 6.3(a) applied to A = A^, we have i n f / ( C G ) =a'
sup(G)

Theorem 6.7. Let X be a locally convex space, G a closed convex subset ofX (with G ^ X), and f: X -^ R a function. Then inf/(CG) =

inf

inf

cI>eX*\{0}

xeX 0(jc)>supO(G)

f(x).

(6.74)

Proof By (1.164) and (1.163), we have G = coG = (AlYAliG)

=

(Aly(X*\m,

and hence by Corollary 6.1 for AG = A^, we obtain (6.74).

D

6.2 Unconstrained surrogate dual problems for reverse convex infimization

183

Remark 6.10. (a) Theorem 6.7 is a "half-space theorem of surrogate duality," since the surrogate constraint sets QsupO(G)

f{x) =

inf

inf

4)6X*\{0}

xeX 0(jc)>supO(G)

f(y).

(6.75)

Proof The first equality holds by Corollary 6.2 below, and the second equality holds by Lemma 1.1. D Remark 6.11. Theorem 6.8, too, is a "half-space theorem of surrogate duality," since the surrogate constraint sets [x e X\(x) > supO(G)} are (closed) halfspaces. (2) For the polarity A^ : 2^ -^ 2^*^^^} defined by (1.166) we have (1.167), and hence the dual objective function (6.27) and the value (6.28) become X^.^m

= inf /(C(A3^)^({CD})) =

mf

f(x)

(O e X*\{0}),

(6.76)

^(x)=sup(G)

P', =

inf

inf f(C(Aly([(G)

Remark 6.12. Now we can also give another proof of the following particular case of Theorem 6.2: If X is a normed linear space, then for any bounded open convex set G C. X and any upper semicontinuous function f: X -> R we have (6.3). Indeed, since G is an open set, by Lemma 1.8, (D(g) < sup (G)

igeG,e

X*\{0}).

(6.78)

Also, by Theorem 1.3, for each x e CG there exists O e X*\[0} such that 4>(jc) = sup 0(G). Thus by Lemma 1.12 (b), G is (A^)^A^-convex. Furthermore, by (6.78) and (1.167), G c nci>ex*\{0}(A^)'({O}) = (A^)'(X*\{0}). Hence by Theorem 6.4(a) (for W = X*\{0} and A = A^), we obtain (6.3). (3) For the polarity AJ^ : 2^ ^ 2^*^^^^ defined by (1.154) we have (1.155), and hence the dual objective function (6.27) and the dual value (6.28) become x;, (CD) = inf /(C(A^)'({cD})) = G

P', = ^^G

inf

O6X*\{0}

inf

inf

xeX 0(A-)>supO(G)

/(C(AM'({0})) ^ ^ ^

=

f(x)

inf

(O e X*\{0}), inf

f(x).

eX*\{0} xeX "^ ' cD(jc)>supcD(G)

'

(6.79) (6.80)

184

6. Unperturbational Duality for Reverse Convex Infimization

Note also that for any O e X*\{0} we have sup 0(G) > —oo, since G ^ 0. On the other hand, if sup 0(G) = +oo, then by (6.79), A^, (O) = +oo. (4) For the polarity A^: 2^ -^ 2^*^^^^ defined by (1.182) we have (1.183), and hence the dual objective function (6.27) and the dual value (6.28) become A^aO) = inf / ( C ( A 4 ^ ) ^ ( { 0 } ) ) =

mf

f(x)

(

R a function. Then inf/(CG) =

inf

inf

CDGX*\{0}

xeX

f(x).

(6.84)

0(x)^0 sup4>(G) sup(I>(G)} = {O € X*\{0}| supa>(G) < +oo}, so the result follows from Theorem 6.13.

D

Combining, in a similar way. Theorem 6.11 and Remark 1.24 (with ^Q and do replaced by O and 1), we obtain the following refinement of Corollary 6.8 for the case 0 G G: Theorem 6.14. Let X be a locally convex space, / : Z -> [—oo, -f-oo) an upper semicontinuous function, and G an open convex subset ofX, with 0 e G. Then inf/(CG) =

inf

max mf{f(y)

- r](^(y) + t]}.

(6.109)

cI>6X*\{0} 7i>0 yeX sup{G) Z. Given a system (X, Z, M), a subset T of Z with M(X) 0 7 / 0 , and a function / : X ^^ /?, we shall consider the primal reverse infimization problem « = ««-'(Cr)./ =

inf

fM-

(6.110)

Remark 6.19. Problem (6.110) is equivalent to problem {P'')of (6.1). Indeed, given a system (X, Z, w) and 7, / as above, let G = U-\T).

(6.111)

f(x) = inf /(jc),

(6.112)

Then inf u{x)e^T

SO problem (6.110) is nothing other than (6.1) with G of (6.111). Conversely, every problem (6.1) can be written in the form (6.110), by taking Z = X,u = Ix, the identity operator in X, and T = G. However, in the study of the "programming problem" (6.110) one can also use the properties of T and u. Now we shall assume that (X, Z, w) is a system, where X and Z are locally convex spaces, with conjugate spaces X* and Z*, 7 is a subset of Z, and f: X -> R is a function. There are several natural ways to introduce unconstrained dual problems to (6.110) that generalize the dual problems of the preceding sections. (1) One can use the polarities A^.,^^^: 2^ -> 2"*^^*^^^^^ (/ = 1, 2, 3, 4) defined by (3.168) and (3.174)-(3.176). For example, replacing the dual objective function (6.70) and the dual value (6.71), respectively, by X'-,

(VI/M) =

inf

^U-UT)

fix)

(vl/ e Z*, V|/M ^ 0),

(6.113)

xeX

^uix)>sup^(uiX)nT)

PI, U-UT)

= inf

inf

^eZ*

xeX

fix),

(6.114)

^u^o ^M(jc)>supvi/(f/(X)nr) we obtain, by Remark 6.10(b) (with Vl^ = tion of Theorem 6.7:

M*(Z*)\{0})),

the following generaliza-

Theorem 6.15. Let iX, Z,u) bea system, where X and Z are locally convex spaces, let T be a subset of Z such that the set {x e X| M(JC) e T] is iu"^iZ*)\[0})-convex (see (9.1)j, and let f: X —> R be a function. Then inf

xeX w(jc)eCr

fix)=

inf

^eZ* vi/M^o

inf

xeX ^u{x)>sup^{u{X)nT)

fix).

(6.115)

192

6. Unperturbational Duality for Reverse Convex Infimization

(2) Instead of A^_,^^^: 2^ -^ 2"*^^*^\{0J of (3.168), one can consider the polarity ^1]T ' 2^ -> 2^*\{^J defined by (3.177), replacing the dual objective function (6.79) and the dual value (6.80), respectively, by r^„ (vl/) =

inf

fix)

(vl/ e Z*\{0}),

(6.116)

^u{x)>sup^(T)

yg;,, =

inf

inf

fix).

(6.117)

Theorem 6.16. Let (Z, Z,u) be a system, in which X is a topological space, Z is a locally convex space, and u: X -> Z is a mapping. Furthermore, let f: X -^ R be an upper semicontinuous function, and T a convex subset of Z, with iniT 7^ 0 and such that U~\CT)

C

w-'(Cr).

(6.118)

Then inf

xeX wU)eCr

fix)=

inf

^eZ*\{0}

inf

xeX ^'u{x)>sup^{T)

fix).

(6.119)

Proof Define G c X by (6.111). If X € C ( A 2 V ) ' ( Z * \ { 0 } ) , i.e., if there exists ^ e Z*\{0} such that ^w(jc) > s u p ^ ( r ) = s u p ^ ( i n t r ) , then by Lemma 1.8 we have uix) ^ intT, whence by (1.20), uix) G C(intr) = Cf. Hence by (6.118), (6.111), and (1.20), jc e

U-\CT)

C

M-i(Cr) = CG = C(int G).

Thus we have intGc(A^^V)^(Z*\{0}).

(6.120)

On the other hand, if JC € CG = U~\CT), then uix) ^ T, whence by our assumptions on T and the separation theorem, there exists ^1/ e Z*\{0} such that sup ^iT) < VI/M(JC), SO X G C ( A ^ V ) ' ( ^ * \ { 0 } ) . Thus we have (A2V)^(Z*\{0}) C G.

Consequently, by (6.120), (6.121), and Theorem 6.4(b), we obtain (6.119).

(6.121)

D

Remark 6.20. In Theorem 6.16 it is not assumed that u is continuous. Nevertheless, under the assumptions of Theorem 6.16, if ini T ^ {^ is replaced by int 7 = 0, then inf

/(jc) = inf/(X).

xeX u{x)eZT

Indeed, by int T = 0 and Lemma 1.2, we have

(6.122)

6.5 Duality for infimization over structured primal reverse convex constraint sets

193

Cr = C(intr) = C0 = z, whence by (6.118),

X c u-\Z) = U-\CT)

C

M-i(Cr).

Therefore, u~^ (CT) = X, and hence since / is upper semicontinuous. inf

xeX

f(x) = inf/(w-i(Cr)) =

mff(X).

M(jc)GCr

In the case that T is an open convex subset of Z, the assumption (6.118) can be omitted. Indeed, we have the following: Theorem 6.17. Let (X, Z,u) be a system in which X is a set, Z is a locally convex space, and u: X ^^ Z is a mapping, and let f: X -> R be a function and T an open convex subset ofZ. Then we have (6.119). Proof. Define G c X by (6.111). If jc e C(A2|y,)'(Z*\{0}), i.e., if there exists ^ e Z*\{0} such that ^u(x) > sup ^ ( r ) , then, since T is an open subset of Z, by Lemma 1.8 we have u{x) ^ T, whence x e u~^ (CT) = CG. Thus, we have G c (A^V)'(Z*\{0}).

(6.123)

On the other hand, if x € CG = w"'(Cr), then u(x) ^ T, whence by the separation theorem, there exists ^ e Z*\{0} such that sup 4^(7) < ^u(x), so X e C(A2J^)^(Z*\{0}). Thus we have (6.121). Consequently, by (6.123), (6.121), and Corollary 6.1, we obtain (6.119). • Corollary 6.9. Let {X, Z,u) be a system in which X is a set, Z is a locally convex space, and u\ X -^ Z is a mapping, and let f: X —> R be a function and T a convex subset ofZ, with int 7 7^ 0. Then inf

xeX M(x)GC(intr)

fix)

=

inf

vi/eZ*\{0}

inf

xeX vi/M(.v)>supvI/(r)

/(JC).

(6.124)

Proof This follows from Theorem 6.17, since int T is convex and open and since sup^(int7) = s u p ^ ( r ) . D Corollary 6.10. Under the assumptions of Corollary 6.9, /f 0 € int T, then inf

f{x) = inf

M(jc)eC(intr)

where 7° is the polar set (1.82) ofT.

inf / ( x ) , vi/w(.v)>l

(6.125)

194

6. Unperturbational Duality for Reverse Convex Infimization

Proof. Since 0 e int T, we have, by Lemma 1.8, supvl/(r) > vl/(0) = 0

(^ G Z*\{0}),

(6.126)

and hence for any ^ G Z*\{0} with sup ^ ( T ) < +oo, we obtain [x e X\ ^u(x) > supvl/(r)} = {x eX\ ^f'u(x) > 1},

(6.127)

where vl/' :=

? vi/; supvl/(r)

(6.128)

on the other hand, if s u p ^ ( r ) = +(X), then with the convention 1/ + oo = 0, (6.127) (for v|/^ of (6.128)) reduces to 0 == 0. Note also that for any ^ e Z*\{0}, we have ^ ' € 7°. We claim that inf

inf

^eZ*\{0}

f(x)=

xeX

inf

inf

^eT°

xeX

f(x).

(6.129)

Indeed, since for each ^ e T° and x e X with ^w(x) > 1 we have ^ / 0 and ^u(x) > s u p ^ ( r ) , the inequahty < in (6.129) is obvious. If this inequality were strict, then there would exist ^o ^ Z*\{0} such that inf

fix) < inf

xeX ^ou(x)>sup^o{T)

^eT°

inf

xeX 4'M(JC)>1

fix);

(6.130)

but then, for ^^ = (1/ sup ^o(T))^fo ^ T° we would have, by (6.127), inf ^eT°

inf

f(x)
\

^

inf

/(x),

xeX -^ vi/oM(jr)>supvI/o(r)

in contradiction to (6.130). This proves the claim (6.129), which together with (6.124), yields (6.125). D Related to Corollary 6.10, let us also prove the following theorem: Theorem 6.18. Let (X, Z, a) be a system in which X is a set, Z is a locally convex space, and u: X -^ Z is a mapping, and let f: X ^ R be a function and T a convex subset of Z, with M(X)nint T 7^ 0. Then for any XQ e X with M(XO) G int T, we have inf

xeX M(jc)eC(intr)

fix) =

inf

^G(T-uixo))°

inf

xeX ^u(x)>l-^^u(xo)

fix).

(6.131)

Proof Let G = u'\iniT),

(6.132)

W = iT - w(jco))° = {vl/ G Z*| sup vl/(r) - ^uixo) < 1}, A(C) = {vl/ e W| ^uic) - ^uixo) < 1 (c G C)}

(C c X).

(6.133) (6.134)

6.5 Duality for infimization over structured primal reverse convex constraint sets

195

If X € C A ' ( W ) , i.e., if there exists ^ e Z"" such that ^u(x) - ^w(jco) > 1 > sup ^(T)-^u(xo), then by Lemma 1.8, M(JC) ^ intT, sox e w"^(C(intr)) = CG. Thus we have G c A\W). On the other hand, if x G CG, then by (6.132), u(x) e C(intr), and hence by the separation theorem, there exists ^ e Z*\{0} such that sup ^ ( 7 — u{xo)) < ^(u(x) - w(jco)). But since 0 e intT - w(xo), by Lemma L8 we have sup ^f(T — w(xo)) > 0, and hence for ^' e Z* defined by vl/' :=

! vy, sup ^ ( 7 — w(jco))

(6.135)

we have vl/^ G (T-u{xo)y = W, and^'(u(x)-u(xo)) > l,sojc G CA^(W).Thus we have A ^ ^ ) ^ G. Consequently, by Corollary 6.1, we obtain (6.131). D Remark 6.21. (a) Theorem 6.18 implies the particular case of Corollary 6.10 in which 0 G u(X) n int T (by applying Theorem 6.18 to any XQ e X with u(xo) = 0). (b) The particular case of Theorem 6.18 in which X is a linear space and u: X -^ Z is a linear mapping can also be deduced from Corollary 6.10, as follows. Define f: X ^~R andT' ^ Zhy f(x):=f{x+xo),

(6.136)

r :=T -uixo).

(6.137)

Then T' is a convex subset of Z with 0 G int T'. Hence by Corollary 6.10, inf

f\x)=

inf

xeX M(jc)eC(int7")

^e(T')°

inf

f{x).

(6.138)

xeX ^u(x)>\

But by (6.136), (6.137), and the linearity of w, we have the equalities inf

f\x)=

inf

xeX MU)GC(intr')

xeX M(jr+jco)eC(intr)

inf f\x)=

f(x-\-xo)=

inf

xeX ^u(x)>\

xeX ^u(x-\-xo)>\+^uixo)

inf

inf

f{x'),

x'eX M(x')eC(int T)

(6.139)

/(x+xo)

fix')

(^ G (ry),

(6.140)

x'eX ^u{x')>\+^u{xo)

which together with (6.138), yield (6.131). Using the polarity A^^^ : 2^ -> 2^*^^^^ of (3.177), we shall give now a sufficient condition for strong duality. Theorem 6.19. Let (X, Z,u) be a system in which X is a set, Z is a locally convex space, and u: X -> Z is a mapping, and let f: X -^ R be a function and T an open convex subset of Z. If problem (6.110) has an optimal solution, i.e., if there exists xo G X with M(XO) G ZT such that /(XQ) = (X{= inf/(w~^(Cr))), then inf

xeX M(jc)€Cr

/(x) =

min

vi/eZ*\{0}

inf

xeX

4/w(A)>SUpVli(r)

/(x).

(6.141)

196

6. Unperturbational Duality for Reverse Convex Infimization

Proof. Define G c X by (6.111). Then by the above proof of Theorem 6.17, we have G c (A^j7^)^(Z*\{0}). Also, since w(jco) G CT, by the separation theorem there exists ^ G Z*\{0} satisfying ^W(JCO) > supvl/(r), i.e., such that xo G C(A^^y^)^({vI/}). Hence since JCQ G 5 ' C ( / ) , from Theorem 6.5 and Remark 6.8(b) we obtain (6.141). D Using the polarity /^fj: 2^ -> 2^*\sup^(T)

Proof Define G c X by (6.111). If jc G C(A22^)^(Z*\{0}), i.e., if there exists ^ G Z*\{0} such that ^u(x) > sup^(T), then clearly, u(x) ^ T, whence x e U-\CT) = CG. Thus, we have G c (A^2^)'(Z*\{0}). On the other hand, if x G CG = U~\CT), then u(x) ^ T, whence since T is a closed convex set, by the strict separation theorem there exists ^ G Z*\{0} such that supvl/(r) < ^u{x), so jc G C(A^2^)'(Z*\{0}). Thus we have G 5 (A^2^)XZ*\{0}). Consequently, by Corollary 6.1, we obtain (6.142). D The following corollary shows that if X is a topological linear space, u. X -> Z is a continuous linear mapping, and T is a closed convex subset of Z, then the assumption (6.118) can be omitted in Theorem 6.16. Corollary 6.11. Let (X, Z,u) be a system, in which X is a topological linear space, Z is a locally convex space, and u: X -^ Z is a continuous linear mapping. Furthermore, let f: X -^ R be an upper semicontinuous function, and T a closed convex subset of Z. Then we have (6.119). Proof. Let W = Z*\{0}. Then since T is a closed convex subset of Z, by Theorem 6.20 we have (6.142). Also, by our assumptions, for all vj/ e Z*\{0} we have ^u G Z*, whence {JC G X\^U{X)

> supvl/(r)} = {x e X\^u{x)

> sup^(T)].

(6.143)

Consequently, from (6.142), (6.143), the upper semicontinuity of / , and Lemma 1.1, we obtain (6.119). D Using the polarity Afj : 2^ -^ 2^*^^^^ of (3.191), we shall prove now the following surrogate duality theorem of hyperplane type: Theorem 6.21. Let (X, Z, u) be a system in which X is a topological linear space, Z is a locally convex space, and u: X ^ Z is a mapping that is either continuous

6.5 Duality for infimization over structured primal reverse convex constraint sets

197

or linear, and let f: X -^ R be an upper semicontinuous quasi-convex function and T a convex subset ofZ, with int T 7^ 0, satisfying (6.118) and inf f(x)
R two proper convex functions. In this section we shall consider the primal infimization problem (6.1) of the form (n

= (P^ f)

a'=a[,f=

inf / ( x ) ,

(6.152)

kix)>0

that is, infimization of a convex function / over a reverse convex strict inequality constraint CG = {X eX\k{x)

>0}

(6.153)

(so G = {x e X\k(x) < 0}). Conversely, given any subset G of X, taking ^ = XG we have (6.153), so problem (6.152) becomes the general reverse convex infimization problem a = i n f / ( C G ) . Note also that problem (6.152) may be regarded as problem (6.110) for the system (X, Z, w), where Z = R, u = k, and T = R_ = [T] e R\r] < 0). However, exploiting the formulation (6.152), we shall now obtain some surrogate and Lagrangian duality results, involving the conjugate functions /*, /c*. As in Section 6.4, our main tool will be again the substitution method. We shall assume that problem (6.152) is feasible, i.e., (dom/) n{x e X\k(x) > 0} # 0. Theorem 6.22. Let X be a locally convex space and f,k:X convex functions such that k = k**. Then

(6.154) -> R two proper

inf fix) = inf max inf {f(y) - r]^(y) -h ry/:*(0)} xeX

k(x)>0

OGX*

n>0

~

(6.155)

veX

'

= inf max{r]k*{(t>) - f^ir]^)}

:=

Proof Since k = /:**, applying (1.99), we obtain the equivalences

fi\

(6.156)

6.5 Duality for infimization over structured primal reverse convex constraint sets

199

k(x) > 0 0 0 0

yeX

^ 3 0 o G X * , r ( O o ) < ^o(x) 0} = {x eX\{^

e X*| r() < cl>(jc)} / 0}

= U^exAx e X\ r ( O ) < ^(x)}.

(6.157)

Hence by (6.157) and Lemma 3.7, we get the following surrogate duality result: a' = inf/(Uo€X*{^ ^ ^1 ^ * ( ^ ) < ^M})

= inf

inf

CI>GX*

f{x).

xeX

(6.158)

k*(0 xeX

k*m 2 ^ « polarity, f e 'R^, and G a A^A-convex subset of X. For an element zo ^ CG, the following statements are equivalent: 1°./(zo) = min/(CG). 2°. We have A(G) c A(A/(,,)(/)).

(7.3)

Prao/ 1° => 2°. By Remark 7.1, if /(zo) = min / ( C G ) , then for any set of functions W c. R and any polarity A: 2^ —> 2^ we have (7.3) (since A is antitone).

204

7. Optimal Solutions for Reverse Convex Infimization

2" ^ r . Since G is A'A-convex, we have A'A(G) = G. Hence if 2° holds, then by (7.3) and since A^ is antitone, we obtain A/(,,)(/) c A'A(A/(,,)(/)) C A'A(G) = G, and thus by Remark 7.1, f(zo) = min / ( C G ) .

•

Corollary 7.1. Let X be a locally convex space, f: X -^ R a function, and G a closed convex subset of X. For an element ZQ € CG, the following statements are equivalent: r./(zo)=min/(CG). 2\ We have (where A^^: 2^ -^ 2^^*\ 2^*\^^^ is the polarity (1.196)) A^^(G)CA«»(A^(,,)(/)).

(7.6)

Proof 1° =^ 2°, by Remark 7.1 and since A^' is antitone. 2° :=^ 1°. If (7.6) holds, then, since (A^^)' is antitone, we obtain, using 0 € G, (1.197), and the assumption that G is closed and convex, A^(,,)(/) c

(A«^)'AO^(A^(,,)(/)) C

(A«^)^A«k(^) = coG = G.

D

Corollary 7.4. Let X be a locally convex space, f: X -^ R a function, and G an evenly convex subset of X with 0 e G. For an element zo G CG, the following statements are equivalent: . r . / ( z o ) = min/(CG). 2°. We have (where A^^. 2^ -^ 2^*^^^^ is the polarity (1.199))

A^HG) C A^HAf^,^)(f)).

(1.1)

7.1 Minimum points of functions on reverse convex subsets of locally convex spaces

205

Proof. The proof is similar to the above proof of Corollary 7.3, using now (1.200). D Remark 7.2. The assumption on G is satisfied, in particular, when G is an open convex subset of Z, with 0 G G. In this case, condition (7.6) becomes G° c A/(^o)(/)°, an inclusion between the usual polar sets (1.82), and it implies (7.7), since A^2(G) = /S.^\G) (by Lemma 1.8). For any X, / , and G as above, we shall denote by 5c(^(/) the set of all optimal solutions of problem {P'') of (6.1), that is, 5CG(/)

:= {^0 G C G | /(ZO) = min/(CG)}.

(7.8)

Lemma 7.1. Let X be a locally convex space, f: X -^ R a convex function, and G a subset ofX, with G ^ X, satisfying inf/(G) < i n f / ( C G ) < +oo.

(7.9)

5cG(/)^bdCG.

(7.10)

Then

Proof Assume that we have (7.9) or only inf/(X) < i n f / ( C G ) < +oo, but not (7.10), so there exist XQ e X and zo ^ ';^ G G H CG c bd G = bd CG, in contradiction to (7.12). But then by the convexity of / , zo ^ o(zo) = supcDo(G), / ( > ; ) = min inf

yeX ^o(>')=supOo(')=supOo(G)

OeX*\{0}

f{y) = /(zo).

/(j),

(7.14) (7.15)

yeX 0(y)=sup(G)

(7.16)

206

7. Optimal Solutions for Reverse Convex Infimization

Proof, r => 2°. Assume 1°. Then since G is convex, with intG ^ 0, and since zo e CG, by the separation theorem there exists OQ e X*\{0} such that supOo(G) < Oo(zo) (< +00). On the other hand, by zo ^ ^ C G ( / ) ' (7.9), and Lemma 7.1, we have ZQ € bdCG c G, and thus ^oUo) < supOoCG) = supOo(G), whence the equality (7.14). Furthermore, by (7.9), Theorem 6.1, (7.14), and 1°, we obtain inf/(CG) =

) = _inf/(C(A«2)^({0})) 1 - O(xo) = - d i s t (xo, {^ e X I cD(>0> 1}) = - l^li Hence by Theorem 7.7, equivalence 1° 0, in terms of 6:-subdifferentials.

7.2 Subdifferential characterizations of minimum points of functions

211

Theorem 7.8. Let X = R^, f,k: X -> R two finite convex functions, and Zo e X such that 0 ^ {jc e X| k{x) > 0} 7^ X and inffiX)

< /(zo) < +00,

(7.36)

k(zo) = 0.

(7.37)

The following statements are equivalent: 2°. We have Uizo)

^

U^>O9.(M/)(ZO)

(S

Proof Let G := A^,,){k), so CG = [x e X\k{x) Remark 7.1, we have 1° if and only if AfuM)

> 0).

(7.38)

> 0}. Then by (7.37) and

^ A,^,om.

(7.39)

We claim that under the assumptions of the theorem, condition (7.39) is equivalent to SfuM)

^ S,^,,,(k).

(7.40)

Indeed, by (7.36), the continuity of / , k, and Remark 1.7, condition (7.39) implies (7.40). In the converse direction, by (7.36), let z e Af(zo)(f)' Then since / is continuous, Af(^Q){f) is open, so there exists a neighborhood V{z) of z such that Viz) c A/(,,)(/) c 5/(,,)(/). Hence if(7.40) holds, then V(z) c 5,(,,)(^). Therefore, since k is continuous, z e iniSk(zo){k) = A^^^^^ik), so (7.39) holds, which proves the claim. Now, by Chapter 4, Remark 4.1, condition (7.40) is satisfied if and only if /:(zo) = max/:(5/(,,)(/)).

(7.41)

But by Theorem 4.9, we have (7.41) if and only if Uizo)

c NASfUo)(fy^ zo)

(£ > 0).

(7.42)

Finally, by (7.36) and Theorem 1.7, ;V,(5/(,,)(/); zo) = U^>oa.(M/)(zo) and therefore (7.42) is equivalent to (7.38).

is > 0),

(7.43) D

We conclude this chapter with a theorem on minimum points for reverse convex infimization problems with a constraint set defined by a (nonstrict) inequality kix) > 0, using subdifferentials of / at the points of some level sets of / (in the spirit of Chapter 4, Theorem 4.8; however, the theorem below is less satisfactory, since it does not give a necessary and sufficient condition.

212

7. Optimal Solutions for Reverse Convex Infimization

Theorem 7.9. Let X, f, k, and ZQ be as in Theorem 7.8. (a) ///(zo) = min^ex,;t(A)>o / U ) , then dk(x) c U^>oa(M/)U)

(X e X, k(x) = 0, fix) = f(zo))-

(7.44)

(b) If f(zo) / min^ex.)tu)>o/(-^), ^^^« there exists x e 5/(zo)(/) satisfying k(x) = k(zo) such that dk(x)^N{Sf^,,^{f);x).

(7.45)

Proof (a) By Theorem 7.8 and its proof, condition f{zo) = minxeXMx)>o fM is equivalent to (7.42). Taking in both sides of (7.42) the intersection over all 6: > 0 and applying Theorem 1.6, we obtain

dk(zo) ^ n,>o^.(5/(zo)(/); ^o) = ^(5/(zo)(/); ^o) = u^>oa(M/)(zo). (7.46) Now, if X G X,k(x) = 0, fix) = fizo), then also fix) = min^'eXMx')>o fi^')Hence by (7.46) applied to zo replaced by x, we obtain (7.44). (b) By Theorem 7.8 and its proof, condition fizo) = mmxeXMx)>o fM is equivalent to (7.41). Hence if this does not hold, then by Theorem 4.8 (with / , G and go of that theorem replaced by k, 5/(^o)(/) and zo, respectively), there exists X e 5/(zo)(/) with kix) = kizo) satisfying (7.45). •

8 Duality for D.C. Optimization Problems

In the preceding chapters we have considered optimization problems with various primal constraint sets and objective functions. In this chapter we shall focus on "d.c. optimization problems," i.e., optimization problems involving differences of convex functions.

8.1 Unperturbational duality for unconstrained d.c. infimization In this section and the next one, we shall consider the following optimization problem: Let X be a locally convex space and let / , /i: X -> Rbc two convex functions. Then, the primal infimization problem (P)

a = inf ( / -f -h){X) = inf {f(x) + -h{x)}

(8.1)

.X€X

is called an unconstrained d.c. optimization problem^ and the objective function f -]- —his called a d.c. function; sometimes instead of "d.c." the term diff-convex is also used. Here the terms "d.c." and "diff-convex" stand for "difference of convex" functions. We use in (8.1) the "difference" / - j — h rather than f — h, since it may happen that both f(x) and /z(jc) are ±oo for some x e X; however, we shall simplify this in the present chapter, starting with the "notational convention" of Remark 8.1(b) below. If X is a locally convex space and / , /i, /, /c: X -> /? are four convex functions, then the constrained primal infimization problem

214

8. Duality for D.C. Optimization Problems (Pe)

a=

inf

xeX lix)+-kix);) -/z(>;)} - supcD(G)}. Consequently, by (8.4) and Theorem 8.1, it follows that, if h = h**, then sup/i(G) = - inf {XGM - h{x)] = - inf {/i*(cl>) - X G ( ^ ) } xeX

= sup cDeX*

OeX*

{-{sup{(j)-/z(j)}-supO(G)}} >eX

= sup {inf [/i(>;) - 0(>;)] + supO(G)}; thus, we have obtained again Theorem 3.14. (b) If G is a (nonempty) subset of X, then for h = —XCG ^ ^ have —XCGU) = - 0 0 for all X G G, whence ( - X C G ) * ( ^ ) = sup^^xl^C-^) " (-XCG)(-^)} =

+^

whenever G / 0, so one cannot apply Theorem 8.1 to reverse convex infimization (see formula (6.9)). X

(c) For any f,heR

satisfying a = inf {fix) - h(x)} > - 0 0 ,

(8.30)

xeX

we have dom/cdom/z.

(8.31)

Indeed, if there exists XQ Gdom / \ d o m h, i.e., /(XQ) < +oo, h(xo) = +oo, then a = inf;,ex{/(-^) - h{x)} = - o o . Note also that if (8.30) holds, then by (8.26), we have dom/z* c d o m / * .

(8.32)

(d) If both / = /** and h = /z**, then applying Theorem 8.1(a) to the functions /z*, / * : X* -> J, and using that (X*, or(X*, X))* = X, we obtain that P = inf {/i*( —oo the following statements are equivalent: 2\ f -h>y; ^•f>h + y. These imply the following statements, equivalent to each other: 4°. Supp / 5 Supp h - (0, y) (where Supp / denotes the (X*, /?)-support set (1.103)); 5 ° . e p i / * 2 e p i / i * - ( 0 , K); 6°./* y. Moreover, if f = /** and h = /z**, then all statements 1°, . . . , 8° are equivalent. Proof The equivalences 1° 4^ 2° 4^ 3° are obvious. 3° =^ 4°. By (1.103), the inclusion 4° means that {(CD, d) eX* X R\^-d

< f}^

{(O, d) e X* x R\^ - d < h} - (0, y)

= [(^,d-y)\(^-d

/(zo) - hiz^) = r - hix)

ix' e X),

whence fix)

> r + hix') - hix) > r + ^(jc' - x)

(JC' € X, O G dhix)).

(8.37)

222

8. Duality for D.C. Optimization Problems

The implication 2° => 3° is obvious. 3° =^ 1°. Assume 3° and let jc G X. We want to prove that f(x) - h(x) > f(zo) -h(zo). Choose r:=h(x)

+ f{zo)-h(zo)-

(8.38)

Case (i): r < f(x). Then f(x) — h{x) > r — h(x) = f(zo) — h(zo). Case (ii): r > /(JC), that is, (JC, r) e epi / . Then by (8.38) we have (8.36), whence by 3°, fix')

> r + 0 ( x ' -x) = h(x) + fizo) - h(zo) +
4>Gdom/?*

=

inf {/(jc)-/i(x)},

(8.45)

xedomf

which proves the assertion. Similarly to Chapter 4, Theorem 4.9, introducing now a parameter e, let us give a characterization of global minima using ^-subdifferentials. Theorem 8.3. Let X be a locally convex space, f: X -^ R a proper convex function, andh: X ^^ R a proper lower semicontinuous convexfunction. For an element Zo ^ dom / n dom/z the following statements are equivalent: r . /(zo) - /lUo) = mm,^x[f{x) - h{x)]. T. We have dsh(zo) C dsf(zo)

ie > 0),

(8.46)

deh{zo)^S,f{zo)

(£>0).

(8.47)

or, equivalently.

Proof 1° ^ 2°. Assume 1°, i.e., that f(zo) - h(zo) < f(x) - h(x)

(X € X).

(8.48)

Let £ > 0 and O € 9e/i(zo), i-e., ^ - /^(x) < -h{zo) - 0(x - zo)

(x e X).

(8.49)

Then by (8.48) and (8.49), we obtain £ + f(zo) - h(zo) < £ + fix) - h{x) < fix) - hizo) - r . Assume that 1° does not hold, i.e., there exists XQ e X such that fixo) - hixo) < /(zo) - hizo).

(8.50)

We claim that hizo). /(zo), fixo) e R and fixo)-fizo)
—oo. Hence by hizo), fizo). fixo) ^ ^ and

224

8. Duality for D.C. Optimization Problems

h(xo) > —00, (8.51) holds (for h{xo) = -hoo this is obvious, and for h{xo) e R this follows from (8.50)), which proves our claim. Consequently, (xo, f(xo) - fizo)) e{Xx

/?)\epi (h - h(zo)).

(8.52)

Hence, since epi (h — h{zo)) is a closed convex set in X x R (because the function h — h{zo) is proper, lower semicontinuous, and convex), by the strict separation theorem and Chapter 1, formula (1.28), there exists {^, /JL) e (X x R)* = X* x /? such that sup

(vl/, /x)(x, d) < (vl/, M)(XO, f(xo) - /(zo)),

(8.53)

(x,d)etpi(h-h(zo))

which means that for v i— ^uP(jc d)Gepi{h—h(zo))^^'' ^^^•^' ^^ ^^ have ^ ( ^ ) + /xJ < V < vI/(xo) + fiifixo)

-

fizo))

ax,d)ecpi{h-h(zo)))-

(8.54)

We claim that we may assume // < 0 in (8.54). Indeed we cannot have /x > 0 (take d -> -\-oo in (8.54)). Furthermore, if (8.54) holds for /x = 0, then ^(x) O.Then^ > Ofor all (x, d) eepi (h — h(zo)), whence by (8.55), for any /x < 0 and all (x, d) Gepi {h — h{zo)) we have ^(x) + /xJ < ^(jc) < v. Consequendy, for any /x < 0 sufficiently near to 0, too, we have (8.54), which proves our claim. Hence for such a /x, dividing by —/x (> 0) and taking d = h(x) — h(zo) in (8.54), we obtain vi/(^) _ h(x) + h(zo) ^ix - zo) - £ fixo) - fizo) < O(xo - zo) - e.

ix e dom/z),

(8.58) (8.59)

Taking x = zo in (8.58), we see that ^ > 0. Also, by (8.58) we have O G dMzo). and by (8.59) we have O ^ a,/(zo), so a,/z(zo) 2 ^^/(zo), and thus (8.47) does not hold. If here £ = 0, i.e., if dhizo) 2 V(^o). then there exists also 6:^ > 0 such that d^'hizo) 2 ^e'fizo) (indeed, otherwise, by (1.131) we would obtain dhizo) = r\e>odshizo) ^ r\8>o^^f(^o) = 9/(zo), a contradiction); thus, (8.46) does not hold either. D

8.3 Duality for d.c. infimization with a d.c. inequality constraint

225

Remark 8.7. (a) Taking e = 0 in (8.47), one obtains again the necessary condition (8.40) for zo to be a global solution of the infimization problem (P) of (8.7). (b) In particular, Theorem 8.3 yields again Theorem 4.9 on convex supremization. Indeed, if G C X, then as observed at the beginning of Section 8.1, for / = XG we have inf ( / — h)(X) = — sup/i(G), so if zo edom /fldom h = G O dom /z, then 1° becomes h(zo) = suph(G); also, by (1.130) and (1.127), for zo ^ G wt have dsXcizo) = {O G X*| cD(x - zo) < XG(X) - xcizo) + s{xe X)} = {O e X*| (D(jc) - 0(zo) <s{xeG)} = N^G; zo),

(8.60)

so (8.46) becomes formula (4.57) of Theorem 4.9 (mutatis mutandis).

8.3 Duality for d.c. infimization with a d.c. inequality constraint In this section we shall consider the constrained primal infimization problem (8.3), that is, with the notational convention (8.6), the primal problem (Pd)

a=

inf

{f(x)-h(x)}.

(8.61)

xeX l(x)-k(x) /? are four convex functions; furthermore, we shall also mention some cases in which the strict inequality < in (8.61) is replaced by < . Remark 8.8. (a) We shall see in the last part of this section that the duality results for the optimization problems (8.61) encompass some important particular cases, identically 0 on obtained by taking one or more of the functions f,h,l,k:X->R X. (b) There are some cases (see Proposition 8.3 below) in which the strict inequality < in (8.61) can be replaced by 0 ^*(0)-(Xdom/)*(0)0

^eX*

i.e., the right-hand side of (8.27). (2) Duality for the problem (6.152) of reverse convex infimization with one reverse convex inequality constraint: Taking h = I = 0 in Theorem 8.4, we obtain again the first part of Corollary 6.12. Indeed, the left-hand side of (8.65) becomes infxeXMx)>o fix). Also, by (8.83) (mutatis mutandis), we have /z* = X{0}, so domh* = {0}, and hence the right-hand side of (8.65) becomes inf max {/z*(0) + r]k*{^) - /*(0 -h rj^)} 4>Gdom^* r]>0 k*m-(xdomfrw)-/*(ryO)}.

(8.84)

edomk* r]>0 A:*(O)-(Xdom/r(O) 0. Hence, for O e dom^*. /:*(0)>0 -oo < 0

r(^)-(Xdom/)*(0)

tfO.O, if CD ^ 0,

so k*(^) - (Xdom/)*(^) < 0 if and only if O 7^ 0. Consequently, (8.84) becomes inf

max {77^(0)-/*(^cD)},

OGdomfc*\{0} r;>0

which is the first part of the right-hand side of (6.164). (3) Duality for the problem of convex supremization, with a constraint set determined by one d.c. inequality: Taking / = 0 in Theorem 8.4, we obtain the following result: Proposition 8.4. IfhJ,k: /:**, then sup

h(x)=

X -^ R are three convex functions, with h = h**, k = sup

xeX

,^eX*

i(x)-k(x)
(iijjH^^),

whence

{f + f2 '''•''•)* - /*°('?i'i)*° • • • ^(^mU*,

(8.109)

where the right-hand side denotes the infimal convolution, defined by

(/*n(^i/i)*n...n(^^/,)*)(vi/):= inf

{/*(vI/o) + (^i/i)*(^i) + --- + (^./.)*(^.)}

(^eX*).

(8.110)

Hence by a = yS, Proposition 8.7, and (8.109), (8.110) with ^ = 0) + YlT=i ^i^i, we obtain (8.108). D Let us give now a result of duality in which the suprema in the right-hand side are attained. To this end, let us consider the following "Slater condition": V(c|)i,..., O^) e Q, 3xo e dom/, l.{xo)-i(xo)+kl(i)0(i e Im)}}. the feasible set of (P2). By (8.112), an equivalent condition to (8.118) is V(cDi, . . . , O,) G ^2, V^ = (r;,, . . . , ^7,) G /?:;', £ rit = 1, m

m

J2mk*(i) < (Xdom/r(5]'7,o,).

(8.119)

From Theorem 8.6 we obtain now the following result: Corollary 8.3. Let X be a locally convex space and let f,h,ki'. X ^^ R (i e Im) be functions with f convex, h(x) = /z**(x) for all x e T{P2), and k\, .,. ,km subdifferentiable on ^(^2)- Furthermore, assume that (8.118) holds. Then for a of (8.117), we have ot

h\^)-Vy^r)ikU^i)

=

(0

a/:*(cD,)n^-'(/?+)/0

Remark 8.17. (a) By (8.121), if the function k\ is differentiable, the constraint a^*(0,) n k;\R^) / 0 in (8.124) reduces to ^i(V^*(Oi)) > 0, where V denotes the gradient. (b) In the case that d o m / = X, condition (8.122) is satisfied provided that inf ^1 (X) / 0 (or alternatively, provided that infk\ (X) is not attained). Let us pass now to the case of strict inequality constraints, i.e., to the primal problem (P
Thus, the set ^ of Lemma 8.1 is now replaced by n ) ^ | dom/:*. Correspondingly, in the following theorem the set dom/i* x ^ of Theorem 8.6 will be replaced by dom/i* X nf^^domit*. Theorem 8.8. Let X be a topological linear space, K a compact convex subset of X, and f.hJi.ki'. X -^ R (/ e Im) finite-valued lower semicontinuous functions with h(x) = h**(x)for all x e T(PK,
8.4 Duality for d.c. infimization withfinitelymany d.c. inequality constraints

243

Proof. Take /z = 0 in Theorem 8.8. Then /?* = X{0}, so dom/z* = {0}, and hence formula (8.135) reduces to (8.140). Furthermore, let JCQ be an optimal solution of (Pj^ /?, or simply the Lagrangian, associated with p by L(x, vj/) := - sup {vl/(z) - p (jc, z)} zeZ

= inf {p (jc, z) - ^iz)}

(xeX,^

e Z*),

(8.147)

zeZ

and the Lagrangian dual problem associated with the perturbation function p as the unconstrained infimization problem (D)

yS := inf A(Z*),

(8.148)

where A: Z* -> P is the dual objective function defined by A.(^) := - sup L(jc, vl/) = inf (-L(jc, vj/)) xeX

(vj/ e Z*).

(8.149)

^^^

Then by (8.148) and (8.149), P=

inf inf ( - L ( x , ^ ) ) .

(8.150)

8.5 Perturbational theory

245

Theorem 8.9. (a) We have 0(jc) < inf (-L(jc, vl/))

(jc G X),

(8.151)

(3/2< hence a = inf 0(X) < inf A(Z*) =

fi.

(8.152)

(b) If px{0) = p**(0) for all x e X, where pxi.): Z -> R are the partial fur :tions PAZ):=P(X,Z)

(xeX^zeZ),

(8.153)

the I

0(x) = inf (-L(jc, ^))

{x e X),

(8.154)

vyeZ*

an hence a = p.

(8.155)

Pr of (a) Observe first that for all JC G X and ^ G Z*, -PI{^)

= - sup{vl/(z) - /7(jc, z)} = L(x, vl/).

(8.156)

zeZ

Hi ice by (8.144), (8.153), and (8.156), for all x G X we have 0, c) = -p{x, 0) =

-PAO)


p,,{0) + vl/o(z) = -a + vi/o(^)

(z e Z),

(8.159)

whence by (8.147), -L(xo, ^o) = sup {vI/o(z) - /7(xo, z)} < a, zeZ

and thus by (8.149), -X(^o) = sup L(x, VI/Q) > L(XO, ^o) > -oi. xeX

Consequently, A,(^o) < —^(-^o, ^o) S a, which together with Theorem 8.9 and (8.148) yields P = X(^o) = -L(xo, ^o) = a,

(8.160)

so ^0 is an optimal solution of (D) and satisfies (8.158). Finally, by p{xo, 0) = —a (which holds by (8.145) and since XQ is an optimal solution of (P)) and the last part of (8.160), we have (8.157). D Remark 8.20. (a) For the unconstrained d.c. infimization problem (8.7), let Z = X and (j) = f — h. Then the perturbation function p: X x X -> R defined by p(x, z) := h(x + z) - fix)

(jc, z G X)

(8.161)

satisfies (8.144), and the perturbational dual problem (8.148) yields the unperturbational dual problem (8.24) of Section 8.1. Indeed, by (8.147) and (8.161), for any X e X and ^ E Z* we have L(x, vj/) = - sup {^(z) - p (x, z)} = - sup {vl/(z) - h(x + z) + fix)} zeX

zeX

= - sup {vl/(x + z) - /z(jc + z)} - fix) + ^ix) zeX

= -/z*(vl/) - fix) + vi/(jc)

ixeX,

^ e Z*).

(8.162)

(vl/ G X*),

(8.163)

Consequently, by (8.149) and (8.162), Xi^) = -sup Lix,^)

= /z*(vI/)-/*(vI/)

XGX

whence by (8.148), we obtain P=

inf {/i*(vl/) -/*(xl/)},

(8.164)

which is nothing other than (8.24). (b) For Z, 0, and p as in (a) above, by (8.161) and (8.162) the equality (8.157) becomes

8.6 Duality for optimization problems involving maximum operators /zUo) = -/z*(vI'o) + ^oUo).

247 (8.165)

Also, by (8.163) and (8.162), the equality (8.158) becomes /(xo) + /*(vI/o) = vi/o(xo).

(8.166)

By the "Fenchel equality" (1.112), we have (8.165) and (8.166) if and only if ^oedf{xo)ndh{xo).

(8.167)

Since the Fenchel inequalities ^o(-^o) < ^(-^o) + ^*(^o) and ^o(-^o) < /(-^o) + /*(^o) always hold, (8.157) and (8.158) are called "extremality relations." An element XQ e X for which there exists ^o ^ ^* satisfying (8.167) is called a "critical point" of f — h. (c) lfh = /z**, then by (8.156), (8.162), (8.161), and (8.153), pTm

= sup {-P:(^)} vj/eX*

= sup L{x, vl/) = sup {-/i*(vl/) - fix) + vl/(x)} ^eX*

vi/eX*

= /i**(x) - fix) = hix) - fix) = pix, 0) = PAO)

ix e X),

Consequently, by Theorem 8.9 and (8.164), it follows that a = inf 0(X) = inf ( / - h)iX) = P= inf {/i*(vl/) - /*(vl/)}, ^eX*

and thus we have obtained another proof of the main part of Theorem 8.1 (b).

8.6 Duality for optimization problems involving maximum operators We shall now consider optimization problems in which the objective function is the maximum of two functions. Let X be a locally convex space and lei f,h: X -> R be two quasi-convex functions. We shall consider the unconstrained primal infimization problem iP)

inf max (/, -h)iX)

= inf max {fix),

-hix)}.

(8.168)

X€X

Thus, comparing problems (8.7) and (8.168), we see that now the difference f + —h is replaced by the maximum max (/, —h), or in other words, the operation -j— is replaced by the operation max — (or equivalendy, -j- is replaced by max). Remark 8.21. By Remark 8.11, problem iP) of (8.168) is a d.c. problem, but in order to apply to it the preceding results of this chapter, one would need to give explicidy the conjugates of the functions whose difference is the function max (/, —/z), which would lead to rather complicated expressions. Therefore, in the sequel we shall apply more direct methods to obtain duality results for problem (P).

248

8. Duality for D.C. Optimization Problems

Note that if G is a convex subset of X, then pc, the "representation function" of (1.224), is convex; indeed, epi pc = {(x, d) e X x R\ pcM < d] = G x R is convex (in contrast, recall that the function -XCG used in formula (8.12) is only quasi-convex, but it need not be convex). Hence the above problem (P) encompasses the following two particular cases: (1) If G is a convex subset of Z, and h: X -^ R is a. quasi-convex function, then for f = PQ the primal problem (8.168) becomes inf max (PG, -h)(X)

= inf(-/z)(G) = -sup/z(G),

(8.169)

i.e., a problem of quasi-convex supremization, studied in Chapters 3 and 4. (2) If G is a convex subset of X, / : X ^- /? is a function, and h. X -^ R'lSdi convex function, then for h = pc the primal problem (8.168) becomes inf max (/, -PG){X)

= inf / ( C G ) ,

(8.170)

i.e., a problem of reverse convex infimization, studied in Chapters 6 and 7. Remark 8.22. It is natural to consider also problem (P) of (8.168), where / is a quasi-convex function and h is a quasi-concave function, i.e., the problem (P)

inf max (/, h)(X) = inf max {/(jc), h(x)},

(8.171)

xeX

where / and h are two quasi-convex functions; this, too, has been studied in the literature (see Voile [290] and the references therein), but we shall not consider it here, since it is equivalent to the problem of quasi-convex infimization.

8,6.1 Duality via conjugations of type Lau The conjugations of type Lau (1.223) constitute a natural tool to study duality for unconstrained primal problems involving maximum operators. Indeed, we shall see in Remark 8.23 that they permit one to recover, as particular cases. Theorems 3.11 and 6.10. Proposition 8.8. Let X be a set, W ^ R , and A: 2^ ^^ 2^ a polarity. Then for X

any functions f^heR

we have

inf max {/(jc), -h^'^^^^^^^'(x)} = inf msix {h^^^\w), -f^^^\w)}. xeX

Proof Let f,heR. obtain

(8.172)

weW

Then by (1.226) (with g = h^^^'>), (1.144), and (1.223), we

8.6 Duality for optimization problems involving maximum operators inf max{/(x), -h^^^^^^^^'(x)]

= inf max|/(jc),

xeX

xeX

I

= inf

inf

= inf

inf

h^^^\w)

u;eCA({jc})

inf

rmix{f(x),h^^^\w)} m2ix{f{x),h^^^\w)}

= inf max\h^^^\w), weW

249

I

inf

f(x)]

jceCA'({u;})

J

= inf m2ix[h^^^\w), -f^^^\w)}.

D

weW

We have the following basic duality theorem for problem (8.168). Theorem 8.11. Let X be a set, W c 'R^, and A: 2^ -> 2 ^ a polarity. (a) For any functions f,heR we have the inequality inf max {/(x), -h(x)] < inf max{h^^^\w), xeX

-f^^^\w)}.

(8.173)

weW j^

(b) For any function h e R the following statements are equivalent: 1°. h is A^A-quasi-convex. 2°. We have inf max {/(jc), -h{x)} = inf maxf/z^^^^u;), -f^^^\w)} xeX

( / e J^).

weW

(8.174) Proof (a) Let fhel^^.By

/z^(^)^(^)' < h, we have -h < -/i^(A)^(^)', whence

inf max{/(x), -h{x)} < inf max{/(jc), -h^^^^^^^^\x)}, xeX

(8.175)

xeX

which together with (8.172), yields (8.173). (b)Let/z G ^ ^ . 1° =^ 2°. If 1° holds, then by (1.227) we have h = /z^(^)^(^)', whence by (8.172), we obtain (8.174). 2° =^ r. If 2° holds, then applying (8.174) to / replaced by /^(^)^(^)' and using that /^^^^ = (/^(A)L(A)y(A)^ ^^ ^^^^^^^ f^^ ^^^ f elR^, inf max {/(jc), -h(x)} = inf max{/i^^^Hu;), xeX

-f^^^\w)}

weW

= inf max{/i^*^*(u;),-(/^*^'^y*^^(w')} = inf max {/^^'^'^'^''(x), -/!(x)}.

(8.176)

Now let / = ps,(h), where d e R. Then by (1.224), the left-hand side of (8.176) is inf max{/05^(;j)(jc), —/z(x)} = xeX

inf [—h(x)} = — sup h(x), xeSAh)

xeSAh)

250

8. Duality for D.C. Optimization Problems

and by (1.228) and (1.224), the right-hand side of (8.176) is inf max{(/)5,(/,))^^'^^^^^^'(x), --h{x)] = inf max{pA'A(5,(/.)(-^), -h(x)} xeX

xeX

=

inf

{—h(x)} = —

xeA'AiSj(h)

sup

h(x).

xGA'A(Sj(h)

Therefore, by (8.176), we have sup

h{x) = sup h(x) < d,

xeA'A(SAh))

xeSj(h)

whence h{x) < d for all x e A'A(Sd(h)), that is, A'AiSAh)) c Sd(h), which, since the opposite inclusion always holds, yields A'A(Sd(h)) = SdQi) for any d e /?. Hence by (1.153), hq(A'A)M =

inf

deR xeA'A(SAh))

d=

inf d = h(x)

(x e X).

deR xGSdih)

D

Remark 8.23. (a) Theorem 8.11 yields Theorem 3.11 as a particular case. Indeed, for f = PG of (1.224), the left-hand side of (8.174) becomes (8.169) and the righthand side of (8.174) becomes, using also (1.225) and (1.224), inf max [h^^^\w), -p^ioiu))}

=

inf

weW

h^^^\w),

IL'GCA(G)

and hence for h e Q{A'A), from (8.174) we obtain (3.109) with / replaced by h. (b) Theorem 8.11 yields also Theorem 6.10 as a particular case. Indeed, for h = PG the left-hand side of (8.174) becomes (8.170), and the right-hand side of (8.174) becomes, using also (1.225), (1.224), inf max{pA(G)(u;),-/^^^\u;)}= weW

inf

(-/^^^\u;)),

weA(G)

and hence, for G e C(A'A) (which is equivalent to PG € Q(A'A), by [254], formula (4.43)) from (8.174) we obtain (6.85). For W = (Z*\{0}) X R and the polarity A = A^' of (1.189), from Theorem 8.11 (b) we obtain the following: ^ Corollary 8.6. Let X be a locally convex space, and f,heR two functions, with h lower semicontinuous quasi-convex. Then inf max {/(jc), —h{x)] inf (O,^)G(A:*\{0})X__ iX*\{Q})xR

max! inf / ( x ) , ^ I^ xeX U)>^

inf h{x)\. xeX ^{x)>d

(8.177)

J

Proof. For the polarity A = A^^ (1.190) and (1.229) hold, so Theorem 8.11(b) yields the result. •

8.6 Duality for optimization problems involving maximum operators

251

Remark 8.24. Applying Corollary 8.6 to f = pc of (1.224), and using (8.169), we get -suph{G)=

inf

maxj inf pcix),-

(ct),J)G(X*\{0})> (ct),J)G(X*\{0})x/?

inf hix)\.

I JCGX (x)>d

xeX (x)>d

(8.178)

J

But inf

-00

if supO(G) > J,

-foo

if sup4>(G) < J.

PG(-^) = {

xex/'^' d

I

r-

V / —

whence by (8.178), sup/z(G) =

sup mini— inf pcM, inf h(x)\ (M)e(X*\m.R I 41^^^, ^1^)^^ J

=

sup

inf /z(jc),

{,d)e{X*\{0})xR^fl supcD(G)>J

*i-^)>^

SO we have obtained again Corollary 3.9 (in Chapter 3 we deduced it directly from Theorem 3.11 applied to the polarity A = A^ ^). One can replace in Corollary 8.6 (X*\{0}) x Rby X* x R, and one can give an extension of the latter to systems. To this end, we shall use the following lemma: Lemma 8.4. Let (X, Z, w) be a system with Z a locally convex space, f: X -> R a function, and h: Z ^^ R a lower semicontinuous quasi-convex function. Then (hu)(x) =

sup

inf h{z)

(x e X).

(8.179)

Proof The inequality > in (8.179) is obvious. In order to prove the opposite inequality, let r < h{u{x)) be arbitrary. Then u(x) ^ Sr(h), where Srih) is a closed convex set (by our assumptions onh). Hence by the strict separation theorem, there exists ^0 € 2* such that supvI/o(5,(/i)) < VI/O(M(JC)).

(8.180)

Let do := sup^o('5r(^))- Then z e Sr(h) implies ^o(z) < do, and hence ^o(^) > do implies h(z) > r. Consequently, inf:.^z.^oiz)>do h(z) > r, whence r


i^,d)eZ*xR^„y%^ , %ou)(x)>d ^^^>^

Hence since r < h(u(x)) has been arbitrary, we obtain the inequality < in (8.179). D Now we can prove the following result:

252

8. Duality for D.C. Optimization Problems

Theorem 8.12. Let (X, Z,u) bea system with Z a locally convex space, f: X ^^ R a function, and h : Z -^ R a lower semicontinuous quasi-convex function. Then inf max{/(jc), —h{u{x))} = inf

(vI/,J)eZ*x/?

max! I

inf

xeX ^{u{x))>d

/(JC), -

inf h{z)\.

zeZ ^(z)>d

(8.181)

J

Proof By Lemma 8.4 and since (/?, (z)>d

^

maxj/(x), sup (—h{z))\

xeX {^J)eZ*xR {^u)ix)>d

=

sup (—h(z))\

(^,d)eZ*xR {^u)ix)>d [

inf

xeX i^u)(x)>d

^^2 ^(z)>d

^

m a x | / ( x ) , sup I

.g2 ^\z)>d

(—h(z))\, J

whence using again the complete distributivity of (/?, 0 and O G X* we have — z;/z*(0) = (—^)(—co) = +00, and hence by (1.92) and (1.84), we obtain inf {-r]/z*(cD) + (i^/)*(vl/ + z;), ^(x) -

f(x)}.

xedom f Gdom/i*

Exchanging here sup^^^om/ ^"d sup^^^^^;^*, and applying (8.183), we obtain [max(/,-/z)]*(vI/) = sup

sup

^edomh*xGdomf

inf {z7[(0 + ^ ) ( ; c ) - / z * ( 0 ) ] - i ^ [ / ( x ) - v l / ( j c ) ] } . ^'^^^

(8.187)

254

8. Duality for D.C. Optimization Problems

Hence using that supinf < inf sup, it follows that [max(/,-/i)]*(vl/)< sup =

inf

sup

sup {^[(cD + vl/)(x)-/z*(cD)]-z^[/(jc)-\[/(x)]} inf |-^/z*(cD)-f

sup {[(^ + ?^)vl/-h y;0](jc) - i^/(jc)}

that is, (8.184). (b) Assume now that / is convex, with d o m / / 0 and /z ^ ±oo, /z = /z**, and let O e dom/i* (hence /z*(0) e /?). Then by Theorem 1.9, with C = d o m / , Z) = {(y;,2^) >0|y/ + ?^ = l},and (p(x. (r], 1^)) = ^[(cD + vi/)(x) - /z*((D)] - nf(x)

-

^(x)l

we have (even if / takes the value —oo), the equality sup

min {y/[(0 + ^)(x) - /2*((D)] - n/M min

- ^(x)]} =

sup {r/[(0 + ^)(Jc)-/z*(cD)]-z^[/(jc)-vI/(jc)]},

^ ' ^ ^ 0 jcGdom/

Hence by (8.187) (where we can replace inf by min, because of the compactness of D = {(T], i^)>0\r] + i} = 1} and continuity), we obtain (8.185). D The assumptions in Proposition 8.9 cannot be entirely omitted. Indeed, even if both / and h are convex, the inequality (8.184) may fail if /i / /z**, as shown by the following example: Example 8.1. Let X = R, fM=\

-1 '

ifx < 0 , r/'"!'

(8.188)

0

ifx < 0, . ' ifx > 0 ,

(8.189)

+00

h(x) = \

+00

if jc > 0,

so max (/, —h)(x) = 0 if jc < 0, — 1 if jc = 0, and +00 if jc > 0, whence [max(/, -h)T(0)

= sup{-max

{/(JC),

-h(x)}} = 1.

AG/?

On the other hand, /z*(0) = sup^^;^ {^M - h(x)} = sup^^o ^M = 0 if O > 0 and +00 if O < 0, whence dom /z* = { O e / ? | O > 0 } . Furthermore, for O edom /z* and yy, z^ > 0, ^ + ?^ = 1, we have (Z>/)*(Z7CD) = sup{z/cD(x) - i^f(x)} = sup{z70(x) - z^(-l)} = z?, xeR

x)} = sup inf (?>/)*(y/0) =

inf ?^ = 0.

Also, if / is not convex, then the equality (8.185) may fail if h ^ /z**, as shown by the following example: Example 8.2. LtiX = R, /(.)=!*;

•'/--;'

1

h(x)=x

(8.190)

i f JC > — 1,

{xeR),

(8.191)

so max(/, —h)(x) = — x if x < —1, and 1 if x > —1, whence [max(/,-/z)]*(0) = sup{-max{/(x),-/2(x)}} = - 1 . xeR

On the other hand, /z*(0) = sup^.^^^ {0(jc) - x] = X{0}(^). whence dom /z* = {0} and /z*(0) = 0. Furthermore, for z^ > 0 we have (z^/)*(0) = sup,.^;j{-z?/(jc)} = 1^, and hence sup

inf {-r]h*((^)-\-{}}/)*(r]^)}=

inf (i>/)*(0) =

cDedom/7* ^'^>^

'?-^>0

inf

i}=0.

/].f?>0

Taking ^ = 0 in (8.185), we obtain as mentioned at the beginning of this section, a duality result for problem (P) of (8.168): Theorem 8.13. Let X be a locally convex set and let f,h\ functions, with h = h**. Then inf max {/(jc), -h(x)] = xeX

inf

X —> R be two convex

max [r]h*{^) - {§fy{7]0

Proof. If / = +00, then both sides of (8.192) are +oo. Assume now that / ^ +00, so (i^fy(r] 0, zy + z^ = 1. By Proposition 8.9(b) (with ^1/ = 0) we have inf max {/(jc), -h(x)} = - [ m a x ( / , -/z)]*(0) xeX

= =

sup inf

min {-r]hH^) + (?^/)*(^0)} max {-{-^/z*(0)-f (z^/)*(r;0)}}.

But since —y;/z*(0) and (i^/)*(r70) are never —CXD, for O G dom/z* and r/, z> > 0, yy + 1^ = 1, the last expression coincides with the right-hand side of (8.192). D

256

8. Duality for D.C. Optimization Problems Let us give an application of Theorem 8.13 in normed linear spaces.

Corollary 8.7. Let X be a normed linear space, and h: X ^^ R a function with h # iboo, h = /z**, h(0) < 0. Then Mmax{\\xh-hix)}= inf

\ '

(8.193)

Proof. Let f(x) = \\x\\

(xeX),

whence, for any ^ e domh* and ^, ?^ > 0, ^ + ?^ = 1, (^/)*(^(D) = sup{rj(x) - z^ lUII} = j ^^J ' I n J i l l t ^ ) = ^^^-*^^^^' where Bx* = {O G X*| ||cD|| < 1} (the unit ball of X*). Then by (8.192), we obtain inf max{||x||, -h(x)} = xeX

inf

max [T]h*((t>) -

x^BxAn^)]

(Pedomh* ?;.i?>0

=

inf

sup r]h*() =

inf

sup r]h*(^).

But by our assumption, h(0) < 0, whence h*() — sup^g;j^{0(x) — /z(x)} > —/z(0) > 0 for all O G X*, and hence the last expression coincides with the righthand side of (8.193). D Returning to locally convex spaces, let us give now an application of Theorem 8.13 to reverse convex infimization. Theorem 8.14. Let G be a closed convex subset of a locally convex space X, and let f: X -^ R be a convex function. Then for any number r e R such that max (r, inf/(G)) > i n f / ( C G ) ,

(8.194)

we have inf/(CG) =

inf

max {r](sup cD(G) + r) - (?^/)*(^?cD)}.

(8.195)

supO(G) r, so inf

maxdijil , -XG-ixo}(y) + r] = r.

(8.200)

yeG-{xo}

Furthermore, since —XG-{XO}(>') -^ f = —oo for y e C(G — xo), we have

258

8. Duality for D.C. Optimization Problems inf

rmix{\\y\\,-XG-{xo}(y) + r}=

>f

:yeC(G-{xo})

M

veC(G-.vo)

= dist (xo, CG).

(8.201)

Hence by (8.200), (8.201), and (8.197), we obtain inf maxdijil, -XG-{xo}(y) + r} = min[ yeX

inf

max{||j||,

-XG-{XO}(J)

+ ^}.

y€G-{.vo}

inf

max{||j|| , -XG-{xo}(y) + f)] = niin[r, dist (XQ, CG)] = dist (XQ, CG),

jeC(G-{xo})

which proves (8.199). Now let h = —r < 0; also, /i*(c|>) = sup {a>(x) -

XG-{XO}M

XG-{XQ}

- r. Then by XQ e G, we have h{0) =

+ d = sup 0(G - {xo}) + r

(cD G

Z*).

xeX

Hence applying (8.193) with h =

XG-{AO}

"" ^^ ^^ obtain

inf max (11,11, - X G - , . „ , W + r } =

Jnf.

sup(G)

sup

inf/(CM).

(9.81)

inf /(CM).

(9.82)

MeM

If in addition, f e Q(M), then sup / ( G ) =

sup MeM GnCM7^0

Proof If M e M , Gr\ (CM) 7^ 0, say ^ G G n CM, then sup/(G) > f{g) > inf /(CM), while if {M G A^ | G n CM) ^ 0} = 0, then the right-hand side of (9.81) is sup 0 = - 0 0 . Assume now, in addition, that / G Q{M). Let us first observe that | J { M G M I g G CM} = {M G A^ I G n CM 7^ 0}.

(9.83)

9.4 Surrogate duality for abstract reverse convex infimization

271

Hence by / G Q(M), (9.20), Lemma 3.7, and (9.83), we obtain sup/(G) = sup /q(^)(g) = sup sup inf / ( C M ) geG

geG MeM geCM

sup

inf /(CM) =

sup

inf /(CM).

D

Applying Theorem 9.7 to various families A4 c 2^ in a locally convex space X, we obtain the following more direct proofs of some geometric corollaries of Chapter 3, Section 3.3: Proof of Corollary 3.10. Let Al be the family of all closed half-spaces in X. Then / is lower semicontinuous and quasi-convex if and only if / G Q(M), and we have {CM \ M e M] = U (of (3.116)). Hence by Theorem 9.7, we obtain (3.116). Proof of Corollary 3.12. Let A4 =U. Then / is evenly quasi-convex if and only if / G Q(M), and we have {CM | M G A 1 } = V. Hence by Theorem 9.7, we obtain (3.120). Proof of Corollary 3.14. Let M = {CH \ H e H}. Then / is evenly quasicoaffine (see Chapter 1, Section 1.1) if and only if / G Q{M), and we have {CM | M £M] = H. Hence by Theorem 9.7, we obtain (3.125). Remark 9.6. Formula (9.82) permits us to define a "dual problem" to ( P ^ (of (3.1)), with the "dual set" {M G >1 | G H CM 7^ 0} c 2^, but in Section 9.2 above we have used some more natural "dual sets," in which the dual variables are functions w: X -^ R (rather than sets M c X). To this end, one could also have applied the method of [236]; namely, given a "dual set" W 2^, one obtains duality theorems for sup/(G) involving W and A by applying Theorem 9.7 above to the family of subsets A^ A of (9.27) and using (9.28) and formula (3.110). However, in Section 9.2 we have given more direct proofs, using the properties of polarities A: 2^ -> 2^.

9.4 Surrogate duality for abstract reverse convex infimization, using polarities AG : 2^ -> 2^ and AG:2^^2^X^ Let us consider the primal infimization problem (P^) of (6.1), where f: X -> R is a function on a set X and G is an abstract convex subset of X. For the polarities A': 2^ -^ 2 ^ (/ = 1 , . . . , 4), introduced in Section 9.1, the dual value (6.26) becomes

272

9. Duality for Optimization in the Framework of Abstract Convexity ^3^3 = inf inf /(CCA^)'({«;})) = inf

inf

f{x),

(9.84)

/(^).

(9.85)

/(x),

(9.86)

W;(A:)>SUP u;(G)

)3;, =

inf inf /(C(A3C)'({"^})) =

inf

inf u;(jc)=supu;(G)

;6;, - inf inf /(C(A^)'({u;})) = inf ^G

w^W

inf

weW

xeX w(x)>supw(G)

yS;^ = inf inf /(C(A^)'({u;})) = inf ^G

weW

inf

weW

xeX w{x)^w(G)

/(x).

(9.87)

We have the following generalizations of some results of Chapter 6 on unconstrained and constrained surrogate duality for reverse convex infimization (the proofs are generalizations of those of Chapter 6): Theorem 9.8. Let Xbe aset,W sup u;(G)

/(x).

^ ^

(9.88)

G e SCA(W) (with G ^^ X), and f: X ->

i n f / ( C G ) = inf

weW

inf

f(x).

xeX w{x)^w(G)

(9.89)

Using (9.88) and (5.13), or alternatively applying Theorem 6.10 in the form (6.86) to the polarity A^^ of (9.37) and using (9.44), we obtain the following corollary: Corollary 9.7. Let X be a set, W ^^^, inf/(CG)=

f e 'R^ , and G e IC(W). Then inf

inf

{w,d)eWxR supw(G)d

Using (9.46), from Theorem 6.10 we obtain the following: Corollary 9.8. Let X be a set, W ^^^, inf / ( C G ) =

inf

inf

(w,d)eWxR xeX wig)d

f e^^

and G e £}C{W). Then

f(x) = inf

weW

inf

xeX w(x)>w(g) (geG)

f(x).

(9.91)

Remark 9.7. One can also give the following alternative proof of Corollary 9.8: Since G G £:/C(W), we have CG = {X e X\ 3w e W, w(g) < w(x) (g e G)}, whence

(9.92)

9.5 Constrained surrogate duality for abstract reverse convex infimization inf/(CG)=

inf

=

f(x)=

inf

xeX -^ 3weWMg)<wix) (geG)

inf

(d,w,x)eRxWxX w(g) 0 (JC G X). Thus by (1.88), h** > h, which, together with (1.209), yields 1°. D

9.6 Duality for unconstrained abstract d.c. infimization

277

Remark 9.13. If G is a subset of X, then for f = XG "^^ have X G ( ^ ) — ^^Vxexi^M + —XG(X)} = sup w(G) (w e W), whence inf {h*(w) + -XG(^)]

= inf |sup{w;(>^) + -h(y)} + - supit;(G)l.

weW

weWly^x

'

*

Consequently, by (8.4), Theorem 9.13, and (1.86), it follows that if/z = /i**, then sup/z(G) = - inf (XGU) + -h{x)} = - inf {h*(w) + xeX

= sup

-XGM)

weW

-

weW ^

sup {w(y) + -h(y)} + - sup wiG) \ I ^yeX

'

^ J

= sup inf [h(y) + -w{y)] + sup u ; ( G ) ; wew^y^^

•

J

thus we have obtained an extension of Theorem 3.14 to the case that X* and the proper lower semicontinuous function / : X ^- i? are replaced by W c i^^^nd any / : X ^ ^ such that / = /**, or equivalently (by (9.29)), that / : X -> ^ is (W + /?)-convex.

10 Notes and Remarks

Chapter 1 1.1. A brief list of some standard books on normed linear spaces and locally convex spaces has been mentioned in Chapter 1, Remark 1.1 (a). From the literature on separation theorems let us also recall that a function O e X*\{0} is said to strongly separate C\ from Ci if sup 0(Ci) < inf 0(C2); clearly, in this case O also strictly separates C\ from C2, but in general, the converse is not true. It is well known (see, e.g., Kelley and Namioka [117, Theorem 14.3 (i)]) that if C\ and C2 are disjoint convex subsets of a locally convex space X, there exists a function O strongly separating C\ from Ci if and only if 0 ^ C2 — Ci. This happens, e.g., when C2 is convex compact, and C\ is convex and closed, with Ci H C2 = 0 (see, e.g., Kelley and Namioka [117, Corollary 14.4]). Applying this to the particular case C2 = {jc}, a singleton, where x ^ Ci, we obtain again Theorem 1.2. Theorem 1.3 was given in [219] for int C / 0, and in [220, Addendum], for closed C. For some notes on evenly convex and evenly coaffine sets, and the corresponding hulls, see, e.g., [254]. Let us mention that necessary and sufficient conditions for a set C to be evenly convex, of a nonseparational character, have been discovered only recently (see Daniilidis and Martinez-Legaz [39]). Lemma 1.5 is due to Ascoli [2]. The general concept of a quasi-support hyperplane of a set in a locally convex space has been introduced in [231, p. 15], using for it the term "support hyperplane." Lemma 1.4 and Corollary 1.1 have been given in [231, pp. 15-16]. The equivalence 1° ^ 3° of Corollary 1.2, Lemma 1.6, and Remark 1.4 have been given in [210,

280

10. Notes and Remarks

Lemmas 1.3 and 1.4] and the remark made after it. In the equivalence 2° O 3° of Corollary 1.2, H can be replaced by any set A (see [210], where a set A has been called a "support set of the ball J5" if we have 2° of Corollary 1.2, with H replaced by A). However, Proposition 1.1 has not been observed in [231] or [210]. Corollary 1.4 has been given in [19], Proposition 1.1. Lemma 1.8, in the more general formulation of Remark 1.6 (b), can be found in [231], Lemma 2.1. Lemma 1.9 and Remark 1.7 are infinite-dimensional extensions, with the same proofs, of parts of Hiriart-Urruty and Lemarechal [104, Ch. 6, Proposition 1.3.3]. Besides (1.73), (1.309), (1.324), (1.426), and (8.111), there are also other "Slater conditions" in the literature, with various applications (see, e.g., Hiriart-Urruty and Lemarechal [104]). Some books on convex analysis containing the other parts of Section 1.1 (on polars, conjugations, and subdifferentials) are mentioned in Chapter 1 (e.g., Ekeland and Temam [54], loffe and Tikhomirov [111], Barbu and Precupanu [14], Holmes [106]). For some notes on the quasi-convex hull /q and on the functional hulls /q, /eq, and /qca of (1.134)-(1.136), see [254], Chapter 4, Section 4.2. Recently, Theorem 1.6 has been considerably refined by Penot and Zalinescu, who have shown (see [178], Proposition 5.4) that it remains valid if the assumptions on / and XQ are replaced by the assumptions that / is a proper convex function and XQ Gdom / , inff(X) < /(XQ), with the convention 0.9/(JCO) = A^(dom f;xo). A short proof of Theorem 1.8 using only separation arguments was given by Borwein and Zhuang [22]. For minimax theorems, see, e.g., the survey of Simons [206]. 1.2. For the theory of abstract convexity and some of its applications to optimization, see the monographs [254] and Pallaschke and Rolewicz [169], Rubinov [193]. The polarities A = A'^^: 2^ -> 2^*^^^^ (/ = 1,2,3,4) of (1.154), (1.160), (1.166) and (1.182) have been introduced and studied in [259]. Lemmas 1.10, 1.11, 1.12, Corollaries 1.5, 1.6, and Lemma 1.13, have been proved in [259]. The polarities A i': 2^ -^ 2(^*\{o^>^^ (/ = l,2,3)of(1.189), (1.191), and (1.193) and formulas (1.190), (1.192), (1.194), as well as the polarities A^' (/ = 1, 2) of (1.196), (1.199), and formulas (1.197), (1.198), (1.200), and (1.201), can be found in [256], Sections 2 and 3. Theorem 1.10 on the "axiomatic" characterization of Fenchel-Moreau conjugates, given in [237], has been the starting point of further research on various concepts of conjugations, obtained by replacing the "second condition" (1.204) with other conditions. Thus in [147], there have been introduced and studied mappings c:~R^ ^W satisfying (1.203) and (/ V df = fA-d

(f e ^ ^ , d e R);

(10.1)

such mappings have been called in [147] "v-dualities." A unified approach to conjugations and v-dualities c: R^ ^^ R^ was developed in [154], [156], namely, that of "dualities associated with a binary operation * on /?" (see (10.186), (10.187) below); for * = + and * = v, these reduce to conjugations and v-dualities, respectively.

10. Notes and Remarks

281

The pseudoconjugates (1.220) and the semiconjugates (1.221) have been introduced and studied in [224] and, respectively, [233]; actually, the semiconjugate has been introduced in order to obtain a conjugate whose biconjugate is the lower semicontinuous quasi-convex hull (formula (1.222)). For some other conjugates of functions f:X->R, useful for duality in convex and quasi-convex optimization, and for some generaUzations, see [234], [243], [254], [260], Atteia [5], Atteia and El Qortobi [6], Crouzeix [34], [35], Elster and Gopfert [59], Elster and Nehse [60], Elster and Wolf [62]-[64], Martinez-Legaz [138]-[141], Passy and Prisman [171], [172], Penot [176], Penot and Voile [177], Voile [287], [288]. The concept of conjugation has been extended to functions with values in the "canonical enlargement" of conditionally complete lattice ordered groups, in [152], [155]. Let us recall briefly this extension, since we shall use it in the Notes and Remarks to Chapter 8. A lattice ordered group A = (A, h(u(x))

(x e domf

z > u(x)).

(10.62)

For Corollary 1.12 (a) see, e.g., Rockafellar [183]. For Corollary 1.12 (b), see, e.g., Rockafellar [183], Ekeland and Temam [54]. For the infimization problem (P) of (1.261), where G is a subset of a locally convex space X and f:X -> i? is a function, another perturbation theory and a corresponding theory of Lagrangian duality have been introduced by Kurcyusz ([125], [126]; see also Dolecki and Kurcyusz [48]), generalizing the "vertical perturbation" (1.442), (1.443). Namely, (1.261) is embedded into a family of perturbed problems (P^)

y^(z):=: inf f(x) xeQiz)

(z e Z).

(10.63)

294

10. Notes and Remarks

where Z is a locally convex space and Q: Z ^- 2^ a multifunction, called 2i perturbation multifunction or constraints multifunction, such that G = Q(0).

(10.64)

The associated Lagrangian function L^: X x Z* ^^ R is defined by L^(jc, vl/) := f(x) + -

sup ^(z)

(jc G X, vl/ E Z*),

(10.65)

zeQ~Hx)

where ^~Ms the inverse multifunction to Q, defined by Q-\x)

:={zeZ\x

e Q(z)},

(10.66)

and the dual objective function X^ and dual value P^ associated with L^ are defined in the usual way. This theory encompasses the important particular cases of Lagrangian duality for linear systems and convex systems (X, Z, u). Indeed ([222], Remark 4.1), if (X, Z, u) is a Hnear system and T is a convex subset of Z, or if (Z, Z, w) is a convex system and 7 = {z G Z| z < 0}, then for the multifunction ^ : Z - > 2^ defined by Q(z) := u-\z

-^T) = {xeX\

u(x) ez + T}

(z e Z)

(10.67)

{x e X).

(10.68)

we have Q-\x)

= {zeZ\

u{x) ez + T} = u(x) -T

Hence by (10.65) and (10.68), we obtain ([240], Theorem 3.3) L^{x, vl/) = f(x) + -

sup

^(z) = f(x) + - sup ^{u{x) - t)

zeu(x)-T

= fix) - ^(u(x)) f mf^(T)

teT

(x eX,^

e Z*),

(10.69)

which is nothing other than (1.416) for /z = xr; it is also the Lagrangian corresponding to formula (1.433). Similar remarks can be also made for various surrogate dual problems to problem (P) of (1.261). Indeed, for the perturbed problems (P^) of (10.63), with ^ : Z -> 2^ satisfying (10.64), one can define [240] the surrogate Lagrangians Lf^Jx, vl/) := fix) + XQi{zezmz)>o})ix) (X G Z, vl/ G Z*), L^ix, vi/) := fix) + xmzezmz)=o})ix)

ix e X,^

e Z*).

(10.70) (10.71)

Then by [240], Theorem 3.3, for ^ : Z -> 2^ defined by (10.67), one obtains Lg^ = Lsurr Of (1.456), and L J = L^ of (1.468). In [228] it has been proved that the perturbational Lagrangian duality theories of Rockafellar and Kurcyusz are, in a certain sense, equivalent, and it has been shown how to pass from one to the other. In particular, given an arbitrary multifunction

10. Notes and Remarks

295

^ : Z ^^ 2^ satisfying (10.64), the perturbation function PQ: X x Z ^^ R defined by Pnix, z) := fix) + Xf2(,)(;c)

{xeX,z€Z)

(10.72)

satisfies p^ix, 0) = f{x) + xcix) (x e X), and for v(z) and L{x, * ) of (1.389) and (1.396), respectively, we have viz) = inf {fix) + xniz)ix)} = v"(z)

iz e Z),

(10.73)

xeX

Lix, * ) - inf [fix) + XQ(z)ix) - *(z)l = L"(x, * ) (x e X, * e Z*). (10.74) Also, by [240], Theorem 3.1, for p ^ : X x Z ^ ^ of (10.72) and the surrogate Lagrangians Lsurr of (1.449) and Lj^ of (1.466) w e h a v e Lsurr — L" , L, = L'J (of (10.70) and (10.71), respectively). Therefore, instead of the Lagrangian (10.65) of Kurcyusz we consider here only the Lagrangian (1.396). In connection with strong duality for PJJ: of (1.467), i.e., with the equaUty inf fix) = max xeX u(x)eT

inf

vi/ez*

f(x),

(10.75)

xeX ^(uix))e^{T)

let us mention that following Rolewicz ([188], [189]), if (X, Z, u) is a linear system and (10.75) holds for all singletons T = {ZQ}, where zo ^ w(^), that is, if for each Zo ^ w(Z) there exists ^o ^ ^* such that inf fix)=

xeX u{x)=zo

inf

xeX ^o(u(x))=^o(zo)

fix),

(10.76)

then (X, Z, M, / ) is said to satisfy the Pontryagin maximum principle, or, briefly, the PMR For systems described by differential equations, the existence of ^o satisfying (10.76) leads to the classical formulation of Pontryagin's maximum principle (see Rolewicz [192]). This concept has been studied by Rolewicz ([188], [189], [191]), who has also extended it ([190], [192]) to the case of "non-one-point target sets," as follows: if T is any closed convex subset of Z, PMP holds if for each ZQ G M(X) there exists VJ/Q 6 Z* such that inf

xeX u{x)ezo+T

fix)=

inf

xeX vi/o(M(jc))=vi/o(2o)4-infvi/o(r)

fix).

(10.77)

Some dual characterizations of linear systems satisfying the PMP have been given in [216], [225], [232]. Fo£example ([225], Proposition 2.1), when (X, Z, u) is a linear system and / : Z -> /? is a continuous convex function, in order that (X, Z, w, / ) satisfy the PMP it is sufficient, and if u is one-to-one, it is also necessary, that a/(jco) n w*(Z*) 7^ 0

(jco e X).

(10.78)

Some new abstract PMPs, corresponding to the other formulas of Lagrangian and surrogate duality have been introduced in [232], as follows: If T is any subset of Z

296

10. Notes and Remarks

with M(X) n r 7^ 0, we say that (Z, Z, w, T, / ) satisfies the PMPi at ZQ e u{X) if there exists ^o ^ Z* such that inf

xeX uix)ezo-hT

fix) = inf {fix) - ^ouix) + inf VI/QCZO + T)}; xeX

(10.79)

the PMP2 at zo ^ " ( ^ ) if there exists ^0 ^ 2* such that inf

fix) =

w(jc)6Zo+7'

inf

/(jc);

(10.80)

^o(uix))>infyl'o(zo+T)

the PMP3 at zo ^ " ( ^ ) if there exists ^0 ^ 2* such that inf

jceX u(x)ezo+T

fix)=

inf

xeX ^o(u(x))e%(zo-^T)

/(jc);

(10.81)

the PMP„ if it satisfies the PMP„ at each zo € M(X). Some dual characterizations of such PMPs, using subdifferentials, have been given in [232]. The following generahzation of problem (P) of (1.413), with applications in the calculus of variations, has been also studied in the literature: Let iX, Z, u) be a linear system and J. X x Z -^ R a. function, and let the primal problem be iP)

a= inf Jix,uix)).

(10.82)

xeX

Defining the perturbation function p: X x Z -> Rby pix, z) := Jix, uix) -z) ix eX.ze Z), (10.83) a duality theory has been developed for problem (P) of (10.82) (see, e.g., Ekeland and Temam [54] and the references therein). In the particular case where y(jc, uix)) := fix) + hiuix))

ix eX.ze

Z),

(10.84)

where f: X ^^ R and h\ Z ^^ R axQ two convex functions, (10.82) and (10.83) reduce, respectively, to (1.413) and (1.415). However, here we did not consider generalizations of this type (although they have been developed also for nonconvex infimization, see, e.g., Toland [280], Auchmuty [10], Gao [76]). In [241] and [242] there have been developed a general theory of surrogate dual optimization problems, and, respectively, a general theory of dual optimization problems, encompassing the known dual problems as particular cases. Let us briefly mention some of the concepts introduced and studied in these papers. Considering the primal infimization problem (P) of (1.386), embedded into a family of perturbed optimization problems (1.389) with the aid of a perturbation p: X X Z -^ R satisfying (10.59) for some fixed zo ^ Z, one defines [241] a dual problem (Z)^^) to (1.386) by (Dpa)

Ppa = s^P^pQ(W). kp^iM^) = mfpiQ^x,zo),^)

(10.85) (^ e W),

(10.86)

10. Notes and Remarks

297

where W is a family of finite functions w: X -^ R and ^(x,zo),^ c. X x Z (\j/ ^ \Y). Observing that (Dp^) is not a surrogate dual problem to (P) of (1.261) (or equivalently, of problem (1.386) with 0 = f+jc)^ in the sensej 10.48), (10.49), but it induces a natural surrogate dual problem (DQ), with value ^Q = fip^, to the optimization problem (P)

5 = inf/7((X,zo)),

(10.87)

with objective function p and constraint set (X, zo) = {(x, zo)\x e X}, both defined in the "extended space" X x Z, and with value a equal to the value a = inf/(G) (by (10.59)), (DpQ) has been called in [241] SL perturbational extended surrogate dual (PES-dual) problem to (P). Of special interest are the so-called decomposed PES-dualproblems to (P), i.e., those in which Z is a locally convex space and ^ix,zo),^ = Xx ^l^^^^

(vl/ e Z*),

(10.88)

where Q^: {zo} x Z* -> 2^ (a multifunction), with zo of (10.59); these encompass, as £articular cases, e.g., the dual problems (Dsurr) of (1.444), (1.445), where zo = 0, ^(x,o),vi> = Xx{z e Z\ vl/(z) > 0} (so ^^^j ^ = {z e Z\ vl/(z) > 0}) , (D,) of (1.461), (1.462), and (Dyr) of (1.463), (1.464).' Another useful class of PES-dual problems is that of "perturbational conjugate dual problems" to (P), obtained as follows: Let W be any family of finite functions w: Z ^^ R, and let us consider any concept of conjugation h ^^ h^ (d e R)by which to each function h: Z ^^ R and each d e R there corresponds a "/x-J-conjugate function" h^: W -^ R. For p. X X Z ^^ R and ZQ ^ Z 2iS above, one defines the (p/i)-dualproblem to (P) as the optimization problem (Dp^)

Pp^=supXp^{W),

(10.89)

V ( ^ ) = - 2^ is defined by ^o,vi. = So,v,

(vl/ e Z*).

(10.94)

Without assuming a perturbation of (P) = (P(^,/) of (1.261), for any dual constraint set W = W^'-^ and any coupling function r = TQ: XxW ^^ R, with values TG(X, W) e R not depending on / , the (Wr)-dualproblem to (P) is defined [242] as the supremization problem (D) = ( D ^ / )

p = ^ ^ / = supXiW),

X(w) = X^{{w) = inf {fix) + TG(X, W)}

(10.95) (W G W).

(10.96)

Thus (D) is a Lagrangian dual problem to (P) in the sense mentioned in Chapter 1, with "penalty term" TG{X, W); here we write TG by abuse of notation, since TG is defined on X x W^'^, but we use this notation in order to emphasize that the values TG{X,W) e R are independent of / . Assuming, for simplicity, that W = W^'^ c R^, and that G Hdom w; 7^ 0, or equivalently, inf u;(G) < +oc (w e W), it turns out ([242], Theorem 2.1) that for r = IG : X x W ^ ^ defined by TG{X, W) := -w{x) f inf w;(G)

(x eX,w

e W),

(10.97)

the Lagrangian dual (1.277), (1.278) to (P), with X* replaced by any W c P ^ (i.e., as in (10.20), when W c P^), coincides with the (lyr)-dual problem to (P). Furthermore ([242], Theorem 2.3)^if W = W^^^ is any set and QG,W ^ X (W e W), then fori = TG: X xW -^'R defined by TG(X, W) := XnaJ^)

(xeX^we

W),

(10.98)

the surrogate dual problem (10.48), (10.49) to (P) coincides with the (Wr)-dual problem to (P). A perturbation function p: XxZ -^ P for problem (1.261), or equivalently, for problem (1.386) with 0 = Z + X G . satisfying (10.59) for some zo ^ ^, is said to be objective function separated [242], if there exists a coupling function TIG'- XXZ -> R with values nG{x,z) € R not depending on 0 such that p{x, z) = PG,ct>{x, z) = (p(x) 4- 7TG(X,

Z)

(X G

X, Z

G Z).

Note that by (10.99) and (10.59), we have 0(x) + 7TG(X, ZO) = (t){x) -\- XG{X)

(X G X),

(10.99)

10. Notes and Remarks

299

and hence by a remark of Moreau ([164], p. 116), Ttcix, zo) = XG(X)

(X e X, 0(jc) e R).

(10.100)

Thus if (pix) e R, then Ttdx, zo) is either 0 or +oo, but ndx, z) may also have other values, for z ^ Zo\ the case in which all values 7TG(X, Z) are either 0 or +oo is also of interest. As has been shown in [242], there are some close connections between the dual problems (10.95), (10.96) associated with TG, and the Lagrangian dual problems associated with TTG of (10.99). Some axiomatic characterizations of perturbational Lagrangian dual objective functions (1.392), and perturbational surrogate dual objective functions (e.g., (1.445), (1.462), (1.464)), and, more generally, of dual problems with respect to objective-function separated perturbations, in the spirit of Theorems 10.8 and 10.1 above, have been given in [151]. Some characterizations of the associated Lagrangian functions have been also given in [151]. In the paper [242], going in the opposite direction, it has been shown that each perturbational dual problem "can be derived from" a suitable unperturbational problem via a "scheme" of "formal replacements." Rather than giving the technical details of the general case, let us mention here that, for example, given the unperturbational Lagrangian dual problem (1.277), (1.278) to (P) of (1.261), and any pair (X, /?), with p: X x Z ^ R, satisfying (1.387), by replacing formally, in this unperturbational dual problem O e X*,jc e X , G , and f(x) by (0, ^) e (X X Zy = X* X Z* (see (1.28)), (JC, Z) e X X Z, (X, 0), and p{x, z) respectively, one obtains P=

sup

inf

{p(x, z) - (0, vl/)(jc, z) + inf (0, ^)(X, 0)}

(0,vI/)€(XxZ)* ix,z)eXxZ

= sup

inf

[p{x, z) - ^(z)},

(10.101)

\lteZ* (x,z)eXxZ

i.e., the perturbational Lagrangian dual problem (1.391), (1.392). Furthermore, given the unperturbational surrogate dual problem (1.376), (1.379) to (P) of (1.261), and any pair (X, p) satisfying (1.387), making the same replacements also in the definition of the surrogate constraint sets, e.g., in QG, of (1.378), one obtains the new surrogate constraint sets ^(x,o),(o,vi/) = {(Jc, z)eXxZ\

v|/(z) = 0} c X x Z

(^ e Z*),

(10.102)

i.e., those occurring in (1.464), and the corresponding perturbational surrogate dual problem (1.463), (1.464), since /3 =

sup

inf

p(x,z)=

sup p(x,z).

(10.103)

^(z)=0

In [248], [252], instead of using the scheme of formal replacements, the "perturbational dual problem corresponding to an unperturbational dual problem" has been redefined by means of explicit formulas, which has permitted to obtain further

300

10. Notes and Remarks

results. This has been achieved by distinguishing between a "problem" and its "instances," much in the way as is done in combinatorial optimization (following, e.g., Garey and Johnson [78], Papadimitriou and Steiglitz [170]), which permits a deeper understanding of the parameters of the dual optimization problems. Let us mention that the idea of regarding and studying the objective function of a dual optimization problem as a function not only of the dual variables, but also of the primal parameters (the primal constraint set and the primal objective function), is useful not only for the above-mentioned "scheme" of "formal replacements," but also for other applications. For example, in [247], considering the instances of {(P), (D)} in which the constraint G of (P) is a singleton, there has been introduced and studied the concept of the "subdifferential of a function / : X ^- 7? at a point XQ e X with respect to a primal-dual pair of optimization problems {(P), (^)}"; in particular, for the unperturbational Lagrangian problem (D) this becomes Balder's subdifferential [13], generalizing the usual subdifferential, while for the unperturbational surrogate dual problem (D) it becomes the "surrogate subdifferential" of [234], generalizing the "quasi-subdifferential" of Greenberg and Pierskalla [95] and Zabotin, Korablev, and Khabibullin [296]. Among other generalizations of convex duality theory let us mention that Rubinshtein [203] (see also [202]) has considered the primal problem (P)

f(x) = sup t

(x e X),

(10.104)

te(a,b) xeGi

where {Gt}te(a,b), —^^ Sci < b < +cx), is a family of subsets of a given set X with certain properties, and has defined a dual problem to it. Also, he has given several applications, e.g., to generalizations of linear optimization, quasi-convex optimization, and best approximation. Burkard, Hamacher, and Tind [28] have shown that duality in mathematical programming can be treated as a purely order-theoretic concept, as follows: Let L = (L,