Quasidifferential Calculus (Translations Series in Mathematics and Engineering)

TRANSLATIONS SERIES IN MATHEMATICS AND ENGINEERING

A.V. Balakrishnan

TRANSLATIONS SERIES IN MATHEMATICS AND ENGINEERING

A.V. Balakrishnan, A.A. Dorodnitsyn, and J.L. Lions, eds.

1983, 267 pp. ISBN 0-911575-00-6

Vistas in Applied Mathematics: Numerical Analysis, Atmospheric

-3;

D-0

Spectral Theory of Random Fields

M.I. Yadrenko

Sciences, Immunology. Collected papers

Mathematical Models in Immunology

dedicated to G.I. Marchuk on his 60th birthday.

1983, 378 pp. ISBN 0-911575-01-4

1986, 396 pp. ISBN 0-911575-38-3

A.A. Borovkov, ed.

V.F. Kolchin

Advances in Probability Theory: Limit Theorems and Related Problems

Random Mappings

G.I. Marchuk

1984, 392 pp. ISBN 0-911575-03-0

1986, 224 pp. ISBN 0-911575-16-2

.60

L. Telksnys, ed.

V.A. Dubovitskij

Z01!

The Ulam Problem of Optimal Motion of Line Segments V-0

1985, 128 pp. ISBN 0-911575-04-9

Detection of Changes in Random Processes 1986, 240 pp. ISBN 0-911575-20-0

V.F. Dem'yanov, and A.M. Rubinov

Statistics and Control of Stochastic

Quasidifferential Calculus

mar

O;5

N.V. Krylov, R.S. Liptser, and A.A. Novikov, eds.

1986, 301 pp. ISBN 0-911575-35-9

Processes 1985, 521 pp. ISBN 0-911575-18-9

V.P. Chistyakov, B.A. Sevast'yanov, and V.K. Zakharov

Probability Theory for Engineers Yu.G. Evtushenko

Numerical Optimization Techniques

R.F. Gabasov, and F.M. Kirillova

°o°

1985, 575 pp. ISBN 0-911575-07-3

1986, 175 pp. ISBN 0-911575-13-8

Methods of Optimization V.F. Dem'yanov, and L.V. Vasil'ev 1985, 472 pp. ISBN 0-911575-09-X

1986, approx. 350 pp. ISBN 0-911575-02-2 acv

Nondifferentiable Optimization

B.T. Polyak

Introduction to Optimization A.A. Borovkov, ed.

Advances in Probability Theory:

1986, approx. 450 pp. ISBN 0-911575-14-6

Limit Theorems for Sums of Random Variables

I.A. Boguslavskij

1985, 313 pp. ISBN 0-911575-17-0

Filtering and Control 1986, approx. 400 pp. ISBN 0-911575-21-9

Vladimir F. Dem'yanov Alexander M. Rubinov

QUASIDIFFERENTIAL CALCULUS

timization Software, Inc. Publications Division, New York

Authors V.F. Dem'yanov Applied Mathematics Department Leningrad State University Leningrad, Staryj Peterhof USSR 198904 A.M. Rubinov Institute for Socio-Economic Problems USSR Academy of Sciences ul. Voinova 50-a Leningrad USSR 198015

Library of Congress Cataloging-in-Publication Data Dem'ianov, V.G. (Vladimir Fedorovich), 1938Quasidifferential calculus.

(Translations series in mathematics and engineering) Bibliography: p. Includes index. 1. Calculus, Differential. I. Rubinov, Aleksandr Moiseevich. II. Title. III. Series. 515.3'3 86-5422 QA304.D45 1986 ISBN 0-911575-35-9

© 1986 by Optimization Software, Inc., Publications Division, 4 Park Avenue, New York, New York 10016. All rights reserved. Published in 1986. Printed in the United States of America.

ABOUT THE AUTHORS

Vladimir Fedorovich

Dem'yanov

(born

1938) received his Ph.D. in Mathematics from the Leningrad State University in 1964

and his D.Sc. degree from the Computing Center of the USSR Academy of Sciences in

1972. He was appointed Professor at the Applied Mathematics Department of the Leningrad State University in 1970. Alexander Moiseevich Rubinov (born 1940) graduated from the Leningrad State University in 1962. He received his Ph.D. in Mathe-

matics from the Novosibirsk Institute of Mathematical Sciences in 1965. Since 1976

he has been with the Leningrad Research Institute for Socio-Economic Problems at the USSR Academy of Sciences.

TABLE OF CONTENTS

Preface

ix

Chapter 1 Preliminaries

1

Chapter 2 Elements of the Theory of Multivalued Mappings

6

Chapter 3 Directional Derivatives

19

Chapter 4 Convex Functions

37

Chapter 5 Sublinear Functions

46

Chapter 6 The Space of Convex Sets

.

.

.

.

65

78

Chapter 8 The Clarke Derivatives

91

00

Chapter 7 Upper Semicontinuously Directionally Differentiable Functions . . . . .

T--4

Chapter 9 Upper Convex and Lower Concave Approximations . . . . . . .

.

104

vii

Table of Contents

viii

Chapter 10 Quasidifferentiable Functions

112

Chapter 11 Examples

122

Chapter 13 A Relation Between the Clarke Subdifferential and the Quasi differe ntial

143

0

Chapter 14 Implicit and Inverse Function Theorems

00

128

0

Chapter 12 Quasidifferentiability of a Composition

155

176

Chapter 16 Necessary and Sufficient Conditions for an Extremum

188

Chapter 17 Rate and Directions of Steepest Descent and Ascent

203

Chapter 18 Saddle Points of Quasidifferentiable Functions

229

Chapter 19 Star-Shaped Sets and Their Applications in Nonsmooth Optimization

236

Chapter 20 Approximate Quasidifferentiability

254

Bibliographic Notes

.

.

.

.

.

.

.

.

.

265 \-0

Z4..

Chapter 15 Cones of Admissible Directions and a Nondegeneracy Condition

269

Index

283

Transliteration Table (Russian-English)

289

000

00

References

PREFACE

W-4

0'+

When studying optimization problems, the mathematician has to make use of certain tools. In the smooth case, for example, one such tool is the derivative; in the nonsmooth case, the derivative concept is naturally replaced by the idea of a directional derivative or one of its generalizations. But there is a catch: any such generalization must be consistent with simpler cases, e.g., if the function is directionally differentiable, then the generalized directional derivative should be the same as the ordinary directional derivative. Otherwise the generalization-even though aesthetically elegant may have little or no relevance to the optimization problem. BCD

'ti

Suppose that a function f is directionally differentiable on some set S in the finite-dimensional space R'2 , i.e. for any X E S and g E R'2 the following limit exists: d f (x)

dg

= 1 im

III

III

f(g)

f(x+ctg)-f(x)

a

a-4+0

Here fx(g) is called the directional derivative of f at x in the direction g E Rn. Let x be fixed. The function

A (alg) = fW + a

dg

is called the first-order approximation of f in a neighborhood of the point x. Then

boo

f(x+xg) = f(x) + a °dg+ o(a) where o(a) = o(a,g, x) is such that °

a a

fi(a,g) + o(a) ,

- 0.

a-4+0

The use of the directional derivative allows us to study some very important properties of the function, e.g., (a) o check necessary (and sometimes even sufficient) conditions for optimality; (b) to find directions of steepest descent or ascent; (c) to construct numerical algorithms.

ix

Preface

x

Suppose that f(g) is a continuous function of g for every fixed x (this

for example, f is a Lipschitzian). In this case, f(g) is also positively homogeneous. But every positively homogeneous continuous func-

00,0

occurs if,

tion can be approximated to within any prescribed accuracy by the differ-

".OW

h(g) = ma x (v,g)

0

'b>

ence of two positively homogeneous convex functions. We know that to study the first-order properties of a function f at a point x it is sufficient to study a good approximation of its directional derivative at the point x. As has been noted, such an approximation can be provided by the difference of two positively homogeneous convex functions. But every positively homogeneous convex function h(g) can be expressed in the form ,

VEA

where A E Rn is a convex compact set. Therefore, the difference of two positively homogeneous functions can be written

ma x (v,g) - m a x (v,g) vEB vEA or, equivalently, m a x (v,g) + min (w,g) vEA WE[-B]

.

anon

Here A and B are convex compact sets in Rn. It should now be clear why it is important to study the class of functions whose directional derivatives can be represented in the form df (x)

dg E

-

min (w,g)

max (v,g) + E

v

,

(1)

(x)

Rn and df(x) c Rn are convex compact sets. 4-.

A function f is said to be quasidifferentiable at a point x if it is directionally differentiable at x and there exist convex compact sets !2f(x) and df(x) such that the directional derivative df(x)/ag of the function f can be is called a represented as in (1). The pair of sets Df(x) = quasidifferential of f at x. It is clear that a function does not have a unique quasidifferential.

We emphasize the fact that to examine the first-order properties of the function, it is sufficient (at least in the case of directionally differentiable Lipschitz functions) to study only quasidifferentiable functions.

If a function f is Lipschitz but not directionally differentiable, we can bbl

consider the upper Dini derivative dD f (x,g) = 1 1 m sup a-4+0

[ f (x+ag) - f (x) ]

Preface

xi

or the lower Dini derivative

a f(x,g)

-

a [f(x+ag) -.f(x)]

These derivatives are consistent with the directional derivative: if f is a directionally differentiable Lipschitzian, then

dDf(x,g) _ d f(x,g)

= f(g).

But, even more importantly, the functions dD f (x,g) and dD f (x,g) are continuous and positively homogeneous (as functions of g). Thus quasi-

differentiable functions can also be used to study the Dini derivatives.

This again demonstrates the importance of studying quasidifferentiable functions.

The concept of quasidifferentiability was introduced in 1979. Since then a

whole theory of quasidifferential calculus has been developed, and many problems traditionally related to classical calculus have been stated and solved for quasidifferentiable functions -formulas for computing quasidifferentials, a chain rule, inverse and implicit function theorems, necesran

sary and sufficient conditions for an extremum, quasidifferentiable mappings, etc.

This book presents the state-of-the-art of quasidifferential calculus. It was written while one of the authors was a member of the International Institute for Applied Systems Analysis in Laxenburg, Austria. The authors express their gratitude to Professor A. Wierzbicki and Professor A.B. Kurzhanski, the former and present Chairmen of the System and Decision Sciences Program at IIASA, for their support, and to thank Helen Gaskin for her careful editing and Elfriede Herbst and Edith Gruber for typing and retyping the manuscript.

Chapter 1 PRELIMINARIES

We recall here some definitions and results which will be used throughout this book. Proofs can be found in any monograph on convex r-,

analysis (see, e.g., [44], [79], [83], [1191, [1271, [1361).

0".

1.1. In what follows, Rn is the n-dimensional Euclidean space. An element x = (X(09 ..., x(n)) of Rn is called a vector (or sometimes a direction or a point, depending on the role it is to play). If x = (x(1) ,...,x(n)) E Rn , y = (y(1)...,y(n)) E Rn, then (x,y) = En=1 x(')y(i) is called the scalar product of vectors x and y. Let X E Rn . Then the quantity

II x II =

( x, x) is called the Euclidean norm

of X.

Vectors x1,...,x2 are said to be linearly independent if the equality r

ITV

(1.1)

LrkXk = 0

k=1

holds if and only if ak = 0 V k E 1:r. Otherwise they are called linearly dependent.

In (1.1), 0 = (0,...,0) E Rn is the zero element of Rn (in the sequel the dimensionality of 0 is usually clear from the context). Recall that in Rn any n+ l vectors are linearly dependent. By Ve (x) c Rn we denote an open ball centered at a point X E Rn with radius . ? 0

Ve(x) =

E R"RnI IIv x II <e} .

By B we denote the unit closed ball centered at the origin:

B=

E R"RnI IIvII _ 0 centered at X E Rn :

B. (x) _{ v E Rn I

11 v -x 11

:5.61 ; 1

Chapter 1

2

CJ`

and by BE we denote a closed ball of radius .6 >_ 0 centered at the origin. is clear that

It

= x+BE = x+eB.

BE(x)

Let A be a set in the space Rn. Then by cl A we denote the closure of A; by int A we denote the set of interior points of the set A. It is clear that BE (x) = cl VV (x). The empty set is denoted by 0. y"'

A set A c Rn is called compact if it is closed and bounded. It is easy to check that

m a x (v,g) =

V v E Rn

1101

gEB

min (v,g) _ -IIvII

VvE Rn

gEB

1.2. A set d2 c Rn is called convex if the inclusions x1 E £2, x2 E £2 imply that

x a = ax1 + (1-a)x2 E i 2

V a E [0,1 ]

.

A set £2 c Rn is called strictly convex if the relations x1 E

x2 E

x1 # x2 imply that

xa = ax1 + (1-a)x2 E int 12

V a E (0,1)

.

For D1 c Rn, £2 c Rn the set £2 =

£1 +

£2

= {x = x1 +x2 I X1 E £21, x2 E D2 }

is called the sum of the sets d21 and £22. If £2 c Rn , 2 E R1, then the set

,(21-AD ={x=Ay iyE,2} is called the product of the set d2 and the number 2. Let £2 c Rn. The set r

co

r

x = I akxk I Xk E 12, ak ? 0, 1 ak = 1, r is a natural number k=1

k=1

is called the convex hull of the set £2. If r

ai =

xi E Rn, ai ? 0 V i E 1: r; i=1

1

Preliminaries

3

then the vector x = Ei = i a1x1 is called a convex combination of the vectors xl ,...,Xr.

Theorem 1.1 (The Caratheodory Theorem). Any vector x E co Sl can be represented as a convex combination of not more than n+l vectors from the set Sl.

Note that the convex hull of a compact set is a compact set. If Sl is a closed bounded set, then

ma x (v, x) = max X E 12

If £21 ,. .. ,is

(v, x)

V v E Rn

XE CO12

are closed bounded sets in Rn and £2 =121 + ... +

,mss ,

then

s

ma x (v,g) _ Y ma x (v,g) . V E12

i=1

V EQt

A

Theorem 1.2 (The Separation Theorem). Let £2 c Rn be a closed convex set and let x0 4 £2. Then there exists a vector go E Rn, Iigo II = 1, and a number a > 0 such that (x-x0 ,g0) < -a V x E £2 .

Corollary 1.1. If £2 c Rn is a closed convex set and x0 is a boundary point of the set £2, then there exists a vector go E Rn , II go II = 1, such that

(x x0 ,g0) < 0

V X E D.

1.3. A set T c Rn is called a cone if the inclusion x E T implies v = Ax c

£2 V2>0.

If T is a cone, then the set

F* = {w E Rn I (w,v) >_ 0 V V E T} is called the cone conjugate to F.

Proposition 1.1. If F c Rn is a cone, then F* c Rn is a closed convex cone.

Theorem 1.3. Let F c Rn be a closed convex cone and let 12 c Rn be a con-

vex compact set. For the cone F and the set 12 not to

intersect, i.e.,

F r, d2 = 0, it is necessary and sufficient that there exist a vector w0 E F such that in a x (w0, x) < 0. XE12

Chapter 1

4

Corollary 1.2. Let T c Rn be a closed convex cone and let 12 c Rn be a convex compact set. In order that int T n d2 = m, T n int 12 = m, it is necessary and sufficient that there exist a vector w0 E F* such that max (w0,x)

A function q is called lower semicontinuous (l.s.c.)

at

a point

x,

if the

inequality lim q(xk) > q(x) holds for any sequence {xk } such that Xk -4 x.

Proposition 2.4. Let a mapping a: X -4 IIc(Rm) be bounded in some neighbor-

hood of a point x E X. Then a is l.s.c. at x if and only if the function q,e is I.s.c, at x for every e E Rm .

P r o o f. (a) Let a mapping a be l.s.c. at x. Take any e E Rm and find y E a(x) such that max

q,e (x) =

y'E a(x)

(.e',y') = (.e,y)

.

The lower semicontinuity of the mapping a implies the existence of a sequence {Yk } such that Yk E a(xk ), Yk -4 y. Since q,e (xk) > V,yk ), we have

lim q1(x)

"M (eyk )

_ (y) = qe (x)

i.e., the function q is l.s.c.

(b) Let the function q,e be l.s.c. at x for any e E Rm. We must show that the mapping a is also l.s.c. at x. Assume the opposite. In this case it is not hard to see that there exist an e > 0, an element y E a(x) and a sequence {xk } such that xk -4 x, a(xk) n (y+Be) = 0. Applying the separation theorem, we shall find, for every k, an element ek such that

max (.ek,v)
0 there exist a 8 > 0 and an a0 > 0 such that

= lf(x+ag) - f(x) - af'(x,g)I < ea .-.

log (a) I

V a E [O,ao], V g E B5(q) .

(3.18)

b'`

cps

P r o o f. (a) Let f be uniformly differentiable, where e > 0 and q E Rn is given. Choose any o > 0 and put r = IIq II + J. It follows from (3.16) that

Directional Derivatives

29

there exists an ao > 0 such that inequality (3.18) holds for all a E (O,ao ] and g E Br. Since B5(q) c Br, this inequality is also valid for g E B5(q). (b) Let property A be satisfied. It then follows from the definition that for every q E S = {x I fix ii = 1 } there exist numbers ao (q) and 8(q) such that < ae V a E (0 900 (q) 19 g E V5(q) (q) ,

log (a) I

where V5(q) is an open ball of radius 5 centered at the point q. The balls V5(q) (q), form an open covering of the sphere S. Since this sphere is compact, there exists a finite covering V5(gI) (q, ),...,V5(q ) (qp ). Let ao be the smallest P

of the numbers ao (q, ),...,a0 (qq ), and let g E S. Then g E Vs(q t) (q) for some i, and therefore l o8 (a)I < ae V a E (O,ao (q j)]

Clearly, this inequality also holds for a E (O,ao ].

.

0 f/1

Remark 3.1. In fact, property A means that f is locally uniformly differ,RS

entiable. Hence, it follows from Proposition 3.3 that locally uniform differentiability is equivalent to "global" differentiability. Unfortunately, in infinite-dimensional spaces this property does not hold and therefore it is necesF-+

V.4

sary to use locally uniform differentiability (see, e.g., [39]).

Next we introduce one important class of uniformly directionally differC13

entiable functions.

qty

Proposition 3.4. Let a function f be directionally differentiable at a point x C X and Lipschitz with a constant L in some neighborhood of this point. Then (a) the derivative fX is Lipschitz (in g) with the same constant L; (b) the function f is uniformly directionally differentiable at the point x.

P r o o f. Consider the absolute value of the difference

l a If(x+a4) -.f(x)] - a V(x+ag) -f(x)] I = l a [f(x+a9) -.f(x+a8)]

(3.19)

0

Assume that a is small enough to ensure that elements ag and aq belong to a neighborhood wheref satisfies a Lipschitz condition. Then

[f(x+aq) - f(x+ag)I 0

Chapter 3

34

there exists a 5 > 0 such that df(x+8yg,y)

d

,

< . V aE (0,05), V

E Y.

(DD

Therefore

joy(a)I

4~

The compactness of R(x) and the continuity of the partial derivative dfldx '0`

imply that a(x) is a compact set. Let d0(x) denote the convex hull of a(x)

(x) = co a(x)

.

We shall call this set the subdifferential of the maximum function 0 at the point x. This set is convex and compact (because it is the convex hull of a compact set). The following equality holds:

O'(x,g) =

max

(v, g) .

VE of (x)

Indeed, =

max

[df(x,y) , g

yE R ( x )

max vE co a(X)

Ox

=

(v,g) = max

max (v,g) /'1

O'(x,g)

vE a (x)

vE f (X)

(v,g).

Chapter 3

36

Proposition 3.7. The mapping x -4 f(x) is upper semicontinuous.

P r o o f. It is sufficient to check that the mapping x -4 f(x) is closed. Let Xk -4 x, Vk -4 v, Vk E f(xk). It follows from the Caratheodory theorem and the definition of the set f(xk) that every Vk can be represented in the form n+1

Vk =

aki

off (xk , yk i )

(3.22)

dx

i =1

where k+1

Yki E R(xk),

aki >_ 09 i =1

aki = 1

E--

The sequences {aki I k E 1:oo} (i E 1:n+1) are bounded. We can assume with-

out loss of generality that aki kt ai and that the sequences {yki} also converge to elements yi E Y. Since the mapping R is u.s.c., we have

yi E R(x) V i E 1: (n+ 1) . Passing to the limit in (3.22) yields

df(x,yi)

n+1

V= i=1

ai

(3.23)

dx

where n+1

yi E R(x),

a, > 0,

But (3.23) simply means that v E f(x). 0

i=1

ai = 1

.

Chapter 4 CONVEX FUNCTIONS

The family of convex functions and the family of maximum functions are

two most important classes of nonsmooth functions, the study of which helped better understand the nature, properties and problems of nonsmooth analysis.

The heart of convex analysis is the concept of the subdifferential, which was discovered and widely used by J.-J. Moreau [104] and R.T. Rockafellar [136,140].

-,:

3C!

Here we shall summarize (without proof) some of the properties of &.4

convex functions to be used later on. For detailed proofs see [44,136].

4.1. A function f defined on a convex set S c Rn is called convex on S if

f(ax1 + (1-a)x2)

0 there exists a 8 > 0 such that

f(z) c 0Y(x) + SE (0)

V z E Si(x)

.

(4.6)

Remark 4.1. It follows from (4.6) and the definition of upper semicontinuity that the mapping x -4o3(x) is upper semicontinuous at any point x E Rn. But, generally speaking, it is not lower semicontinuous.

Remark 4.2. It is also possible to show that if S is an open set in R12, and x E S, then

f(x) = {v E Rn I f(z) - f(x) >_ (v,z x) V z E S} ,

(4.7)

i.e., the subdifferential concept is a local one (the subdifferential of f at x is completely defined by the properties of the function f in a neighborhood of the point x).

Theorem 4.4. The following relation holds:

f(g)

=

df(x) dg

=

max VE

(X)

(v, g)

.

(4.8)

Convex Functions

Corollary 4.2. Let X E Rn , g E Rn ,

v1

39

f(x). If oaf (x)/dg = -a < 0

E

and max

(v1, g) =

f(x)

--

(v, g)

df (x) dgg

then forge = -g we have

df(x) gl i.e.,

d=

max (v,g1) > (v1,g1) = (-v1,g) _ vE f (x)

a> 0,

if a convex function f decreases in some direction, then it necessarily

increases in the opposite direction.

4.3. Let functions f , i E 1:N, be defined and convex on Rn. It is easy to show that the functions N

ai > 0

ai, f (x) ,

0 1(x) _

i =1

and '0,

02 (x) = max{f (x) i E 1:N} I

are convex on Rn.

Theorem 4.5. The following relations hold: N

d 1(x) =

ai Ai (x)

(4.9)

i =1

'0,

&2 (X) = CO fei(X) I i ER(x)j ,

(4.10)

where

R(x) = {i E

1:N I f(x) = 02 (x)} .

Corollary 4.3. Let a function f be convex continuously differentiable in a neighborhood of a point x. Then

f(x) _ {f'(x)} .

(4.11)

Remark 4.3. Observe that the multiplication of a convex function by a negative number does not preserve the convexity.

Chapter 4

40

Theorem 4.6. If a function f is convex on Rn, then it is Lipschitz on any convex bounded set, i.e., for any bounded set G c Rn there exists an L < oo such that 11(z) - f(x)l

0 and set

fi(x) = {v E R" I f(z) -f(X) >_ (v,z-x)-E V Z ER"}

.

(4.16)

The set fix) thus defined is called the e-differential of the function f at the point x. Any element v E fi(x) is called an e-subgradient of f at x. The r-,

mapping J: [0,oo) x Rn -417(R") is called the E-subdifferential mapping.

Theorem 4.10. For any fixed x E R" and e > 0 the set fi(x) is nonempty, closed, convex and bounded.

Corollary 4.5. The mapping (ex) -4 fi(x) is bounded on any bounded set from [0,oo) x R" .

Definition. Let e ? 0, g E Rn , and let a function f be convex on Rn . The quantity

max

vE aef(x)

(v,g)

is called the e-derivative of the function f at the point x in the direction g.

(4.17)

Chapter 4

42

Theorem 4.11. The following relation holds: de.f(x) = dg

i of 1 ff(x+a8) -f(x) + E] .

a>0 a

(4.18)

A point X E Rn is called an c-stationary point of f on Rn if

0 E &f(X)

(4.19)

.

Theorem 4.12. Relation (4.19) is equivalent to 0

!5 f(x) - f* !5 e

9

(4.20)

where

f* = lrifnfly). ye R

Corollary 4.6. If for some X E Rn , .6 >0 and g E Rn we have

d,f (x) dg

0

then the funcrion f can be decreased in the direction g by not less than E. The direction ve(x) g,c (x)

-

II v x II '

where IIve(x)II =

min

IIvIi

VE def(X)

is called the c-steepest descent direction of the function f at the point x (and is, again, unique). The following property is most important.

Theorem 4.13. The e-subdifferential mapping (e, x) - d f (x) is H-continuous in .6 and x on (0,oo) x Rn.

Convex Functions

43

4.6. Let S c Rn be an open convex set and let Q c S be a convex set. Let a function f be defined and convex on S. Fix X E 12, e ? 0 and set m

dn}'(x) = {v E R" I f(Z) -,t'(x) >_ (v,z-x) V z E DI ff(x) _ {V E Rn I 1(z) - f (X) The sets o"(2f(x)

and dnf(x)

(V,Z-X)

,

- E d Z E D}

(4.21)

(4.22)

the conditional

are called (respectively) 0E.

subdifferential and the conditional E-subdifferential of the function f at the `CJ

,CD

point x ED with respect to the set D. An element v E f(x) is called a conditional subgradient, and an element v E dnf(x) is called a conditional e-subgradient of f at x with respect to D. These concepts are useful in studying constrained optimization problems.

Lemma 4.2. If X E int .fl, then for any foxed c >_ 0 the set do f(x) is nonempty, convex, closed and bounded. Moreover, if f is defined and convex on R", then o"f(x) = o'(x). Theorem 4.14. Let c >0, x ED. Then the following relation holds: d,12f(x) = cl BBf(x)

(4.23)

,

where Ble2f (X) = J V E R" 13Z E Sl: V E C2f (Z), f(z) - f (X)

(V,Z X) - E} .

Theorem 4.15. Let f = fl +f2, where functions fl and f2 are convex on an open set S C R" . Let .fl C S be a convex set and let c >_ 0. Then

0f(x)

=

1zo'

2

>_o

[d fi (x) + d 12f2(x)]

.

(4.24)

+

e1 +e2=e

Let x E 1, e > A, g E Rn,

II g II = 1. Set

ma x vE dnff(X)

(v, g)

do f (x) dg

sup (v,g) VEdff(x)

The quantity dnf(x)ldg is called the conditional derivative of the function f at the point x ED in the direction g with respect to the set .fl; the quantity dnf(x)ldg is called the conditional e-derivative of f at x in the direction g

Chapter 4

44

with respect to 12. Let

y(x) _ {V = 2(Z-x) I A > O, Z E D} , x E D . It is possible to show that a12

fcX

_

d-dga if g E cl r(x)

,

(4.25)

anf(x) dg

sup (v,g) _ V E dd2 f

RIB

Lemma 4.3. If X E 1, g E y(x), II g II = 1, then inf a>O

(x)

[f( x+ag) -f(x) + e]

(4.26))

x+agE £2

Corollary 4.7. For any e > 0 dg .f (x) dg

-

+C>O

Vg4y(x).

(4.27)

Lemma 4.4. Let a function f be convex on S and x c= .fl. Then O-Y(x) - I' (X)

--a f (X)

where T*

(x) = {w I (w,v) >_ 0 V v E AX)J . I'.

The cone I'*(x) is conjugate to the cone Ax) (see Chapter 1).

Theorem 4.16. Let a function f be convex on a convex set .fl. For a point x E S2 to be a minimum point off on .fl it is necessary and sufficient that 0 E dOf(x)

.

(4.28)

HIV

Let e > 0. A point x E S2 is called an e-stationary point of a convex function f on a convex set .fl if 0E

(4.29)

Convex Functions

45

Lemma 4.5. Relation (4.29) is equivalent to

.f(X) -}*

0

(4.30)

E,

where

PyE n Corollary 4.8. If 0 4 an f(x), then there exists a direction g E y(x) such that (4.31) and

i of

a>O

f(x+ag) < f(x) -.c

.

x+agE£2

Let X E 12, g(x) E Rn, II g(x) II = 1 be such that

d nf(x) d 8(x)

inf u g a= i

dg f (X) dg

0, then the mapping dj f is H-continuous (Hausdorff continuous) at the point [e,x].

Chapter 5 SUBLINEAR FUNCTIONS

5.1. A very important subclass of convex functions

is

that of sublinear

`CC's"

functions, i.e., convex positively homogeneous functions. There are nu-

merous reasons for their importance, of which we shall mention only two. First, the derivative of a convex function is convex and positively homogeneous, i.e., sublinear; second,

sublinear functions

are convenient for

describing convex compact sets. Minkowski duality is used for this purpose (see Section 5.2).

Let K be a convex cone in the space Rn. A function p defined on K is called sublinear if it is subadditive, i.e.,

p(x+y) < p(x) + p(Y)

V x,y E K

and positively homogeneous, i.e.,

p(2x) = 2p(x) V X E K, VA > 0. We shall now summarize some elementary properties of sublinear functions:

1. A sublinear function is convex. This follows immediately from the definition: if x,y E K, a E [0,1], then p(ax+(1-a)y)

8) ? P(8)

0

i.e., v E K*. Hence (-dp) c K*. Since K* is a closed convex cone, we have

c K*

cl cone (-o

r-,

Check the reverse inclusion. Let v 4 [-cl cone obi], or equivalently,

(-v) 4 cl cone dp . Since cl cone dp is a convex closed cone, the element (-v) can be separated from the cone cl cone dam, i.e., there exists a g E Rn such that

-(v,g) > 0

=

(w,g) max we cl cone dp

.

(5.2)

Relation (5.2) implies the inequality and hence

0 t max (w,g) wedp

p(g)

Since p(g) _ 4, Wk E U.

Without loss of generality, we can assume that the limit lim Wk = w exists. Thus w E U (since U is compact). Since 0 4 U, we have w # 0. Therefore the sequence {2k } is bounded (otherwise

II vk II

-4 +oo). We can also assume

BCD

that 2k -4 A >_ 0. It follows from the equality v = 2w that v E A, and hence the cone A is closed. 0

Note that if 0 E U, then the conical hull cone U is not necessarily 0

closed.

Note also that the relation 0 4 )7 holds if and only if there exists an element g such that p(g) < 0. Indeed, if such an element exists, then the relation

p(g) = max (v,g) < 0 V dp

,-+

implies that 0 4 dp.

Arguing from the other direction, if 0 4 p, then it follows from the 4-4

separation theorem that there exists an element g such that

p(g) = max (v,g) < (v,4) = 0 . V

dp

IoW

r..

5.2. Let M be the family of all convex compact subsets of the space Rn, and let P be the family of all sublinear functions defined on Rn. Theorem 5.1

shows that for every p E P there exists an element U E M (viz. U = p) such that

p(x) = max (v,x).

(5.3)

Take an arbitrary compact set U E M and define the function

pu(x) = max (v,x). The function pU defined on Rn by formula (5.4)

(5.4)

is

called the support

Sublinear Functions

51

function of the compact U. It is easy to verify that this function is sublinear. Indeed, pu (xl +x2)

= m a x [(v,xl) + (v,x2 )] VEU

Cue

< m a x (v,xl) + m a x (v,x2) = pu(xi)+pu(x2), vEU

VEU

pU (2x) = m a x (v,2x) _ Amax (v,x) _ 2p(x) V2 > o. VEU

VEU

Hence, pu E P. What can be said about the subdifferential dpu of the function pU ? Proposition 5.3. The following equality holds:

dPa

= u.

P r o o f. For any fixed v E U it follows from (5.4) that (v,x)

PU (x)

VXEX

.

Hence, v E 39 and therefore U c off.

Let us now assume that there exists an e E such that e 4 U. Since U is a convex compact set, it follows from the separation theorem that there must exist an element x such that

(v,x) > m a x (w,x) = p(x)

,

which contradicts the inclusion v E off. This contradiction completes the proof.

0

C4"

pU.

044

0,,

Consider a mapping 0: P -) M which associates every subdifferentiable function p with its subdifferential off. This mapping operates on the entire set M: every compact set U c M is the subdifferential of some sublinear funcIn addition, it follows from Proposition 5.3 that 0 is a one-totion one mapping. The mapping thus defined is called Minkowski duality. This mapping makes it possible to study convex compact sets using the properties of sublinear functions, and conversely, to study sublinear functions s..

using the properties of convex compact sets. We shall now summarize some of the properties of support functions and subdifferentials which are related to algebraic operations.

Chapter 5

52

The support function of the sum of sets is equal to the sum of

1.

the support functions: PU1+U2 = PU1 +PU2 .

Proof. PU 1 + U 2 (x)

=

max [(v1,x) + (v2 ,x)]

(v,x) =

max

v1 E U 1 V2 E U2

vE U 1 +U2

max (v1 , x) + max (v2 , x) = pU l (x) + pU 2 (x) , p

V1 EU1

V2 EU2

This property can be extended to an arbitrary number of summands by induction.

2. The subdifferential of the sum of sublinear functions is equal to the ^`.

sum of subdifferentials of these functions. ran

This property follows immediately from Theorem 4.5, but we shall give another proof which makes use of Minkowski duality. Without loss of generality, we can consider the case of only two functions. Let pt , p2 E P. The subdifferentials of these functions dal and 0)72 are

convex and compact, and therefore their sum U =da + dp2 is also convex and compact. It follows from property 1 that

pU (x) = m a x (v,x) = V EU

max (v1,x) +

v1 EaPI

max (v2,x) = P1 (x) + p2 (x)

V2 EaP2

Thus, dpU = a(pl +p2 ). On the other hand, by virtue of Proposition 5.3, PU = U = d-P I + -0392

0

3. If A > 0, U E M, then piU = 2pu

IfA>O,pE P, then d(A ,)=Ads I

IV

This property can be verified directly. o r-,

4. Let U = co [ U1 U U2 ] =

u [aU1 +(1-a) U2 ] . Then 0_ pU 1(x), then this maximum coincides with pU 2 (x). 0

By induction, it is possible to show that if U = co pU (x)

= max

fpU 1(x),...,pU

m

u Uj, then

ie1:m

(x)} .

5. Let p(x) = max {p1(X),...,P(X)}. Then

o5 = co u dpi . i=1:m Proof. LetUi=o i V i E 1:m,and

U=co u Ui U. i 1:m

Then U is a convex compact set and it follows from property 4 that max {pu (x),...,PU m W1

Pu(x)

1

= max {pl (x),..., pm (x)} .

Thus pu = p and therefore U = o U = ff.

El

6. Let p 1(x) >_ p2 (x) V x E Rn. Then Pi

P2

This property follows immediately from the definition of the subdifferential. 0

Chapter 5

54

7. Let Ul and U2 be convex compact sets. If Ul D U2, then PU 1(x) >_ pU 2(x)

V x E Rn

.

This follows from the definition of the support function. o

Finally, we have one more relation between a convex compact set U and its support function pU.

8. Let lip II = max p(x), where p is a sublinear function. Clearly Ilx II < 1

p(x) 5

lip 11

V x E R"

Ilx 11

.

then

=CD

It is easy to see that if II U II = p(U,O), where p is the Hausdorff metric, and II U II = II pU II .

In fact,

max max (1,x)

= max pU (x)

IIpU II

Ilxll max OWI IIX its i

On the other hand,

= max p(x)

Ilp II

max IuXNSi

IIX osi

Therefore Ilp II

= max

(p(x)I

IIX os i

Let L be any Lipschitz constant for the function p. Then

IP(x)I = Ip(x) - p(0)j 5 Lllxll and therefore Ilp 11 5 L. This completes the proof. D

Essentially we have proved the following:

Theorem 5.2. (i) Let U1,...,Um E M. Then P

i 1:m

U

71

.(x) = I PU i(x) , :

i

(x) = max {Pu1 (X),..., pU m (x)}

.

If A > 0, U E M, then pi U (x) _ 2pu (x) V x E Rn I f Ul D U 2, then pU 1(A) >_ pU 2 (x)

(ii) Let p, ,...pm E P. Then IQ,

d(I ppi) d max pi i l:m

Io = co u

39i

V x E Rn

Chapter 5

56

Here (max pt)(x) = max pt (x). (iii) Let U E M. Then the quantity IIUII

= max I1.ell .AEU

coincides with the quantity II pU II

= max pU (x) , iiX 11s i

which is the smallest Lipschitz constant for the function pU.

(iv) If .Z > 0, p E P, then d(Ap) _ Adp.

If Pi W > P2 (x) V x E P, then dal DO192 We shall now prove the following proposition, using Minkowski duality. Proposition 5.4. Let U, V, W be convex compact sets such that

U+W = V+ W. Then

U=V.

P r o o f. The support functions pl and p2 of the compact sets U + W and V + W coincide. But

pi = pU+w = pU + pw P2 = pv + w = pv + pw It is clear that pU = Pv and therefore U = V.

0

jai

5.3. Now let us study sublinear functions defined on some convex cone K in the space Rn. Such functions can, generally speaking, be discontinuous on the boundary of K. In what follows we shall consider only lower semicontinuous (l.s.c.) sublinear functions.

Proposition 5.5. Let p be a l.s.c. sublinear function defined on a convex cone K c R" . Then its epigraph

epi p = {(x,A) E K x RI is a closed convex cone.

I p(x) >_ Al

Sublinear Functions

57

P r o o f. The fact that epi p is a convex cone follows immediately from the sublinearity of p. We must prove that this cone is closed. Let sequences {xk} and {Ak } be such that

(xk,Ak) E epi p

dk,

xk-X, 2k-4A. Since p is l.s.c., we have P(x)

0

W'E K*

8m?

Inequality (5.10) indicates that x E K**. But the cone K is closed so that K** =KandhencexE K.

III

Now we can rewrite (5.9) in the form (v,x) > p(x), which is impossible since v EdX p. This contradiction shows that u = dK p. 0 is not difficult

to show that an

x=0

=w,

'CA

Remark 5.3. Using Proposition 5.7 it

analogue of Theorem 5.2 holds for the subdifferentials of sublinear functions defined on a cone. Note the following property, which will be used later on: If p, and P2 are l .s.c. sublinear functions defined on a cone K, then d K(PI+P2)

_K P2)

Cl

Proposition' 5.8. Let p be a l.s.c. sublinear function defined on a convex closed cone K. Then P(x) , x E K ,

sup

(v, x)

VEdKP

+oo

,

xcL K

.

'0"a

Q,0

Ado

P r o o f. It is only necessary to consider the case x 4 K. Since K is a convex closed cone, we can use the separation theorem to find an element W E K*

such that (w,x) < 0. Let V E dK p. Since the set dK p is (-K*)-stable, we have

(v 2w) E dK P

V A> 0

.

62

Chapter 5

Therefore

sup (v,x) VE

>_

dKp

sup

v'=v-2w

(v',x) _ (v,x) + sup 2(-w,x) = +00

.

R>0

0

R>_0

BCD

USU.

,per

Let p be a sublinear function defined on the space Rn, and let K be a convex closed cone in Rn. The function PK defined on the cone K by the equality

PK W = P(x) is called the contraction of the function p to the cone K.

The following property of dKPK is a very useful technical tool for studying convex sets contained in a cone, and will be widely used later on.

Proposition 5.9. The subdifferential dKPK of the contraction of a sublinear function p defined on the whole space R" to a convex closed cone K is given by the formula c4'

d pK =

Proof. Let U =

p-K*

.

- K*. The set U is convex since it is the sum of Let a se-

convex sets p and (K*). Let us prove that this set is closed.

may

quence {vi } be such that vi -- v, vi E U V i. Any vi can be represented in the form vi = v + v Ed , E (K*). Since the set p is comCAI

pact, we can assume that I i m v = v' and v' E. Since vi' = vi - vj, the 1-400 sequence {v"} also converges to some element v" E (-K*). Thus v = v' + v" E (off-K*), which implies that U is closed.

The set U is stable with respect to the cone (K*). Indeed, let v E U, v = v' + v", where v' E d5 v" E (K*). Take any W E (-K*). Then

v + w = (v'+v") + w = v' + (v"+w) E (-K*) since v"+w E (K*). Finally, if x E K, then s u p (v, x) =

s ud

vE U

V'E

[(v', x)+(v ", x)l

V"E (- K*)

orb

= sup (v',x) + V'E dp

= p(x) < +0 .

sup

V"E (-K*)

(v",x) = sup (v',x) v'E dp

Sublinear Functions

63

Thus, all the assumptions of Proposition 5.7 are satisfied for the set U. Therefore U is the subdifferential of the function x

(v,x) vE U

VxE K,

which coincides with PK . 0

a.,

Remark 5.4. Proposition 5.9 can easily be proved by means of theorems on l.s.c. convex functions which take infinite values (see, e.g., [79,136]). But the proofs of these theorems are rather complicated, and therefore we shall not use them here.

Remark S.S. Most of results in this section follow from the theorems in Sec,,,

tion 4.6 given therein without proofs. 0

a".

5.4. A function q defined on a convex cone K in the space Rn is called superlinear if the function p = (-q) is sublinear. In other words, q is superlinear if

q(2x) = 2q(x) V X E K, VA >_ 0, q(x1 +x2) >_ q(x1) + q(x2)

V x1, x2 E K .

ten.

All of the above results concerning sublinear functions can be reformulated (with appropriate modifications) for the superlinear case. We shall not state 0

all these results here, but rather give only the definition of the superdifferential. We shall confine ourselves to the case of a superlinear function q defined on the whole space. In this case,

q(x) = min (w,x), W Edq

where dq is a convex compact set called the superdifferential of the function q and is defined by the equality

Let U be a convex compact set in the space Rn. In addition to the support function

pu(x) = m a x (v, x) , VEU

we can also introduce the function

qU (x) = min (w, x) . WE U

Chapter 5

64

The function pU is sublinear, the function qU is superlinear and dU = dqu = U. Note that

qU (x) = min (v, x) _ - m a x (-v, x) VEU

VEU

ma x (w, x) WE [- U]

P- v (x) _ -Pa (x) Alternatively qu (x)

min (w, x) _ - m a x (v, x) WEU

vEU

-PU(x)

Conversely, if

q(x) = -p(x) V X E Rn , where q is a superlinear function and p is sublinear, then o p = dq.

Chapter 6 THE SPACE OF CONVEX SETS

6.1. Let M be the family of all convex compact subsets of the n-dimensional space Rn. We can introduce the operations of addition and multiplication by a nonnegative number in a natural way (see Chapter 1): Let A c Rn and B c Rn be convex sets and let c >_ 0. Then

A+B = {xIx=y+z, YEA, zE B}, M = JXIX=Cy, YEAJ. It is well known that these operations have the following properties: if A, B, C cM,c1 >_0,c2 _ 0, then

(A+B)+C = A+(B+C),

A+B = B+A, c(A+B) = cA+cB, (Cl +c2)A = c1A+c2A,

(6.1)

(Cl c2 )A = c1(c2A)

=A, 0 A = {0} . All these properties are easily proved. Let us prove, for example, the equality

65

Chapter 6

66

Here it is assumed that c1,c2 ? 0. If c1 = c2 = 0, then the property is obvious. We shall therefore assume that c1 + c2 > 0. Let x c (Cl +c2 )A. Then x = (Cl +c2 )y, where y E A, and therefore

x = (C1y+C2y) E c1A +c2A . This proves the inclusion

(Cl +c2)A c [c1A + c2A] . To check the reverse inclusion, let us take X E [c1 A+ c2A], i.e., x = c1 x1 +C2 x2 , where x1 E A, X2 E A. We have C1 X1 + C2 X2 = (Cl +C2)

C2 C

-Cl -FC2

xl + Cl -1-C2 x2J 0

and since A is a convex set, the term in the square brackets belongs to A, so that X E (Cl +c2 )A. 0 0.O

Remark 6.1. The equality cA = {x I x = cy, y E Al allows us to define the multiplication of elements of the set M by any real number. But in this case some of the properties mentioned above no longer hold. For example, let B be the unit ball of the space R'z. Then -B = (-1)B = B and therefore 4-+

B+B = B + B = 2B. Assuming at the same time that (6.1) is valid, we get

B + B = (-1+ 1)B = O -B = {0}

.

The operation of addition in M has the property of contraction. This means that if A, B, C EM, then the equality

A+C = B+C implies that A = B. This property was proved in Chapter 5 (see Proposition 5.4).

H.6

6.2. The set M is not a linear space (multiplication by (-1) is not allowed .,..,

and therefore we cannot subtract). This is not very convenient, and therefore we have to consider whether it is possible to extend this set in such a way that subtraction is always feasible. Mathematicians have faced many analo.,.,

gous problems; to solve them a special algebraic technique has been develop-

The Space of Convex Sets

67

ed involving the imbedding of a semigroup with contraction into a group. We shall illustrate the idea with a simple example.

Suppose that (a) we are already familiar with the concepts of a natural number and of zero, (b) we know how to perform algebraic operations and

(c) we are aware of the fact that the operation of subtraction on the set N = J09 1,2,...l is not always feasible. We want to extend this set to make subtraction possible.

Consider all ordered pairs [a,b], where a,b E N, i.e. the direct product N x N. Let us define the operations of addition and multiplication by a natural number and zero in N x N:

[a,b] + [c,d] = [a+c,b+d]

,

r-,

p[a,b] = [pa,pb] V P EN . Let us also introduce the equivalence relation =:

a+d=b+c.

[a,b] = [c,d] ate.

r-,

A pair (x,y) such that [a,b] + [x,y] = [c,d] is called the difference of pairs [a,b] and [c,d].

The operation of subtraction on the set of pairs is always feasible. For example, we can take [x,y] = [b+c,a+d]. One inconvenience inherent in such

a definition is the nonuniqueness caused by the use of equivalence rather than equality. In addition, it is necessary to link the new objects with the original set N. (IQ

To do this, let us factorize the set of pairs so that a class of equivalent pairs will be referred to as a single element. We shall denote the set of all classes of equivalent pairs by Z. Let a,,8 E Z. How should we define their sum? It is easy to check that if [a1,a2 ] and [a1 ,a2 ] are two pairs from the class a and [b1,b2 ] and [b1 ,b2 ] are two pairs from the class f3, then their sums [a1,a2 ] + [b1,b2 ] and [al ,a2 ] + [bl ,b2 ]

Rte

are equivalent pairs. This means that the sum (up to the equivalence relation) is completely defined by classes a and f3 and does not depend on specific repre-

sentatives of these classes. The class a + 6 to which the sums of pairs from a and f3 belong is therefore called the sum of classes a and f3.

`-'

ran

s.,

5'O

We define the multiplication of a class by a natural number and by zero in an analogous way. The operations thus defined have all the usual properties. Thus, for any two elements a,,6 E Z we define the difference of a and f3 as the class which contains the pair [b+c,a+d], where [a,b] E a, [c,d] E f3.

Chapter 6

68

0

0

The role of zero is played by the class which contains all pairs of the form [a,a]. Let us denote this class by 0. Let aQ denote the class which contains the pair [a,O]. It is clear that aQ = ab if and only if a = b, aQ + ab = as + b, ap.Q = pa, Thus, algebraic operations over classes of the form as are complete-

ly defined by operations over natural numbers. This allows us to identify a class aQ and a number a, and therefore to consider the set N as being imbedded in Z.

d..

Special attention should be paid to the class a containing the pair [O,a]. This class is the inverse of the class aQ : aQ + aQ = 0.

Elements of the set Z are called integers: the integer as containing the v)'

Coca

pair [O,a] is called the number -a (here a E N).

It is obvious that the definition of negative numbers is rather complicated. However we just forget their complicated origin when operating with them, making use only of the very fact of their existence and their properties.

An analogous construction with appropriate modifications is used to define the difference of convex compacts. Since the concept of numbers (including negative ones) is assumed to be familiar, we shall immediately 4-+

define the multiplication of pairs of sets by an,y real number. Let us first consider the direct product M x M, i.e., the family of pairs of the form [A,B], where A E M, BEM. We shall define the algebraic operations of addition and multiplication by a real number, using the following relations: [A 1,B 1 ] + [A2 ,B2 ] = [A 1 +A2 ,B 1 +B2 ]

[cA,cB]

ifc >_ 0

,

P-4

ifc _ 0 is associated with the class acA . Identifying the set A with the class aA, we can consider the family of all convex compact sets M as being

imbedded in the linear space M; to be more precise, M is a convex cone in M. Furthermore, M is a reproducing cone, i.e., every element of M can be represented as the difference of two elements of M. Indeed, let a EM and let the class a contain a pair [A,B]. Representing [A,B] in the form

[A,B] = [A,0] + [0,B] = [A,0] - [-B90] and using the definition of algebraic operations in M, we find that a = aA - a- B ,

i.e., a E M - M. Thus, M is a linear space containing M as a convex reproducing cone. For this reason the space M is called the space of convex sets (or, more precisely, convex compact sets).

ten

6.3. At first glance the construction above looks rather complicated and unnatural. But in fact the opposite is the case. This will become clear from P".

the functional realization of the space M, which is given below.

'=0

Let P be the family of all sublinear functions defined on Rn. If a funcis contained in M. Note that every convex compact A is the subdifferential of some function PA E P

tion p belongs to P, then its subdifferential

-this is the support function of the set A and is defined by PA (x) = m a x (v,x) v EA

V X E Rn

Let us also consider the family Q of all superlinear functions defined on Rn. The superdifferential dq of every function q E Q belongs to M; if AE M, then A = ?qA for the function qA E Q defined by the equality qA (x) = m i n (w,x) V X E Rn wEA

Chapter 6

70

Every set A E M0" is associated with a sublinear function pA and a super0

linear function qA. Since

max(v,x) _ -min(-(v,x)), v EA

V EA

we have (see Section 5.4)

PA W = -qA (x) ;

-4(- A) W

PA W

Let us consider the set L = P + Q of all functions which can be represented as the sum of a sublinear function and a superlinear function. It is not difficult to show that this set is a linear space. We shall prove only that if e E L, then-,e E L. Indeed, let e = p + q, where p E P, q E Q. Then .I = (-p) + (-q). Since (p) E Q, (-q) E P, we must have (.e) E L. We shall now describe the relations between L and M. Consider a pair [A,B] which belongs to M x M. Put

4A,B1(X) = max (v, x) + m i n (w, x) . VEA

WEB

It is clear that 1[A ,B ] EL and (1[A1,B1] + I[A2,B21 )(x)

gee

= max (v, x) + min (w, x) + max (v, x) + min (w, x) V EA1

max (v,x) + a>4

=

wEBI

vE [AI+A2]

v EA2

wEB2

min (w,x) _ 'e[A I+A2B I+B2I(x)

WE [BI+B2]

_ ZIAI)BII+[A2,B2](x)

Thus, every sum of pairs is associated with a function from L, which is equal to the sum of the corresponding functions. For c >_ 0 we have CI[A,B ] (x)

= c (m a x(v,x) + m i n(w,x)) = v EA

_ 'e[cA,cB ](x) _

wEB

c[A,B ](x)

max (v,x) + min (w,x)

v EcA

wEcB

The Space of Convex Sets

71

`k'

-Icl(m a x(v,x) + m i n(w,x)) WEB

V EA

(max (v, x)

+

vE I CIA

min (w, x) WE 1cJB

min (-v, x) +

max (-w, x) WE I cIB

VE JcIA

= min (v, x) + max (w, x) = vEcA

[cB ,cA]

(x) = 'ec [ A , B ] (x}

Thus the operation of multiplying a pair by a number (see formulas (6.3)) is associated with the multiplication of the corresponding function by the corresponding number. Let pairs [A 1,B 1 ] and [A2 ,B2 ] be equivalent, i.e., A 1-B2 = A2-B1. Then

max(v,x) + max

v e Al

WE (-B2)

(w,x) = max(v,x) +

max (w,x) we (-B1)

v E A2

.

Therefore

Z[A 1,B 1 ] (x) = max (v, x) + m in (w, x) V E Al

we B1

max (v, x) -

max (w, x) WE(-B1)

vEAl

(6.4)

max (v, x) v e A2

max

WE (-B2)

(w, x)

= max (v,x) + m i n (w, x) _ l[A 2,B 2 ] (x) . v EA2

m

WEB2

Thus, the equivalence of pairs implies equality of the corresponding functions. It follows from (6.4) that the converse is also true: If 1[A 1,B 1 ] = .e[A 2,B 21

,

then [A1,B1 ]

' [A2,B2]

''-h

Since elements of the space of convex sets are classes of equivalent pairs, it follows from the above argument that these elements are completely defined by functions from L. To be more precise, the mapping yf: M - L defined by

Chapter 6

72

the equality

where

1(x) = max (v, x) + m in (w, x) wEB

V EA

4-4

and (A,B) is an arbitrary pair from the class a, is a one-to-one mapping on the entire space L. Note that every pair (A,B) belonging to the class a is associated with a particular decomposition of the function e into the sum of a sublinear function p and a superlinear function q (namely, p = PA, q = qB ); conversely, every representation e = p + q is associated with a particular pair (A = dp, B = dq) from the class a. As shown above, algebraic operations over pairs from a (and hence over

classes of equivalent pairs) are associated with the corresponding algebraic operations over functions from L. In other words,

VI(al+a2) = Kal) + V02) 9

:tea

Since the set L is a linear space and yi is a one-to-one correspondence M - L, Q''

it is clear that M is also a linear space. Furthermore, yi can be regarded as f-+

an isomorphism of linear spaces M and L. Let A E M and let aA be the class from M which contains the pair [A,0]. Then where

1(x) = max (v, x)

.

v EA

Thus, the isomorphism yi associates every class aA with the support function PA of the set A. Identifying, as well as above, the class aA with the set A means that we have indeed identified this set and its support function.

Let us summarize the results. Any function e which can be represented as the sum of a sublinear function and a superlinear function can be associated with a class of equivalent pairs of sets. Furthermore, algebraic operations over functions are associated with algebraic operations over the corresponding classes."', The family of all equivalence classes is a linear space which contains the set M of all convex compact sets as a convex reproducing cone.

The Space of Convex Sets

73

6.4. We shall now discuss some properties of the space L and M which have to do with ordering relations. N

Definition 6.1. A set Z is called ordered if an ordering relation is introduced into Z. This is a relation >_ with the following properties: if z1 >_ z2, z2 z3, y"'

Nor

then z1 >_ z3; the relations z1 >_ z2 and z2 >_ z1 are satisfied if and only if z1 =

Z2-

o°'

A set Z is called an ordered linear space if it is both a linear space and an ordered set, and its linear operations and ordering relation are consistent in the following sense:

thenz1 +z _ Z2+z V Z E Z

and CZ1

CZ2

b4+

if Z1 _ Z2,

V C>O.

If Z is an ordered linear space, then the set K = {Z EzZ>_I 0} is a convex cone which does not contain straight lines. This cone is usually called the cone of positive elements. gyp

Conversely, if we choose some convex cone K which does not contain straight lines from a linear space Z, then Z becomes an ordered linear space. An ordering relation can be introduced as follows: z1 Z2 if and only if

Z1-Z2EK. +.+

Nor

Now let us consider the space L. An ordering relation can be introduced in a natural way: the relation el >_ 12 implies that el (x) >_ .e2(x) V X E Rn. Clearly, this relation transforms L into an ordered linear space. The cone K -44,

of positive elements consists of functions e which are nonnegative on Rn. r1'

Our next aim is to show that L is a lattice. First we give the necessary definitions.

coo

N

Let Z be an ordered set. A subset U of this set is said to be upper

(lower) bounded if for some Z E Z the following inequality holds: x_ Z)

VXE U.

X00

The element z is called an upper (lower) bound of the set U. Let U be an upper (lower) bounded set. If the family of all upper (lower) bounds of the set U contains a smallest (largest) element, this is called the supremum (infimum) of the set U and is denoted by sup U (inf U). Sometimes we use the term `the exact upper (lower) bound' of U instead of the term `supremum' ('infimum'). In addition to sup {ai ,...,an }, sup ay, the notation a 1 v a2 v yE r

,..

v an' yE r ay is also used. The forms inf {a,b} and a A b are equivalent.

A set Z is called a lattice if every finite subset of Z has its own supremum

and infimum. Clearly, if a supremum and an infimum exist for all two-element subsets of Z, then Z is a lattice.

Chapter 6

74

Let Z be a set of functions defined on some set T, where the relation zl >_ z2 means that >_

z2 (t)

V t E T. +.,

z1(t)

If in addition to functions zl and z2 the set Z also contains functions

z(t) = max{z1(t),z2(t)},

z(t) = min{z1(t),z2(t)}

then Z is a lattice, where z1 v z2=z, z1 Az2=z.

,°"S

Nay

Remark 6.2. If for some z1, z2 E Z the corresponding elements z and z do not belong to Z, it does not necessarily imply that Z is not a lattice. For example, the set of convex functions defined on some convex set is a lattice,

although z does not necessarily belong to this set. If z1, z2 E Z, then the function zl A z2 is defined as follows: the epigraph of this function is such that it coincides with the convex hull of the union of the epigraphs of z 1 and z2.

Lemma 6.1. The space L is a lattice.

Proof. Letej,Z2 EL and let Li = pi + q1; 12 = P2 + q2, where P1,P2 E P,

qi, q2 E Q Q.

Let us show that the functions .e(x)

= max {e1(44 (x) } V xE Rn

:g(x) =min {.e1(x),.e2(x)}

belong to L. First note that

.e(x) = max {pl(x)-g2(x),p2(x)-ql(x)} + (ql(x)+q2(x}) fi(x) = (Pi(x)+Pa(X)) + min {q2(x) Pi(x),4i(x) P2(x)} .

Consider, for example, equality (6.7). Let x E Rn be such that 1i (x)

e2 (x)

,

p l (x) +q l (x) > P2(x)+q2(X)

,

(6.6) (6.7)

The Space of Convex Sets

75 4..

In this case, the expression on the right-hand side of (6.7) coincides with p2(x)-q2(x) = 12(x). At the same time, .:g(x)

= min

{11(x),12 (x) } = 12 (x)

.

If 11(x) _ a2 if and only if yi(ai) >_ yr(a2). Let W(al) = 11 = pi + q1, y(a2) = 12 = p2+q2, where p1,p2 E P and ql 9 q2 E Q.

Then

a1 =

1(11) contains the pair [dp1,dg1]

a2 = V _1(,e2) contains the pair [p2,dg2]

,

Chapter 6

76

The inequality ;Y(al) >_ V/(a2) implies that

pl -q2 ? P2 -ql Let us consider the sublinear function p = pl -q2. coincides with the set

Its subdifferential

1-dq2. Moreover, the subdifferential d(-q2) of the ,,p

sublinear function -q2 coincides with (-dq2). Therefore

dff

d(p I +(-q2))

dp I + d (-q2)

d-P I - q2

-

Analogously the subdifferential p' of the sublinear function p' = P2 - ql coincides with d P2-dq1. The inequality yr(a1) >_ V/(a2) is equivalent to the inequality

pl -q2 > p2 -ql which (from the above) is equivalent to the inclusion

P2 - dql

P1 - dq2

Let [A1,B1 ] be an arbitrary pair from the class a1 and let [A2,B2] be an arbitrary pair from the class a2. Putting

pi(x) = m a x (v,x); qi(x) = m i n (w,x) we B

V E A.

1:2

,

we get VI(a1) =

p1 + ql

;

V/(a2) = P2 + q2

Ai = pi, Bi = dqi; V i E 1:2. Thus, the relation a1 >_ a2 implies that the inclusion Al - B2

A2 - B1

holds for arbitrary pairs [A1 ,B 1 ] E a1, [A2,B2 ] E a2.

It follows immediately from the definition that the ordered spaces L and M are isomorphic. Since L is an ordered linear space, M is also an ordered c13

linear space. Since L is a lattice, M is a lattice as well.

Using formulas (6.8) and (6.9), it is easy to find the exact bounds of a finite set of elements from M. Let I be a finite set of indices, where pi E P, qi E Q. It follows from Theorem 5.2 that

d v pi = co u dpi . iE I

iE I

The Space of Convex Sets

77

Applying an analogue of this theorem to superlinear functions yields

= co u dqi

A qi

iEI

iE I

Here q1 E Q V i E I.

Let ai E M V I E I and let (Ai,B1) be any pair from the class ai. Then the supremum iE I ai coincides with the class containing the pair [A,B],

where

kE I

iE I i* k

(6.10)

'-%

A = co u IAk- 2: Bi B = 2: Bi iE I

A ai coincides with the class containing the pair [C,D], p,;

The infimum

iE I

where

C = IiEAi I D = co u Bk - 2: Ai iE I

kE I

i# k

Let us prove (6.10). It follows from (6.8) that an element e c L related to the class v ai can be represented as the sum of the sublinear function iE I

p(x) = max L(X)

(qi (x)

+ iE I

,he

i# k

and the superlinear function

q(x) =

iE I

qi (x) ,

where we can take pi(x) = m a x (v,x), qi(x) = m a x (w,x). Using the linearity

VEAW E B.

of Minkowski duality and the rules for computing the subdifferentials and superdifferentials of exact bounds, it is now easy to obtain formula (6.10).

Chapter 7 UPPER -SEMICONTINUO USLY DIRECTIONALLY DIFFERENTIABLE FUNCTIONS

7.1. Let us consider the n-dimensional space Rn equipped with some norm II. Let X be an open set in this space and let a function f be defined, continuous and directionally differentiable on X. We say that the function f is II

°,4

upper-semicontinuously directionally differentiable (u.s.c.d.d.) at a point xo E X if for any fixed g E Rn the function x -4f'(x,g) is upper semicontinuous (in x) at this point and is bounded in some neighborhood of x0. The latter property means that there exists a number C < 00 such that [f'(X'g)J

:5

(7.1)

C11g11

for all g E Rn and every x in some neighborhood of x0. Let us consider two examples of u.s.c.d.d. functions.

Example 7.1. Let f be a convex function defined on Rn. Then it is directionally differentiable on Rn and df (x)

dg

= f'(x,g) =

max

VEdf(x)

v (,g)

where fi(x) is the subdifferential of f at x. The mapping x - e(x) is u.s.c. ...

and bounded (see [44,136]). It follows from Proposition 2.3 that the function x -4 f'(x,g) is upper semicontinuous for all g. The boundedness of the mapping x -4 e(x) on any bounded set in R" implies the existence of C < C>0 such that (7.1) holds. Hence f is an u.s.c.d.d. function.

Example 7.2. Let f be a maximum function, i.e.,

f(x) = m a x O(x,y) yEY

V X E X,

where the function 0 is defined on X x Y, Y is a compact set in some space, and 0 and its partial derivative doldx are jointly continuous in both variables. 78

Upper Semicontinuously Directionally Differentiable Functions

79

The function f is directionally differentiable and

f'(x,g) =

max (v,g) vEd f(X)

where

f(x) = co fd'vx R(x)

y e R(x)}

,

= {y e Y I O(x,Y) =f(x)}. r.,

dam

(D0

It follows from Proposition 3.7 that the mapping x -4 f(x) is u.s.c. The boundedness of Y and continuity of do/dx imply that the mapping x -4 d(x) is bounded. Therefore the function f is u.s.c.d.d. It turns out that these examples are quite typical.

Definition 7.1. We say that a function f defined on X is subdifferentiable at a point x E X if it is directionally differentiable at x and if its directional derivative fX is a sublinear function (as a function of direction). We denote by f(x) the subdifferential of f at x. By definition,

f(g) =

max (v,g)

vEd f(X)

V x E Rn

Recall that the subdifferential is a convex compact set.

Proposition 7.1. If a function f is u.s.c.d.d. at a point x E X, then it is also subdifferentiable at this point.

P r o o f. The positive homogeneity of the function f(g) = f'(x,g) is obvious. Let us check its subadditivity. Take gi, 92 E Rn . Then there exist functions Sri (a) and V2(a) such that

ui (a) a and

0

0

V2 (a) a

--0 0

I

Px'gi) = a [f(x+agl) -f(x)] + Vj(a) f'(x,gi +g) = a [f(x+agi+a82) f(X)] + V2(a)

Chapter 7

80

yielding

f'(x,g+g) -f'(x,gl)

=

V3 (a)

a [f(x+ag1+ag2) -f(x+agl)] + V3(a) , V2 (a)

w1(a)

a+0

0

.

Fix some a > 0, put xa = x + ag1 and define

Ma =

sup f'(xa+/Jg2,g2) 0 0 there exist e > 0 and ,u > 0 such that the mapping x -4 (Hx)* dEaf(H(x))

of the function 0 on the set X1. Here *

is a continuous denotes transposition.

The proof is based on the following lemma.

Lemma 7.1. Under the hypotheses of Theorem 7.1, for any 8 > 0 there exists an e > 0 such that

H(x) + Be c H(x+B5)

H(Bs(x)) V x E X.

ma'r'

Q`.

O.0

P r o o f. First let us show that for every point x E X2 the image of any of its neighborhoods contains some ball centered at the point H(x). Assume for the sake of argument that the minor which does not vanish (see condition (ii)) corresponds to the first n indices. Let

x = (x(1),...,x(n),Z

E X2

Consider the set X2 = {y = (y(1),...,y(n)) E Rn I

1),..., (m)) E X2}

Upper Semicontinuously Directionally Differentiable Functions

87

and the mapping H defined on this set by the equality

H(y) = H(x)

,

where x =

rya

Since the Jacobian of this mapping does not vanish at the point y = ((1) ,...,x-(n) ), it follows from the inverse function theorem that in some neighborhood of this point there exists a continuous mapping H-1 which is the inverse of H. The continuity of H 1 implies that the image of every

sufficiently small neighborhood of the point y in the set X2 (under the mapping H) contains a ball centered at the point H(Y) = H(xi). Furthermore,

the image of any neighborhood of the point x in the set X2 (under the mapping H) contains a ball centered at the point Y.

Fix S > 0. Let e(x) denote the supremum of the set of numbers e > 0 such that

H(x) + Be c H(x+BS) V X E X2

.

Here BS and B., are open balls, of radius 5 and e, respectively, centered

at zero. It follows from the above argument that e(x) > 0 for all x. Let us show that the function e(x) is l.s.c. Setting out to prove the opposite, we can find a sequence {xk } and numbers e', e" > 0 such that Xk -4 X,

V k.

Xk E X2, e(x) > E" > E' > e(xk)

(7.8)

The inequality e' > E(xk) implies that there exist elements {yk} such that I I H(xk) - A 11 < i',

yk 4 H(xk +B s)

(7.9)

.

Since the sequence {H(xk )} converges, the sequence W1 is bounded. We can assume without loss of generality that the limit lim Yk = y exists. Since II H(x) - y II = lim II H(xk) - yk II

0. 0

P r o o f of Theorem 7.1. Let O(x) = f(H(x)). Since f is Lipschitz, it follows from Theorem 3.1 that

O'(x,g) = fj (HX (g)) =

max (v, HX (g)) v r= d f(H(x) )

max ((HX)*v, g) =

vE af(H(x))

max (v',g) v'E (H')* (a.f(H(X)))

We conclude that 0 is an u.s.c.d.d. function and that

&(x) = (HX) *(e(H(x)))

.

Let numbers 6 > 0, v > 0 be given. Find an e > 0 which corresponds to 05 (and whose existence is guaranteed by Lemma 7.1) and choose a ,u such that the inequality 11A

p1I(HX)*II < v holds. Take a continuous (e,p)-subdifferential d f of the function f. Then

f(H(x)) c 4f(H(x)) c f(Hx + BE) + ,uB *

.

Applying the operator (HX) * to these inclusions yields

(HX)* f(H(x)) c (HX)*d ,uf(H(x)) c (HX) *f(H(x) +Be) + ,u (HX) *B

c (HX) * f (H(x+BS)) + vB * .

Uppersemicontinuously Directionally Differentiable Functions

89

Making use of the equalities

&(x) = (Hx")*e(H(x)) dO(x+Bj) =

u

9

(HX)*e(H(x+Bj))

11X'- xa 0 a function f has a continuous (e,0)-subdifferential 4f, then for any J > 0 there exists an e > 0 such that the mapping (Hx)(dx o(H(x)))

is a continuous (e,0)-subdifferential of the function 0 = f(H) on the set X. This follows directly from the proof of Theorem 7.1, which allows us to

describe a continuous (e,0)-subdifferential for one class of finite maximum functions.

coy

Theorem 7.2. Let functions hl ,...,hn be defined and continuously differentiable on an open set X c R" (where m >_ n) and let

O(x) = max he(x) V x E X

.

ie 1: n

Assume that the Jacobian matrix

ah;

' O)} has a minor of the nth order which

does vanish on the closure cl X of some bounded open subset X of the set X. Then for any 8 > 0 there exists an e > 0 such that the mapping d o (x) defined below is a continuous (5,0)-subdifferential of 0. The mapping d p is described by the relation n

n

y E Rm I y= I Vi dhi (x) 1

i=1

d

,...,

dhi (x) vi

i =1

dx< m) J

v=(v1,...,vn)E Ve(x)

9

Chapter 7

90

where n

V (x) =

vi = 1, vi >_ 0; vi = 0 if i 4 R2 (H(x))

v E Rm I i =1

fi(x) _

lim qg (xk)

.

Consider a subsequence {k5 } such that

lim qg (xks) = lim qg (xk)

,

and let es - +0. Since =

(xk

Tim

[f(y+ag) - f(Y) I

y-4Xk S a -)+0

there exist, for any s, a vector ys and a number as such that - xks II<e

Ilys

qg (xks }

0 such that

(Vf(y),g) < P(8) + e V Y E B S (x)\ T .

(8.8)

Fix Y E Ba12(x). Let My be the intersection of the line {y+ag I a E R1 } with the ball Ba(x). My is nonempty. Put hy P9

(a) = fly+ag)

-

c43

If the function f is differentiable at y + ag, then the following derivative exists:

h;)9 (a) = (VAY+ag)g)

-

pica

Clearly, hy,g is Lipschitz. It was noted above (see property (b)) that for almost all y the intersection

L of the interval M with the set T is a set of one-dimensional measure zero. Considering the function hy ,g , using the fact that .f is differentiable on My \ Ly and noting property (c), we obtain (for almost all y)

a

f(y-i-ag)-f(y) = by>s(a) - by>s(0) = f0(Vf(y+flg),g)48. Here the number a is sufficiently small to ensure that y+ag E Bo(x). It follows from (8.8) that for almost all ,(3 E [O,a] the inequality :5 P(8) + e

(V.fCY+iaB),8)

.o°

is satisfied. Therefore

f(y+ag) - f(y)

f(g)

d 8 E Rn ,..,

0

holds for any upper convex approximation p, but there is no lower concave approximation q such that q(g)

_>

q(g)

V gE R'2

,

V qE Q,

.,..,

where Q is the family of lower concave approximations of the function f at x.

Example 9.3. Let f be a concave function. Then the derivative fX exists, is continuous and superlinear. Moreover,

d fz = d f (x)

,

Chapter 9

106

where a*) is the superdifferential of f at x. It is possible to prove that a sublinear function p is an u.c.a. if and only if r1 affix)

0

,

.fl

and a superlinear function q is a l.c.a. if and only if dq =)df(x). A concave function has a maximal lower concave approximation fX but no minimal upper convex approximation.

Example 9.4. Let a function f be Lipschitz in a neighborhood of a point X E X

and let f(g) = f°(x,g) be the Clarke derivative (more precisely, the upper Clarke derivative) of the function f at x in a direction g (see Chapter 8). Assuming that f is directionally differentiable at x, it is easy to see that

I °(x,g) ? f;(g). Since the function f ° is also sublinear and continuous, this function is an u.c.a.

of f at x. Thus, the Clarke subdifferential do 1f(x) is only one of many sets which provide a local approximation of f at x. The Clarke lower derivative fo(x,g) can be taken as a l.c.a. (see Chapter 8).

9.3. Since an individual u.c.a. is not a satisfactory approximation of a function, it is appealing to consider those families of u.c.a.'s which provide an exact local approximation.

Let A be an arbitrary set. A family {p, I A E A }, where pp is an u.c.a. of a function f at a point x, is called an exhaustive family of u.c.a.'s of f at x if

i n f p, (g) _ fx(g)

AE A

V g E Rn

In other words, if

f(x+ag) = f(x) +

a E f pA (g) + og (a)

V g E Rn

where

og (a)

a

- 0.

a -9+ 0

U..4

Analogously, a family {q I A E A }, where qA is a l.c.a. of a function f at a point x, is called an exhaustive family of l.c.a.'s of f at x if

sup q, (g) = f(g)

AE A

V g E Rn

Upper Convex and Lower Concave Approximations

107

i.e., if

f(x+ag) = f(x) + a E p

qz (g) + 08 (a)

V g E R"

The existence of an exhaustive family of u.c.a.'s means that the directional derivative fX can be linearized (described by linear functions) by means of the "minimax" operation:

f(g) =

i n f max W,g) .

Ae A uE ape,

The existence of an exhaustive family of l.c.a.'s implies that ff can be linearized by means of the "maximin" operation:

f(g) = sup min (v,g). 2e A vE aq2

Theorem 9.1. Let a function f be defined on an open set X c R" and be direc-

tionally differentiable at a point x EX. Assume also that the derivative fX is continuous (as a function of direction). Then the function f has exhaustive families of u.c.a.'s and l.c.a.'s at the point x. This theorem follows immediately from the next result. obi

Theorem 9.2. Let a function h be defined, positively homogeneous and continuous on R". Then there exist families of sublinear functions {pa I 2 E fl} and superlinear functions {q I A E A} such that

h(g) =

h(g) = s up 9j (8)

i n f pA (8)>

.

.eA

.EA

(9.2)

The same set of indices A may be chosen for all h. ~.,

P r o o f. Suppose that for any continuous positively homogeneous function h there exists a family of superlinear functions

jqA I A EA I

Ma

j

I A EA I

such that the second equality in (9.2) is satisfied. Then

(-h}(g} = sup q_ h(g) , 2A

h(g) = in f aEA

(-q-h)(8) A

Chapter 9

108

A E A } of Thus, the first equality in (9.2) holds for the family {p, sublinear functions (where pt = -qq h). This implies that it is sufficient to I

prove the existence of a family of superlinear functions. First let us show that the function h is bounded on the unit sphere S = {x I

= 11. Since, in addition, h is continuous at zero, there exists an i7 > 0

III

Ilx II

such that Jh(x) 1

M+ 1;

ge S

1 - L)(h(u) - E) > M .

(9.3)

Check that 1, has the following properties: ..,

(1) .eA (u) = h(u) - e (this follows from the definition of 4); (2) .eA(g)

II u-g II

h(g) V g E S.

then h(g) > h(u) - e > 1, (g). Here we use the ,-,

Indeed, if

i n f h(g) - 1 > M, gE S

which may be derived from the definition.

Upper Convex and Lower Concave Approximations

If

109

Il u-g II >_ 8, then

4(g) = - Ilu-g112(h(u)-e-11 + (h(u)-e) 0, /32 0, then the families {cp j I A E A } and {cqu

p E M} are

c},

exhaustive families of u.c.a.'s and l.c.a.'s of fi at x.

If c < 0, then the family {cq'u

,u E M} is an exhaustive family of

u.c.a.'s of A at x. Indeed, Orb

(fi)x(S) _ -I c[fx(8) _ -1cl sup q (S) ,u e M

= Icy i n f (-9,u (g)) = in f c9,u (g) ,u e M

#r= M

4'"

W0.

Analogously, it can be shown that the family {cps I A E A } is an exhaustive family of l.c.a.'s of fl at x. (b) Let {pj I A E A} and {p" 1,u E M} be exhaustive families of u.c.a.'s of functions fi and f2, respectively, at a point x. For [AP] E A x M let us put p'I,u (g) = pi (g) + pu (g)

Also, let f = fl +f2. Then f(g) _ (fl )X (g) + (f2) ,r (g) = i n f p(g) + i n f pu (g) IEA # EM

i n f [p,j (g) + p (g)] =

AEA ,uEM I

PIE u (g) ,

[.Z,u] E A x M} is an exhaustive family of u.c.a.'s of

the function f at the point x. We can construct an exhaustive family of l.c.a.'s in the same way.

cps

i.e., the family {pju

inf [A,fl] E A XM

0

Upper Convex and Lower Concave Approximations

111

poi.

(c) Using (a) and (b) it is not difficult to write down formulas for computing exhaustive families of u.c.a.'s and l.c.a.'s for a linear combination c1 fl + c2 f2, the product f1 f2, and the quotient fl /f2. r0-

Of course, these formulas are of practical interest only if we know exhaustive families of u.c.a.'s and l.c.a.'s of functions Ii and f2, which, however, can be easily constructed.

The concepts of an u.c.a. and a l.c.a. are very convenient in the study of extremal problems. We shall illustrate this by the following example. Suppose

that f is defined on an open set X c Rn and is directionally differentiable at x* E X. For the point x* to be a minimum point of f on X, it is necessary that

`d g E Rn

.

If p is an u.c.a. of f at x*, then p >_ fX and therefore

p(g)?0

`dgE Rn

.

(9.6)

00.0

U''"' T.,

This inequality is equivalent to the inclusion O Edp. Thus, a necessary condition for a minimum has a very simple and natural form: the subdifferential o)7 must contain zero.

If {pj I A E Al is an exhaustive family of u.c.a.'s, then the necessary condition (9.6) is equivalent to the condition O Edp,j `d 2 E A. Analogously, a necessary condition for a maximum can be expressed in terms of l.c.a.'s. Thus the problem of studying necessary conditions is reduced to that of finding u.c.a.'s (or 1.c.a.'s) and the exhaustive families of these approximations.

tea)

In the next chapter we shall describe one sufficiently rich class of ,,r

functions for which exhaustive families of u.c.a.'s and l.c.a.'s constructed in a rather simple canonical way (and with relative ease).

can be

Chapter 10

Q UASIDIFFERENTIABLE FUNCTIONS

10.1. A function f defined on an open set X c RR is said to be quasidifferentiable at a point x E X if it is directionally differentiable at this point and if its derivative of(x)ldg can be represented in the form df (x)

max (v,g) + m i n (w,g) ,

fX' 1 (9)

WE W

VEV

where V and W are convex compact sets in Rn.

Remark 10.1. It is possible to show that representation (10.1) is equivalent to the following property: there exists bounded sets Zl and Z2 in the space Rn such that

df(x) d -= sup (v,g) + i n f (w,g) vEZ,

g

.

WEZ2

Let a function f be quasidifferentiable at a point x (i.e., representation (10.1) holds). Put

p(g) = m a x (v,g); q(g) = m i n (w,g) VEV

.

WEW

-h~

Clearly, the function p is sublinear and the function q is superlinear. Thus, the directional derivative of a quasidifferentiable function can be represented as the sum of a sublinear function and a superlinear function and therefore the derivative is continuous as a function of direction. Moreover, this derivative is Lipschitz since both sublinear and superlinear functions are Lipschitz (see Chapter 5).

It is clear of course that if the derivative fx' can be represented as the sum of a sublinear function p and a superlinear function q, then f is quasidif-

ferentiable at x; in formula (10.1) we can take the subdifferential p of the function p as V and the superdifferential dq of the function q as W. 112

Quasidifferentiable Functions

113

?:'

Let us establish a relation between quasidifferentiability and the concepts of upper convex and lower concave approximations.

Theorem 10.1. Let a functionf be defined on an open set X c Rn and be directionally differentiable at a point X E X. The following propositions are equivalent:

(a) the function f is quasidifferentiable at the point x; (b) the function f has an exhaustive family of u.c.a.'s I W E W}, where pw = w + p, W is a convex compact set, p is a sublinear function; (c) the function f has an exhaustive family of l.c.a.'s {q (v E V}, where q,, = v + q, V is a convex compact set, and q is a superlinear function.

P r o o f. (a) G=* (b). Let f be quasidifferentiable at x so that relation (10.1) holds. Put a x (v,g) . p(g) = m VEV R'1

The function p is sublinear and therefore for any W E W the function p,, = w+p is also sublinear. Thus

f;;(g) = max (v,g)+mi n (w,g), VEV

WEW

4-4

b0>

i.e., the family fpw I W E W} is an exhaustive family of u.c.a.'s. However, if fPw I W E W} is an exhaustive family of u.c.a.'s (where W is a convex compact set and pw = w + p), then

f(g) = m i n ((w,g) + p(g)) = p(g) + m i n (w,g) = p(g) + q(g) WEW WEW

Since the function q(g) = mEi (w,g) is superlinear, the function f is quasidifferentiable at x. The equivalence (a) ==* (c) may be proved in an analogous way. 0 a)4

10.2. The nonuniqueness of a pair of sets defining the directional derivative of a quasidifferentiable function seems to be quite natural if viewed in terms

0'0

of the space of convex sets. Indeed, quasidifferentiability means that the derivative fX can be represented as the sum of a sublinear function, and a superlinear function, i.e., belongs to the space L (see Chapter 6). But, as observed in Chapter 6, every function from L is associated with a class of

cep

pairs of convex compact sets. Two pairs [A 1, B 1 ] and [A2 , B2 ] belong to the same

class if and only if they are equivalent, i.e.,

A 1 - B2 = A2 - B1

.

(10.2)

Chapter 10

114

ran

Ir.

Definition 10.1. Let a function f be quasidifferentiable at a point x. The class of equivalent pairs of sets [A, B] such that

f(g) = max (v,g) + m i n (w,g) v EA

V g E R1z

WEB

`.,

is called the quasidifferential (QD) of the function f at the point x and is denoted by Df(x). Any pair of sets belonging to this class is also called a quasidifferential of f at x and is denoted by Df(x). This ambiguous usage of Df(x) should not cause confusion.

BCD

Remark 10.2. Using the terminology of the space of convex sets, Df(x) is just an element a of the space of convex sets such that j/i(a) = fx, where yi

'v0

,,r

is the mapping defined by formula (6.5). It is also a pair of sets which belongs to the class of a. If Df(x) = [V,W] then the set V is called a subdifferential of the function P"'

X01

f at the point x and is denoted by fi(x), and the set W is called a superdifferential of f at x and is denoted by af(x). Note that the sets must be considered as a pair Df(x) = (a quasidifferential): they cannot

[f(x),df(x)] such that df(x) =

f(g) =

max (v,g) +

mar'

`ti

be studied and used separately. By definition, a quasidifferential of f at x is a pair of convex sets

min (w,g)

.

4.,1

dg vedf(z) weO f(x) This pair is defined up to the equivalence relation (described by (10.2)).

10.3. MAIN FORMULAS OF QUASIDIFFERENTIAL CALCULUS. It is understood that algebraic operations over quasidifferentials are performed in the same way as operations over elements of the space of convex sets (or, equivalently, over pairs of sets). Recall that (see Chapter 6) [AI,BI] + [A2,B2] = [AI+A2, BI+B2] [cA,cB]

ifc _ 0,

[cB,cA]

ifc = afl W + aft W (ii) If fi E fi(x) and c is a real number, then f = cfi E fi(x) and

D(cfl)(x) = cDfl (x)

Df(x)

In other words, if Dfl(x) =

.

(10.5)

r7fi(x)], then

Df(x) = where .,..

c

A

if c

Al

.,.,

{ af(x) _ (x)

if c

0

.

(iii) If fif2 E O(x), then f =fl f2 E fi(x) and

Df(x) = f1(x)Df2(x) + f2(x)Dfi (x) .

(10.6)

(x)] and Df2(x) = [e2 (x),df2 (x) ], then

In other words, if Dfi (x) _

Df(x) =

Chapter 10

116

where

df(x) =

,

f1df2+f2fi

lf.fi

0 ,.fi ? 0

,

f1df2+f2df1

Lf.fi

0 , .fi

f1f2-i-f2df1

iffi ? 0 , fi

IA

f1f2+f2f1 Iffi?0 ,.fi?0

0

,

0

,

f1df2+f2df1

1.ffi ? 0 , fi ? 0

,

f1f2+f2df1

ffi

0 , fi ? 0

,

fif2+f2f1

f.fi

0 9f2

f1df2+f2f1

f fi ? 0 , fi ? 0

0 .

Here the values of the functions and their subdifferentials and superdifferenrials are computed at the point x.

Df(x)

nil

fi

I

f E A(x) and 2

ski

(iv) If fl E O(x) and f1(x) # 0, then f =

Df, (x)

A (X)

In other words, fDf1(x) = [fi (x), afi (x)], then

Df(x) = [fi(x), df(x)] where (x)

fl

afi (x)

df(x)

,

112(x)

ei fix)

P r o o f. We shall prove only properties (i) and (ii). Properties (iii) and (iv) can be proved in an analogous manner.

Quasidifferentiable functions

117

(i) Since fi, f2 E fi(x), there exist derivatives dfi(x)lag and af2(x)lag, where df 1 (x)

=

g

df2(x)

dg

ma x (v,g ) + m in (w,6) , VEd f1(x) vEdf1(X)

=

,g

= fj(x,g) =

max ve d f 2 (x)

(v,g) +

min

ve d f2 (x)

(w,g) coo

d

.

Since the directional derivative of a sum is equal to the sum of the directional derivatives we have

d f(x} dg

-

df I (x)

dg

--

df2(x)

d

g

max (v,g) +

ve df 1 (x)

max (v,g)

VE f2(x)

min (w,g) + min (w,g) wed f, (x) wEd f2 (x)

(10.7)

m

+

+

+!Y2

max

(w,g) coo

(v,g) +

max VE 1 ! Y 1

.

WE [af1(X) +af2(X)]

But (10.7) implies (10.4). This proves (i). (ii) Let f = cfl, fi E O(x). If c > 0, then dfi (x)

dg -c dg -c

(max

V E f (x) /'Y

of (x)

we df (x)

(v,g) +

ve [Caf (x)]

(w,g))

min

CPO

max

(v,g) + min

(w,g)

.

(10.8)

WE [Caf (x)]

Ifc_ 0, m(S) = 1.

Let X be an open set in Rn and a functionfs be associated with every S E S,

0.0

where fs is defined on X and is quasidifferentiable at x E X. Assume that for every y EX the function s -4 fs(y) is summable. Let

1(y) = ffstm.

(10.14)

&a00

We shall now find conditions under which the function f is quasidifferentiable and describe its quasidifferential. .ti

Let every s E S be associated with a sublinear function ps defined on Rn and the function s -4 p s (g) be measurable for every g E Rn. (In this case the mapping s -, ps is called weakly measurable.) It is not difficult to show that in this case the function s -4 ii p s ii = v'r1 I p s (g) I , is also measurable; 8

the continuity of p s implies that is summable. Since

II p

s

II < +00. Assume that the function s -4

II p

s

II

VgERn,

Ips(g)I
- (v,y-yo) e y E r} ,

afZCvo,zo) _ {w E Rn2 I f(yo, z) -f(Yo,zo)

(w,z-zo) V z E Z}

(11.2) (11.3)

a0.

The set Dfy(yo,zo) is the subdifferential of the convex function hl(y) = f(y, z0) at the point yo E Y, and the set dfZ(yo,zo) is the superdifferential of the concave function h2(z) = f(yo, z) at the point zo E Z. These sets are nonempty, convex, closed, and bounded. In [47, Section 4, Chapter II] it was proved that where the function f is quasidifferentiable on X and Df(xo) =

Y0(X0) _ [,(yo,zo, On219 at'(xo) _

[only

(11.4)

are zero elements of spaces Rn' and R n2 , respectively. Thus the first formula in (11.4) means ans that f(xo) is the direct product of sets o y(yo,zo) c R ' and {02} c R 2. Here On

1

and On 2

Examples

125

Example 11.8. Let a function f be defined on an open set X c Rn and let

F(x) = sat f(x) =

if f (x) 2 1 , if -1 S f(x) S 1 if f (x) 5 -1 .

1+1 f (x) L-1

Since Fl(x) = max {fl(x),-1}, where fl(x) =min {f(x),1}, quasidifferentiability of the function f at a point xo E X implies quasidifferentiability of the function F at xo.

Applying formulas (10.12) and (10.13), we get DF(x) _ [o'(x),df(x)], where

if f(x) >_ 1 or f(x) < 1

1{O}

f(x) _

{f' (x)}

if I

0

1co if (X), 01

I y if xW > y( V i E 1: m. In other words, we treat Rm as an ordered space (see Section 6.4) in which the cone of positive elements of the space Rn coincides with the cone + Rm

_ {xE

R"

IE

1.,

If an operator P is sublinear, then it is (i) subadditive, i.e., P(x+y) _< P(x) + P(y) V X, Y E Rm ; (ii) positively homogeneous, i.e., P(Ax) = 2P(x) V A > 0, V X E Rn .

Let P: Rn - Rm be a sublinear operator. The subdifferential dP of P con-, sists of all nxm-matrices composed of rows i which belong to the subdifferentials dp of the coordinate functions pi. Clearly, dP is a compact and operationally convex set and P(x) = m a x Ax. AEdP It follows from the definition that

dP = {A I Ax 5 P(x) V x E Rn }

.

A mapping Q; Rn -4 Rm is called superadditive if

Q(x+y) > Q(x) + Q(y) V x,y E Rm A mapping which is both superadditive and positively homogeneous is called a superlinear mapping (or a superlinear operator). The superdifferential dQ

of a linear operator Q consists of all nxm-matrices A such that Ax > Q(x) V x E Rn. Clearly, Q(x) = m i n Ax. A E aQ

Let X be an open set in Rn. It is clear that a mapping H: X - Rm is quasi-

differentiable at a point x E X if and only jf (i) it is directionally differentiable and (ii) its directional derivative HX can be represented in the form HX = P + Q, where P is a sublinear operator and Q is a superlinear operator. Moreover, the subdifferential dH(x) coincides with the subdifferential dP of the operator P, and the superdferential dH(x) coincides with the superdifferential dQ of the operator Q.

12.3. First we shall discuss the composition of a sublinear function and a sublinear operator.

Let T: Rn - Rm be a sublinear operator and let p be a sublinear function defined on Rm. Unfortunately, the composition g(x) = p(T(x)) is not neces-

Quasidifferentiability of a Composition

131

sarily a sublinear function. This can be guaranteed only in the case where p has the property described below.

We say that a function p defined on Rm is increasing if the inequality x ? y implies p(x) ? p(y). If a sublinear function p is increasing, then the composition g(x) = p(T(x)) is sublinear. Indeed, since T is sublinear, we have yl = T(xl+x2)

f3(x) = x(') , f4(x) = -x(2)

.f(x) = max {fi(x),.fi(x)}, fi (x) =min {f3(x),.fa(x)}

.

and f4 are continuously differentiable, we can take , Since f2f3 Df2(xo) =

af(x0)J, ei(xo) = {(-1,1)}, af(xo) = {(0)4)}

D.f3(x0) = 13(x0), df3(xo)],

f3(xo) = {(1,D)}, dfs(xo) = {(0 ,0)}

Dfa(xo) = [a(xo), afa(xo)]>

%(xO) = {(0,-1)}, o'fa(xo) = {(0,0)} .

Using the rules of quasidifferential calculus (see Chapter 10), we have

Dfl(xo) = [e&O)q fj(xo)]

9

where

2fI(XO) = Wq-l)j

9

dfi(xo) = co J(0,1), (-190)1

9

and

Df(xo) = &(xo), f(xo)]

,

where

f(xo) = co {(1,-1) - (0,0); (-1,1) - co {(0,1), (-1,0)} = co {(1,-1); co {(0,1), (-1,0)}} = co o '(xo) = (0,0) + co {(0,1), (-1,0)1 = co {(0,1), (-1,0)} We observe that f E M(xo).

(0,1), (-1,0)1

,

The Clarke Subdifferential and the Quasidifferential

153

It is clear from Fig. 13.1 that we can take

C(f) = C(f) _ Ig E R2 1 IIg II = 1 } \ u

iE 1:4

Ai

,

where

A, =

-

9

9-

A3

(T2

Tj ) 1

1

( 42-

,

9

T2)

A2 =

(2 T 9

5

1

T5

I-

A4 NF5

T 5)

Find

fi(g) = Arg max {(v,g) (v e f(x0)} O(g) = Arg min {(w,g) I w e df(xo)} . con

It is clear that for g E A1A2 (where AB is the arc on the unit circle which connects points A and B but does not include them) we have (see Figs. 13.1 and 13.2) O(g) = {(0,1)}, O(g) = {(-1,0)} and hence 0(g) = {(-1,1)} .

x2

Figure 31.1

Figure 13.2

Chapter 13

154

e-1

m

coq

For g e A2A3 we have {(1,-1)}, O(g) _ {(-1,0)} and hence O(g) = {(0,-1)} m

010

For g E A3A4 we have {(1,-1)}, O(g) _ {(0,1)} and hence 0(g) = {(1,0)}

For g e A4A 1 we have

fi(g) = {(-1,0)}, O(g) = {(0,1)} and hence 0(g) Applying (13.11) we finally obtain (see Fig. 13.3) x2

x1

Figure 13.3

do Lf(xo)

= co {O(g) I g E C} = co {(-1,0), (0,-1), (1,0)} .

The direct evaluation of do z f (xo) using formula (13.1) gives the same result.

Remark 13.2. The function (13.22) was described by F. Clarke in [31, Section 25].

Chapter 14 IMPLICIT AND INVERSE FUNCTION THEOREMS

CAD

14.1. The implicit and inverse function theorems are essential components of classical differential calculus. Here we describe analogous theorems for quasidifferentiable functions.

First we shall consider the case in which there is only one equation. A system of equations will be examined in Section 14.3. Let z = [xy] E Rm + 1

,

where x e R"', y E R1, and consider a given function f(z) = f(x,y). Let zo = [xo,yo] be a solution of the equation f(z) = 0, i.e.,

f(xo,yo) = 0 .

(14.1)

Fix any g E Rm, 11g II = 1, and consider the function 010

F(a,y) = f(x0+ag,y), where 0

a continuous function y(a) defined on [O,ao] can ,..

a

be found, such that

F(a,y(a)) = 0

V a E [O,ao]

,

(14.2)

then we say that equation (14.2) describes an implicit function y(a) = y(a,g).

In this case a neighborhood of the point zo contains a solution to the equation

F(xo+ag,y) = 0

.

(14.3)

The question of existence and uniqueness of implicit functions has been studied in some detail in the case where the function f is smooth. Now let us 155

Chapter 14

156

N

assume that f is jointly quasidifferenti able in both variables at the point zo and continuous in some neighborhood of zo. Then

f(z0+a) = f(zo) + a

max (v,77) + min (w,R) vEd f(ZO) WEd f( 2O)

+ o(a) .

(14.4)

arm

Here 17 = [g,q] (recall that g E Rm is fixed). We assume that there exists a qo ER' such that f(zo,770) =

af( zo) drb

max

min

(v,i7o) +

VE d f (ZO)

(w,i7o) = 0

,

(14.5)

WE d f( ZO)

where 77o = [g,qo] Let that function o(a) = o(a,g) in (14.4) be such that o a,

a

-4

a-+0

(14.6)

0

uniformly with respect to q in some neighborhood of the point qo E R1.

This assumption is quite natural and is satisfied in many cases of practical importance. Put

v = [vl,v2],

w = [w1,w2] ,

where v2 E R 1,

V1 E Rm ,

wl E Rm ,

W2 E R 1,

and consider the function

h(q) =

max

vEd f(ZO)

((v1,g) + v2q) +

min

wEd f(ZO)

((wl,g) + w2q) .

(14.7)

Relation (14.5) implies the equality

h(9o) = 0 .

(14.8)

Implicit and Inverse Function Theorems

157

Theorem 14.1. Let h(qo) = 0. If the function h(q) is strictly monotonic in a neighborhood of the point qo, then there exists a function y(a) which is defined and continuous for a E [O,ao], where ao > 0, and such that equality (14.2) is satisfied. Furthermore, the right-hand-side derivative of the function y(a) exists at the point a = 0 and

(0) a yyo y+ C) y(O) =

1im

,

a-+0

a [y(a)) -- yC0 )]

= qo

(14.9)

0

P r o o f. It follows from (14.1) and (14.4) that f(xo+ag, yo+a(qo+,8)) = ah2(a,A

,

(14.10)

where

h2(a,fl) = hl(,8) +

o a,fl)

hiA = h(qo+,8), hl(0) = 0 .

9

Since the function h(q) is strictly monotonic in a neighborhood of the point qo, the function hl(#) is strictly monotonic for ,l3 sufficiently small. Therefore there exists an Eo > 0 such that for any e E (O,eo] we can find 131(e) and Q2(E) which satisfy the equations

hl(#2(e)) = -e

h I(#, (e)) = e I and the relations ,81(e)

0

fl2(e) - 0

0,

.

m

Fix any e E (O,EO]. It follows from (14.6) that

oa a

OAA

_

a

-4

0

a-+0

uniformly with respect to,8 (if the absolute value of fl is sufficiently small). Therefore there exists an ao(E) > 0 such that e

NAM

0

V e E Aad(e)]

't3

Chapter 14

15 8

Thus WIN 2

29

The continuity of the function h2(avg) in ,8 implies that there exists a /3(a) such that h2(a,,Q(a)) = 0, i.e., f(xo+ag, yO+a(qo+fl(a))) 010

0

Clearly,

fl(a)

-4

a 4+0

0

Y(a) = Yo + a(9o+Q(a)) is a solution to equation (14.2) such that y+ (0) = qo.

0

14.2. Now let us consider the case m = 1, i.e., the case in which f(x,y) is a function of two variables. Let z = (x,y) E R2 and let zo = (xo,yo) be a solution to the equation f (z) = 0, i.e., O15,

f(XO'YO) = 0

-

Put h(q)

(v1g+v2q) +

max

=

(v1,V2)E

(ZO)

min

(w1,w2)E af(zO)

(w1g+w2q)

h(q)+l(q). Here g = +1 or -land

h(q) =

max (v1)v2)E f (ZO)

min

(w1g+w2q)

.

(w1,w2)E ?f (ZO)

and B =Pry [4f(zo)] be the projections of the sets e(zo)

Let A = Pry

respectively, on the axis y (A and B are obviously intervals).

r.1

and

(v1g+v2q), h(q) =

Implicit and Inverse Function Theorems

159

Proposition 14.1. if A n B

m

(14.11)

,

then the equation h(q) = 0 has a unique solution and the function h(q) is strictly monotonic (for both g = +1 and g = -1).

P r o o f. Fix any q E E1 and let A = [a1,a2 ], B = [b1,b2 ], where a1 < a2, b1 S b2. Take a direction s E El (s = +1 or s = -1). Then

max vs+

(v1,v2) E R(q)

mi n (wl,w2) E Q (q)

ws 2

14. ()

where

R(q) = {v = (v1tv2) I V E f(zo); v18 + v29 = h(9)}

Q(9) = {w = Mtw2) I W E df(ZO); x'18 +x'29 = h(9)} . It follows from (14.11) that

as

>_ max {b1-a2, a1-b2} = c >0 .

(14.13)

It is clear from (14.12) that if

for a direction s, then the inequality

necessarily holds for the direction s1 = -s. Since (14.13) is valid for any q, there exists a unique qo such that h(qo) = 0. Inequality (14.13) also implies that the function h(q) is monotonic.

Chapter 14

160

Corollary 14.1. If f is a subdifferentiable function, then Df(zo) = &(zo),O] and condition (14.11) is equivalent to the condition 0 4 A.

If f is a superdifferentiable function, then (14.11) is equivalent to the condition 0 4 B. If f is a smooth function, then B = O1, A = f(z0) and condition (14.11) becomes the classical condition

f(xo,yo) # 0 . Remark 14.1. Proposition 14.1 can be generalized to the case x E Rm. Put HQ7) = df (zo)/d17 and find the solution to the equation

H(17) = 0 .per

subject to the condition

(14.14)

11 17 II = 1. -14

Assume that equation (14.14) has a finite number of solutions where 77k = (P 1 k'p2 k )

LA,

CIO

p,,

Of course it may happen that there is either an infinite number of

solutions or none at all. For example, if

4f(zo) c int J(zo) or e(zo) c int [--af(zo)]

>

..4

then equation (14.14) has no solution. Take a solution 77k = (Plk'P2k). If P1 k > 0, then q1 k = P2 k'Pl k is a solution to the equation max

(w1 +w2q) = 0 .

min

(v 1 +v2q) +

IUD

h1(q) -

(14.15)

(w1,w2) E af(Z0)

(v1,v2) E af(ZO)

If, in addition, the function hl (q) is strictly monotonic in a neighborhood

of the point ql k, then there exists a continuous function y(a) defined on [O,ao ], where ao > 0, such that

y(o) = yo,

f(x0+a, y(a)) = 0

V a E [0,ao]

.

(14.16)

The function y(a) is differentiable from the right at the point a = 0 and

y+(o)=qlk

If pl k < 0, then q2 ,k = P2 k/ Pl k is a solution to the equation

h2 (q) =

max

(vIIv2) 6 d(Z0)

(-v1 +v2q) +

min (w1,w2) 6 f(ZO)

(-w1 +W2 q) = 0 . (14.17)

Implicit and Inverse Function Theorems

161

4-+

If, in addition, the function h2(q) is strictly monotonic in a neighborhood of the point q2 k, then there exists a continuous function y(a) defined on [O,ao], where ao > 0, such that .

.-,

V a E [O,ao] w

y(O) = Yo, f(x0-a, y(a)) = 0

(14.18)

This function y(a) is differentiable from the right at the point a = 0 and y+(0) = q2 k If p1 k = 0, then the equation f(xo+ag,y) = 0 may have a solution (for g = +1 .-r

or g = -1) which has an infinite derivative at zero (this solution corresponds to the point p7k ). r,,

If pl k > 0 V k E 1,.e, then the equation f(xo-a,y) = 0 has no solution for a > 0 sufficiently small (in a neighborhood of yo). If pl k < 0 V k E 1:.e, then the equation f(xo+a,y) = 0 has no solution for a sufficiently small (in a neighborhood of yo). ?7

,...,i7

can help us to solve the inverse problem: find

Q,^

CD's

4.4

The points

max r01

hi(g) =

BCD

4-A

solutions to the equation f (x, yo+aq) = 0 (where q = +1 or q = -1). In particular, if P2 k > 0 and the function

(v1g+v2) +

min

(w1$+w2)

(w1,w2) E f(z0)

(v1,v2) E af(ZO)

is strictly monotonic in a neighborhood of the point g1 k = Pi k'P2 k, then there exists a continuous function x(a) defined on [0,ao], where ao > 0, such that

x(0) = xo, f(x(a), yo+a) = 0

V a E [0,ao]

(14.19)

and x+ (0) = gl k.

Analogously, if P2 k < 0 and the function

h2(g) =

max (v1,v2) E af(ZO)

(v1$-v2) +

min

(wlg-w2)

(w1,w2) E f(zO)

is strictly monotonic in a neighborhood of the point 92k = Pl k'-p2 k, then there exists a continuous function x(a) defined on [0,ao], where ao > 0, such that

x(0) = xo, f(x(a), yo-a) = 0 and x+ (0) = 92k-

V a E [0,ao]

(14.20)

Thus, in the two-dimensional case, studying a subdifferential of the function enabled us to prove the existence of an implicit function for both directions, g = +1 and g = -1.

Chapter 14

162

Example 14.1. Let f(xy) = jxj - jyj + 2y and zo = (0,0). Clearly, RZO) = f(OVO) = 0

-

We have RZ) = A (Z) + f2 (Z)

,

where

fi(z) =

f2(z) = -lvi

ICI + 2Y,

The function f, is subdifferentiable at the point zo and we can take

f1(zo) = co {(-1,0),(1,0)} + (0,2) = co {(-1,2),(1,2)} df, (zo)

JO)Oj

The function f2 is superdifferentiable at zo and we can take

f(zo) = {(0,0)}, af2(zo) = co {(0,-1),(0,1)}

.

Therefore

f(zo) =

Jj(zo),

afcZO> = af2(=o)

In Figure 14.1, the set f(zo) is bounded by a bold line and the set [-f(zo)] .t)

by a dashed line (-df (zo) = f(zo)) (in this case 4-f (zo) = df(zo)).

,.0.
0 and y+'(0) _ - 1/3. Next we consider the inverse problem. Since the second components of the vectors 1i and 112 are negative, there exist two continuous functions xl (a) and x2 (a) (defined for sufficiently small a > 0) such that

x1 (0) = xpt f(x(a),y0-a) = 0

V i E 1:2.

Furthermore, xi+ (0) = 3, x2+ (0) = 0.

At the same time, there is no solution to the equation f(x, yo+a) = 0 (where a > 0) in any neighborhood of the point xo.

Example 14.2. Let f(x,y) = Ixl - lYl, zo = (0,0). It is clear that f(zo) = 0. We can take

f(zo) = co {(-1,0),(1,0)}

,

df(zo) = co {(0,--1),(0,1)}

Again, 4f(zo) = f(zo). V V

Figure 14.1

Figure 14.2

Chapter 14

164

It is clear from Fig. 14.2 that the equation H(r7) = 0 has four solutions:

27 1

L - F2

\F2

272

t

L

NF2

- \F2

1

273

1

272

t

-\F2 '

-%F2

Since pl 1 > 0 and p2 2 > 0, equation (14.15) has two solutions: q1 1 = 1, q12 = -1.

The function h1 (q) is strictly monotonic in a neighborhood of the points

q1 1 and q12 . Therefore there exist two functions yl (a) and y2 (a) such that

yi(0) = 0, f(xo+a,y(a)) = 0 V i E 1:2 for all sufficiently small a > 0. In addition, yi + (0) = 1, y2.} (0) = -1. Analogously, since p13 < 0 and P14 < 0, equation (14.17) has two solu-

tions q13 = +1, q14 = -1, and the function h2(q) is strictly monotonic in a neighborhood of points q1 3 and q1 4 .

Thus, there exist two functions y3(a) and y4(a) such that

V i E 3:4

yi (0) = yo = 0, f(xo-a, yi (a)) = 0

.-r

r-,

for sufficiently small a > 0. In addition, y3 + (0) = 1, y4 + (0) = -1. It is not difficult to check directly that this is really the case.

Example 14.3. Let f(xy) = sin xj - sin 12yj and zo = (0,0). It is clear that f(zo) = 0. We have RZ) = A (Z) + f2 (Z)

,

where

f1(z) = sin xj = max {sin x, -sin x} , f2(z) = -sin 12yj = -sin (max {2y,-2y}) . The function fi(z) is subdifferentiable at the point zo and we can take

f1(zo) = co {(cos 0, 0), (-cos 0, 0)} = co {(1,0), (-1,0)} t(0)0)1

Implicit and Inverse Function Theorems

165

0

The function f2(x) is a smooth composition of quasidifferentiable functions, i.e., f2(x) = q5(y(z)), where

0(t) = sin t, yl(z) = 12y1. Obtaining the quasidifferential of a composite function in the usual way, we have Df2(zo) _ 0"(Y1(zo))DY1(zo) = -cos 0 Dyr(zo) _ -Dyr(zo) D Vl(zo) = [r2o(zo),

where

dyf(zo) = co {(0,2), (0,-2)} d yf(zo) _ {(0,0)} Therefore

t'2(zo) _ {(0)0)} N

df2(zo) _ -dyf(zo) = co {(0,2), (0)-2)} Thus 0

f(zo) = co {(1,0),(-1,0)} 0

df(zo) = co {(0,2),(0,-2)} . It is clear from Fig. 14.3 that the equation H Q7) = 0 has four solutions:

171 =

(45

e

2 273

_

I,

VJ

1

VJ $ ;5-

s

272

r > -VJr )

s

L yJ

272 = (

2 %F5

-

)

.

yJ

For each of two directions, g = +1 and g = -1, there exist two functions yi (a), y2(a) satisfying equations (14.16) and (14.18), respectively (for all sufficiently small a > 0), and such that y1 (0) = yo (i E 1:2). In each case, one of these functions has a right-hand-side derivative equal to 1/2, while the second has a right-hand-side derivative equal to -1/2.

Chapter 14 C13

166

Example 14.4. Let f(x,y) = Ix +y2-21 - Jx2-2y+i1 and zo =

(1,-1). Clearly,

f(go)=0. `'h

According to the rules of Quasidifferential Calculus (see Chapter 10), we can take Df(zo) = &(zo), af(zo)J, where

f(zo) = co {(1,2), (-1)-2)}, df(zo) = co {(2,-2), (-2,2) . It follows from Figure 14.4 that the equation df(zo)/dij = 0 (111711

1) has

four solutions:

771= (&) 773 = (0,1) ,

V1-7-

l/

774 = (0,-1) .

Y Y

Figure 14.3

Figure 14.4

The equation f(x0+a, y(a)) = 0 has a solution related to the point 77, such that y(O) = yo = 1, y,'(0) = 1/4.

Similarly, the equation f(x0-a, y(a)) = 0 has a solution related to the point 172 such that y(O) = yo = 1, y.} (0) = - 1/4.

The points 273 and 774 indicate (since p13 = P14 = 0) the possibility that there are solutions to equations (14.16) and (14.18) which have infinite

derivatives at the point a = 0. It is not difficult to check directly that such

Implicit and Inverse Function Theorems

167

M-+

solutions exist. Since p2 * 0 V i E 1:4, the equation f(x, yo+a) = 0 (for all sufficiently small a > 0) has two solutions x1(a) and x2 (a) related to the points 711 and 772 and such that

xi + (0) = 4,

x1(0) =x2(0) = xo = It

x2+ (0) = 0

.

Similarly, the equation f(x, yo-a) = 0 has two solutions x3(a) and x4(a) related to the points 773 and N and such that

x3+(0)= -4,

x3(0)=x4(0)=xo= It

x4+(0)=0.

G."

ti,

Example 14.5. Let f(x,y) = Ix+y2-21 + 1x2-2y+1I and zo = (1,1). It is clear that f(zo) = 0. The function f is subdifferentiable at the point zo and we can take

f(zo) = co {(3,0), (-1,4), (1,-4), (-3,0)} df(zo) _ {(0,0)}

vow

Since [-rdf(zo)] c int f(zo), the equation df(zo)ld1 = 0 (where 47 11 = 1) has no solution. Therefore there are no solutions to equations (14.16) and (14.18) in a neighborhood of the point (1,1).

14.3. Now let us consider the multidimensional case. Let z = [x,y], x E Rm, y E Rn, and let the function f (z), i E I:n, be quasidifferentiable on Rm + n Consider the following system: r-.

f (x,y) = 0

V i r= 1:n .

This can be rewritten in the form

f(z) = 0

(14.21)

,

CT'

where f = (f ,...,fn ), 0 E Rn. The problem is to find a function y(x) such that

f(x, y(x)) = 0

ViE 1:n,VxE Rn

.

Unfortunately we cannot solve this very general formulation of the problem for an arbitrary quasi differentiable system of type (14.21). But what we shall try to do is to solve this problem for a given direction g E Rm. We shall call this a directional implicit function problem.

Chapter 14

168

Suppose that zo = [xo,yo] is a solution to system (14.21), i.e.,

J(zo) = 0

V i E 1:n

.

Consider the system of equations

f(x0+ag, y(a)) = 0

(14.22)

,

where a > 0. Since the functions f are quasidifferentiable for any q E Rn, we have from (14.21) that dfi (zo )

f (xo+ag, yo+aq)

= f (xo,yo) + a d

=ad

[g,q]

+ oi(a,q)

i(zo) + of (a,q) [g,q]

(14.23)

where

dfi (zo)

d[g,q]

=

max

viEdfi(z0) [(vi

i 99) + (v2 i ,q)]

min

+

[(wt i,g) + (w2 i,q)] .

(14.24)

W E o fi (Z0)

Here Df i (z) = [ f (z), d f (z)] is a quasidifferential of fi at z; f (z) c R'n + n , N

df (z) c R'n+n are respectively subdifferentials and superdifferentials off at z (convex compact sets); vi = [vii, v2 i] and wi = [wi i, w2 i]. Let qo E Rn be a solution to the quasilinear system afi (zo)

d [g,qo ]

=0

Vi E

1: n

.

(1425) .

Suppose that in (14.23) of (a,q)

(14.26)

Implicit and Inverse Function Theorems

169

uniformly with respect to q E S8(g0)

Jq E Rn I ll(J-(jp 11 < } s

where 8 > 0 is fixed.

Is it possible to find a vector function r(a) with ao > 0 such that

V i e I :n, a E [O,ao]

f (xo+ag) yo +a[qo+z(a)]) = 0

(14.27)

,

where r(a) E Rn V a E [O,ao]? Take e 2t 0 and introduce the sets vi e ei(zo) I (vl i,g) + (v2 i,q0) 2 R ie

=

max

Ift,

vie d f i

[(vi i ,g)+(%i ,qo)] - e (ZO)

wi E d f (ZO) I (wl i,g) + (w2 i )qo)


0 there exists a 51 > 0 such that
qo+Z) (v2i(Z)+w2i(z),z) + r1 i(z) ,

where

r1 i(z) = (v1 i(Z), g) + (v2 i(z), qo) + (w1 i(z), g) + (w2 i(z), qo) , Vi(T)

_ [v1 i(Z), v2 i(z)] E R i(z)

wi(z) _ [w1 i(z), w2 i(r)] E R i(Z)

Since R i(z) and R i(Z) are upper semicontinuous, if zs wi (ZS)

s400

s400

0, vi (ZS)

S-400

vi, and

wi, then vi E R i (0), wi E Ri (0). This means that

rI i(O)

dfi (zo)

d

and the rl i(z) are continuous. It follows from (14.25) that

rl i(0) = 0

V i E l:n

.

(14.29)

Thus, from (14.23) f (xo+ag) yo+a(qo+z)) = a[(v2 i (z)+w2 i (z), z) + ri (a,z)] , where

ri(a,Z) = r1 i(s.) +

oi(a , qo+Z)

a

Consider the functions Fia (Z) = (v2 i (Z) + w2 i(2'), Z) + ri(a,2) . Here v21(r) E "21(')' w2 i (Z) E

"2i()' where

V2i(Z) _ {v2i 13 v1i E Rn: [vli,v2i] E Ri(Z)} W2 i(Z) = {w2 i 3 wl i E Rn : [w1 i,w2 i] E R1(z)}

(14.30)

Implicit and Inverse Function Theorems

171

The mappings vi i(r) and w2 i(r) are upper semicontinuous. Now introduce

the set M(z) of matrices such that A E m(r) if A is a matrix with ith row [v2 i(z) + w2 i(Z)]T, where

v2 i(z) E V2 i(z) and w2 i(Z) E W2 i(z)

The mapping M is convex valued and upper semicontinuous. Let us denote by M. (where e >_ 0) the set of matrices such that A2

Me = IA =-

:

Is

Ai = [v2 i+w2 i]T , V2 i e

R, W2 i E R ie

V

An

From (14.28) it is clear that

M(z) C M£ d z E S' 61 ( 0 )

(14.31)

.

Note that if 51 =di(e) in (14.31), then (14.28) is satisfied.

Theorem 14.2. If for some e > 0 we have

mi n detA>0,

(14.32)

AEMc

then for a positive and sufficiently small a there exists a solution to system (14.27) or, equivalently, to the system

Fia(z) = 0

Vie 1:n.

P r o o f. Let us construct the mapping M-1 (r) r(a,r) _ Oa (z)

,

where

M- I (r) = JB = A- I I A e M(r) I

.

From (14.31) and (14.32), it follows that 0a (z) is upper semicontinuous (for any fixed a e Mao]) in z e Sal (0) and that Oa (S81(0)) C S461 (0) .

Chapter 14

172

It is easy to see that 0a (2) is convex for all z. This means that all of the conditions of the Kakutani theorem (see [81 ]) are satisfied and therefore there exists at least one point 2(a) which is a fixed point of the mapping Oa (z): Z(a) E 0a ma)) ANC)

From (14.26) and (14.29) it is also clear that Z(a)

a0

which together with (14.30) yield Fi a (Z(a)) = 0

.

0

000

Corollary 14.2. If qo is a solution to (14.25) and condition (14.32) of

'0

III

Theorem 14.2 is satisfied, then system (14.22) has a solution y(z) defined on [O,ao] (where ap > 0) and such that im

a

fy(a)

0

= qo

goo

We shall call Theorem 14.2 a directional implicit function theorem. Of course, there could be several solutions to (14.25), or none at all. It is important to be able to solve systems of equations of the form max

vi Eali

[(VI i,g) + (v2 i,q)] + m i n [(wl i,g) + (w2 i,q)] = bi Wi Ea21

V i E 1:n

,

where vi = [vl i, v2 i], wi = [w1 i, w2 i ], and al i c Rm + n and 0121 c Rm + n are convex sets. We shall call systems of this type quasilinear.

In some cases (for example, if 61 i and 621 are convex hulls of a finite number of points) the problem of solving quasilinear systems can be reduced to ;,.,

that of solving several linear systems of algebraic equations (we shall illustrate this later on).

Implicit and Inverse Function Theorems

173

14.4. Now let us consider a special case of the problem, namely, where system (14.21) is of the form

x+O(y) = 0,

x(i) + Oi(y) = 0

(14.33)

V i e 1:n

,

where

X = (x(1),...,X(n)) E Rn, y = (y(l),...,y(n)) E Rn and the function Oi are quasi differentiable on Rn. Suppose that zo = [xo yo] E R2 n is a solution to (14.33), i.e.,

Choose any direction g E R. We now have to consider two questions: 1. What conditions are necessary for the existence of a positive ap and a continuous vector function y(a) such that the expressions V a e [O,ao]

y(0) = yo, xo + ag + r(y(a)) = 0

(14.34)

are satisfied? 2. If y(a) exists, does

.-.

Y'' (0) = 1 i m a-)+ 0

1 a Cy(a) - yC0)]

necessarily exist? CDR

To answer these questions we turn to Theorem 14.2 and its corollary. Let DOi (y)= [d Oi (y), du(y) be a quasidifferential of Oi at y. We then have

max Vie d O i (y0)

(vi,q) + -{-

Oi(yo+a) _ oi( O) + a

min

(wi,q)

W iE d O i (yo)

+ o; (a,q) .

(14.35)

Chapter 14

174

In this case equation (14.25) takes the form max

(wi,q) _ -g1 V i E 1:n

min

(vi,q) +

.

(14.36)

W 1E d Oi (Yo)

v1E d Ot (yo)

Suppose that qo E Rn is a solution to (14.36) and that in (14.23) o.(a)g)

a

a 4+0

0

uniformly with respect to q E Sg(go) We now introduce the sets

= {vi E do (yo)

R

I

(vt,q) >_

max

(vi,q) - e}

v1E d Ot (yo)

_ {wt E dO (yo) I (wi ,q)

R ie

_min

(wi,q) + e}

.

W 1E d 0t (Yo)

Let M. be a set of matrices such that [At

Me

A

,

Ai = [vi + w,]T , Vi E Rte, wt E kig, V

i

A_ An

where e >_ 0.

Theorem 14.3. If for some e > 0 we have

min det A > 0

,

(14.37)

AEMe

then there exist an Cro > 0 and a continuous vector function y(a) such that

and y+ (0) = qo.

Implicit and Inverse Function Theorems

175

Remark 14.2. In the case where each of the sets ao;(yo) and dOt(yo) (for all values of t) is a convex hull of a finite number of points, it can be shown that Theorem 14.3 is valid if (14.37) holds for e = 0. An analogous result can also be obtained for Theorem 14.2. L3,

Remark 14.3. Suppose that [xo,yo] is a solution to (14.34). Then to solve the directional inverse function problem, it is necessary to find all solutions to (14.36) and check whether condition (14.37) is satisfied.

Chapter 15 CONES OF ADMISSIBLE DIRECTIONS AND A NONDEGENERACY CONDITION

cor

15.1. In studying and solving extremal problems, it is often convenient to U1,

approximate those sets over which optimization is to be performed. Approxima0

tions of this type can also be useful in studying other problems. There are a number of ways of making such an approximation; we shall mention only those we shall be using in later chapters.

000

Let Q be a set in the space Rn and let x be a point in the closure of Q. A vector g E Rn is called an admissible direction with respect to the set at the point x if there exists an ag > 0 such that x+ag E Q V a E (0,ag ). A vector g E Rn is called a tangent direction with respect to the set Q at the point x if for any a > 0 there exists a vector

a

-4

0

(a) such that x+ag+yr(a) E £2.

0.

(15.1)

Let V1 be the set of functions yf operating in Rn, defined for sufficiently small a > 0 and such that (15.1) holds. Elements of the set V/1 will also be denoted by o(a).

Thus, a direction g is tangent (with respect to a set Q at a point x) if there exists a function f E V/1 such that x + ag + Vl(a) E Q or, equivalently,

if x+ag+o(a) E Q. A vector g is feasible with respect to a set Q at a point x if there exist sequences {gk } and {ak } such that gk E Rn ,

ak > 0, gk -4 g, ak -4 +0, x + ak gk E Q .

The following definition is equivalent: a vector g is feasible if for any e > 0,

a0 > 0 there exists an element w E g+B,, and a number a E (0,a0) such BCD

that x+aw E 12. Let y(x,A2) denote the set of directions which are admissible with respect

to a set .2 at a point x; let K(x,Q) denote the set of directions which are tangent to a set d2 at a point x, and let r'(x,Q) denote the set of directions which are feasible with respect to a set Q at a point x. 176

Cones of Admissible Directions and a Nondegeneracy Condition

177

Clearly, each of the set y(x,Q), K(x,Q) and r(x,sl) is a cone (although not necessarily a convex cone). We shall call these sets the cones approximating a set d2 at a point x. It follows directly from the definition that

K(x,sl) =

U K V (x,(2) , VIE V1

where

KxQ= n

a> 0

l Q-x- a.

Similarly,

n rxa = C> n 0 a0>0 u aE (O,ao)

1 Q-x + eB.

Let

Pa (y) = in f Ily-z ll 2E2

Since x e cl fl, we have 012 (x) = 0. It is clear that g e y(x,i2) if and only if there exists an ag > 0 such that pfj (x+ag) = 0 V a E (O,ag). Property 15.1. The following equality holds:

K(x,!l) _ {g e Rn I (pa )'(x,g) = o}

(15.2)

.

P r o o f. By definition, '(,g) x = 1 i m 1 [p1-i ( x+ag) -P(x)] = 1 i m Let

lim 1p(x+ag) =0 a and let va E Q be such that Ilx+ag-va II < Pa (x+ag) + a2 .

a P (x+ag)

.

Chapter 15

178

Put V/(a) = x+ag-va. Then x+ag+br(a) E d2 and simultaneously yr E V1. Thus g E K(x,Q). Analogously we can show that the relation g E K(x,d2) implies (pp)'(x,g) = 0. For the cone T(x,d2) we have SIa

a

fPn (x+aS) -PD (x)] =

0}

(15.3)

or, equivalently,

a JJni a -,+0

y(x, Q) c K(x,Q) c T'(x, w) .

(15.4)

If x is an interior point of Q, then all three cones coincide with Rn ; if x is an isolated point, then all three cones contain only zero.

(DEC

0.0

The cone T(x,Q) is always closed. Indeed, let g be a limit point of this cone. For every .6 > 0 and ao > 0 it is necessary to find a v E BB (g) and an a E (O,ao) such that x+ av E D. Let w E T(x,Q) be such that IIw-g II < .6/2. Then there exist a v E B,12 (w) and an a E (O,ao) such that x+ag E d2. This is precisely what we required, i.e., T(x,Q) is closed.

Col'

If the function pp is directionally differentiable at x E cl d2 and its

fir,

directional derivative is continuous as a function of direction, then the cones K(x,Q) and T(x,Q) coincide. A corollary of this is that the cone K(x,Q) is closed (in general this cone is not assumed to be closed). Note also that the following inclusions hold:

I'(x,Q) c cl (cone(Q--xw)), y(x,Q) c cone

(Q-x)

.

(15.5)

tea)

.fl

Indeed, if g E T(x,Q), then there exists a sequence {gk } such that gk -4g and akgk E [Q x] for some ak > 0. This implies the first inclusion in (15.5). The second inclusion can be checked in a similar way. The following property follows from relations (15.5) and (15.4): If y(x,Q) = cone (Q -x), then cl y(x,Q) = T(x,Q)

.

..,

0

These three cones describe the local properties of d2 in a neighborhood of x. Indeed, it follows from the definition that for any e > 0 we have

Cones of Admissible Directions and a Nondegeneracy Condition

170

y(x, i2) = y(x, Q r Bo (x) )

K(x,(2) = K(x,(2 r Bo(x))

,

T(x,Q) = T(x,Q r

.

B,,(x))

CAD

Why is it necessary to use different cones? The cone of admissible directions seems to be the simplest and most convenient of the cones considered, but unfortunately in many cases it is empty or contains only zero. For example, if d2 is described by an equality, i.e., d2 = {x h(x) = 0}, and if the graph of the function h does not contain "linear" segments in a neighborhood of the point [xof(xo)], then the cone y(xo,D) consists only of zero. In some cases the cones K(x,a) and T(x,!2) provide much more informa0

I

CSC

tion about the set a than does the cone y(x,a), but they are much more !"f

difficult to construct. Thus, if a is the intersection of a number of sets

n 'Ii

iE 1:N

then y(x,(2) = n y(x,ai) and, thus, if we know the cones y(x,Qi), it is not iE 1:N

difficult to find y(x,Q). At the same time, the equalities K(x, (2) =

r) K(x, (2i ), T (x, Q) =

iE 1:N

o T (x, Qi )

ie 1:N

(/S

do not necessarily hold and it is not easy to find the conditions under which they do. Observe in passing that

r x, v d2j) iE 1:N

but that y (x,

v ai

iE 1:N

v T (x,Qi) ,

ie 1:N

does not always coincide with

u y(x9d2i). However,

ie 1:N

extremal problems in which a set a is given as the finite union of its subsets are not encountered too frequently. &n'

There is quite a large class of sets for which cl y(x,.rl) = T(x,a). This class includes all convex sets and "inequality sets," i.e., sets of the form 12 = {x I h(x) < O}, where h is a function satisfying certain natural properties. We shall now describe approximating cones for some special classes of sets.

Chapter 15

180

15.2. First let us consider the case where d2 is a convex set.

Proposition 15.1. Let Q be a convex set and X ED. Then

cone (.- x)

A> 0

T (x, Q) = K(x, S2) =

cl

42-x) ,

y(x, a)

(15.6)

.

Proof. Let g E cone (a-x). Then aog E (Q--x) for some ao > 0 or, equivalently, x+ago E fl. In addition, since X E !2, we have x+ag = [/3(x+ao g) + (1--,(3)x] E !2

V a E [O,ao ]

,

where ao

Thus g E y(x,Q), i.e., cone (fl-x) c y(x,Q). The reverse inclusion follows from (15.5), proving (15.6). Using (15.4), (15.5) and (15.6), we arrive at the equality cl y(x,Q) = T(x,Q). Let us prove that K(x,Q) = T(x,Q). Consider the function psl (y) = p(x,a) which represents the distance of a point y from the set 42. The convexity of

4 implies that po is a convex function (this can easily be checked by the reader). Therefore pn is directionally differentiable at the point x and hence, in particular, we have

111

=

r-'

'x

l i m a[pa( 1 x+ag)-pa (x)]

a-,+0

r-'

l i m 1 [pa Cx+a ( -,On g) p.(X) () a+ a

V g

E Rn .

a)04.4

From (15.2) and (15.3) we deduce that K(x,Q) = T(x,12). 0

",p

In what follows, the cone y(x,Q) will be called the cone tangent to a convex set !2 at a point x. It will be denoted by Kx(Q) or simply by KX (where this does not cause confusion). t10

Let us now describe the cone conjugate to KX in the case where 42 is a convex set.

Cones of Admissible Directions and a Nondegeneracy Condition

181

Definition 15.1. Let .fl c R" be a convex set and x e .Cl. The cone

NX = NX (i2) _ {v E R' I (v,x) = ma x (v,y)}

yEl)

is called the cone normal to the set a at the point x. In other words, v E Nx(A2) if (v,x) = pa (v), where pa is the support function of the set D. Nx is a cone since, if (v,x) = pD (v), we have (AV9 X) = A (V) X) = APD (V) = PD (AV)

VA 2t 0

.

0

Let us prove that NX is a convex cone. It is necessary to show that if v1,v2 E NX, then VI +V2 E N. We have

(vi+v2,x) = (vi,x) + (v2,x) = pn (vi) + pn (v2) >_ pry (vi+v2)

.

At the same time, since x E .Q, we have

(v1+v2,x) < ma x (v1+v2,y) = p.Q(v1+v2)

Yen

Thus, (vl+v2,x) = pQ (vi+v2), which implies that v1+v2 E Nx.

Proposition 15.2. The following relation holds:

NX = - KX

.

Pro o f. (i) Let I E (KX ). Applying the equality Kx = cone (Q-x) yields (v,y-x) 0, we have (v,x+ag) < (v, x), i.e., (v,g) < 0. This implies that v E (-Kr). 0

15.3. Consider the case where d2 = {y E X I h(y) 0 there exists an element q such that ,.p

IIq-g II < e,

hr(q) < 0

.

Put

max (v,y), p2(y) =

...

p, (y) =

vE a h(x)

max (v,y) vE [-a h(X)]

V Y E Rn

The functions p, and P2 are sublinear and

hz(Y) = Pi(Y) -P2(Y)

V y E R"

.

Therefore pi (g) = p2(g)-

Since the pair [dh(x), -dh(x)] is in a general position, there exists a v

such that vE

Gg (dh(x))

v 4 Gg

(15.10) (15.11)

Making use of (15.11), the separation theorem and formula (15.9), we find an element w E Rn such that

(v,w) >

max v'E G (ah (X)) 9

(v',w) _ (p1)(w)

Chapter 15

184

Therefore for sufficiently small a > 0 we have (V,W) >

I (pj(g+aw)-pj(g)]

or, equivalently,

Pt(B+ax') < P1(g) + a(v,w) .

N-+

r-1

(15.12).

From (15.10), the definition of the max-face and the equality p1(g) = p2(9), we deduce that

pi (g) + a(v,w) = (v,g) + a(v)w) = (v, g+aw) < p2(g+aw) . Putting q = g + aw, where a is sufficiently small, so that 11q-g11 < e, we have from (15.12) that h,r (q) = pI (q) - P2 (q)
0 and e > 0 such that

x+av 4 Sl VaE (O,ao), VvE B'C (g),

r-.

h(x+av) # 0 if 0 < a < ao,

II v-g II < e

.

We shall show later that in this case h(x+av) always has the same sign. For the sake of argument one needs to assume that h(x+av) >_ 0. Then

h(x+av) = h(x) +ah'xv +oav = a (h'(x,v) +>_ 0. Dividing by a and taking the limit as a - +0 yields Al

h'(x, v) > 0

V V E BC (g)

.

However, these inequalities contradict the assumptions of the proposition. We obtain a contradiction also in the case h(x+av) < 0.

Let us now prove that the function h(x+av) always has the same sign if a E (O,ao], V E B,(g). Assuming the contrary, we find a1,a2 E (O,ao] and VI V2 E B, {g} such that

h(x+al vl)

< 0,

h(x+a2v2) > 0

.

Let

0(t) = h(x+talvl +(1-t)a2v2)

V t E [0,1]

.

The function 0 is continuous and is of different sign at 0 and 1. Therefore O(to) = 0 . Then

toai vi + (1-to)a2v2 = av 9

`"'

there exists a point to E (0,I) such that (15.17)

Cones of Admissible Directions and a Nondegeneracy Condition

187

where

a = to al + (1-to)a2 E V=

ta1

vi +

(1

a)a2

(O,ao]

v2, E BE(D)

But (15.17) implies that h(x+av) = 0, which contradicts the assumption that

g 4 T(x,a). 0

Theorem 15.2. Let a function h be uniformly quasidifferentiable at a point x E .Q, where .f2 is described by (15.15). If the pairs [d h(x), -N(x)] and [dh(x), 4h(x)] are both in general positions, then relation (15.16) holds.

P r o o f. It follows from the proof of Proposition 15.3 that if the pair [dh(x), -dh(x)] is in a general position, then for any e > 0 there exists an element v E B,(g) such that hX (v) < 0. _

Analogously, if the pair [dh(x), -dh(x)) is in a general position, then for some W E BB (w) we have (--h)X (w) > 0 or, equivalently, hX (w) < 0.

To conclude the proof it is sufficient to refer to Proposition 15.4. 0

Remark 15.3. Equality (15.16) is automatically satisfied under the same conditions as equality (15.14) (see Remark 15.2.).

Chapter 16 NECESSARY AND SUFFICIENT CONDITIONS FOR AN EXTREMUM

"tea

16.1. It is convenient to describe necessary conditions for an extremum in terms of the cones approximating the set over which optimization is to be performed.

0:.

The following necessary conditions follow directly from the definition of an extremum point.

Theorem 16.1. Let a function f defined on an open set X c R" be directionally differentiable at a point X E X. Let d2 be some subset of X and x E .fl. Then for a point x to be a minimum point of f on Sl it is necessary that min f(g) gE r (x,.?)

=0.

(16.1)

For a point x to be a maximum point off on !2 it is necessary that max f(g) gE Y (X,a)

=0.

(16.2)

If, in addition, the directional derivative fx' is continuous in g, then the cone y(x,s2) in (16.1) and (16.2) can be replaced by its closure cl r(x,s2).

ate'

P r o o f. We consider only the case of a minimum. Let g c= y(x,Q). Then for sufficiently small a > 0 we have x+ag E d2 and therefore f(x+ag) > f(x). This inequality implies that f(g) > 0. Since, in addition, 0 E y(x,Q) and f(0) = 0, condition (16.1) must be true. 0 116

Definition 16.1. A point x E .C2 satisfying (16.1) is called an inf-stationary point of the function f on the set d2.

A point x E S2 satisfying (16.2) is called a sup-stationary point of the function f on the set d2.

188

Necessary and Sufficient Conditions for an Extremum

189

Theorem 16.2. Let all the conditions of Theorem 16.1 be satisfied. Assume that the function f is uniformly directionally differentiable at a point x E Sl

and that its derivative f(g) is continuous in g. Then for a point x to be a minimum point off on Sl it is necessary that min f(g) gEr(X,Q)

=0

(16.3)

.

For a point x to be a maximum point off on dl it is necessary that max

gEr(X,Q)

f(g) = 0

(16.4)

.

P r o o f. We shall again only consider the case of a minimum. Take any g E r(x,dl). Then there exist sequences {ak } and {9k} such that

gk E R' , gk -4 g, x + ak gk E 12 . m

ak -4 +0,

We have

.fix) _ f(x+ag) _ .f(x) + akfX (8k) + 0(ak9k) , where IIvO 0

IIV11

Taking the limit yields fX (g) > 0.

O

o

Corollary 16.1. Let f be Lipschitz on S5(x) ri 12, where 8 > 0. If

f(g) > 0

V g E r(x,Q), g# 0,

then x is a strict local minimum of the function f on the set 12.

P r o o f. We must show that there exists an r > 0 such that

f(y) > f(x)

V Y E Sr (x) r1 Q Q.

Assuming the contrary, there must exist a sequence {xt } such that

X.

x, xt E 12, f(x) < f(x) -4 -400

d1

Chapter 16

190

We can assume without loss of generality that

xi -x ai

gi

-4

:400

g,

where ai = Ilxi x I I . Clearly, g E T(x,Q), u g h I = 1. We have

f(x) -f(x) = f(x+ag) -f(x) = [f (x+ai g) f (x)] + [f (x+aigi) f (x+ai g) ]

The function f is Lipschitz and therefore jf(x+a1g1) -f (x+ai g) J < Lai II gi-g II

Since 1

[(

:

-f(x)] 'a

i-4

f(g),

we have 1

On the other hand, our assumptions.

1

a

.

Cf(xt) -f(x)]

f;(g).

[f(x)-f(x)] < 0 V i. Thus fX(g) < 0, which contradicts

00c

ado

Remark 16.1. Clearly, Theorems 16.1 and 16.2 are still valid if we replace the cones r(x,Q), cl r(x,sl), T(x,Q) by subcones of these cones. For example, if T(x,sl) is not convex, it is more convenient to use convex subcones of T(x,Q) rather than T(x,sl) itself. Now consider the case where the function f is quasidifferentiable.

Theorem 16.3. Let a function f be quasidifferentiable at some point x and let K be a convex cone in R2 . Then (i) the relation m i n f,(g) = 0 holds if and only if r7f (x) (= f(x) - K* ;

geK

( ii) the relation ma x f(g) = 0 holds if and only if -df'(x) c df(x) + K* .

geK

Here, as usual, K* is the cone conjugate to K and Df(x) = [fi(x), df(x)] is a quasidifferential off at x.

Necessary and Sufficient Conditions for an Extremum

191

P r o o f. Let

pl (g) =

max (v,g), p2(g) = (v,g) max vEdf(x) VE[-of(x)]

V g E Rn

.

(16.5)

The functions p, and P2 are sublinear and

max (v,g) +

vE d f (x)

min (w,g) = pi(g) - p2(g)

w r: d f(x)

Let m i n f(g) = 0. Then gEK f(g) = p i (g) - p2 (g)

0

VgEK m

f,(g) =

.

ooh

Hence Pi K > P2 K , where PiK is the contraction of the function pi to the cone K. Since P i K > P2 K) we have P2 K Making use of Proposition 5.8, iK we then obtain (16.6)

It is easy to check that this inclusion is equivalent to

dpi - K* D P2 .

(16.7)

Since f (x) and - if (x) are convex compact sets, equation (16.5) implies that pi = f(x), 0p2 = -, if(x). The required inclusion f(x) - K*

(16.8)

-df (x)

then follows from (16.7). BCD

n°'

Next let us prove the opposite inclusion. Let (16.8) hold. Inclusion (16.7) is also satisfied, implying inclusion (16.6), i.e., P i K P2 K This is equivalent to PI K (g) > P2K (g) V g E K, which in its turn is equivalent to f,(g) > 0 V g E K. This proves the first part of the Theorem. The second part can be proved in a similar way. 0 0

The following necessary conditions for the unconstrained case were stated by L.N. Polyakova (see [1201):

Chapter 16

192

Theorem 16.4. Let a function f be quasidifferentiable at X E int Q. Then for point x to be a minimum point off on .fl it is necessary that --df (x) c

f(x).

(16.9)

For the point x to be a maximum point off on Sl it is necessary that

f(x)

df(x)

(16.10)

Pro of. Since X E int £2, the cone r(x,Q) coincides with the entire space Rn, and therefore y*(x,Q) _ {0}. Applying Theorems 16.1 and 16.3 now yields the required result.

16.2. Let a function h be defined and quasidifferentiable on an open set X c Rn. Put

S l = {yE XI h(y) - dh(x), dh(x) - afcx>}

P r o o f. Let h(x) = 0 and let x be a minimum point of f on Q. Consider the function

F(y) = max {f(y) f*, h(y)} , where

f* = f(x) = mi n f(y)

.

YED

The equality F(x) = 0 implies that x is a minimum point of F on R'2. Since

the function F is quasidifferentiable (as the maximum of two quasidifferentiable functions), applying the necessary condition for an unconstrained minimum (condition (16.9)) leads to -9F(x) c d F (x), where

aF(x) = co {afcx>-ah(x), ah(x) - afcx>} aF(x) = df(x) + ah(x)

,

.

`dam

(These regulations follow from the rules of quasidifferential calculus; see Chapter 10).

11

Necessary and Sufficient Conditions for an Extremum 16.4. Let Sl = {y I h(y) 0. In this case, UX = [O,v x]. The star shape of U with respect to its zero follows immediately from the above and is equivalent to either the relation I

AU c U

V2E [0,1]

or the relation

AU D U V A z 1

.

Let .fl be a set in Rn and let 0 E int .fl. The function ICI

IxJn

= inf {A > 0 1 x E ASl}

(19.1)

is called the gauge of the set d2 (or the Minkowski gauge function). If d2 is v)'

convex, then the gauge coincides with the gauge function familiar from convex analysis; if d2 is a ball, then the gauge is a norm corresponding to CS`

this ball.

Theorem 19.1. Let s be a functional defined on Rn. The following propositions are then equivalent: (a) the functional s is positively homogeneous, nonnegative and continuous; (b) s coincides with the gauge of a star-shaped set .fl, where

Chapter 19

238

4-:

P r o o f. (a) Let s be a positively homogeneous, nonnegative, continuous functional and let £2 = {x I s(x) < 11. Then

It is easy to check that the set d2 is star-shaped. Indeed, the continuity of s implies that the set £21 = {x s(x) < 1 } is open and therefore zero is an interior point of £2. Let x # 0. If s(x) = 0, then the entire ray PX belongs to d2i , i.e., to the interior of £2, and therefore does not intersect its boundary. If s(x) > 0, then the intersection of the ray PX and the boundary of d2 consists I

+,,

of only one point Ax, where A = 1/s(x). At this point, s(2x) = 1. (b) Let s coincide with the gauge of a star-shaped set £2. Since £2 is star-

shaped, it follows from the definition that s(x) < 1 if x E £2 and that if s(x) < 1, then x E £2. Since £2 is closed, we have £2 = {x I jxi < 1 }. It is clear that the gauge is both positively homogeneous and nonnegative.

Let us now show that the gauge is continuous. Since the gauge is positively homogeneous, it is enough to check that the set B1 = {x (xi < 1} is closed and that the set B2 = {x (xI < 1 } is open. (Indeed, in this case the sets {x I lxi < Al are open for all A > 0 and the sets {x I (xi < Al are closed for all A > 0. Therefore the gauge is continuous on the whole space.) B1 is closed since it coincides with D. Suppose now that B2 is not open, that x E B2 and that there exists a sequence {xk } such that xk -4 x, ixk I >_ 1. We can assume I

+''

I

ate

0

without loss of generality that lim (xk I = V >_ 1. Take yk = xk/Ixk J. Then (yk I = 1

U4)

ItU

ate

and therefore yk is a boundary point of £2. Since yk - x/v is also a boundary point of £2. If x # 0, it follows that the ray PX intersects the boundary of d2 at at least two different points, x/ixi and x/v, which is impossible.

call

4-.

If lxi = 0, then the ray PX lies entirely in d2 and (from the definition of "star-shaped") does not contain any boundary points of £2. Thus the gauge of a star-shaped set must also be continuous and the theorem is proved. 0

{x

I

0.."

Remark 19.1. Since the gauge is continuous and int £2 coincides with the set

lxi < 11, £2 must be regular, i.e., it coincides with the closure of its

interior. Q,,

Let us denote by S the set of all star-shaped subsets of the space Rn, and by K the family of all nonnegative, continuous, positively homogeneous functions defined on Rn. The following propositions may then be deduced.

Proposition 19.1. A mapping i/: S -4 K which associates a gauge with every star-shaped set is a bijection.

The set K is a cone in the space C0(Rn) of all continuous, positively ...

homogeneous functions defined on Rn. Since every function from Co(Rn) is

Star-shaped Sets and Their Applications in Nonsmooth Optimization

239

I

0

0

'+~

0330
0 such that IIx II > r for any U E I and any x such that IxIU = 1. This follows immediately from condition (i). Since X is compact, there exists an R < 00 such that IxI 0 there exists a 5 > 0 such that the relations

IIx II

=

Ily II = 1 and 11x -y II < Simply that Ixlu - Iymu I

< e V U E E.

Putting e' = re, we shall find a 8' corresponding to e' such that (ii) is satisfied. Let 5 = 8'/R, take elements x and y such that IIx II = Ily II = 1, top

11x -y II < 8 and choose any U E E. Let

A=

1

,

IxIU

x = 2x,

y

= Ay .

It is clear that 11x'11 = 11y'11 = A. Since Ix'I U = 2I xIU = 1, we have x E U C X and hence A = 11x'11

R.

Thus

11x' -y'11 = A11x y1I < R8 _ 8' From (ii) we have y' E dU , i.e., LY'Iu - IxIU

< C' .

Star-shaped Sets and Their Applications in Nonsmooth Optimization

249

Since Ix'IU = 1, we have Ilx'll = A > r and therefore 1

- lulu _

I1XPIU

I

- LY 'U

I

r E' = e


0 and let e' < dIR. We shall find a 8' such that the relations IIx' II

=

= 1,

Ily' II

<S

Ilx' y' II

imply that IIx'IU

VUE

- VIu < .6' I

E.

Choose a 5 such that 5 < (e-Re')r, 5 < rS'. Let U EE and X E dU, i.e., Iu = 1. Assume also that

ily x II

E S and put x' = x/ Ilx ii, y' = y/ ily II. The

inequality 2

IlyII

Iix ii

II

Ilx II

Ilx y 112 1 Ily 11

implies that iix' y' ll < 5'. At the same time, since ily' ii = 1, we have Iy'IU

IlxlU

- lulu = I

1

r and hence

IIX'IU . lix u - Iy, JU uy" 111

I[x'IU' lix II

0 .

Now let us consider star-shaped convex sets U1 and U2. Since 0 E int Ut, the polar U is compact. Since the gauge I It of the set U1 coincides with the support function of the polar Ui*, the following relation holds: 1x11 - Ix12 =max ° (l,x) - m a x ° (l,x) I EU1 I EU2

max ° (l,x) + min I

E U1

°

IE [-U2]

(l,x)

.

(19.10)

Star-shaped Sets and Their Applications in Nonsmooth Optimization

251

Let yf and 'P be mappings defined by formulas (19.5) and (19.9), respec-

tively, let a be an element of the space T containing the pair (U1,U2), and let /3 be an element of the space M containing the pair (Ul , -UU ). It follows from (19.10) that yia = (Pf3 and hence /3 = (0-1 ;V) (a). The operator it = 0). It therefore seems useful to have some analogous approximation for quasidifferentials. Here we describe an approach suggested by V. Gorokhovik [61,62].

20.1. Let a function f defined on an open set X c Rn be directionally differentiable at a point x E X, and let e > 0. We say that a pair of compact convex sets DEf(x) = [r7 f(x), aEf(x)] is an e-quasidifferential of the function f at the point x if

x f(g) -

(v,g) + min

max

(w,g) ski

W E def(x)

L''E def(x)

where B is the unit ball. Equivalently, we can define De f (x) _ [dE f (x), ^..,

de f(x)] as an e-quasidifferential of f at x if max

[yE def(x)

(v,g) + min

(w,g)

WE def(x)

< eIIgII

IV g Rn

.

ski

x (x) -

ago

Clearly, if a pair [de f (x), de f (x)] is an e-quasidifferential of f at x, then any equivalent pair (as defined in Chapter 6) is also an e-quasidifferential of f at x. Thus, an e-quasidifferential is an element of the space of convex sets. Note that even in this space an e-quasidifferential is not uniquely defined, since even nonequivalent pairs may be e-quasidifferentials of f at x. We say that a function f is approximately quasidifferentiable at x if f has

an e-quasidifferentiable at x for any e > 0. It is clear that any quasidiffer4-4

entiable function is also approximately quasidifferentiable, and a quasidifferential is also an e-quasidifferential for all e > 0. Theorem 20.2 shows that the class of approximately quasidifferentiable

3a)

functions is quite rich. But first let us recall a fundamental result from functional analysis. 254

Approximate Quasidifferentiability

255

Theorem 20.1 (Stone-Weierstrass theorem). Let Z be a set of functions defined and continuous on a compact set S such that (1)

Z is a linear set;

(ii) if zl, z2 E Z, then the functions

z(x) = max {z1 (x), z2(x)} and z(x) = min {zl(x), z2(x)} also belong to Z; cow

(iii) Z contains the identity element, i.e., the function e defined b y the equality e(x) = 1 V X E S. coo

If a function h is continuous on S, then there exists a sequence {zk Zk E Z V k} which converges uniformly to h.

This theorem is proved, for instance, in [87].

Theorem 20.2. Let a function f be defined on an open set X c Rn, X E X. The function f is approximately quasidifferentiable at the point x if and only if f is directionally differentiable at x and its directional derivative f(g) is %o'

continuous (as a function of g).

P r o o f. (i) Let L be the set of all functions defined on R'z which can be represented as the sum of a sublinear function and a superlinear function. Let Z be the trace of L on the unit sphere S = {g E R'z II g II = 11 (z E Z if and only if there exists a function e EL such that z(g) = 1(g) V g E S). Since L is a linear space, Z is also linear. From Lemma 6.1, the set L satisfies condition (ii) of Theorem 20.1, and therefore Z also satisfies this N-,

I

condition.

Finally, the identity element (the function e -the trace of the

sublinear function p(g) = II g II on S) belongs to Z. Thus all the conditions of Theorem 20.1 are satisfied.

'a'

p,,

Let a function f be directionally differentiable at x and the function f(g) be continuous in g. It follows from Theorem 20.1 that for any e > 0 there exists a function Z E Z such that Lfx'(g) - Z(g) I

<EVgES.

Let a function I E L be such that its trace on S is z and consider a N

sublinear function p and a superlinear function q such that p + q = I. Then

Chapter 20

256

This relation immediately implies that the pair [dam, aq] is an e-quasidifferential of the function f at x and therefore f is approximately quasidifferentiable. (ii) Let a function f be approximately quasidifferentiable at x. It follows from the definition of approximate quasidifferentiability that for any natural number k there exists a funcrion ek EL such that

[f(g)

-

Vg

k Ilg II

ek(8)I

.

This means that for any R > 0 the sequence {.2k(g)} converges to f(g) uniformly on BR (0), where

BR(O) = {g E Rn I

Ilg II

_< R}

,

i.e., fX is the uniform limit of a sequence of continuous functions and therefore fX is also continuous. This completes the proof. 0 20.2. We shall now give a theorem on the e-quasidifferentiability of a composition.

Let functions hl,...,hm be defined on an open set X c Rn and where ej > 0. Let De .hi(xo) _ [de hi(xo), do .hi(xo)] be an e1-quasidifferential 1

of the function hi at a point xo, and H(x) _ (hl (x),..., hm (x)}. Consider a 4-.

function f defined on an open set in Rm containing the point yo = H(xo) and such that f is uniformly directionally differentiable at yo. Suppose that for some > 0, f has an quasidifferential:

Dj(yo) _ [d .f(yo), d.f(yo)] Put CO

_ em+ IIeIIC,

where

M=

max IIHX (v) II

=

IIvII

where m

WW=

\.y

(0) =

[V i (2i+pi) - vi2i -" vi,ui ]; V = (V19 ... )VM) E de(0),

r..

i =1

Ai E dei(O), Pi E dei (O)

,

m

do (0) =

ww=

[Vi(/Z.i+,lli) + v /.i + VI;ui]; v = (Vi,...,Vm ) E de(0}, i =1

Ai E

dei(O),,Ui E dei(0)

and

v' < v < v" V V E de(0) u de(0)

.

Since the pair

[A,B] _ [d.f(yo), d.f(yo)] is a quasidifferential of e at zero and the pair [Ai,Bi] = [d .hi(xo), d£ .hi(xo)]

is a quasidifferential of ei at zero, &(0) coincides wih the set D and 9O(0) coincides with the set D. 0 Using Theorem 20.3, it is not difficult to derive formulas for the e-quasidifferentials of a sum, a product, a quotient, a pointwise maximum and a pointwise minimum. For example, let us compute the e-quasidifferential of a sum. Consider the function f(y) = yi + --- + y11. In this case ! = 0, d N f (y) = (1,...,1) and d f (y) = 0.

Chapter 20

260

20.3. A point x0 is called a local e-minimum point of a function f if there exists a neighborhood N(x) of x0 such that

f(xo) < f(x) + Elix-xo II

V x E N(xo), x # xo

.

(20.4)

Here e > 0. Clearly any point xo satisfying (20.4) is a local minimum of the function

h. (x) = f(x) + e II x -

xo II

and, conversely, any local minimum point of h£(x) is a local e-minimum point of the function f. This definition of a local e-minimum point seems to be more natural than

the one used in convex analysis: there an e-minimum point is defined as a point satisfying f(xo) 0 an e-approximation of a minimum point can be rough enough.

Theorem 20.4. Let a function f defined on an open set X c R" be directionally differentiable at a point X0 E X, and let its directional derivative f'(xo,g)

be continuous as a function of direction. Let e > 0 and let x0 be a local .6-minimum point of the function f. Take El > 0 and let

Del.f(xo) =

[de1f(xo), de1f(xo)]

be an El-quasidifferential off at x0. Then de1f(xo) + (e+e1)B D [--a£1 f(xo)]

.

P r o o f. Let .e(g) =

max vE a£1f(XO)

(v,g) + min

(w,g)

.

wE d£1f(xo)

Then Lf'(xo,g) - 1(g) I _ -e and therefore

1(g) =

(v,g) + min

max

(w,g) >- - (e+e1) _ - (e+e) II g II

wE d£1f(xo)

VE d£1f(xo)

VgE S. Putting e' = e + e1, we can rewrite this inequality in the form e' II g II

+

max

(v,g)

-min

>_

WE [-d£1 f(xo )]

vE d£1f(xo)

(w,g)

or, equivalently, max

VE [d£1f(xo)+£'B]

de , g(xo)

(v,g)

-min

WE [-d£1f(xo)]

+ eB c: [-d. , f(xo) 1

(w,g)

,

0

Corollary 20.1. If xo is a local minimum point off, then ,.f(x0) + E1B

[-df1.f(x0)]

b El > 0

.

Remark 20.1. A point xo is called a local e-maximum point of a function f if there exists a neighborhood N(xo) of xo such that

f(x0) > f(x) + Ellx-xo II

V x E N(xo), x # xo

.

0

In an analogous way it is possible to prove that if xo is a local E-maximum point of f, then

a f(xo)

[-d

+ (e+ei)B] .

In particular, if xo is a local maximum point, then

a f(x0) c [--d£,f(xo) + e1B]

V el >- 0

.

262

Chapter 20

20.4. Constrained optimization problems can be treated as in Chapter 16. The 4.,

following analogue of Theorem 16.3 holds.

Theorem 20.5. Let a function f defined in a neighborhood of a point x be directionally differentiable at the point x, and let f'(x,g) be continuous in g. +.4

Let K c Rn be a convex cone. Then ,.,

(i) the equality

min f'(x,g) = 0 g EK

holds if and only if

Ve>0;

ae.f(x) + eB - K* (ii) the equality

m a x f '(x,g) =0 g EK

holds if and only if

df f (x) +,6B + K*

[-d f (x)]

Ve>0.

P r o o f. We shall consider only case (i). Let e > 0 and

f'(x,g) >_ 0 V g E K. Al

(20.5)

Then max r01

vE df (x)

(v,g) -

max

WE [- If (x)]

- 0VgEK

(w,g) + e I i g I I >

Put

pi(g) = max vE

p2(g) =

(v,g) + e II g II =

max W r= [-;)f W]

max w r= [af(x) +£B ]

(X)

(w,g)

.

(v,g)

.

Approximate Quasidifferentiability

263

As in Theorem 16.3, it can be shown that

dpi - K* z

,

which implies that !2f(x) + eB - K*

VE>0

(20.6)

.

Arguing the same way as in proving Theorem 16.3, it is possible to prove that (20.6) implies (20.5). D

The development then proceeds as in Chapter 16. Let f attain its minimum on the set

at a point xo. Let f and h be directionally differentiable at xo and let their derivatives

f'(xo,g) and h'(xo,g) be continuous in g. Fix e > 0 and let

Y£1

(w,g) < ell g II g I max (v,g) + min vE d£h(xo) WE d£h(xo)

Assume that Y i is not empty. Then

h'(xo,g)