Optimality and Stability in Mathematical Programming (Mathematical programming study)

MATHEMATICAL PROGRAMMING STUDIES

Editor-in-Chief R.W. COTTLE, Department of Operations Research, Stanford University, Stanford, CA 94305, U.S.A. Co-Editors L.C.W. DIXON, Numerical Optimisation Centre, The Hatfield Polytechnic, College Lane, Hatfield, Hertfordshire ALIO 9AB, England B. KORTE, Institut fur Okonometrie und Operations Research, Universitiit Bonn, Nassestrasse 2, D-5300 Bonn I, W. Germany T.L. MAGNANTI, Sloan School of Management, Massachusetts Institute of Technology, Cambridge, MA 02139, U.S.A. M.J. TODD, School of Operations Research and Industrial Engineering, Upson Hall, Cornell University, Ithaca, NY 14853, U.S.A. Associate Editors E.L. ALLGOWER, Colorado State University, Fort Collins, CO, U.S.A. V. CHVATAL, McGill University, Montreal Quebec, Canada J.E. DENNIS, Jr., Rice University, Houston, TX, U.S.A. B.C. EAVES, Stanford University, CA, U.S.A. R. FLETCHER, University of Dundee, Dundee, Scotland J.-B. HIRIART-URRUTY, Universite de Clermont II, Aubiere, France M. IRI, University of Tokyo, Tokyo, Japan R.G. JEROSLOW, Georgia Institute of Technology, Atlanta, GA, U.S.A. D.S. JOHNSON, Bell Telephone Laboratones, Murray Hill, NJ, U.S.A. C. LEMARECHAL, INRIA-Laboria, Le Chesnay, France L. LOVASZ, University of Szeged, Szeged, Hungary L. MCLINDEN, University of Illinois, Urbana, IL, U.S.A. M.W. PADBERG, New York University, New York, U.S.A. M.J.D. POWELL, University of Cambridge, Cambridge, England W.R. PULLEYBLANK, University of Calgary, Calgary, Alberta, Canada K. RITTER, University of Stuttgart, Stuttgart, W. Germany R.W.H. SARGENT, Imperial College, London, England D.F. SHANNO, University of Arizona, Tucson, AZ, U.S.A. L.E. TROTTER, Jr., Cornell University, Ithaca, NY, U.S.A. H. TUY, Institute of Mathematics, Hanoi, Socialist Republic of Vietnam R.J.B. WETS, University of Kentucky, Lexington, KY, U.S.A. e. WITZGALL, National Bureau of Standards, Washington, DC, U.S.A. Senior Editors E.M.L. BEALE, Scicon Computer Services Ltd., Milton Keynes, England G.B. DANTZIG, Stanford University, Stanford, CA, U.S.A. L.V. KANTOROVICH, Academy of Sciences, Moscow, U.S.S.R. T.e. KOOPMANS, Yale University, New Haven, CT, U.S.A. A.W. TUCKER, Princeton University, Pnnceton, NJ, U.S.A. P. WOLFE, IBM Research Center, Yorktown Heights, NY, U.S.A.

MATHEMATICAL PROGRAMMING STUDY19 A PUBLICATION OF THE MATHEMATICAL PROGRAMMING SOCIETY

Optimality and Stability in Mathematical Programming

Edited by M. GUIGNARD

August (1982)

NORTH-HOLLAND PUBLISHING COMPANY - AMSTERDAM

© The Mathematical Programming Society, Inc. -1982 All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior permission of the copyright owner. Submission to this journal of a paper entails the author's irrevocable and exclusive authorization of the publisher to collect any sums or considerations for copying or reproduction payable by third parties (as mentioned in article 17 paragraph 2 of the Dutch Copyright Act of 1912 and in the Royal Decree of June 20, 1974 (S. 351) pursuant to article 16b of the Dutch Copyright Act of 1912) and/or to act in or out of Court in connection therewith.

This STUDY is also available to non-subscribers in a book edition.

Printed in The Netherlands

PREFACE The last decade has seen a number of improvements regarding the theory of optimality in mathematical programming, in particular concerning characterization of optimality for broad classes of problems. Increasingly sophisticated types of optimality conditions are studied and more general types of problems can now be considered: nonsmooth, nonconvex, nondifferentiable, etc. New tools have been developed which permit these advances, such as generalized gradients, subgradients, generalized equations, cones of directions of constancy, faithful convexity, normal subcones, semiconvexity, semidifferentiability, just to name a few. Characterizations of optimality have been proposed with or without constraint qualifications. At the same time, linear complementarity theory grew out of infancy both from a computational and a theoretical viewpoint. For either type of problem, characterizations of the existence of bounded solutions and/or bounded sets of multipliers and the effect of small perturbations of the data on the solutions, the multipliers and/or the optimum have been thoroughly studied, and considerable progress has been made. This Study is a collection of papers dealing with the various aspects of the theory mentioned above. The following is a brief description of these papers which tries to place them into perspective. They are arranged by alphabetical order of authors. Sufficient conditions for a globally exact penalty function without convexity by Mokhtar S. Bazaraa and Jamie J. Goode. Penalty functions are used for transforming a constrained nonlinear programming problem into a single unconstrained problem or into a sequence of unconstrained problems. This paper extends earlier results on the existence of an exact penalty function when there are additional set constraints. It also proves the existence of a globally exact penalty function in the nonconvex case under appropriate conditions. Optimality in convex programming: A feasible directions approach by A. Ben-Israel, A. Ben-Tal and S. Zlobec. This is a survey of the optimality theory for convex programming developed by the authors. Optimality is characterized by the absence of usable directions, yielding primal optimality conditions. These multiple conditions can be used to improve the classical feasible direction methods. Also they can be restated as dual optimality conditions, dealing with the existence of multipliers. Optimality is thus equivalent to a family of conditions, which can actually be reduced to a single one associated with the minimal index set of binding constraints. The paper then describes a parametric representation of nonascent directions to v

vi

Preface

characterize optimality, and this approach is used in a parametric feasible direction method, less expensive than the former ones. Finally the authors study arbitrarily perturbed convex programs. A unified theory of first and second order conditions for extremum problems in topological vector spaces by A. Ben-Tal and J. Zowe. The problem considered here is to characterize weak minima of quite general optimization problems in ordered topological vector spaces. A general necessary condition is proved and first and second order conditions are derived for nondifferentiable as well as differentiable problems. Sections of the paper are devoted to the computation of relevant sets of directions of decrease and of tangent directions. In particular the objective function is vector valued and because "inequality" constraints are defined over a convex cone, the concept of active constraints must be extended. Characterization of optimality without constraint qualification for the abstract convex program by J.M. Borwein and H. Wolkowicz. The abstract convex program consists in minimizing a real valued function over a convex set subject to cone-defined constraints. The authors prove a "Standard Lagrange Multiplier Theorem". They can characterize optimality without using any constraint qualification and do not require that the infinum be attained, or that the interior of the cone or the intrinsic core of the feasible set be nonempty. Several kinds of optimality criteria are given, using directional derivatives, subgradients and the Lagrangean function. Several examples and applications illustrate the theory and include semi-infinite programs and linear operator constrained programs. Differential properties of the marginal function in mathematical programming by J. Gauvin and F. Dubeau. The mathematical programming problem studied in this paper contains a vector of parameters or perturbations in the objective function and in the constraints. The marginal function is the optimal value of the program considered as a function of the perturbations. The paper analyses the differential properties of this function locally in the vicinity of a point where it is finite. The lower and upper Dini directional derivatives are first shown to be bounded from above and from below under some regularity assumption. The marginal function is thus locally Lipschitz and the authors study its Clarke generalized gradient. If among other things the gradients of the binding constraints are linearly independent the marginal function is proved to be Clarke regular. Refinements of necessary optimality conditions in nondifferentiable programming II by J.-B. Hiriart-Urruty. This paper considers the minimization of a locally Lipschitz function over a closed subset of a finite dimensional real space, defined by one or a finite number of equality constraints. Necessary conditions for a local minimum are described, in terms of normal subcones. The first part of the paper is devoted to the study of the relative generalized Jacobian matrix of a locally Lipschitz function and of

Preface

vii

the normal subcone to a set defined by equalities. These tools are then used to obtain estimates of the normal subcones appearing in the optimality conditions. Necessary conditions for e-optimality by P. Loridan. The object of this paper is to study optimization with errors. One considers the minimization over a nonempty closed subset of a real Banach space of a real valued functional which is bounded from below and l.s.c.. The author derives necessary conditions for regular approximate solutions stationary up to e. If the constraint set is defined by a finite number of equalities and inequalities, and if the functional and the constraint functions are locally Lipschitz, generalized Kuhn-Tucker conditions up to e are proved with no constraint qualification. Finally the concept of Lagrangean duality is extended to duality up to e for problems without equality constraints. Characterizations of bounded solutions of linear complementarity problems O.L. Mangasarian. This paper studies the solution set of an LCP. Eight constraint qualification or stability conditions on the feasible region are shown to be equivalent without any assumption on the matrix of the problem. If this matrix is copositive plus, these conditions and seven others are equivalent characterizations of problems with bounded solutions. Among these is Slater's constraint qualification as well as a condition that can be checked by solving a simple linear program. Another one is a stability condition that for all small perturbations of. the data which maintain copositivity plus, the perturbed LCP's have solutions which are uniformly bounded. Finally for an LCP with a copositive plus matrix, the existence of a nondegenerate vertex solution is proved to be sufficient for the solution set to be bounded. On regularity conditions in mathematical programming by J.-P. Penot. This paper unifies and relates a number of recent results. The problem considered is the minimization of a real valued function over a subset of a Banach space defined by implicit (h(x) E C) and explicit (x E B) constraints. The qualification conditions under consideration correspond to generalized KuhnTucker conditions expressed in terms of the explicit constraint, using tangent and normal cones and tangential approximations. The author studies essentially three types of conditions, a linearizing condition, a transversality condition, and a variant of the latter, and shows how they are related. The case of finite dimensional constraints is treated separately, as the results are simpler. The infinite dimensional case is considered next. All of this sheds a new light on the relationships between the (more or less) classical constraint qualifications and Robinson's condition which is another form of the variant of the transversality condition. Generalized equations and their solutions, Part II: Applications to nonlinear programming by S.M. Robinson. The problem considered is the minimization of a real-valued function of n real variables, subject to explicit constraints and cone-defined constraints. Under

viii

Preface

suitable regularity and differentiability conditions, necessary optimality conditions involving multipliers can be written and they can be viewed as a generalized equation. Given a local minimizer, for sufficiently smooth perturbations of the constraint and objective functions, the set of local stationary points (i.e. satisfying necessary optimality conditions) is nonempty and continuous. The set of multipliers is also shown to be u.s.c. Finally if some polyhedrality assumptions hold, the distance from a stationary point of the perturbed problem to the initial minimizer, or from an associated multiplier to the set of original multipliers, obeys a kind of Lipschitz condition. Kuhn-Tucker multipliers and nonsmooth programs by J.-J. Strodiot and V. Hien Nguyen. This paper deals with the existence of a bounded set of multipliers for the general nonconvex, nondifferentiable locally Lipschitz programming problem in the absence of prior constraint qualification. The authors treat some of the constraints jointly as an implicit constraint so that the constraint qualification of Hiriart-Urruty is satisfied and a bounded set of Kuhn-Tucker multipliers exists. In the convex case, their optimality condition is not only necessary but also sufficient, and the Ben-Israel, Ben-Tal and Zlobec characterization of optimality appears as a special case of these results. Finally, I would like to thank the authors and the referees for their invaluable help during the preparation of this volume. Monique Guignard

CONTENTS Preface

v

(1) Sufficient conditions for a globally exact penalty function without convexity, M.S. Bazaraa and J.J. Goode . . (2) Optimality in convex programming: A feasible directions approach, A. Ben-Israel, A. Ben-Tal and S. Zlobec

16

(3) A unified theory of first and second order conditions for extremum problems in topological vector spaces, A. Ben-Tal and J. Zowe.

39

(4) Characterization of optimality without constraint qualification for the abstract convex program, J.M. Borwein and H. Wolkowicz

77

(5) Differential properties of the marginal function in mathematical programming, J. Gauvin and F. Dubeau

101

(6) Refinements of necessary optimality conditions in nondiflerentiable programming II, J.-B. Hiriart- Urruty . . . .

120

(7) Necessary conditions for e-optimality, P. Loridan

140

(8) Characterizations of bounded solutions of linear complementarity problems, D.L. Mangasarian

153

(9) On regularity conditions in mathematical programming, J.-P. Penot

167

(10) Generalized equations and their solutions, Part II: Applications to nonlinear programming, S.M. Robinson

200

(11) Kuhn-Tucker multipliers and nonsmooth programs, J.-J. Strodiot and V.H. Nguyen . . . . . .

222

ix

Mathematical Programming Study 19 (1982) 1-15. North-Holland Publishing Company

SUFFICIENT CONDITIONS FOR A GLOBALLY PENALTY FUNCTION WITHOUT CONVEXITY

EXACT

M o k h t a r S. B A Z A R A A *

School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA, U.S.A. Jamie I. G O O D E

School of Mathematics, Georgia Institute o[ Technology, Atlanta, GA, U.S.A. Received 2 July 1979 Revised manuscript received 2 December 1980 In this paper, we consider the nonlinear programming problem P to minimize f(x) subject to g~(x) 0 such that f ( x ) > f('2) for each x E X such that x ~ '2, Ilx - '21t < ~, and g,(x) 0- such that O(x, ;t) >i 0(,2, ;t) for each x E X such that IIx < ~. is a strict local minimum for problem P(~,) if and only if there exists e > 0 such that O(x, A) > 0('2, ~,) for each x E X such that x # '2 and IIx -'211 < ~. Next, we need to provide suitable tangential approximations to the set X at a point x ~ X. Following Rockafellar [15], consider the contingent cone K ( x ) and the cone of hypertangents H ( x ) defined below: y E K ( x ) if and only if there exist a sequence {yk} converging to y and a positive sequence {Ak} converging to 0 such that x + )tkyk E X for each k. y E H ( x ) if and only if for each sequence {xk} in X converging to x, there exists a positive sequence {Xk} converging to 0 such that Xk + ?,y ~ X for all x ~ (0, xk). Note that H ( x ) is a c o n v e x cone which is not necessarily closed and that K ( x ) is a closed cone, but not necessarily convex. Further, H ( x ) C K(x). T h e o r e m 2.1 below gives a sufficient condition for the existence of an exact penalty strict local minimum, where the closed c o n v e x cone C(x) is defined by:

y E C(x) if and only if ~Tgi(x)ty ~ 0 so that f o r all )~ >>-,%, ~ is a strict local m i n i m u m [or p r o b l e m P(;~). Proof. Suppose by contradiction to (i) that there exists a sequence {Xk} converging to ~ such that XR# ~, Xk E X , gi(Xk) 0. If ~i"=l g~(x)+ > 0, m then x c=. E ( v ) for any v ,< ~i=1 gi(x)+. T h e r e f o r e , there exists a finite subcover, say A N X C E ( e ) U / ~ for some 9 > 0 . In other words, if x E X is such that .f(x) 0: k E {0} O ~(x*)} not all zero ~/,oV/~

*)

+ ke~x*) A k V f k ( x * ) = 0,

(1.2)

(KKT) The K a r u s h - K u h n - T u c k e r su~icient condition [23, 24, 25]: =l{Ak ~ 0: k E ~(x*)} ~ Vf~ *) + ~

Akvfk(x *) ----0,

(1.3)

kEgP(x*)

where ~(x*) is the index set of binding constants at x*

fk(x*) =.0}.

g,(x*) = {k e ~':

(1.4)

In general, the implications (KKT) ~ x* optimal :~ (F.J)

(1.5)

cannot be reversed, as illustrated by the following example.

Example 1.1 inf

--X,

s.t.

f'(x) ~ 0,

where fl(x)

fx e, x ~0, 1.0,

x ', ' = ' , ' ~< ',

etc.

we denote

D~ELA~O~(x) = {d ~ R": =16l> 0 D f(X + ~ d ) R E L A T I O N f(x), V~ E (0, ~]}.

(2.2)

In particular, we call

D~(x) D~(x)

descent the set of directions of

D~(x)

constancy

of f at x.

non-ascent

If {fk : k E I2} is a set of functions indexed by a set O, we abbreviate

DRELATION(x)L /Jlg~RELATION'[x)" DRELATION(x)L ~'~ D~ELATION(x), kEO where D~ ELAxI~ is interpreted as R". The directions of descent, constancy and non-ascent of c o n v e x functions were studied in [7, 9]; their main properties are collected in the following two lemmas.

A. Ben-Israel et al./ Optimality in convex programming

20

Lemma 2.1. Let f : R" ~ R be convex and let x E dom f = {x E R" :f(x) < oo}. (a) D~(x) is a convex cone; (b) D~(x) = D~(x) U DT(x); (c) D~(x) = {d :f'(x, d) h ( 0 ) > 0 , d ~= d({4}), a direction along a 'flat' part of the feasible set, is chosen as a usable direction. The optimal solution is in the neighborhood of xt = - 0 . 2 5 6 , x2 = 0.650, x3 = 0.999, x4 = 1.060, x5 = 1.064 with the approximate optimal value f~ = 0.854. Let us mention that standard devices for speeding up the convergence of the feasible direction methods are also applicable to the new methods. For other numerical implementations see Section 6.

4. Optimality conditions using the minimal index set of binding constraints By Corollary 2.4, the optimality of x* G S is equivalent to the condition that for every subset O C ~(x*), the intersection < , D~(x*) O D~,).n(x ) n D~(x*) = ~.

(2.7)

Using the projection-reduction idea of [1], it was shown in [2] that only one subset O of ~ ( x * ) has to be checked in (2.7). This subset, which carried all the information needed to check optimality, is the same for all feasible x, and thus is a (global) characteristic of the c o n v e x program (P). It is denoted by ~ : and defined as follows:

Definition 4.1. The minimal index set of binding constraints ~ = of the feasible set

S={xER":

ff(x)~O,

kG~}

(1.1)

xGs~fk(x)=O}

(4.1)

is defined as

~=a={kG~: or equivalently, by (2.3), A

~- = ~

~(x).

(4.2)

Since ~= c ~ ( x ) for every x E S, we denote the complement of ~= relative to ~ ( x ) by ~ 0 for this special j. Thus n+l

n+l

n+ 1

n+l

= ,2,

I

> 0.

We reach a contradiction to (3.6). Necessity: Put S := n7=1 s~ and suppose for the m o m e n t that S+s ~. Then S and S,+1 are non-empty convex sets, S is open and by (3.4), S n S,+1 = ~. Hence by a standard separation theorem we have with a suitable x* E X*, x* # 0, x*x~<x*~

for a l l x E S , ~ E S , + ~ ,

i.e. x* ~ A(S), - x * E A(Sn+O and

We apply L e m m a 3.1 t o s e e that with suitable x ~, E A(S~), i = 1, 2 . . . . . n, x * = x~ + . . . + x*,

(3.5) and (3.6) follow if we put x,+~ : = - x * . Note that, if S = n[':~ s~ = ~, then there exists l~<m < n such that A L ~ s ~ # ~ and n ~ , ~ s j = ~ . We apply the result just proved with n replaced by m and put x*§ . . . . . x*+~ = 0. For later use we add the following:

46

A. Ben-Tal and J. Zowe/ First and second order conditions

Remark. If s , , 0, then x~ ~ 0 in L e m m a 3.2. To prove this let x0E N~--+21Si and suppose (3.5) and (3.6) hold with x~' = 0. Copying the first part of the above proof one gets a contradiction to (3.6). Suppose the sets $1 ..... S,+1 in L e m m a 3.2 are (in addition) cones. Then (3.6) becomes the trivial inequality 0~ 0 such that t-2g(xt) ~ - K - v

for0t O.

(8.4)

Proof. Suppose first, h'(xo) is not onto. Then R(h'(xd) and an arbitrary element in W not belonging to R(h'(xo)) can be separated, that is, there is some

if* E W*, v~* ~ 0, such that ~ * 9 h'(xo)x >t 0

for all x ~ X.

It follows that ~* 9 h'(xo) = 0. (8.2)-(8.4) hold with u* = 0, v* = 0 and w* = if* or w* = - a,*. Hence we may assume throughout the following that h'(xo) is onto. Now, suppose (8.1) holds with a given d. By Propositions 6.1 and 7.1, d satisfies (2.9) in Theorem 2.1. Propositions 7.2 and 6.2 together with (6.3) and (6.4) imply that Q1(x0, d), Q~(x0, d) and Vh(Xo, d) are convex; moreover, Vh(xo, d) is non-empty. If also Qs(x0, d) and Qg(x0, d) are non-empty, then f, g and h are d-regular and Theorem 2.1 applies. The assertion follows from Theorem 2.1 and Proposition 8.1. Now suppose Q~(x0, d ) = I~ (if Qi(x0, d ) = ~, then a similar argument applies). Then the non-empty convex sets {g'(xo)z + ~g"(Xo)(d, d): z E X}

and

- i n t Kg~xo),g'Cxo)d

can be separated and thus there exists some v* ~ 0 in V* such that for all z ~ X, v E K and )t, tx ~>O, v*(g'(xo)z + 89

d)) >- v * ( - k - hg(xo) - Ixg'(xo)d).

It follows that v* E K §

v*g(xo) = O,

v*g'(xo)d = 0

and, finally, v*g"(xo)(d, d) >~O. (8.3) and (8.4) follow if we put u* = 0 and w* = 0. We add a constraint qualification (compare Section 4, Remark 3): h'(x0) is onto and there exists z E X such that CQ(d)

g'(xo)z + g"(Xo)(d, d) E - int K~txo).g' 0 and Yi >10 for i E J(Xo, d) and a functional w* ~ W*, not all zero, such that Yof'(Xo) +

~

yg~(Xo)+ w* 9 h'(xo) = 0,

(9.6)

iEJ(xO,d)

(Y0f"(x0)+

: i~J(xo, d)

yg'[(Xo)+w*'h"(Xo))(d,d)>~O.

(9.7)

A. Ben-Tal and J. Zowe/ First and second order conditions

65

Remark 1. F o r dim W < ~ the range of h'(x0) is always closed and this assumption can be omitted. Theorem 9.3 reduces to a result by Ben-Tal [3]; see also Cox (in Hestenes [13, Chapter 6, Theorem 10.4]) where a similar statement is proved for finite dimensional X and for those d's only satisfying (9.5) with equality. Remark 2. Suppose under the assumptions of Theorem 9.3 that (9.5) holds for d. Then the corresponding y0 can be chosen as 1 if the following constraint qualification is met (see Corollary 8.3 and Proposition 5.3): h'(xo) is onto and there exists z E X for which

CQ(d)

g~(xo)z + g'[(Xo)(d, d) < 0

for i E J(xo, d),

h'(xo)z + h"(xo)(d, d) = O.

Remark 3. For d = 0 and W = R m, Theorem 9.3 reduces to the Fritz-John condition. The condition guaranteeing y0 = 1 in (9.6) becomes the MangasarianFr-mowitz constraint qualification (see e.g. Gauvin and Tolle [9]): h~(xo), j = 1 . . . . . m, are linearly independent and

there exists z E X such that

CQ(0) g~(xo)z < 0

for i E I(xo),

h)(xo)z = 0

for j = 1, 2 . . . . . m.

Remark 4. If there is ~i I> 0 for i E J(x0, d), (9.5)), then the set of from (9.6) (put yi: = 0

a d in (9.5) for which (9.6) holds with a multiplier Y0> 0, and ~* (e.g., whenever CQ(d) is met for at least one d in directions d satisfying (9.5) looks more familiar. One gets for i E I ( x o ) \ J ( x o , d)):

{d I f'(xo)d 0, and for d = (~, - 2yo >/O.

66

A. Ben-Tal and J. Zowel First and second order conditions

Together with y ~ > 0 this implies (Yo, Yl, Y2, Y3) ~ (0, O, O, 0).

y0 = yt = y2 = y 3 = 0

in contradiction to

The question arises under what conditions one can state a necessary condition where the same multipliers can be taken for all d. One such case will be when the multipliers in the Euler-Lagrange equation (9.6) are determined uniquely by (9.6). This is quaranteed by the condition CQ1

{(g'(xo)x + k, h'(xo)x): x E X, k E R" and ki = 0 for i E I(x0)} = R" • W.

This condition could be rephrased omitting k and using the active components of g only. But note that our formulation also makes sense for an arbitrary cone relation g ( x ) E - K (here k has to be taken from Kg(x0)N-Kg~x0)). The same applies to the next condition CQ2. For dim X < oo and h = (hi . . . . . hm) (i.e., W = Rm), CQ1 is equivalent to the well-known condition (see e.g. Fiacco and McCormick [8] or Luenberger [22]): The gradients Vgi(Xo) and Vhi(x0), i E I(xo) and j = 1. . . . . m, are linearly independent. Another type of regularity assumption is the following For every d satisfying (9.5) there is z E X and k ~ R" with ki = 0 for i E I(xo) such that CQ2

g'(xo)z + g"(Xo)(d, d) + k = O, h'(xo)z + h"(xo)(d, d) = O. For dim X < co and dim W < 0% CQ2 is implied by the so-called second order constraint qualification given by Fiacco and McCormick [8, p. 28]. We get the following result which for dim X < oo and dim W < oo can be found e.g. in [8, p. 25; 22; 13, p. 27].

Corollary 9.4. Consider problem (MP) with twice differentiable functions f, gi and h and Banach spaces X and W. Let Xo be optimal and suppose that either of the following conditions holds (i) CQ1, (ii) CQ2, the range of h'(xo) is closed and Y0 can be taken as 1 for all d satisfying (9.5). Then there are multipliers Yi for i E I(xo) and w* ~ W* such that f'(Xo)+ ~ Yigl(xo) + w*h'(xo) = O, iel(xo)'

(9.8)

(r,(x0)+ie;(x E o)y,gT(xo)+

(9.9)

A . B e n - T a l and J. Zowe/ First and s e c o n d order c o n d i t i o n s

67

for all d satisfying g~(xo)d = 0 for i E I(xo) and yi > 0, g~(xo)d o~

h'(xo)do = lim t ~ l ( h ( x . ) -

h(xo)) = O.

n--->oe

Hence do satisfies (11.1). Further do # 0. If (i) holds, then we reach a contradiction. Hence suppose (ii) holds, that is, (11.2)-(11.4) hold with suitable u~, v~ and w~. Now by definition of the derivative f ( x . ) - f(xo) = f'(xo)z. + ~f"(Xo)(Z., z.) + o(t~) and, because of (11.5),

f'(xo)z. + ~f"(xo)(Z., z.) + o(t~) e - C. Similarly, g'(xo)z. + 89

z.) + o(t2.) E - K~(xo),

h'(xo)z, + 89

z.) + o ( t b = O.

Multiplying these equations with u~, v~, w~ and summing up while using (11.3), we get (note that v~ E Kg(xo) since v~ E K + and v~Jg(xo) = 0): +

(u~f"(xo) + v~g"(xo) + w~h"(xo))(z~, z,) + o ( t b ~ O. Dividing by t 2 and letting n ~oo we get (u ~f"(xo) + v ~g"(xo) + w ~ h"(xo))(do, do) ~< 0 which contradicts (11.4). A simple modification of the above proof gives us a similar condition which for U = R and finite-dimensional V and W becomes [13, Chapter 4, Theorem 7.3]. The assumption dim X < ~ is essential for the above statement; see the following example.

Example 1. Let X be the real sequence 12 and I = R. Choose some s E X with

A. Ben-Tal and J. Zowe/ First and second order conditions

71

c o m p o n e n t s s, > 0 for all n and consider the free problem oo

minimize

oo

f (x ) = ~= six 2 - ~= x 4.

L e t x0 be the origin in 12. Then f'(x0) is the zero-functional on 12 and

f"(Xo)(d, d) = 2 ~ sid~ for all d E 12. H e n c e (11.2)-(11.4) hold with u* = 1 for all d E 12, d ~ 0. Nevertheless, x0 is not optimal. T a k e the sequence x n := 2N/J, 9I n for n = 1, 2 . . . . . where l" is the vector with all c o m p o n e n t s 0 except the n t h c o m p o n e n t which is 1. Then x ~ -->x0, but

f(x")=4s 2-16s 2 - [31lyl]}.

Then xo is strictly optimal. For conditions guaranteeing (11.7) see Maurer and Z o w e [25]. In [24] the relation of the a b o v e condition to well-known sufficient second order conditions in control theory is studied.

A. Ben-Tal and J. Zowe/ First and second order conditions

72

12. S u m m a r y of n e c e s s a r y a n d sufficient c o n d i t i o n s

N e c e s s a r y c o n d i t i o n s f o r the general p r o b l e m (P) x0 is a n o p t i m a l s o l u t i o n f o r (P) Statement

Source

Remark

Theorem 2.1

If CQ(d) holds, then It is not the zerofunctional (Remark 2 in Section 4).

Theorem 8.2

X, W Banach spaces, h'(xo)X closed.

Nondifferentiable case For every d ~ D1(x0) N Dg(xo) O Th(xo) for which f, g and h are d-regular there are continuous linear functionals lr ~EA(Q1(x0, d)),

lg E A(Qg(xo, d)),

Ih E A(Vh(xo, d)),

not all zero, such that l~+ Ig + lh = 0, 8"(1~ I (Qf(xo, d)) + ,~*(lg I Qg(xo, d)) + 8*(In I Vh(Xo, d)) 0 for i E ](xo, d)

Theorem 9.3

and a functional w* E W*, not all zero, such that yof'(x0)+

ylg~(xo)+ w*h'(xo) =0,

~ i E J ( x o, d)

(yof"(x0)+

,ig'[(xo)+w*h"(xo))(d,d)~O.

~ iEJ(x O, d)

If either CQ1 or CQ2 + CQ(d) for all d satisfying (*) holds, then there are multipliers y~ for i E I(xo) and w* • W* such that

f'(xo)+ ~

yig~(x0)+ w*h'(xo)=O,

iEl(x o)

yig'[(xo)+w*h'(xo))(d,d)>~O

(f"(x0)+ ~ i E l ( x o)

for all d satisfying

g~(xo)d = 0 for i E I(xo) and yi > 0, g~(xo)d 0 with

h(x + ad) 'relation' h(x) for all 0 < a -< c~}. These are the cones of directions of constancy, decrease, nonincrease, increase and nondecrease respectively. For simplicity of notation, we let D'relation'- x 'relation' q iX) = Dqg (x) D~elation'(x) =

n Ltqr"'relati~ qEQ

Furthermore, if Q c S § we let

go(x) = sup qg(x) q~Q

and set

D(x) = Dg~=(x).

f o r q E Y*,

for Q c Y*.

J.M. Borwein and H. Wolkowicz/ Characterizations of optimality

83

The following l e m m a gives several useful properties of the 'uniform' function go. Note that dom(h) denotes the points at which the function h is finite. Lemma 2.4. If Q c S § is compact, then: (a) go is convex and dom(g o) = X (in particular, gq is continuous on lines); (b) if in addition g is 'weakly' lower semicontinuous (i.e. qg is lower semicon-

tinuous for each q in Q) and X is barrelled, then g is continuous. Proof. (a) For 0 0, and choose ~ to satisfy

EF

and

g(~) E - i n t Q+.

(4.9)

This ~ always exists by L e m m a 2.2 and Corollary 2.1 and m o r e o v e r satisfies go(~) < 0.

(4.10)

N o w let xA=,~x+(1-X)~

for0- 0, which implies that g~-(x,) = 0

J.M. Borwein and H. Wolkowicz/ Characterizations of optimality

92

for small A > 0. Now, since g~=(xx) -O, for all x ~ R:. Now, T h e o r e m 4.1 yields the optimality condition: D~(O) N C~(o)(O) = ~. However,

O~ Of(O)+ 3s+g(O) for any s § ~ S+(0), since 0f(0) = (1, 0) while as+g(O) = s§ This shows that part (iii) of Theorem 3.1 characterizes optimality while parts (iii) and (iv) fail to. The reason for this is that the cone of subgradients B~(0)(0) is not closed and thus not equal to - (C~(o)(O)) +.

Example 5.5 This example illustrates Corollary 4.1. Consider the semi-infinite program (see Fig. 2) minimize

f(x) = -x,

subject to

h(x,t) So, let Y = 12 a n d set S = { n o n n e g a t i v e s e q u e n c e s in Y}. W e c a n n o w r e w r i t e the a b o v e p r o g r a m in the a b s t r a c t f o r m u l a t i o n minimize

y(x),

s u b j e c t to

g(x)E-S.

(P) L e t us c h o o s e t h e g e n e r a t i n g set ~ = { e i : i = 1 , 2 , 3 .... }, the set of c o o r d i n a t e f u n c t i o n s in 12. T h e n c o n e ~ = S § S. N o t e t h a t S l a t e r ' s c o n d i t i o n fails, s i n c e int S = ~, b u t N = = tt. T o a p p l y C o r o l l a r y 4.1, w e m u s t c h o o s e the c o m p a c t set Q = { e i : i = 1,2 . . . . . k } K ~ < =

~,

while the r e m a i n d e r ~={ei:i=k+l,l+2

.... }.

Thus ( F ~ - 1) + = c o n e ( F ~ - 1) -- R_ and our optimality conditions yield: 0 E 0f(1) + Os+g(1) - ( F ~ - 1) + = { - 1 } - R_, since S+(1) = {0}, as t h e r e are no b i n d i n g c o n s t r a i n t s .

References [1] R.A. Abrams and L. Kerzner, "A simplified test for optimality" Journal of Optimization Theory and Applications 25 (1978) 161-170. [2] M.S. Bazaraa, C.M. Shetty, J.J. Goode and M.Z. Nashed, "Nonlinear programming without differentiability in Banach spaces: Necessary and sufficient constraint qualification", Applicable Analysis 5 (1976) 165-173.

100

J.M. Borwein and H. Wolkowicz/ Characterizations of optimality

[3] A. Ben-Israel, A. Ben-Tal and S. Zlobec, "Optimality conditions in convex programming", in: A. Pr6kopa, ed., Survey of mathematical programming, Proceedings of the 9th International Mathematical Programming Symposium (The Hungarian Academy of Sciences, Budapest and North-Holland, Amsterdam, 1979) pp. 137-156. [4] A. Ben-Tal, A. Ben-Israel and S. Zlobec, "Characterization of optimality in convex programming without a constraint qualification", Journal of Optimization Theory and Applications 20 (1976) 417--437. [5] A. Ben-Tal and A. Ben-Israel, "Characterizations of optimality in convex programming: The nondifferentiable case", Applicable Analysis 9 (1979) 137-156. [6] J. Borwein, "Proper efficient points for maximizations with respect to cones", SIAM Journal on Control and Optimization 15 (1977) 57-63. [7] J. Borwein and H. Wolkowicz, "Characterizations of optimality without constraint qualification for the abstract convex program", Research Report 14, Dalhousie University, Halifax, N.S. [8] B.D. Craven and S. Zlobec, "Complete characterization of optimality of convex programming in Banach spaces", Applicable Analysis 11 (1980) 61-78. [9] M. Day, Normed linear spaces, 3rd ed. (Springer, Berlin, 1973). [10] I.V. Girsanov, Lectures on mathematical theory of extremum problems, Lecture notes in economics and mathematical systems 67 (Springer, Berlin, 1972). [11] F.J. Gould and J.W. Tolle, "Geometry of optimality conditions and constraint qualifications", Mathematical Programming 2 (1972) 1-18. [12] F.J. Gould and J.W. Tolle, "A necessary and sufficient qualification for constrained optimization", SIAM Journal on Applied Mathematics 20 (1971) 164-171. [13] M. Guignard, "Generalized Kuhn-Tucker conditions for mathematical programming problems in a Banach space", SIAM Journal on Control 7 (1969) 232-241. [14] R.B. Holmes, A course on optimization and best approximation, Lecture notes in mathematics 257 (Springer, Berlin, 1972). [15] S. Kurcyusz, "On the existence and nonexistence of Lagrange multipliers in Banach spaces", Journal of Optimization Theory and Applications 20 (1976) 81-110. [16] D.G. Luenberger, Optimization by vector space methods (J. Wiley, New York, 1969). [17] L.W. Neustadt, Optimization--A theory of necessary conditions (Princeton University Press, Princeton, N J, 1976). [18] B.N. Pshenichnyi, Necessary conditions for an extremum (Marcel Dekker, New York, 1971). [19] A.P. Robertson and W.J. Robertson, Topological vector spaces (Cambridge University Press, Cambridge, 1964). [20] R.T. Rockafellar, Convex analysis (Princeton University Press, Princeton, N J, 1972). [21] R.T. Rockafellar, "Some convex programs whose duals are linearly constrained", in: J.B. Rosen, O.L. Mangasarian and K. Ritter, eds., Nonlinear programming (Academic Press, New York, 1970) pp. 393-322. [22] H. Wolkowicz, "Calculating the cone of directions of constancy", Journal of Optimization Theory and Applications 25 (1978) 451-457. [23] H. Wolkowicz, "Geometry of optimality conditions and constraint qualifications: The convex case", Mathematical Programming 19 (1980) 32-60. [24] J. Zowe, "Linear maps majorized by a sublinear map", Archly der Mathematik 26 (1975) 637-645. [25] J. Zowe and S. Kurcyusz, "Regularity and stability for the mathematical programming problem in Banach spaces", Preprint No. 37 Wiirzburg (1978).

Mathematical Programming Study 19 (1982) 101-119. North-Holland Publishing Company

DIFFERENTIAL PROPERTIES OF THE MARGINAL FUNCTION IN MATHEMATICAL PROGRAMMING Jacques GAUVIN

D~partement de Math~matiques Appliqu~es, Ecole Polytechnique de Montreal, Montreal, P.Q. Canada Franqois DUBEAU Departement de Math~matiques et de Statistiques, Universit~ de Montreal, Montreal, P.Q. Canada Received 13 September 1979 Revised manuscript received 14 January 1981 This paper consists in a study of the differential properties of the marginal or perturbation function of a mathematical programming problem where a parameter or perturbation vector is present. Bounds for the Dini directional derivatives and estimates for the Clarke generalized gradient are obtained for the marginal function of the mathematical program neither assumed convex in its variables or in its parameters. This study generalizes some previously published results on this subject for the special case of right-hand side parameters or perturbations.

Key words: Nonlinear Programming, Marginal or Perturbation Analysis, Differential Stability.

1. Introduction In this p a p e r w e c o n s i d e r a m a t h e m a t i c a l p r o g r a m m i n g p r o b l e m of t h e f o r m : max s u b j e c t to

f ( x , y),

x E R n, y E R " ,

gi(x, y)~ 0 such that

(i) a(/3)= c~, (ii) H(a([3), [3) = 0 on N, (iii) H ( a, [3) = 0, Is - a([3)l < E on N hold only in the case a = ct([3). If H (a, [3) is continuously differentiable on S, then a([3) is continuously differentiable on N.

104

J. Gauvin, F. Dubeau/ Differential properties of the marginal function

Part (ii) of the (M-F) regularity condition gives the following result which is a slight modification of [7, Theorem 5.2, p. 163]. Lemma 2.2. If at (~, Y) the Jacobian matrix Vxh(~, Y) has full row rank q, then for any direction (L 5) E R n x R m, there exist neighborhoods U of (~, Y) and V of (~, g), a { > 0 and a continuous function

a(t;x,y;r,s)

onO=]-t,t[xUxV

with values in R ~ such that a(O;x,y;r,s)=O

onU•

V,

(2.3)

-~-(t; x, y; r, s)

(2.4)

exists and is continuous on 0 with ~t ( O ; x , y ; r , s ) = O

onU•

V.

(2.5)

Furthermore the function X ( t ; x , y; r, s) = x + a ( t ; x , y; r, s)+ tr

(2.6)

satisfies h ( X ( t ; x , y; r, s), y + ts)= = h(x, y) + t{Vxh(x, y)r + Vyh(x, y)s}

on O.

(2.7)

Proof. Consider the function H (t, oJ;x, y; r, s) = h(x + Vxh(x, y)To~ + tr, y + t s ) - h(x, y) - t(Vxh(x, y)r + Vyh(x, t)s)

(2.8)

which is continuous with values in R q. As H ( 0 , 0 ; ~ , y ; ~ , g ) = 0 and rank[V~H(0, 0; ~, Y; L 5)] = rank[Vxh(~, y)Vxh(~, y)T] = q, from L e m m a 2.1 there exists an E > 0 , a neighborhood N = ] - t , t [ x U x V of ( 0 , ~ , y ; L 5 ) and a continuous function oJ(t; x, y; r, s) such that (i) to(O;x,y;r,s)=O, (ii) H(t, t o ( t ; x , y ; r , s ) ; x , y ; r , s ) = O o n N , (iii) (d/dr)to(t,x; y; r, s) exists and is continuous on N. If we let

a(t;x, y; r, s) = Vxh(x, y)Tto(t;x, y; r, s), we then have (2.3) from (i), (2.4) from (iii) and (2.7) follows from (ii) and (2.8). Finally if we take the derivative of (ii) or (2.8) with respect to .t at (0; x, y ; r, s),

J. Gauvin, F. Dubeau/ Differential properties of the marginal [unction

105

we obtain

]

V~h(x, y) ~-~-(0; x, y; r, s ) + r + Vyh(x, y)s = Vxh(x, y)r+Vyh(x, y)s, therefore Vxh(x, y)Vxh(x, y)X ~ t (0; x, y; r, s) = 0. Since we can choose U to have rank[Vxh(x, y)] = rank[Vxh(g, 9)] = q it follows that Vxh(x, y) Vxh(x, y)~ is regular on U and we must have (d/dO to(0; x, r; r, s) = 0 and (2.5) follows. We denote by S*(y) = {x E R": gi(x, y) < O, i = 1..... p, h(x, y) = O} the strict interior of the feasible set S(y). The next lemma will be fundamental in the sequel. Lemma 2.3. If the (M-F) regularity condition is satisfied at some g ~ S(9), then for any direction g E R", there exist a { > O, a neighborhood N ( g ) of g and a continuous function X ( t ; g) on ]0, t[x N ( g ) such that X(0; s) = g, X ( t ; s ) E S*(9 + ts) with (d/dt)X(t;O) continuous on ]0, t [ x N (g). Fu'rthermore there exists an 9 > 0 such that S(y) is nonempty if ]]y - y[[ ~< 9 Proof. For any s E R m, consider the direction r ( s ) E R n defined by

r(s) = x~ - Vxh(~, y)~yh(~, y)s,

(2.9)

where Vxh(x, y) ~ = Vxh(x, 9)T[Vxh(x, 9)Vxh(x, y)T] -1

is the pseudo-inverse of the full row rank Jacobian matrix Vxh(~, Y) and where F is given by the (M-F) regularity condition. Immediately we have Vxh(g, Y) r(s) + Vyh(~, ~)s = 0.

(2.10)

For the active inequality constraints i E I(g, 9),

Vxgi(~, Y) r(s) + Vyg~(~, y)s = = XVxgi(~, 9)~ + {Vyg~(~, 9 ) - Vxg~(~, 9)Vxh(~, 9)* Vyh(~, y)Js. For s on a bounded neighborhood N ( g ) of g, we can choose A > 0 big enough to have Vxgi(~, ~) r(s) + Vygi(g, y)s < O, i E I(~, ~), s E N (g).

(2.11)

For s E N(g), r(s) is in a neighborhood of r(g), therefore we can consider the

106

J. Gauvin, F. Dubeau/ Differential properties of the marginal function

function

X(t, s) = ~ + a(t; ~, Y; r(s), s) + tr(s),

(2.12)

where a(t;X, Y; r(s), s) is given by L e m m a 2.2. From (2.3) we have X(0, g) -- X. From (2.7) and (2.10) we have

h(X(t, s), ~ + ts) = 0 on [0, to[ x N0(g)

(2.13)

for a certain to> 0 and N0(g) C_N(g). For i ~ I(X, ~), since gi(x, ,~) < 0, by continuity there exists a 7~> 0 and Ni(g) such that

gi(X(t, s), ~ + ts) < 0 in ]0, t-i[ • Ni(g).

(2.14)

For i E I(~, y), since g~(~, Y) = 0, we have by the mean value theorem

gi(X(t, s), y + ts) = = t {Vxgi(X(~', s ) , y + ,s)[~-~ (~'; 2, Y; r(s), s)+ r(s)] +

+ Vygi(X(r, s), y + rs)s } for a certain ~-E [0, t], r depending on s. By continuity, (2.5) and (2.11), there exist ti > 0 and N~(g) such that

gi(X(t, s), y + ts) < 0 on ]0, {~[ x N~(g)

for i E I(~, 9).

(2.15)

We finally obtain the first part of the result from (2.12), (2.13) and (2.14) with t- = min{~: i = 0 . . . . . p} and N(g) = nf=o N~(g). To obtain the last part of the result we just need to consider a ball of radius e > 0 in the neighborhood t'N (g) of g = 0. The nonemptiness of S(y) near y is also demonstrated in [12, Theorem 1].

3. Continuity of the marginal function and of some related point-to-set mappings We will need some concepts from the theory of point-to-set maps. For a more detailed exposition the reader is referred to Hogan [8] or Berge [2]. In the following, F will represent a point-to-set mapping of E C_R k into the subsets of R I. Definition 3.1. The map F is upper semi-continuous at X E E, if for any open set 0 containing F(~) there is an open neighborhood N(X) of ~ such that F(x)C_ 0 for each x E N(X) n E.

J. Oauvin, F. Dubeau[Differential properties of the marginal function

I07

The concept of upper semi-continuity will be more suited to our purposes if phrased in terms of sequences.

Definition 3.2. F is said to be closed at ~ ~ E if {x~}C E, x. ~ ~, y~ ~ F ( x . ) and y. ~ ~ imply that y E F(~).

Definition 3.3. F is uniformly compact near ~ E E if there is a neighborhood N(~) of ~ such that the closure of Ux~Nt~) F ( x ) is compact. With the above definitions we can relate the concepts of closed maps and upper semi-continuity. The proof of the following proposition is found in [8].

Proposition 3.1. Let F be uniformly compact near ~ ~ E. Then F is closed at ~ if and only if F(~) is compact and F is upper semi-continuous at ~. A first result on continuity is given in the following lemma. Lemma 3.2. If S(y) is nonempty and S(y) is uniformly compact near y, then S(y) and the marginal function v(y) are upper semi-continuous at ~. Proof. Since the function gi(" ," ) and hi( 9 9) are continuous, S(y) is closed at and it follows from Proposition 3.1 that S(~) is compact and S(y) is upper semi-continuous at .~. From Hogan [8, Theorem 5] it follows that v(y) is upper semi-continuous at y. Theorem 3.3. Suppose that S(y) is nonempty, S(y) is uniformly compact near y, and the (M-F) regularity condition holds at some ~ E P ( ~ ) . Then v(y) is continuous at ~. Proof. Consider a sequence {y.), y. --} y, such that 8 = lira inf v(y) = lim v(y.). y--~

n--~

Let s. = ( y . - Y)/(IlY.- YlI) and tn = [lY.- Y]I. Without loss of generality, we can assume s.--} g, for some direction g. From L e m m a 2.3, we have the function X ( t , s) such that X(0, g) = ~ and X(tn, s.) E S*(y + t.s.) for n sufficiently large. Therefore v(y.) t> f ( X ( t . , s.), y.), I> lim f ( X ( t . , s.), y.) = f(~, y) = v(y) n-~o0

which shows that v(y) is lower semi-continuous at y. This, with L e m m a 3.2, gives the result. In the next theorems it is demonstrated that the (M-F) regularity condition is

108

J. Gauvin, F. Dubeau/ Differential properties of the marginal function

preserved under small variations of the parameters and that the set K(y) of Kuhn-Tucker vectors is locally well behaved. Theorem 3.4. Assume the (M-F) regularity conditions holds at some ~ E P(~). Let {y.} and {x.} be sequences such that y. ~ ~, x. E P(y.) and x. ~ ~. Then for n sufficiently large (M-F) holds at x. and there exist subsequences {(u m, vm)}, {xm} with (u m, v m) ~ K(x,.; ym) such that (u m, vm)~(fl, ~) for some (ft, 6) E K(~, ~).

Proof. (Gauvin and Tolle [5, Theorem 2.7]). Since x. ~ ~, for n sufficiently large it follows that I(x., y.) C_I(~, y) and {Vxh(x., y.)} still have full row rank. Let given by (M-F) at (~, y) and let 5. = [I -Vxh(x., y.)'Nrxh(x., y.)]E

(3.1)

For n sufficiently large, we have, as ~. ~ L

Vxgi(x., y.)~. < 0,

i ~ I(~, y),

Vxh(x., y.)f. = 0.

(3.2)

Since I(x., y.) C_I(~, y), we have (M-F) satisfied at (x., y.). For each (x., y.) take (u., v.) ~ K(x., y.). Then Vxf(x,, y,) = uTVxg(x,, y.) + vTVxh(x,, y,),

(3.3)

v T = [Vxf(X,, y , ) - UTVxg(X,, y,)]Vxh(x,, y,)'~.

(3.4)

From (3.2) and (3.3), we have Vxf(x., y,,)~,, = uTVxg(X., y,,)~. ~< uTVxgi(x., Y.)L, for any i E I(~, y). Then

v~f(x., y.)~. u7 0, (M-F) holds at each x E P(y) with IlY - Y[[~< & Furthermore the point-to-set mapping K(y) is closed at y and uniformly compact near y and therefore upper semi-continuous at ~. It is also shown in [12, Theorem 1] that the (M-F) regularity is preserved under small perturbations.

4. Dini directional derivatives of the marginal function

The lower and upper Dini directional derivatives of v(y) at y in the direction s ~ R" are respectively D+v(y; s) = lim inf [v(y + t s ) - v(y)]/t, t~0 +

D+v(y; s) = lira sup [v(~ + ts) - v(y)]lt. t--~0+

In the next two theorems we obtain lower and upper bounds for D+v(y; s) and D+v(~; s). Theorem 4.1. If the regularity condition (M-F) is satisfied at some optimal point

E P(?), then for any direction s E R m, D+v(y; s)/>

rain

(u, v)EK(.~, ~,)

{VyL(~,~; u,v)s}.

110

J. Gauvin, F. Dubeau/ Differential properties of the marginal function

Proof. For the given direction s E R m, the linear program rain subject to

--{gTVyg(.~, y) + 19TVyh(.~,y)}S, UTVxg(-g, y) q- t]Tvxh(x, y) = Vxf()C, y),

(4.1)

uTg(.~, V) = O, U ~>0, is feasible and bounded as the set of Kuhn-Tucker vectors K(~, y) is nonempty and bounded under the (M-F) regularity condition. Therefore the dual max r,w

subject to

Vxf(~, y)r, Vxg(~, y)r + toTg($, y) ~< -Vyg(~, y)s,

(4.2)

Vxh('2, y)r = -Vyh(3c, y)s, ~ R p,

r E R",

is also feasible and bounded. Take (r(s), w ( s ) ) an optimal solution of the dual (4.2) and define for h > 0 r(h; s) = AP+ r(s), where ~ is given by the (M-F) regularity condition. We then have for any h > 0,

Vxgi(~, y)r(h ; s) + Vygi(x, y)S < O,

i E I(~, y),

(4.3)

Vxh(~, y)r(h ; s ) + Vyh(~, y)s = 0,

(4.4)

vg(~, ~)r(X ; s) = = hVxf(2, ~)~ +

(4.5)

min

- {uXVyg(g, ~) + vVVyh(g, ~)}s.

(u, v)EK(Y,, ~)

From (4.4) and (4.3), (rOt; s), s) satisfies (2.10) and (2.11) of L e m m a 2.3 for any h > 0 and, as in L e m m a 2.3, we can show that for the function

X (t, s) = ~ + a(t; 2, y; r(h; s), s) + tr()t ; s), we have, for some { > 0, for any h > 0,

X ( t , s) E S*(y + ts),

t ~ 10, ?[.

Then

v(y+ts)>-f(X(t;s),y+ts),

t~lO,?[

and as X(O, s) = :~, v(y) = f(~, y), we have D+v(q; s) t> lira f f ( x ( t , s), y + ts) - f ( x ( o , s), y)l/t t-.O ~

Vxf(x, y)[-~-, (0, x,y, r(h, S), s)-t-r(h, S)]+ Vyf(X, y)S. LUt

.!

J. Gauvin, F. Dubeau/ Differential properties of the marginal .function

I11

From (2.5) of L e m m a 2.2 and (4.5) we obtain D+v(y; s)/> I> AV..f(~, ~)P +

min

(u, v)~K(~, ~)

= AVer(g, y)~ +

- {uTVyg(~, .~)s + vrVyh(~, ~)s}+ Vyf(~, ~)s

rain

(u, v)~K(~,y)

{VyL(~, ~); u, v)s}.

The result follows by letting X--, 0. If we assume that the (M-F) regularity holds at every optimal point g E P(.~), we obtain an upper bound for the Dini directional derivative. Examples show that this derivative can be infinite if this condition is not satisfied at one of the optimal points.

Theorem 4.2. S u p p o s e S(y) is nonempty, S(y) is uniformly c o m p a c t near y and the (M-F) regularity condition holds at every ~ E P(y). Then f o r any direction s ~ R ~', there is a ~ E P(y) such that D+v(y;s)~
0 where U(~, y), V(?) are neighborhoods respectively of (~, y) and ?, respectively. By (2.7) in L e m m a 2.2, we have h ( X ( t ; x, y; r, - s ) , y - ts) = = h(x, y) + t{Vxh(x, y)r - Vyh(x, y)s} If we let X , ( t ) = X ( t ; x n , yr + tns - tns = y we obtain

on 1"].

yn;Pn,--S) we have Xn(0)=xn, and as y n - t n s =

h(Xn(t~), y~) = h(xn, Yn) + tn{Vxh(x,, Y,)Pn - Vyh(xn, y~)s} = h(xn, y,) = 0

(4.12)

by (4.11), (4.9) and (4.8). For i ~ 1(4, ~), gi(X~(t,), y - ) g i ( x , Y ) < 0 , hence for n sufficiently large g~(Xn(t~),Yp)~O,

i ~ I(~, ~).

For i ~ I(~, y), g i ( X n ( t . ) , Y) = g i ( X , ( t . ) ,

yr + tns - tns)

d = g~(Xn(O), fr + t~s) + tn ~ GT(z,t,) for some "rn ~ [0, 1], where GT(t) = g~(Xn(t), y~+ (tn - t)s) with 67(0) = g~(X.(0), y. + t.s), GT(t.) = g~(X.(t.), y~). Since

-•t

GT(~t,) =

= V~(X.(~-~t.), y + ( 1 - ~-.)t.s) ~-(~-.t,; x., y., ~.,-s) + ~. -Vygi(X.('r.tn), (1 - "r~)t.s )s.

(4.13)

J. Gauvin, F. Dubeau/ Differential properties of the marginal function

113

By (2.5) of L e m m a 2.2 and (4.11) we have lim d GT('r.t.) = Vxgi(~, 9)~ - Vygi(~, 9)s .-.~ dt = ;t Vxg~(~, 9)~ + V~g~(~, y)r(-s)- Vyg~(~, Y)s

~<XVxg~(~, 9)~ < 0 by definition of r ( - s ) and F. Therefore, for n sufficiently large,

g~(X.(t.), y) ~-O, ck ~ ]a, b[, Mk E o~F(ck) for all k, ~,~=l ak = 1, and

ak

F ( b ) - F ( a ) = ~ l a~M~(b - a).

(1.8)

J.-B. Hiriart-Urruty/ Optimality conditions in nondifferentiable programming

125

Now let us consider a sequence {x.} converging to x0 and a sequence {h.} C R+* converging to 0. We set E. = {q~[F(x. + And)] - q~[F(x.)]}h~ ~. According to the above theorem, there exist F. ~ ]F(x.), F ( x . + And)[ and u. aq~(F.) such that E . = ( F ( x . + And) - F ( x . ) , u.)A~ ~.

(1.9)

Now, by applying the same mean-value theorem to F, we get that [ F ( x . + And) - F(x.)]h~ 1 = ~ ak..Mk..d,

(1.10)

k=l

where ak.. >--O, ~ = 1 ak.. = 1 and Mk.. E ~ F ( c k . . ) for some Ck.. E IX., X. + And[. Since the set-valued mappings ar and .,r are upper-semicontinuous, we may suppose (subsequencing if necessary) that u. ~ u E aq~(F(xo)), P

for all k (~--i ak = 1),

ak,. ~ ak

MR,. ~ Mk E ~ F ( x o )

for all k.

Consequently, we derive from (1.9) and (1.10) that lim s u p E . -< max{(MTu, d) [ u E aq~(F(xo)), M E ~r n~

Hence, a(~p o F)(xo) C co{Mru I u ~ &p(F(x0)), M E oCF(x0)},

(1.11)

and the result (1.7) is thereby proved. The next examples illustrate the foregoing results. Example 2. Let F : R 2 ~ R 2 of the form (f, _f)x where f ( x l , x2)= max(xb x2), let ~o:REoR be defined by ~0(yl, y2) = lYll+lY2[. The concerned point is here x0 = (0, 0) while Q is taken to be the whole space. We have

CF(xo)=

co{(I), (I), (ll)(I)}' {['-~ ~ 0 0 for all d in the cone T(S; xo) of adherent displacements for S from x0 (see [6, Theorem 2]); (ii) in a dual form: if x0 solves (P) locally, then

0 ~ Of(xo) + R+ Ods(xo).

(3.1)

132

J.-B. Hiriart-Urruty/ Optimality conditions in nondifferentiable programming

This latter result is obtained by noticing that x0 is a local minimum (on R p) of f + kds for k Lipschitz constant of f around x0 (see [2, L e m m a 2; 5, Th6or~me 5]). In fact, the condition that Of(xo) n N ( S ; xo) ~ fJ (derived from (3.1)) has been proved to be a particular case of a more general optimality condition, dualization of the primal condition (i) [cf. [6, Theorem 3]). As pointed out earlier [8, Section 3.2], the concept of normal cone is not well-adapted to handle optimization problems with equality constraints. Therefore, we shall pursue our earlier study on the normal subcone to a set [8, Section 3.2] be refining conditions yielding an estimate of it when the set is represented as one equality-type constraint, and by generalizing the concept to the case where more than one equality constraint is involved. 3.1. Let Sh be a constraint set defined as Sh = {X E R p I h(x) = 0},

(3.2)

where h is a (real-valued) continuous function. We shall begin by presenting in a unified fashion necessary conditions for x0 E Sh to solve locally (P). Starting with optimality conditions in a geometric form, we refine and cover the results established earlier for optimization problems of the above form. Let S~ and S~ denote the following subsets: S-~ = {x I h(x) >- 0},

Sh = {x I h(x) 0 satisfying sup{(x*, d + ) l x * E Of(xo)+ N+}- 0 . For every point u E V satisfying J ( u ) 0

and by using the cone of feasible directions to K at u, denoted b y F ( K , u) as in [1], we have the following definition:

Definition 3.4. If J has a directional derivative at u E K for e v e r y d in F ( K , u), we shall say that u is stationary up to E if J'(u ; d) + "X/-~Ildll >- 0 for all direction d ~ F ( K , u). Proposition 3.1. If the hypothesis (H) is satisfied, if J has a directional derivative at u for all [easible directions d E F ( K ; u), if u, is a regular approximate solution up to E for (P), then it is also a stationary point up to E. Proof. By using Corollary 2.1 with v = u, + td, where d is some feasible direction at u, and t > 0 such that u, + td E K, we obtain: J(u~) J(u). Remark 4.1. A c o n v e x functional on V is also E-semiconvex. For E = 0, we have the concept of semiconvexity defined by Mifflin [9]. By including the notion of E-quasidifferentiability, we obtain the next proposition similar to [9, Theorem 8]. Proposition 4.1. If J is E-semiconvex on a convex set K, u E K and u + d E K, then

J(u + d) + x/~ Ildll -< 1(u) implies that J ' ( u ; d) + ~/~ [Id[[-< O.

Proof. The proof is similar to the one given in [9]. We suppose, for contradiction purposes, J(u + d) + X/-~ Ildll -< 1(u) and J'(u; d) + X/~ I[dll > 0. Then, there exists t>0suchthatt 0,

and by setting f ( t ) = J(u + td) + X/-~ t[ldl], we obtain f(t) > [(0). But we have supposed f(1) -< f(O). So if /- maximizes the continuous function f ( t ) over [0, 1], we obtain:

f(1) = 1(u + d) § V ~ Ildll ~ J(u) = f(o) < f({).

P. Loridan [ Necessary conditions for e-optimality

145

Then, by the maximality of {, we have

J'(u + i-d; d) + ~/~ Ildll -< 0,

(4.1)

J'(u + ? d ; - d ) + x/~lldll ~ 0 .

(4.2)

By the E-quasidifferentiability of J, we obtain from (4.1) and (4.2):

J~

+ ?d; d)-O, l _ - V - ~ IIx - x(~)llo

for all ;~ E K * ,

n

J(uA + ~= X,(e) Gi(u,) - L(u,, X) - X/~ IX - ,~(e)lln for all X ~ K*. This gives the first inequality for a quasi saddlepoint. We now use the result (5.3) by noting that there exists z* E 0J(uA, z~, E 0G~(u~), 1 --- i -< n, and b* ~ B* such that n

z* + ~ ;~i(e)z~,, +X/~ b* = 0. By using the characterization of the generalized gradient of Clarke [2] and with the hypothesis of quasidifferentiability for d and Gi, we finally obtain: n

n

--- -x/~lldll for all d E V, where ( . , .) denote the canonical bilinear form on V* x V, where V* is the topological dual space of V. By using the definition of E-semiconvexity at u,, with d = v - u,, v ~ V, we then deduce:

P. Loridan/ Necessary conditions for ~-optimality n

151

n

s ( v ) + ~ h,(e) c,(v) + x G IIv - u, ll-> J(u~) § ~ h,(e) C,(u,) So we obtain the second inequality for a quasi saddlepoint:

L(v, h ( e ) ) + V ~ l l v - u, ll-> L(u,, h(e))

for

all v E V

and the proof is achieved. We also have, trivially, the following result: Corollary 6.1. If the hypothesis (H') is satisfied, if J and Gi, 1 -O,

x>-O,

xT(Mx+q)=O,

(1)

where M is a given n x n real matrix, q is a given vector in R" and T denotes the transpose. We shall refer to this problem as LCP(M, q). Existence of solutions for the linear complementarity problem has been investigated by many authors [3, 5, 7, 8, 15]. One well-known existence result [2, 5] is that if the feasible region of (1), S(M, q) = {x I Mx + q >- O, x >- 0}

(2)

is nonempty, then the solution set of (1), S(M, q) = {x I Mx + q >- O, x >- O, xT(Mx + q) = 0} * Research supported by National Science Foundation Grant MCS--7901066. 153

(3)

154

O.L. Mangasarian/ Bounded solutions o[ linear complementarity problems

is nonempty when the matrix M is copositive plus, that is, (a) (b)

x - 0 implies xTMx >-0 (copositive), x>-O, xTMx=O i m p l y ( M + M X ) x = 0 (plus).

(4)

Copositive plus matrices include positive semidefinite matrices and positive matrices. Other existence results are given in [5, 11, 12, 17]. Global uniqueness of solution of the linear complementarity problem has been investigated in among others [1, 14, 15] and local uniqueness in [13]. Robinson [20] and Doverspike [4] have characterized nonemptiness and boundedness of S(M, q) by the nonemptiness of S(/f/, ~]) for all (/~/, q) sufficiently close to (M, q) for the cases when M is positive semidefinite and copositive plus respectively. In addition, for positive semidefinite M, Robinson has further characterized the nonemptiness and boundedness of S(M, q) by the nonemptiness and uniform boundedness of ~(~7/, 4) for positive semidefinite ~7/. One of our characterizations, Theorem 2(xvi), extends this result of Robinson to the copositive plus M and ~/. Another characterization of the nonemptiness and boundedness of S(M, q), Theorem 2(x), extends the corresponding boundedness results of Williams [23] for a dual pair of linear programming problems to the linear complementarity problem with a copositive plus matrix. A useful feature of another characterization, Theorem 2(vii), is that without knowing whether (1) has a solution and without knowing any of its solutions we can determine if its solution set S(M, q) is nonempty and bounded from the following equivalence

S(M,q) is nonempty} ~ [ M a x { e T u l M T u _' 0 , qTu_0}=0 and bounded copositive~ plus

(5)

where e is a vector of ones in R n. Note that the right-hand side of the equivalence (5) can be easily checked by solving a simple linear programming problem. Another interesting characterization of the nonemptiness and boundedness of S(M, q) for copositive plus matrices, Theorem 2(iii), is that the feasible region S(M, q) be stable [19] or equivalently that it satisfies the Slater constraint qualification [9]. The principal results of the paper are contained in Theorem 2 which gives a number of equivalent characterizations for the nonemptiness and boundedness of the solution set S(M, q). The first eight characterizations of Theorem 2 are stability or constraint qualification conditions for the feasible region S(M, q) and they do not require any assumptions on the matrix M. The last eight characterizations of Theorem 2, however, make essential use of the copositivity plus of the matrix M. The equivalence between (ix) and (xv) of Theorem 2 is due to Doverspike [4] and is stated separately as Theorem 1. Corollary 1 shows that whenever a linear complementarity problem with a copositive plus matrix has a nondegenerate vertex solution, its solution set must be bounded. Corollary 2 establishes the characterization (5) stated above. Corollary 3 characterizes the

O.L. Mangasarian/ Bounded solutions of linear complementarity problems

155

subclass of copositive plus matrices which is in Q, that is the class of matrices for which the linear complementarity problem has a solution for each q in R". Corollary 4 specializes Theorem 2 to the case of a symmetric M and uses the same condition as that of [10, Thoerem 2.2] which ensures the boundedness of the iterates of the algorithms of [10]. We briefly describe now the notation of this paper. All matrices and vectors are real. For the m x n matrix A, row i is denoted by Ai and the element in row i and column j by Aij. For x in the real n-dimensional Euclidean space R ", element j is denoted by xj. All vectors are column vectors unless transposed by the superscript T. For I C {1..... m} and J C {1..... n}, A~ denotes the submatrix of A with rows Ai, i E / , A~j denotes the submatrix of A with elements A0, i E / , j E J, and xj denotes x~, i E J. Superscripts such as A ~, x ~denote specific matrices and vectors and usually refer to elements of a sequence. For simplicity we shall write A iT for (Ai) T. The Euclidean norm (xTx) in of a vector x in R" will be denoted by [[xl[ and the corresponding induced matrix norm maxllxll=~llAxl]will be denoted by IIAII. The vector e will denote a vector of ones usually in R n. A partition {I, J} of the set of integers {1. . . . . n} is defined as I C{1 . . . . . n}, J C { 1 ..... n } , I tA J ={1 . . . . . n} and l A J = O .

2. Principal results We begin with a theorem which was established by Robinson [20] for the case when M is positive semidefinite and extended by Doverspike [4] to Eaves' class L of matrices [5] which includes the copositive plus case. This theorem will be utilized in establishing one of the equivalences of Theorem 2. Theorem 1. (see [4]). Let M be an n x n copositive plus matrix and let q be in R n. The following are equivalent: (i) The solution set S(M, q) of the linear complementarity problem (1) is nonempty and bounded. (ii) There exists an ~ > 0 such that the solution set S(I(,I, (1) of the perturbed linear complementarity problem LCP(Jf/, (1) is nonempty for max{ll/f/- MI[, I1(1 - qll} < ,.

Our principal result which follows establishes the equivalence of a number of conditions for the nonemptiness and boundedness of the solution set S(M, q) of the linear complementarity problem (1). Theorem 2. For any n x n matrix M and any vector q in R ~ the statements (i) to (~,iii) below are equivalent. If in addition M is copositive plus, then the statements (i) to (xvi) below are equivalent.

156

O.L. Mangasarian/ Bounded solutions o[ linear complementarity problems

(i) The system Mx+q~>O,

x>-O,~>-O,

has a solution (x, ~) in R ~+l. (ii) The system Mx+q>O,

x>-O,

has a solution x in R ~. (iii) The system Mx+q>O,

x>O,

has a solution x in R ~. (iv) For each h in R ~ the system Mx+q+yh>-O,

x ->0, 3, > 0 ,

has a solution (x, 3') in R n§ (v) There exists a ~ > 0 such that the system Mx + fl >_O,x >-O, has a solution x for each F:l in R ~ such that UFt- qll 0 such that the system lf/lx + r >-O, x >-O, has a solution x for each n • n matrix 1Q1and each ~1 in R n such that max(llM

- MII, 114 - qll}

-< ~.

(vii) The system MTu 0, R > 0, ~-> 0. Because the open set {(x, ~) I Mx + q~ > 0} contains (R, 2) it must also contain (R, ~ + 3) for some sufficiently small positive 8 and hence MR + q(~+ 3 ) > 0 . The point R[(~+ 3) solves the system of (ii). (ii) ~ (iii). Obvious. (ii) ff(iii). Let R satisfy MR + q > 0, R -> 0. Because the open set {x I Mx + q > 0} contains R it must also contain R + ~e for some sufficiently small positive & The point R + ~e solves the system of (iii). (i)r (vii). This follows from Motzkin's theorem of the alternative [9]. (iv) r (vii). Condition (vii) is equivalent to the system MTu-0, x >-0, has no solution for i = 1,2 .... By

158

O.L Mangasarian/Bounded solutions of linear complementarityproblems

Motzkin's theorem of the alternative this is equivalent to MiTt/ m~O, qiTu < 0 , u ->0, having a solution u i for i = 1,2 ..... Since ui~ 0 it follows that MiT U i

^

" Wi

Ui

Hence there exists an accumulation point ~ such that MTI~0,

qT/~-O, O:~z>-O,

qTz 0 and ~ is a vertex of the feasible region S ( M , q), then assertion (xv) of Theorem 2 holds. If in addition M is copositive plus, assertions (i) to (xvi) of Theorem 2 hold as well.

Proof. Since ~ is a nondegenerate vertex solution of (1) there exists a partition {I, J} of {1..... n} such that Mjj is nonsingular and Mjj~j + qj = 0, MIj~j + qt > O, ~:

> 0.

By the Banach perturbation lemma [16, p. 45] for any n • n matrix ~/ satisfying Ilgt:~ - M~II < IIM;;I1-1, M,~ is nonsingular and

IIM Y;II-
0,

has a solution (x, ~) E R n+l.

~ -->0,

Because the set {(x, ~) I Mx + q~ > 0, (x, ~) ~ R"+I} is open, the last condition is equivalent to the set {(x, ~) [ Mx + q~ > 0, ~ > 0, (x, ~) ~ R "+1} being nonempty, which in turn is equivalent to Mx + q > 0 having a solution x in R". Hence Theorem 2(x) holds if and only if Mx + q > 0 has a solution x in R". The corollary now follows from Theorem 2. Remark 1. The equivalence between (ix) and (x) for the case when M is a skew-symmetric and hence copositive plus matrix degenerates to Williams' characterization [23, Theorem 3] of bounded solutions to a pair of dual programs. For the pair of dual linear programs (a)

min{c~Y I Ay -> b, y ~ 0},

(b)

max{bTz [ ATz --0},

y

(7) Z

Williams' boundedness characterization for both the primal and dual solutions sets is that (a)

Ay ~ 0,

cTy --0,

has no solution y

(8)

and (b)

AXz--O, O~z>--O,

has no solutionz.

We establish now that the negation of (8) is equivalent to the negation of Theorem 2(x) with M=

(o

0

'

q=

(c) -b'

x=

(,)

.

The negation of (8) is (a)

Ay-0,

cry-0,

hasasolutiony (9)

or

(b)

AXz-O,

0~z->0,

hasasolutionz.

We show now that (9) is equivalent to

Ay>-O,

AXz-O,

has a solution (y, z)

O~(y,z)->O, (10)

which is the negation of Theorem 2(x) for the linear programming case. If (9a) holds take z = 0 in (10), if (9b) holds take y = 0 in (10), and hence (9) implies (10). Suppose now (10) holds. If either y or z is zero, then (9) holds. If both y and z are nonzero,

O.L. Mangasarian/ Bounded solutions of linear complementarity problems

165

then again (9a) holds when cTy-- 0 because bTz >-- cVy > 0. H e n c e (9) and (10) are equivalent and T h e o r e m 2(x) degenerates to Williams' condition (8) when M and q are specialized to the linear programming case. It is worthwhile to point out an interesting subtlety in connection with conditions (8). Taken together, conditions (8a) and (8b) are equivalent to T h e o r e m 2(x) and hence guarantee the existence and b o u n d e d n e s s of the solution sets to b o t h of the dual linear p r o g r a m s of (7). H o w e v e r , taking (8a) or (8b) one at a time merely guarantees the b o u n d e d n e s s but not the existence of a solution set to the corresponding linear program. For example, for A = 0, c = - 1 , b = - 1 , condition (8a) is satisfied b y y = 1, but the linear program (7a) has no solution because its objective is u n b o u n d e d below. R e m a r k 2. F r o m the proof that (ix) f f (x) in T h e o r e m 2 we conclude that if the linear c o m p l e m e n t a r i t y problem (1) with a copositive plus M has an unbounded solution set, then each of its solutions ~ lies on a ray of solutions ~ + Ax where A > 0 and x is any nonzero element of the c o n v e x cone {x ] Nix >- O, qTx --0}.

Remark 3. It can be shown that Cottle's theorem 3.1 (from [2]) relating a solution ray of LCP(M, q) and the c o m p l e m e n t a r y cones of M follows f r o m the equivalence of (x) and (xiii) of T h e o r e m 2.

Acknowledgment I am indebted to Dr. W a y n e HaUman for pointing out the counter-example of [6] which shows, a m o n g other things, that LCP(~7/, q) m a y not be solvable for all (/~7/, q) sufficiently close to (M, q) even if LCP(M, q) is solvable for all ~ in R".

References [1] R.W. Cottle, "Nonlinear programs with positively bounded Jacobians", SIAM Journal on Applied Mathematics 14 (1966) 147-158. [2] R.W. Cottle, "Solution rays for a class of complementary problems", Mathematical Programming Study 1 (1974) 59-70. [3] R.W. Cottle and G.B. Dantzig, "Complementary pivot theory of mathematical programming", Linear Algebra and its Applications 1 (1968) 103-125. [4] R. Doverspike, "Some perturbation results for the linear complementarity problem", Mathematical Programming 23 (1982) 181-192. [5] B.C. Eaves, "The linear complementarity problem", Management Science 17 (1971) 612-634. [6] L.M. Kelly and L.T. Watson, "Erratum: some pertubation theorems for Q-matrices", SIAM Journal on Applied Mathematics 34 (1978) 320-321. [7] C.E. Lemke, "Bimatrix equilibrium points and mathematical programming", Management Science I1 (1965) 681-689.

166

O.L. Mangasarian f Bounded solutions o{ linear complementarity problems

[8] C.E. Lemke, "On complementary pivot theory", in: G.B. Dantzig and A.F. Veinott Jr., eds., Mathematics of the decision sciences, Part 1 (American Mathematical Society, Providence, RI, 1968) pp. 95-114. [9] O.L. Mangasarian, Nonlinear programming (McGraw-Hill, New York, 1969). [10] O.L. Mangasarian, "Solution of symmetric linear complementarity problems by iterative methods", Journal of Optimization Theory and Applications 22 (1977) 465-485. [11] O.L. Mangasarian, "Characterizations of linear complementarity problems as linear programs", Mathematical Programming Study 7 (1978) 74-87. [12] O.L. Mangasarian, "Simplified characterizations of linear complementarity problems solvable as linear programs", Mathematics of Operations Research 4 (1979) 268-273. [13] O.L. Mangasarian, "Locally unique solutions of quadratic programs, linear and nonlinear complementarity problems", Mathematical Programming 19 (1980) 200-212. [14] N. Megiddo and M. Kojima, "On the existence and uniqueness of solutions in nonlinear complementarity theory", Mathematical Programming 12 (1977) 110-130. [15] K.G. Murty, "On the number of solutions to the complementarity problem and spanning properties of complementary cones", Linear Algebra and its Applications 5 (1972) 65-108. [16] J.M. Ortega and W.C. Rheinboldt, Iterative solution of nonlinear equations in several variables (Academic Press, New York, 1970). [17] J.-S. Pang, "A note on an open problem in linear complementarity", Mathematical Programming 13 (1977) 360-363. [18] J.-S. Pang, "On Q-matrices", Mathematical Programming 17 (1979) 243-247. [19] S.M. Robinson, "Stability theory for systems of inequalities, Part I: Linear systems, Part II: Differentiable nonlinear systems", SIAM Journal on Numerical Analysis 12 (1975) 754-769 and 13 (1976) 497-513. [20] S.M. Robinson, "Generalized equations and their solutions, Part I: Basic theory", Mathematical Programming Study 10 (1979) 128-141. [21] S.M. Robinson, "Strongly regular generalized equations", Mathematics of Operations Research 5 (1980) 43-62. [22] R.T. Rockafellar, Convex analysis (Princeton University Press, Princeton, NJ 1970). [23] A.C. Williams, "Marginal'values in linear programming", Journal of the Society for Industrial and Applied Mathematics 11 (1963) 82-94.

Mathematical Programming Study 19 (1982) 167-199 North-Holland Publishing Company

ON R E G U L A R I T Y CONDITIONS IN M A T H E M A T I C A L PROGRAMMING Jean-Paul PENOT D~.partement de Math~.matiques, Universit~ de Pau, 64000 Pau, France Received 3 October 1980 Revised manuscript received 15 March 1982

Constraint qualifications are revisited, once again. These conditions are shown to be reminiscent of transversality theory. They are used as a useful tool for computing tangent cones, by the means of generalized inverse function theorems. The finite dimensional case is given a special treatment as the results are nicer and simpler in this case. Some remarks on the nondifferentiable case are also presented.

Key words: Nonlinear Programming, Implicit-inverse Function Theorem, Polar Cones, Tangent Cones, Multipliers, Constraint Qualification, Farkas' Lemma.

O. Introduction

Nonlinear programming can be viewed as the problem of minimizing a functional f under implicit as well as explicit constraints. More precisely, we consider the following classical nonlinear programming problem (P)

minimize f(x), subject to x E A = B fq h-~(C).

Here B (resp. C) is a subset of a Banach space X (resp. Y) and f : X ~ R , h:X~Y (or only h : B ~ Y ) . Taking for C a closed convex cone, or if Y = Y~ x Y2, the product of {0} and a closed convex cone C2 in I"2, we obtain the usual formulation with inequality (and/or equality) constraints. We do not (always) make differentiability assumptions on f, but it is not the chief aim of this paper to relax differentiability assumptions (for such a purpose see [7, 8, 13, 17, 25, 29, 30, 46-48, 54, 55, 61, 63]). We mainly focus our attention on regularity (or qualification) conditions which enable one to write a necessary condition of optimality in terms of the explicit constraint B, eliminating the implicit constraint h-~(C), at the cost of perturbing the objective function f by y* o h, where y* E Y* is a Lagrange-Kuhn-Tucker multiplier. This problem has received much attention (see [1,2,4,6,9,15,24,33, 34,37,41,53,65,71] for a sample of a multitude of papers). Here we focus our attention on the existence of true (or normal) Lagrange multipliers (with the exception of Theorem 4.8). The case of Fritz John multipliers is treated in [34] along similar lines. The best results known to us are those of Guignard [24] (see also 167

168

Jean-Paul Penot[On regularity conditions

[9, 11, 22, 23, 44]) on one hand and those of Duong and Tuy [16] and Robinson [58] on the other hand. Here we try to relate these two lines of thought and present simple proofs of their fundamental results. Our point of view consists in reducing the whole question to the study of simple rules for calculating tangent cones. We point out an analogy with transversality theory; this connection with one of the main pillars of differential topology is not fortuitous. As our purpose here is to shed light on this parallel, we do not insist on the connection with another (fast growing) topic: the generalized differential calculus. This can be made through epigraphs and indicator functions, if one consents to leave differentiable or Lipschitzian functions ([29, 30, 46-48, 54, 61, 63]). The organization of this paper is as follows. We first recall general optimality conditions in terms of tangent and normal cones and introduce the idea of tangential approximation to a set or a function at some point. The second section is devoted to the calculus of polar cones and its applications to multipliers. In Section 3 we focus our attention on regularity or transversality conditions and their significance. We then specialize our study in Section 4 to the case of finite dimensional constraints, while the following section is devoted to the general case. We end this paper with some special cases in which differentiability or regularity conditions can be relaxed. Throughout the paper X and Y denote Banach spaces unless otherwise specified. For any subset A of X and r > 0 we denote by Ar the set of points of A whose norm is not greater than r. Given x ~ X, d(x, A): = inf{d(x, a) I a ~ A} and the Hausdorff distance d(A, B) between A and another subset B of X is d(A, B): = max(e(A, B), e(B, A)) with e(A, B): = sup(d(a, B) I a E A). The closure (resp. interior) of a subset A of X is denoted by cl A or A (resp. int A or _~); its closed c o n v e x hull is denoted by c--o(A).

1. Tangential approximations and optimality conditions We believe that classical tangent cones (and the corresponding subdifferentials) are still the best tool for what concerns necessary conditions. This is not only attested by the numerous papers using them (or derived variants) but proved by elementary examples. H o w e v e r the usual tangent cone concept suffers from the unpleasant drawback of lack of convexity. There are two ways of avoiding this difficulty. The first one consists in restricting one's attention to tangentially convex sets and tangentially c o n v e x functions; these classes include the convex and the smooth classes. The second one follows an idea of Guignard [24] (for subsets) and Pshenichnyi [55] (for functions) which consists in choosing convex cones (resp. convex functions) replacing the given data. This treatment is versatile enough to encompass at the same time the tangentially convex situation and the peritangent-peridifferential approach of Clarke [13] (see also

Jean-Paul Penot/ On regularity conditions

169

[7, 8, 25, 29, 30, 53, 61, 63]) in which convexity is automatically brought into the picture, even when the situation is naturally nonconvex. We recall the following classical definition which dates back to the thirties, with Bouligand, if not earlier. It is also useful in other topics as flow invariance for dynamical systems, or fixed point theory [18, 38, 49]. Definition 1.1. The tangent cone TaA to a subset A of X at a point a in the closure of A is the set of right derivatives of continuous curves c : [ 0 , 1]-~X emanating from a with sufficiently many values in A (more precisely, such that 0 is an accumulation point of c-~(A)) for which the right derivative exists. Equivalently v E TaA iff there exists a sequence (t., an) in (0, +oo) x A with limit (0, a) such that v = lim t-,l(a, - a) or iff 1 lim inf 7d(a + tv, A ) = O. t-~O+

We shall also use the radial tangent cone T~A to A at a, which is the set of v E X such that a + t,v E A for some sequence (t,) in (0, +oo) with limit 0. Obviously Tr~A C TaA, and if A is starlike at a E A, in particular if A is convex, then T a A - - c l ( T ~ A ) , hence T~A and T~aA have the same polar cone which is called the normal cone to A at a: N~A = (T~A) ~ = {x* E X [ (x*, x) - 0 for each x E X, hence 0 E Of(a).

f(a)< +oo, then

We get in this way an elementary f r a m e w o r k which encompasses at the same time the classical differential calculus and the subdiferential calculus for convex functions. H o w e v e r , the computing rules (or an easy direct analysis) give only the following optimality condition which includes the classical normality condition (N)

- f ' ( a ) ~ N~A.

Lemma 1.4. [47, Proposition 4.1, Theorem 4.6]. If f:X-->R" attains a finite (local) minimum on A at a ~ A, then for each v E T~A dr(a, v)>-O. If T~A is convex, if dr(a,. ) is convex and finite at 0 there exists some y* E Of(a) with (iq)

- y * ~ NaA.

There is an analogous statement with d_f(a,-) and (IaA) ~ where IaA = X - ~ To(X "~ A) is the interior tangent cone (or the cone of 'interior displacements') but we can assert that there is a y * E O_f(a) A ( - N a A ) only when _0f(a) = af(a); this is the case when f is semi-diferentiable at a. Note that the assumption dr(a, 0 ) < + ~ (resp. 0f(a, 0 ) > - ~ ) is lacking in [47 T h e o r e m 4.6]. Some flexibility can be introduced by the way of the following notions of approximation. Note that they play a key role in sufficient conditions of optimality [40]. Many variants [39, 40, 43] of these notions exist in the literature, the closest one being [10]; however, the differentiability assumption in [10] is more restrictive.

Definition 1.5. A subset C of X is an approximation to a subset A of X at a point a E A if there exists a map 4> : C --> A tangent to Ix (the identity of X ) at a with ~b(a) = a. It is a strict approximation to A at a if ~b is strictly tangent to Ix at a. We call C a continuous approximation of A at a if ~b is continuous on a neighborhood of a. In fact it suffices to define ~b in a neighborhood of a in C. The following characterization can be useful. Note also that tk can be extended by Ix on X ~, C so that the extended map is still differentiable at a (but not necessarily con-

Jean-Paul Penot] On regularity conditions

171

tinuous elsewhere); of course a strict approximation is a continuous approximation. Lemma 1.6. C is an approximation to A at a iff lim

d(x, A)/d(x, a) = O.

x~a

x~C~{a}

Proof. As d(x, A ) f ( a ) + f ~ x - a) one gets a g ( a ) = 8/(a), the periderivative of f at a (or generalized gradient of Clarke [13]). However, taking P as large as possible and g as close as possible to f yields better results. We give only two elementary examples.

Example 1.11. (The Christmas garland). L e t us consider the one dimensional unconstrained problem of minimizing the function f:R--> R given by f ( x ) = I x l - Isin x I.

Using the necessary condition 0 E ~f(a) in terms of the periderivative of f we find an infinite number of candidates (the set ,rZ) whereas the inclusion

0 ~ 0I(a) has only a = 0 as solution, and 0 is the global minimizer of [.

Example 1.12. L e t X = R 2, A = R • {0} O {0} x R, and let f be any continuously differentiable function on X. The necessary condition 0 E 8[(0) + (ffoA) ~ = f'(O) + X *

using the peritangent cone is satisfied by any f, whereas the necessary condition -f'(O) E (ToA) ~

is satisfied only if f'(O) = O. A similar result holds for A = {x E R2 [ max(x 1, X 2) ~---0}.

2. Generalized Farkas' lemma and multipliers In this section we consider the general nonlinear programming problem (P); h is supposed to be differentiable at a, a local solution to problem (P).

Definition 2.1. L e t g be an upper c o n v e x approximation to y at a, a local solution to problem (P). A continuous linear form y* on Y is said to be a multiplier for

174

Jean-Paul Penot] On regularity conditions

problem (P) at a (with respect to g) if y* E Nh(a)C and (Mg)

0 E Og(a) + y* o h'(a) + NaB.

Again Og(a) denotes the subdifferential of g in the sense of convex analysis. If f is differentiable at a, this can be written, with g(x) = f ( a ) + ['(a)(x - a): (M)

0 ~ f'(a) + y* o h'(a) + NaB.

In other words, if y* is a multiplier, a is a critical point with respect to B of the perturbed function f + y* o h: the implicit constraint has disappeared. We shall obtain the existence of multipliers under a general regularity condition with the help of a generalization of the Farkas' lemma. This is a very classical device; however we need to improve our knowledge of the computing rules for polar cones. We begin with the following well-known lemma. Lemma 2.2. Let M (resp. P) be a closed convex cone in X (resp. Y ) and let u ~ L(X, Y ) with transposed map u* E L ( Y * , X*). Then the polar cone Q0 of Q = M n u-l(P) is the weak*-closure of M ~ u*(P~ This result follows easily from the bipolar theorem applied to the cone L = {(x, u(x)) I x E M, u(x) E P}. Now we prove that under a general condition we have Q0 = M0+ u,(p0); this condition is much more general than the classical one u ( M ) A i n t P ~ 0, as it does not require that int P g 0. The proof we give below is an adaptation of the (incomplete) proof of [34, Theorem 2.1]; for a quite different proof see [50]. Theorem 2.3. Let X, Y, M, P, Q be as in the preceding lemma. Suppose u ( M ) - P = Y. Then

(M n u-I(P)) 0= M ~

u*(P~

Proof. Using the lemma, it suffices to show that M ~ u*(P ~ is weak*-closed. We invoke the Krein-~mulian theorem t o show this by proving that for any closed ball B* with center 0 and radius r in X* the set B* n (M~ u*(P~ is weak*-closed in B*. Let x* ~ B* be the weak*-limit o f a net (x*)i~z c B* with x~f=w~f,+u*(~),

w*EM ~

~Ep

~

Given any y ~ Y we can find x E M and z ~ P with y = z - u(x). Hence

(z~,, y)

= ( ~ , z - u ( x ) ) max(r, s) y = t ( u ( r t - ~ ( b - a)) - s t - l ( d - c)) E t ( u ( B - a ) - ( C - c))

and Y = (0, + ~ ) ( h ' ( a ) ( B - a ) - C + h ( a ) ) .

178

Jean-Paul Penot[ On regularity conditions

Thus we get an interpretation of Robinson's regularity condition [58] in terms of radial transversality. In Section 5 we give a direct proof of the fact that this condition (R) implies the linearizing property (L)

TaA = TaB n h'(a)-~(Th~a)C)

when B and C are convex and h is strictly differentiable. On the other hand it is well known that (L) holds true when B and C are differentiable submanifolds and h &B C [3, 21, 31, 35]. A number of classical results of convex analysis can be cast into the framework of r-transversality. We give here only one such restatement; for other important instances connected with subdifferential calculus see [50]. Theorem 3.4. multifunction jection p : X x neighborhood

[16, 58, 64]. Let X and Y be Banach spaces and let F : X --, Y be a with closed convex graph G. If for some g = (a, b ) ~ G the proY~Y maps T~a,b)G onto Y, then F is open at (a,b): for any N of a, F ( N ) is a neighborhood of b.

The analogy with the classical open mapping theorem is made clear; this may suggest other results in this direction. Now we shall use this result to study the relationship of condition (R) with the boundedness of the set K of Lagrange-Kuhn-Tucker multipliers. This problem has already been considered by Gauvin [19], Gauvin and Tolle [20] in the finite dimensional case and by Zowe and Kurcyusz [71] in the infinite dimensional case. The novelty here lies in the fact that f is nondifferentiable and that the boundedness of K is obtained under the condition (R) instead of the stronger condition (Rr). Let us start with a simple result. Lemma 3.5. Given convex cones M in X, N = - P in Y, and u E L ( X , Y), the following two conditions are equivalent. (5) cl(u(M) + P) = Y; (b) u*-~(M ~ n pO = {0}.

Proof. It is easily verified that ( u ( M ) + p)O = u.-~(MO) n pO.

Then, the bipolar theorem gives the result. Theorem 3.6. Let F * be a bounded convex subset of X*, and let M (resp. P ) be a convex cone in X (resp. Y ) and u ~ L ( X , Y). Suppose

K = {y* E p0 [ 3 w* E F*: u*(y*) + w* ~ M ~ = p0 n u * - l ( M ~ F*) is non void. Then the conditions (~) and (b) of the lemma are equivalent to condition (c) and are implied by condition (d) below.

179

Jean-Paul Penot / On regularity conditions

I f M and P are closed condition (d) is a c o n s e q u e n c e o f condition (a) (a) u ( M ) + P = Y ; (c) the a s y m p t o t i c cone T ~ K = {y* ~ Y* I V Y* E K V t - 09* + ty* E K} o f K is

{0}; (d) K is bounded. (2) r

(b) (c) ~

(d) ~

(a).

Proof. It is clear that (d) implies (c). To p r o v e the other implications let us first

observe that if B is a bounded c o n v e x set and if C is a closed c o n v e x cone, then, ifB+C~0 T=(B + C) = C.

The inclusion C C T~(B + C) is obvious. Conversely, if v ~ T~(B + C), then there exists a sequence (b,, c,) in B • C and a E B + C such that a + nv = bn + c..

H e n c e v = lim n-~(b. - a + c.) = lira n-~c. ~ cl C = C. L e t Y* be an arbitrary element of K. We know [60] that T ~ K = { y * E Y * ]V t > - O g * + t y * E K } .

Thus T=K = {y* E Y* ] V t ~ 0 y* + ty* ~ p0, u*(y*) + tu*(y*) E M ~ - F*} = T~P ~ n u*-~(T~(M ~ - F * ) ) = p0 N

u*-l(M~

and (b) and (c) are equivalent. When M and P are closed, the multifunction F : X - - - ~ Y given by F ( x ) = u ( x ) + P if x ~ M , F ( x ) = 0 if x E X - - - M has a closed c o n v e x graph G and 0 E c o r e ( F ( X ) ) if u ( M ) + P = Y. Using T h e o r e m 3.4 we can find r > 0 such that any y U Y~ can be written y = u(x) + z

withx~Mr=MnX, z~Pr=PnYr. L e t y* and y* be arbitrary elements of K. L e t w* and if* in F * be such that x* = u * ( y * ) + w* ~ M ~

:~*=u*(9*)+ff*EM

~

Then, for any y = u ( x ) + z in Y~, with x E Mr, z E Pr, we have ( y * - 9", y) = ( y * - 9", u(x)) + ( y * - 9", z ) = (x* - ~*, x ) -

( w * - ~ * , x ) + ( y * - 9", z )

_< (~*, - x ) + (~* - w*, x) + (9", - z ) -< r[~*[ + dr + r[9*[

180

Jean-Paul Penot/ On regularity conditions

where d is the diameter of F*. Then [y* - y*[ = SUpy~v,(y* - y * , y ) is bounded.

Corollary 3.7. Let g be a continuous convex upper approximation to f at a, a local solution to problem (P). Suppose B and C are tangentially convex at a and h(a) respectively and h is differentiable at a. Suppose the set K of multipliers with respect to g at a is non void. Then (a) if K is bounded, then h ' ( a ) ( T a B ) - Th(~)C is dense in Y ; (b) conversely, if h ' ( a ) ( T o B ) - Th(o)C is dense in Y, then T~K = {0}; (c) /f h ' ( a ) ( T a B ) - Th(~)C = Y, then K is bounded.

Proof. This follows from Theorem 3.6 and the fact that F * = Og(a) is bounded as g is convex and continuous. This corollary can be applied when f is tangentially convex and f is semiderivable at a in the following sense: Of(a," ) = dr(a,. ). As we have just seen, the regularity condition (R) is almost equivalent (and, when Y is finite dimensional, is actually equivalent) to the boundedness of the set K of multipliers, provided K is non void. Now, our purpose consists in showing that K is non void as soon as (R) is satisfied and Y is finite dimensional or, when (R ~) is satisfied. Before turning to this aim, let us relate conditions (R) and (R r) and relate them to a classical Slater type condition. Our assumptions are slightly weaker than Zowe's ones in [70, 71]; see also [34, 53]. The following lemma [70, Theorem 2.2] will be useful; we present a simple proof for the sake of completeness.

Lemma 3.8. I f N and P are two cones o[ a t.v.s, and if P is convex with a non void interior, then int(N + P ) = N + int P.

Proof. As N + int P is open and included in N + P, we have int(N + P ) D N + int P. Let x ~ int(N + P ) and let q E int P. There exists t > 0 small enough so that x - tq E int(N + P). Let (n, p) E N x P be such that x - tq = n + p. Then x = n + p + tq E N + P + i n t P C N + i n t P

as P + int P C int(P + P ) = int P since P is a convex cone.

Proposition 3.9. (a) (R r) implies (R) and is equivalent to it if B and C are convex and if Y is finite dimensional or if Z : - h ' ( a ) ( T r B ) - Trhca)C contains a finite codimensional subspace W of Y. (b) If h ' ( a ) ( T ~ B ) and Trh(a)C are convex and dense in h'(a)(TaB) and Th(a)C

Jean-Paul Penot/ On regularity conditions

181

respectively and if one o f the latter cones has a non void interior, then (R~ and (R) are equivalent. (c) I f B and C are closed convex subsets and if one o f them has a non void interior, (R r) and (R) are equivalent. (d) If C is a c o n v e x cone (R r) can be written h ' ( a ) ( T ~ B ) + R h ( a ) - C = Y. (e) I f B and C are c o n v e x (R r) is equivalent to R o b i n s o n ' s condition (Ro)

0 E c o r e ( h ' ( a ) ( B - a) + h ( a ) - C)

(f) I f C is a c o n v e x cone with non e m p t y interior, (R r) is equivalent to (S)

there exists v ~ T r B such that h ( a ) + h ' ( a ) v E int C.

Proof. (a) The implication (R r) ~ (R) is immediate as T~B C TaB and TrcC C TcC with c = h(a). When B and C are convex and (R) holds Z is dense in Y as T~C.=cI(T~C) and f ( c l S ) C c l f ( S ) for a continuous map f. If p : Y ~ Y [ W denotes the canonical projection, this argument shows that p ( Z ) is dense in Y / W hence coincides with Y [ W [60, Corollary 11.7.3]. Hence Z = p - ' ( p ( Z ) ) = Y and (W) holds. (b) Under the assumption of (b) and condition (R), the convex cone Z is dense in Y, hence coincides with Y as int Z = int cl Z = int Y = Y. (c) Suppose B and C are convex and (R) holds. If int C is non empty, assertion (b) can be applied. If i n t B is non empty, then i n t T ~ B = int(0, + ~ ) ( B - a) also is non empty. Let us apply Theorem 3.4 with F ( x ) = h ' ( a ) ( x ) - (C - c) for x E X and g = (b, h'(a)(b)), where b is some point of int B. As (0, + ~ ) ( F ( X ) -

h ' ( a ) ( b ) ) = h ' ( a ) ( X ) - TrcC = Y,

F is open at g: for some r > 0 we have F ( B ) D h ' ( a ) ( b ) + Yr.

Hence Z -- h'(a)((O, + ~ ) ( B - a)) - (0, + ~ ) ( C - c) D F ( B ) - h ' ( a ) ( a )

has a non void interior. It follows as in (b) that Z = Y. (d) This rewritting follows from T~cC = (0, + ~ ) ( C - c) = C - (0, + ~ ) c = C + [0, +~)c

(e) Follows from lemma 3.3.

- (0, +~)c

= C -

Rc.

182

Jean-Paul Penot/ On regularity conditions

(f) Using l e m m a 3.8 we see that int Z = h ' ( a ) ( T ~ B ) - int(C - Rc) = h ' ( a ) ( T r B ) + R c - int C. If (S) holds, 0 in interior to Z, hence Z = Y and (R r) is satisfied. Conversely, if (R r) holds, int Z = int Y = Y contains 0, so that there exist w E T r B and r U R with h ' ( a ) w + rc E int C. If r > l, we take v = r-~w, so that h ( a ) + h ' ( a ) v = r-~(h'(a)w + rc) E int C whereas forr- 0, small enough if necessary, we can find ai E A

(i = 1. . . . . 2n) with f'(0)(a~) = b~,

f ' ( 0 ) ( a ~ + . ) = - b~,

i = 1 . . . . . n.

Then we define h : Y ~ X b y h

yibi =

y~ ai + i=l

yTai+, i=l

with t § = max(t, 0), t - = m a x ( - t , 0) for t E R. The unit ball Y~ of Y is m a p p e d b y h into A. M o r e o v e r , writing f ( x ) = f'(O)(x) + r(x)

Jean-Paul Penot/ On regularity conditions

183

with limlxl~0+.x~A r(x)/lx I = 0, we see that k = f o h is differentiable at 0 with derivative Iy. L e t p E (0, 1) be such that for e a c h y E Yp

Ik(y)- yl

lyl

and k is c o n t i n u o u s on Yp. L e t v U V = Yr

W e define k~ : Yp ~ Y b y

k~(y) = v + y - k(y). T h e n ko is c o n t i n u o u s and m a p s Yp into itself as Iko(Y)l --- Ivl + ly - k(y)l -< 89 + 89 0 small e n o u g h one has

f(a + X , ) ~ f ( a ) + Y,. In this case, or, m o r e generally, w h e n e v e r ['(a)((-TaA) fl TaA)) = Y, g can be c h o s e n to be differentiable at f(a). R e m a r k 4.3. (a) O n e can extend T h e o r e m 4.1 to a p a r a m e t r i z e d v e r s i o n with essentially the s a m e proof. (b) It suffices to s u p p o s e f is H a d a m a r d - d i f f e r e n t i a b l e at a instead of F r e c h e t differentiable at a. M o r e o v e r X could be a n y locally c o n v e x space. Corollary 4.4. Let X, Y be n.v.s, with Y ]inite dimensional and let B C X with a

convex continuous approximation a + M at a E B. If f : B --* Y is continuous and differentiable at a with f'(a)(ToM)= Y, then there exists a neighborhood V of

184

Jean-Paul Penot/ On regularity conditions

f ( a ) in Y and a mapping g: V o B g(f(a)) = a.

semi-differentiable at f ( a ) with f o g - ~ I v ,

Proof. Again we s u p p o s e a = 0, f ( a ) = 0. L e t $ : M ~ B be an a p p r o x i m a t i o n m a p p i n g with $ ' ( 0 ) = Ix. T h e n fl = f o~b is differentiable at 0, c o n t i n u o u s in a n e i g h b o r h o o d of 0 and satisfies fi(O)(ToM)= f'(O)(ToM) = Y. H e n c e there exists a n e i g h b o r h o o d V of 0 in Y and g ~ : V ~ M semi-differentiable at 0 with g l ( 0 ) = 0 , f ~ o g ~ = I v ; thus g = ~ b O g l satisfies g ( 0 ) = 0 , f o g = f o ~ b O g l = f ~ ~

Iv. T h e following result is r e m i n i s c e n t of the classical implicit f u n c t i o n t h e o r e m and will be useful f o r our purposes. T h e o r e m 4.5. Let A be a convex subset of a n.v.s. X, let Y be a finite dimensional n.v.s, and let f : A ~ Y be continuous on a neighborhood of a E A and differentiable at a. If f ' ( a ) ( T a A ) = Y, the subset S = [-l(f(a)) of A admits the convex approximation Z = (a + N ) n A at a, with N = K e r f'(a). In particular, if N n TaA = c l ( N n T~A), then

TaS = N n TaA. Proof. As in the p r o o f of T h e o r e m 4.1. we s u p p o s e a = 0 , b = 0 and we c o n s t r u c t a right i n v e r s e h : Y ~ X of f'(0) such that h ( Y O C A, h is positively h o m o g e n e o u s and t h e r e exists m --- 1 with h ( Y O C Xm. L e t us set, for (t, y, z) E (0, 1] • Y~ x Z~ k,(y, z) = k(t, y, z) = t - l f ( t h ( y ) + (1 - t)z). Then, for each t ~ (0, 1], kt is differentiable at (0, 0) and Dk,(0, 0) = (Iy, 0). In fact, if r ( x ) = f ( x ) - f ' ( 0 ) x and if a : R + ~ R + is a n o n d e c r e a s i n g f u n c t i o n with a ( 0 ) = lira,_.0 a(r) = 0 and

Ir(x l--< (Ixl)lxl for each x ~ X small enough, we h a v e [kt(y, z) - y[ = [t-~r(th(y) + (1 - t)z)[

0 shows. Suppose we have defined a sequence (x.) in B for n - p with x. E F(x,_,) for 1 - n - p, d(xn, xn-O 0 and a

Jean-Paul Penot[ On regularity conditions

192

neighborhood U of a in B such that for each b E U we have, with A =

B n h-~(C), d(b, A) O. We can now use these observations to formulate an appropriate set of conditions. To do so, we note that the requirements on h involving the objective function and the Lagrangian of (2.2) are independent of the scale of h, but those involving the constraints (i.e., that x0 + h ~ C and g(xo) + g'(xo)h E QO) are not. To simplify the latter, we enlarge slightly the class of vectors h considered by requiring that h E Tc(xo) and g'(xo)h E Too(g(xo)), where Tc(xo) denotes the tangent cone to C at x0 (i.e., the polar of the normal cone aOc(Xo)), and similarly for Q~ Note that, since C and Q~ are convex, any h satisfying the earlier requirements will necessarily satisfy the latter. For small enough h, the converse is true if C and Q are polyhedral, but generally not otherwise. The steps just described therefore lead us to the following general secondorder sufficient condition: Definition 2.1. Suppose (x0, u0) is a point satisfying (1.2). The second-order su17icient condition holds at (Xo, Uo) with modulus /x > 0 if for each h ~ Tc(xo) with

g'(xo)h ~ TQo(g(xo)),

f'(xo)h = O,

one has (h, Sf"(Xo, uo)h ) >! I~ Ilhll 2. We remark that it would not have changed Definition 2.1 if we had written f'(xo)h ~ O, since that inequality implies f'(xo)h = 0 in the presence of (1.2). To see this, note that h and - [ f ' ( x 0 ) + g'(xo)*Uo] belong respectively to the tangent and normal cones to C at x0. Thus, 0 ~< (f'(x0) + g'(xo)*Uo, h)

= f'(xo)h + (Uo, g'(xo)h).

S.M. Robinson[ Generalized equations, Part H

205

However, since g(xo) belongs to the normal cone to Q at Uo, Uo also belongs to the normal cone to Q~ at g(xo) (see [22, Corollary 23.5.4]). As g'(xo)h belongs to the corresponding tangent cone, we have (u0, g'(xo)h) p Ilhll2 = p,

and the theorem now follows by contraposition.

206

S.M. Robinson~Generalized equations, Part H

Theorem 2.2 shows that a stationary point of (1.1) satisfying the second-order sufficient condition is a strict local minimizer. A reasonable question to ask is whether it is also an isolated local minimizer: that this, whether there is some neighborhood of xo containing no other local minimizer of (1.1). As we shall now see, this is not so, even for very simple problems. Consider, for instance, the problem minimize

89 2,

subject to

x 6sin(I/x) = 0

/sin(I/0):= 0].

(2.5)

Here C = R = Q ; the feasible region is {0}U{(nTr) - 1 1 n = - 1 , - + 2 .... }. The second-order sufficient condition is satisfied at the origin; however, the origin is a cluster point of the feasible region and every feasible point is a local minimizer. One might reply that such bad behavior is not very surprising since (2.5) is a bad problem. This is quite true, and we shall see that a result of the type we seek can indeed be proved if we exclude bad problems: that is, if we impose a constraint qualification. The qualification we shall use is the requirement that the constraints g(x) E Q~ x~C

of (1.1) be regular at x0 in the sense of [18]; that is, that 0 ~ int{g(x0) + g'(xo)(C - xo) - Q~

(2.6)

We have shown in [17] that (2.6) is a sufficient qualification for the derivation of optimality conditions; and in fact for the standard problem (1.4), the condition (2.6) is equivalent to the well-known constraint qualification of Mangasarian and Fromovitz [12]. We shall now show that if the constraints of (1.1) are regular at a point x0 which, together with some u0, satisfies (1.2), then the second-order sufficient condition ensures that x0 is actually an isolated stationary point of (1.1). In order to prove this, it will be convenient to establish first certain continuity results that we shall use here and later in the paper. We therefore introduce at this point the perturbed optimization problem minimize

f ( x , p),

subject to

g ( x , p ) E Q~

(2.7)

x~C,

where p is a perturbation parameter belonging to a topological space P, and x is the variable in which the minimization is done. The functions f and g are defined from D x P to R and R m respectively, where 1-2, Q and C are as previously defined. We shall identify (1.1) with the particular case of (2.7) arising when p is

S.M. Robinson/Generalized equations, Part II

207

some fixed P0 ~ P ; thus x0 is a stationary point of (2.7) for p = p0. Our interest here will be in predicting, from information in (1.1), aspects of the behavior of (2.7) when p varies near P0, such as solvability, location of minimizers, etc. In all of what follows we make the blanket assumption that for each p ~ P, [ ( . , p) and g ( . , p) are Fr~chet differentiable on s that [, g, [' and g' are continuous on g] x P, and that f ( - , po) and g( 9 po) are twice continuously Fr6chet differentiable on g~. The stationary-point conditions for (2.7) are

0 E f'(x, p) + g'(x, p)*u + O~bc(X), 0 E - g(x, p) + a~0o(u),

(2.8)

and we define U0: = {u E R m [ (x0, u) satisfies (2.8) for p = P0}. We note that by the results of [16], if the constraints

g(x, p) ~ QO, x ~ C,

(2.9)

of (2.7) are regular at x0 for the fixed value p = p0, then there are neighborhoods M1 of x0 and N1 of p0 such that for any (x, p ) E M1 x N1 satisfying (2.9), the system (2.9) is regular at x (for the given, fixed, value of p). We shall now show that in addition the multipliers in (2.8) are uniformly bounded, and in fact the set of all such multipliers is an upper semicontinuous function of (x,p). This extends a result in [ll], also given in [5]. Theorem 2.3. If the system (2.9) is regular at Xo for p = P0, then there exist

neighborhoods ME of Xo and N2 of Po, such that if U : M E X N 2 ~ R m and SP : N 2 ~ M 2 are multifunctions defined by U(x, p) := {u E R m [ (x, u, p) satisfies (2.8)} for (x, p) E ME X N2, SP(p ) := {x ~ ME I for some u, (x, u, p) satisfies (2.8)} for p E N2, then U and SP are upper semicontinuous. Proof. We first show that U is locally bounded at (Xo,Po). Assume, on the contrary, that there are sequences {xi} C g~ and {pi} c P converging to x0 and p0 respectively, and a sequence {ui} C R m with limi_.~llu~l]= + 0% such that for each i the triple (x~, ui, p~) satisfies (2.8). With no loss of generality we can suppose that uJllu~ll converges to some y. For each i, (2.8) implies that u~ E Q (a cone) and (ui, g(x~, p~)) = 0; dividing by Ilu~lland passing to the limit we find that y E Q and (y, g(x0, p0)) = 0. Again from (2.8), we have for each i,

f'(Xi, pi) + g'(xi, pi)*Ui E -- Oqic(Xi); again dividing by ]lui[I and passing to the limit, using the fact that OqJcis a closed multifunction whose values are cones, we obtain g'(xo, po)*y E-Oq~c(Xo). By regularity, for some e > 0 there is a point x~ E C with g(xo, po)+g'(xo, po)

208

S.M. Robinson/Generalized equations, Part H

(x~ - x0) + ey E Q~ Using our information about y, we find that 0 >t (y, g(Xo, Po) + g'(x0, p0)(x, - x0) + ey) dyll 2 > 0, a contradiction. It follows that there must exist neighborhoods M 2 of x0 in 12 and N2 of P0 in P, and a compact set K C R m, such that if (x,p)~ M 2 x N2, then U(x, p) C K. Without loss of generality we can suppose that M 2 and N2 are small cnough so that M 2 is compact and that for (x, p) e M 2 x N2 the values llg(x,p)ll, IIf'(x, P)[I and IIg'(x, P)ll all satisfy some uniform bound. Now let

h(x, p, u) := [ f'(x, p) + g'(x, p ) * u ] ; I

- g(x,

p)

we note that there is some compact set L such that if (x, p, u ) E M2x N2 x K, then h(x, p, u) E L. Now denote by G the graph of the multifunction 3qJc x 3~bo, and define H := {(x, p, u, v) [ (x, p) E M2 • N2, (x, p, u, v) ~ graph h, (x, u, - v) E G}. The continuity assumptions and the properties of Otkc and O~o imply that H is closed in M2 x N: • R ~' x R "§ However, if (x, p, u, v) ~ H, then (x, p) ~ M2 x N2 and (x, u, p) satisfies (2.8), so that u ~ K and therefore v E L. As K and L are compact it follows that the projection of H on the space of the first three, o r o f the first two, components of (x, p, u, v) is closed. The first of these projections is the graph of the multifunction U, and the second is that of SP-% Thus U and SP are closed; however, as the image of U is contained in the compact set K and that of SP in the compact set M2, it follows that U and SP are actually upper semicontinuous. This completes the proof. The next theorem gives conditions under which a stationary point of (l.l) is isolated. In fact, we shall prove isolation for stationary points of an extended form of (1.1), namely: minimize subject to

f ( x ) + ~(x - x0, E(x - Xo)), g(x) ~ QO

(2.10)

x ~C, where E is any sufficiently small n x n symmetric matrix. The presence of E may seem superfluous, but we shall use this form of the theorem in Section 4. In the meantime, readers who do not care about E may take it to be zero, in which case (2.10) becomes (1.1). Theorem 2.4. Suppose that [ and g are twice continuously Fr~chet differentiable on 12, and that the regularity condition (2.6) holds at a stationary point Xo of

S.M. Robinson/Generalized equations, Part H

209

(1.1). Suppose also that for each Uo such that (Xo, Uo) satisfies the stationary conditions (1.2), the second-order sufficient condition of Definition 2.1 holds for (1.1) at (Xo, Uo) with some positive modulus. Then there exist a neighborhood W of Xo and a positive number ~, such that for each n x n symmetric matrix E with IIEI]< e, and for each stationary point x of (2.10), either x ~ W or x = Xo. Proof. By contraposition. Suppose that there are sequences {xi} of points in R n -~ {x0} and {Ei} of n x n matrices such that {xi} converges to x0, {E~} converges to the zero matrix (denoted by 0), and for each i, xi is a stationary point of (2.10): that is, for some ui, 0 E f'(xi) + (xi - xo)Ei + g'(xi)*ui + a~bc(Xi), 0 E - g(xD + 0tpQ(ui).

(2.11)

Now regard P as the space of n • n symmetric matrices under the Euclidean norm, and take po = 0. Applying T h e o r e m 2.3, we find that there are neighborhoods W0 of x0 and V0 of 0 (in P ) such that if x i E W o and E~EV0, then ui E U ( x , E~); further, the multifunction U is locally bounded. Thus the u~ are uniformly bounded, so by passing to subsequences if necessary we can assume that: (a) for each i, xi E Wo, Ei ~ Vo and ui E U (xi, El), (b) (x, - xo)/llx, - xoll converges to some h E R n ; (c) ui converges to some u0. We note that by closure of U we have uo~ U(xo, O), which is the set of admissible multipliers for (1.1) at xo. Now from (2.11) we find that for each i, g(xO ~ QO and x~ E C; reasoning as in the proof of T h e o r e m 2.2 we can establish that h ~ Tc(xo), g'(xo)h E Too(g(xo)) and ['(xo)h >~O. From (2.11) we see that for each i, 0 = (ui, g(xi)) = (ui, g(xo))+ (ui, g'(xo)(Xi - Xo)) + o(llx, - xoll), where we have used the fact that the u~ are uniformly bounded. However, as ui E Q and g(xo) E Qo we have (u, g(xo)) 10. Using (2.11) again we find that for each i,

f'(xd

+ (x~ - xo)E~ + g ' ( x i ) * u ~ ~ -

a~c(x~),

S.M. Robinson/Generalized equations, Part H

210

so that 0 >! f ' ( x i ) ( x i - Xo) + (xi - Xo, Ei(xi - Xo)) + (ui, g'(xi)(xi - Xo)),

and upon division by

IIx,- x011and

passage to the limit we find, using (2.12), that

.f'(xo)h 0 consider the scalar function s defined for t E [0, 1] by --

X0

-

g(x,)

where (xt, ut):= ( 1 - t)(xo, u0)+ t(xi, ui). This function is continuous on [0, 1] and (continuously) differentiable on (0, 1). Further, since (x0, u0) satisfies (1.2) and (xi, ui) satisfies (2.11), we have s(0)~ 0 ~ s(1). By the mean-value theorem, there is then some t E (0, 1) (depending upon i), with

0~>s'(t)=

x~-xo u~ -

uo

[~e"(x,,u,)+E~ ' l_

-

g'(x,)

g'(x,)*][x~ 0

II_ui -

uo

= (xi - Xo, [~"(xt, ut) + Ei](xi - Xo)).

(2.13)

Dividing (2.13) by Ilxi - x0112and passing to the limit, we find a contradiction to the second-order sufficient condition and thereby complete the proof. We have thus shown that when regularity is added to the second-order sufficient condition, the stationary points x0 of (1.1) are isolated. It is an easy consequence of this fact that if the conditions of Theorem 2.4 hold in some compact region of R n, that region contains only finitely many stationary points, if any. However, this result says nothing about the behavior of stationary points, or minimizers, of the perturbed problem (2.7). Such behavior is treated in the next section.

3. Local solvability of perturbed nonlinear programming problems In this section we establish two continuity properties of the perturbed problem (2.7). The notation remains the same as in Section 2; in addition, we denote the distance from a point a to a set A by d [ a , A ] := inf{lla - a'll I a E A} (+ oo if A = ~), and we write B for the Euclidean unit ball whether in R" or R " ; the space involved should be clear from the context. Our first theorem shows that if the second-order sufficient condition holds at a local minimizer at which the constraints are regular, then that local minimizer persists under small perturbations.

Theorem 3.1. S u p p o s e t h a t f o r p = po, (2.7) satisfies the s e c o n d - o r d e r sufficient c o n d i t i o n at Xo a n d s o m e uo E Uo a n d its c o n s t r a i n t s are r e g u l a r at Xo.

S.M. Robinson[Generalized equations, Part H

211

Then for each neighborhood M of Xo in 0 there is a neighborhood N of po such that if p E N, then (2.7) has a local minimizer in M. Proof. By hypothesis, for p = po the constraints of (2.7) are regular at Xo. By [18, Theorem 1] there are neighborhoods M0 and No of Xo and p0 respectively, and a constant ~, such that for each x E C n M0 and p E No,

d[x, F(p)] ~< ~d[g(x, p ), Q~ where F(p) := {x E C [ g(x, p) E Q~ L e t E0 be the modulus for the second-order sufficient condition at (x0, u0); let E E (0, e0) and choose a positive 8 so that (x0 + 28B)C M0 n M n v, where V is the neighborhood of x0 given by Theorem 2.2 for the chosen ~. Select a neighborhood N6 of po with N6 C No, and some positive a, such that ff p E N6 and if xl, x2 E xo + 26B with IlXl- x211~ 5, then II/(xl, p ) - [ ( x 2 , po)l[ < E~2/16 =:/3. Next, find a neighborhood N of P0 with N C N~ and such that if p E N, then

C sup{llg(x, p) - g(x, p0)ll I x ~ x0 + ~B) ~<min{a, ~8}. Denote/(x0, p0) by qb0. Now choose any p E N. If F ( p ) n {x I IIx - x011-- ~ is not empty, let x' be any point in it. One has x' E C N {x0 + ~B} and g(x', p) E Q~ so

d[x', F(p0)] ~< ~d[g(x', po), Q~ ~< ffllg(x', p0)- g(x', p)ll ~<min{a, ~}. Thus there is some x ~ F(po) with IIx'-x~ll~<min{a, 89 and therefore with IIx;- xoll >I IIx'- xoll- llx'- x~ll/> ~. Accordingly, by Theorem 2.2 we have

f(x~, Po) >1 60 + ~ ( ~ ) 2 = cko + 2/3. Also,

IIx6- x011~< IIx'- xoll § IIx'- gill ~ ~ § ~ = ~, so x; ~ x0 + 28B; as IIx6- x'll ~ ~ we have IIf(x', p) - S(x6, p0)ll < ~. Therefore

f(x', p) >! f(x6, Po) -IIf(x', p) - f(x6, > &0 + 2/3 -/3 = 6o +/3.

po)l] (3.1)

It is also true that

d[xo, F(p)] ~< ~d[g(xo, p ), OO] < ~llg(x0, p) - g(xo, po)ll ~<min{a, 89 Accordingly, there is some x " ~ F ( p ) with

IIx0- x"ll ~<min{a, ~}.

Thus I~(x", p)

-

[(x0, p0)ll 0 . i~M

On the other hand, the segment [~; x - Y`] is contained in Q; and consequently,

f~(y`;x-g)-----O

for each i E M .

Hence f'(y`; x - Y`) -> O, and Y` is optimal. We will now establish two 'explicit' formulas for M c and N(Quc; Y`), which will imply that T h e o r e m 8 includes, as a special case, [2, T h e o r e m 5.1] and [3, Theorem 5.3].

The complement of the largest element M of ~ in I is equal to the set of constraints which are identically zero on the entire feasible set:

L e m m a 9.

M c = {i [ i = 1. . . . . p; f,(x) = OVx ~ Q}.

(4.2)

Remark. The set of the right-hand side of (4.2) was introduced by Abrams and

J.-J. Strodiot and V. Hien Nguyen/ Kuhn-Tucker multipliers and nonsmooth programs

237

Kerzner in [1] where the reader can also find an algorithm for constructing this set in the case of differentiability. See also [15] for another computational method. Proof. For simplicity of notation, let us denote by J= the set in the right-hand side of (4.2). M c is contained in J =. Let i ~ M c, and let x E Q, x ~ 4. Since Qi c is convex, the direction d - - x - ~ belongs to the tangent cone T(Qic; 4). By the definition of M c, d E Di. H e n c e f~(~, d) -> 0. It follows now from [11, Theorem 23.1] that f~(~+td)>0 t

for a l l t > 0 .

In particular, if we take t = 1, then f~(~ + d)>-0, or equivalently, f~(x)>-O. The opposite inequality is immediate because x E Q. Hence f~(x) = 0. That is, i E J=. J= is contained in M ~. Let i ~ J~. Then, of course, i E I. Suppose that i ~ Me; that is,

Di f) T(QLG 4) ~ O. Take an element d in this intersection. Since Di~ 0, by L e m m a 1, 19, C int

T(Q~; 4).

It follows from L e m m a 2 that d E T(Qx; 4). Then by using the definition of tangent cone in the sense of convex analysis, we have that there exist sequences {tj} ~ 0 and {dj} ~ d such that ~ + ti dj ~ QI for all j. Moreover, as {tj dj} ~ 0 and fk(~) < 0 for all k ~ I, we can deduce from the continuity of fk, k ~ I, that + tj dj ~ Q

for j large enough.

So, Q being convex, the segment [4; ~ + tjdj] is contained in Q. Since f~(x)= 0 for all x E Q, the directional derivative f~(~; dj) -- 0

for j sufficiently large.

In the limit j -~ ~, we get f'~(~; d) -- 0, and thus d E D~. But, this contradicts the assumption that d ~ D~. Hence J ~ C M L Lemma 10. The normal cone N(QM~; ~) to Qu~ at ~ is equal to the negative polar cone of the intersection iEM c

where D~(~) is the cone of directions of constancy of f~ at 4: D~(~) = {d ~ R ~ I there exists r > 0 such that f,(~ + ~d) =f,(~) V a ~ ]0, ~[}.

238

J.-J. Strodiot and V. Hien Nguyen/ Kuhn-Tucker multipliers and nonsmooth programs

Proof. For simplicity of notation, let us set

N(QMo;4) is contained in [D~c(2)]-. It is clear that each direction of constancy of fi, i E M c, belongs to the tangent cone T(QM0; 4). Hence, to conclude, it suffices to take the negative polar cone for obtaining that

[D~c(2)]- ~ [T(QMc; 4)]- = N(QMc; 4). [D~c(2)]- is contained in N(QMc; 4). Let us consider a d E [D~e(2)]-. By the definition of normal cone in the sense of convex analysis (see [11]), we have to prove that ( y - 2 , d)--O, i E M, such that 0 E 3f(X) + ~ , tx, Of,(~) + [D;=(J/)]-, iEM

where M = I \ Y = and J= is defined as in the right-hand side o f (4.2). So, b y an entirely different t e c h n i q u e , we h a v e obtained, as a special c a s e of our results, the c o m p l e t e c h a r a c t e r i z a t i o n of optimality in [2, T h e o r e m 5.1] and [3, T h e o r e m 5.3].

Acknowledgment The a u t h o r s wish to t h a n k J e a n - B a p t i s t e H i r i a r t - U r r u t y ( C l e r m o n t - F e r r a n d ) for suggesting s o m e i m p r o v e m e n t s in the presentation.

References [1] R. Abrams and L. Kerzner, "A simplified test for optimality", Journal of Optimization Theory and Applications 25 (1978) 161-170. [2] A. Ben-Israel, A. Ben-Tal and S. Zlobec, "Optimality conditions in convex programming", in: A. Pr6kopa, ed., Survey of mathematical programming (North-Holland, Amsterdam, 1979) pp. 153-169. [3] A. Ben-Tal and A. Ben-Israel, "Characterizations of optimality in convex programming: The nondifferentiable case", Applicable Analysis 9 (1979) 137-156. [4] A. Ben-Tal, A. Ben-Israel and S. Zlobec, "Characterization of optimality in convex programming without a constraint qualification", Journal of Optimization Theory and Applications 20 (1976) 417--437. [5] F.H. Clarke, "Generalized gradients and applications", Transactions of the American Mathematical Society 205 (1975) 247-262. [5] F.H. Clarke, "A new approach to Lagrange multipliers", Mathematics o[ Operations Research 1 (1976) 165-174. [7] F.H. Clarke, "Generalized gradients of Lipschitz functionals", Mathematics Research Center Technical Summary Report No. 1687, University of Wisconsin, Madison (1976).

240

J.-J. Strodiot and V. Hien Nguyen/ Kuhn-Tucker multipliers and nonsmooth programs

[8] J.-B. Hiriart-Urruty, "On optimality conditions in nondifferentiable programming", Mathematical Programming 14 (1978) 73-86. [9] J.-B. Hiriart-Urruty, "Tangent cones, generalized gradients and mathematical programming in Banach spaces", Mathematics of Operations Research 4 (1979) 79-97. [10] V.H. Nguyen, J.-J. Strodiot and R. Mifflin, "On conditions to have bounded multipliers in locally Lipschitz programming", Mathematical Programming 18 (1980) 100-106. [11] R.T. Rockafellar, Convex analysis (Princeton University Press, Princeton, N J, 1970). [12] R.T. Rockafellar, "Clarke's tangent cones and the boundaries of closed sets in R"", Nonlinear Analysis 3 (1979) 145-154. [13] R.T. Rockafellar, "Directionally Lipschitzian functions and subdifferential calculus", Proceedings of the London Mathematical Society 39 (1979) 331-355. [14] H. Wolkowicz, "Geometry of optimality conditions and constraint qualifications: The convex case", Mathematical Programming 19 (1980) 32-60. [15] S. Zlobec and B. Craven, "Stabilization and calculation of the minimal index of binding constraints", Mathematische Operationsforschung und Statistik, Series Optimization 12 (1981) 203-220. [16] S. Zlobec and D.H. Jacobson, "Minimizing an arbitrary function subject to convex constraints", Utilitas Mathematica 17 (1980) 239-257.