Operators, analytic negligibility, and capacities

2.4.

OPERATORS, ANALYTIC NEGLIGIBILITY,

Let o(T) Hilbert space Hilbert space normal T on H reducing T on

AND CAPACITIES*

and Op(T) denote the spectrum and point spectrum of a bounded operator T on a H. Such an operator is said to be subnormal if it has a normal extension on a K, K m H. For the basic properties of subnormal operators see [I]. A subis said to be completely subnormal if there is no nontrivial subspace of H which T is normal. If T is completely subnormal, then Op(T) is empty. A

necessary and sufficient condition in order that a compact subset of a completely subnormal operator was given in [2].

~

be the spectrum of

If X is a compact subset of ~ , let R(X) denote the functions on X uniformly approximable on X by rational functions with poles off X. A compact subset Q of x is called a peak set of R(X) if there exists a function f in R(X) such that f = I on Q and Ifl < I on X \ Q ; see [3, p. 56]. The following result was proved in [4]. THEOREM.

Let T be subnormal on H with the minimal normal extension

~=Ild~

on K, K c

H. Suppose that Q is a nontrivial proper peak set of R(o(T)) and that E(Q) # 0. Then E(Q)H (~} and H, the space E(Q)H reduces T, TIE(Q)H is subnormal with the minimal normal extension E(Q)N on E(Q)K, and o(TIE(Q)H) c Q. Further, if it is also assumed that R(Q) = C(Q), then TIE(Q)H is normal. Thus, in dealing with reducing subspaces of subnormal operators T, it is of interest to have conditions assuring that a subset of a compact X(=o(T)) be a peak set of R(X). Problem I. Let X be a compact subset of ~ and let C be a rectifiable simple closed curve for which Q = clos [(exterior of C) n X] is not empty and C N X has Lebesgue arc length measure 0. Does it follow that Q must be a peak set of R(X)? In case C is of class C 2 (or piecewise C2), the answer is affirmative and was first demonstrated by Lautzenheiser [5]. A modified version of his proof can be found in [6, pp. 194-195]. A crucial step in the argument is an application of a result of Davie and Cksendal [7] which requires that the set C ~ X be analytically negligible. (A compact set E is said to be analytically negligible

if every continuous function on

~

which is analytic on an

open set V can be approximated uniformly on V U E by functions continuous on ~ and analytic on V U E; see [3, p. 234].) The C 2 hypothesis is then used to ensure the analytic negligibility of C n X as a consequence of a result of Vitushkin [8]. It may be noted that the collection of analytically negligible sets has been extended by Vitushkin to include Liapunov curves (see [9, p. 115]) and by Davie [10, Sec. 4] to include "hypo-Liapunov" curves. Thus, for such curves C, the answer to Problem I is again yes. The question as to whether a general rectifiable curve, or even one of class C z, for instance, is necessarily analytically negligible, as well as the corresponding question in Problem I, apparently remains open, however. As already noted, Problem I is related to questions concerning subnormal operators. The problem also arose in connection with a possible generalization of the notion of an "areally disconnected set" as defined in [6] and with a related rational approximation question. Problem 2 below deals with some estimates for the norms of certain operators associated with a bounded operator T on a Hilbert space and with two capacities of the set o(T). Let y(E) and ~(E) denote the analytic capacity and the continuous analytic capacity

(or

AC capacity) of a set E in ~ . (For definitions and properties see, e.g., [3, 11, 9]. A brief history of both capacities is contained in [9, pp. 142-143], where it is also noted that the concept of continuous analytic capacity was first defined by Dolzhenko [12].) It is known that for any Borel set E, ( ~ , ~ s ~ ) ~ ( ~ ) ~ ( ~ ) ing was proved in [13]. *C. R. PUTNAM.

2142

; see [11, pp. 9, 79].

Purdue University, Department of Mathematics,

West Lafayette,

The follow-

Indiana 47907.

THEOREM.

Let T be a bounded operator on a Hilbert space and suppose that

(T-~)(T-~) ~ D >0

(1)

holds for some nonnegative operator D and for all z in the unbounded component of the complement of o(T). Then IDL I/2 ~ y(o(T)). If, in addition, (I) holds for all z in ~ and if, for instance, OD(T) is contained in the interior of o(T) [in particular, if Op(T) is empty], then also IDII/~ ~ ~(o(T)). Problem ~: Does condition (I), if valid for all z in ~ , but without any restriction On Op(T), always imply that IDI i/2~e(o(T)), or possibly even that IDI