No title

Vol. 101, No. 1 DUKE MATHEMATICAL JOURNAL © 2000 APPROXIMATE SPECTRAL SYNTHESIS IN THE BERGMAN SPACE S. M. SHIMORIN ...

Author: Jonathan Wahl (Academic Editor)

5 downloads 414 Views 4MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Vol. 101, No. 1

DUKE MATHEMATICAL JOURNAL

© 2000

APPROXIMATE SPECTRAL SYNTHESIS IN THE BERGMAN SPACE S. M. SHIMORIN

1. Introduction. From a hard-analysis point of view, the main result of the present paper is an approximation theorem for extremal (Bergman-inner) functions in terms of finite zero divisors. This corresponds to the famous Carathéodory-Schur theorem from the early 1900s, which states that every function in the unit ball of H ∞ (the uniform algebra of bounded holomorphic functions in the unit disk) can be approximated by finite Blaschke products, in the topology of uniform convergence on compact subsets of the disk. Along the way, we find a theorem about kernel functions for weighted Bergman spaces of the type that estimate them away from the diagonal. The motivation for these results is the study of z-invariant subspaces in the Bergman space. It is known that the lattice of z-invariant subspaces in Bergman spaces has a very complicated structure. But one may single out among all z-invariant subspaces those of simplest nature, the zero-based ones. What z-invariant subspaces can be approximated by zero-based ones? What z∗ -invariant subspaces can be approximated by finite-dimensional ones? In this paper, we answer these questions (see Theorems 1 and 2), and we discuss some relations with rational approximation and cyclic vectors for the backward shift. Let X be a Banach space of functions analytic in the unit disk D = {z ∈ C : |z| < 1} of the complex plane C. Suppose that X is invariant with respect to the operator Mz of multiplication by the independent variable. Many important problems concerning the structure of the lattice of subspaces of X that are closed and invariant with respect to Mz (or, simply, z-invariant) are related to problems of spectral synthesis. Let Y be the space dual to X. Then the functionals kλ of evaluation of functions in X at the points of D, kλ : f → f (λ); λ ∈ D, are eigenvectors of the operator Mz∗ : λI − Mz∗ kλ = 0; (n)

the functionals kλ of evaluation of the derivatives (n)

kλ : f → f (n) (λ);

n ∈ Z+ , λ ∈ D,

Received 15 May 1997. Revision received 16 December 1998. 1991 Mathematics Subject Classification. Primary 46E20; Secondary 30H05. Author’s research partially supported by Russian Foundation for Fundamental Studies grant number N 96-01-00693. 1

2

S. M. SHIMORIN

are root vectors of Mz∗ . Here Z+ is the set of nonnegative integers. Assume that all (n) root vectors of Mz∗ are of the form kλ , n ∈ Z+ , λ ∈ D. (This is not the case, e.g., (n) if X = H ∞ (D), the space of bounded analytic functions in D.) The vectors kλ ∗ form a weak -complete system in Y . The spectral synthesis of a subspace of Y that is closed and invariant with respect to Mz∗ (or, simply, z∗ -invariant) is the process of reconstructing it, starting with these root vectors. If J ⊂ Y is a finite-dimensional z∗ -invariant subspace, then the classical Kronecker theorem states that J is the linear (n) span of the root vectors kλ contained in J . For z-invariant subspaces in X, this means that if I ⊂ X is a z-invariant subspace of finite codimension, then there exists a function v : D → Z+ (a divisor of I ) such that v(λ) = 0 only for finitely many λ ∈ D, and I = Iv = f ∈ X : f (n) (λ) = 0, λ ∈ D, 0 ≤ n ≤ v(λ) − 1 . (1) This observation gives rise to the classical notions of a zero-based (or divisorial) z-invariant subspace and a synthesizable z∗ -invariant subspace. For any function v : D → Z+ (a divisor function), the z-invariant subspace Iv generated by v is defined as in formula (1). For any z-invariant subspace I ⊂ X, the divisor vI of I is defined as vI (λ) := max n ∈ N : ∀f ∈ I, ∀k = 0, 1, . . . , n − 1, f (k) (λ) = 0 . (We agree that the maximum of the empty set is zero.) Here N stands for the set of positive integers. A z-invariant subspace I ⊂ X is said to be zero-based if I = IvI . In this case, if vI (λ) = 0 only for finitely many λ ∈ D, then I is said to be finitely zero-based. A z∗ -invariant subspace J ⊂ Y is said to be synthesizable in some topology if (n) (n) J= k λ : kλ ∈ J , where the symbol denotes the closure of the linear span. It is easy to check that a z-invariant subspace I ⊂ X is zero-based if and only if its annihilator I ⊥ ⊂ Y is synthesizable in the weak∗ topology of Y . It is well known that there exist z-invariant subspaces that are not zero-based (and, consequently, nonsynthesizable z∗ -invariant subspaces). The classical example is given by the subspace H p in the Hardy space H p , where is an inner function with nontrivial singular part. This means that the procedure of forming the closure of the linear span of root vectors is too rigid to yield all z∗ -invariant subspaces, so that one needs another, more flexible, procedure of their reconstruction by root vectors; possibly, this procedure could be applicable to all z∗ -invariant subspaces. Such a procedure, called approximate spectral synthesis, was suggested by Nikolskii in 1978 [21]. His idea was to use not only the operation of closure of the linear

APPROXIMATE SPECTRAL SYNTHESIS

3

span but also the passage to the limit in order to obtain an arbitrary z∗ -invariant subspace. To describe this procedure in more precise terms, we recall the notion of the lower limit of a sequence of subspaces in a Banach space. If (En )n≥1 is a sequence of subspaces in a Banach space X, then the lower limit E of this sequence is defined as E = lim En := x ∈ X : ∃ xn ∈ En : x = lim xn n→∞ n→∞ = x ∈ X : lim dist(x, En ) = 0 . n→∞

It is easy to verify that the lower limit of an arbitrary sequence of subspaces is always closed, and if each En is invariant with respect to a certain bounded linear operator T ∈ L(X), then E is also T -invariant. The following definitions describe different versions of approximate spectral synthesis, that is, synthesis via forming linear spans and passing to the lower limit. Definition 1. A z∗ -invariant subspace J ⊂ Y is said to admit weak approximate spectral synthesis if there exists a sequence Jn of z∗ -invariant subspaces such that dim Jn < +∞ and J = lim n→∞ Jn . Definition 2. A z-invariant subspace I ⊂ X is said to admit weak approximate spectral cosynthesis if there exists a sequence (In )n≥1 of z-invariant subspaces such that dim(X/In ) < +∞ and I = lim n→∞ In . Definition 3. A z∗ -invariant subspace J ⊂ Y is said to admit strong approximate spectral synthesis if there exists a sequence (Jn )n≥1 of z∗ -invariant subspaces such that dim Jn < ∞, J = lim n→∞ Jn , and J⊥ = lim n→∞ (Jn )⊥ . Definition 4. A z-invariant subspace I ⊂ X is said to admit strong approximate spectral cosynthesis if there exists a sequence In of z-invariant subspaces such that dim(X/In ) < +∞, I = lim n→∞ In , and I ⊥ = lim n→∞ In⊥ . Here E⊥ denotes the preannihilator of a subspace E ⊂ Y in the space X. Obviously, if J ⊂ Y is weak∗ -closed, then J admits strong approximate spectral synthesis if and only if I = J⊥ admits strong approximate spectral cosynthesis, and a z-invariant subspace I ⊂ X admits strong approximate spectral cosynthesis if and only if I ⊥ admits strong approximate spectral synthesis. However, for weak approximate spectral synthesis and cosynthesis, the similar property fails, as we see later. (The main reason is that the relation E = lim n→∞ En only implies the inclusion E ⊥ ⊃ lim n→∞ En⊥ , but not the equality.) Note that the supplementary condition Jn ⊂ Jn+1 in Definition 1 (and In ⊃ In+1 in Definition 2) implies that J is synthesizable (correspondingly, that I is zero-based). In [21], Nikolskii proposed the following conjecture. Conjecture 1. Any weak ∗ -closed z∗ -invariant subspace of Y admits weak approximate spectral synthesis.

4

S. M. SHIMORIN

In the case where X is the Hardy space H 2 , this fact was proved by Douglas, Shapiro, and Shields [6]. In this context, it is easy to verify that any z∗ -invariant subspace also admits strong approximate spectral synthesis. This result is based on the famous Beurling theorem, which states that any z-invariant subspace in H 2 is of the form H 2 , with some inner function , and on approximation of inner functions by finite Blaschke products. The calculation of lower limits of z∗ -invariant subspaces is closely related to problems of rational approximation. Indeed, suppose that Y is identified with a space of functions analytic in D in such a way that the duality between X and Y is determined by the Cauchy pairing f, g := fˆ(n)g(n), ˆ n≥0

where fˆ(n) and g(n) ˆ are the Taylor coefficients of the functions f and g, and the infinite sum on the right is understood in an appropriate sense. In this case, the (n) functionals kλ correspond to the functions (n)

kλ (z) =

n! zn , (1 − λz)n+1

and if v : D → Z+ is a divisor such that v(λ) = 0 only for λ belonging to some finite set ⊂ D, then the annihilator Iv⊥ ⊂ Y is the collection of rational functions r having poles of order at most v(λ) at the points 1/λ, λ ∈ \ {0} and such that the function zr(z) has a pole of order at most v(0) at infinity. Therefore, the problem of calculating lim n→∞ (Ivn )⊥ for a sequence of divisors (vn )n≥1 is equivalent to the problem of describing those functions in Y that can be obtained as limits of rational functions with preassigned poles determined by the divisors vn . For the weighted Lp spaces on T, rational approximation with preassigned poles was studied by Tumarkin [29]–[31] and Katsnelson [17], [18]. In [10], Gribov and Nikolskii calculated the lower limits lim n→∞ (n H 2 )⊥ in H 2 for arbitrary sequences of inner functions (n )n≥1 (see also [22, Chapter 2]). In the study of rational approximation with preassigned poles, the notion of the capacity of a divisor of poles is important. (The divisor of poles of a rational function r is the function v : C → Z+ such that v(λ) = v ≥ 1 if r has a pole of order v at λ, and v(λ) = 0 otherwise.) As shown in [17], [18], and [29]–[31], for the Hardy spaces and the weighted Lp spaces on T, the following capacity is of importance: 1 , v(λ) 1 − cap(v) = |λ| λ∈D−

where D− = C \ D, and v : D− → Z+ is the divisor of poles. In this case, the calculation of the lower limits lim n→∞ (Ivn )⊥ depends on the behaviour of the capacities cap(vn ). Different notions of capacity related to arbitrary spaces Y were introduced


5

in [10]. In what follows, we use the following version of the definition: CapY (v) := inf

|λ|≤1/2

−1 (0) , distY kλ , Iv⊥

where v (λ) = v(1/λ), and Iv ⊂ X is defined by formula (1). In [10], it was proved that, for any sequence of divisors vn : D− → Z+ , the relation lim n→∞ (Ivn )⊥ = Y is equivalent to limn→∞ CapY (vn ) = +∞. By duality, we immediately obtain

−1 1 = sup Re g(λ) : g ∈ Iv , g ≤ 1, |λ| ≤ . (2) CapY (v) 2 Thus, estimates of capacities related to Y are equivalent to certain qualitative versions of uniqueness theorems for the functions in X. Another problem associated with weak approximate spectral synthesis and rational approximation is that of describing the noncyclic vectors for the operator Mz∗ in the space Y . With respect to the Cauchy pairing, Mz∗ is the operator S ∗ of the backward shift: ∗ f (z) − f (0) S f (z) = . z For the Hardy space H 2 , S ∗ -noncyclic vectors were described by Douglas, Shapiro, and Shields in [6]. They obtained two versions of the description: in terms of pseudocontinuations and in terms of rational approximation. The latter description reads as follows: An element f ∈ H 2 is noncyclic for the backward shift if and only if f = limn→∞ rn and supn≥1 cap(vn ) < +∞, where rn are rational functions with poles in D− and vn is the divisor of the poles of rn . A similar description of S ∗ noncyclic vectors in the general situation is a weaker version of Conjecture 1 (which was also suggested as a problem in [21]). Conjecture 2. A function f ∈ Y is noncyclic for S ∗ if and only if f = limn→∞ rn and supn≥1 CapY vn < +∞, where rn are rational functions with poles in D− and vn is the divisor of the poles of rn . The “if” part of Conjecture 2 easily follows from the definition of the capacity CapY (see [21] or [22, Chapter 2]). The “only if” part is a consequence of Conjecture 1. Indeed, if f ∈ Y is S ∗ -noncyclic, then we can form the z∗ -invariant subspace n J= Mz∗ f, n ≥ 0 = Y. We then take an approximation J = lim n→∞ Jn with dim Jn < +∞, and we choose some λ satisfying |λ| ≤ 1/2 and a subsequence (nk )k≥1 such that

(0) inf dist kλ , Jnk > 0. k≥1

6

S. M. SHIMORIN

Finally, we write f = limk→∞ rnk , rnk ∈ Jnk . Our aim in the present paper is to study the problems related to approximate spectral (co)synthesis in the case where X is the Bergman space. For p > 0, the space p p La (D, dm2 ) (or, simply, La ) consists of functions analytic and area-integrable with power p in the disk D. If f p := |f (z)|p dm2 (z), D

p

p

then · is a norm in La for p ≥ 1, and f − gp is a metric in La for p ∈ (0, 1). Here dm2 denotes the normalized area measure in D. The first simple observation pertaining to this situation (this observation appeared in [23] as a referee comment) concerns weak approximate spectral cosynthesis. Suppose that X satisfies the additional condition (the division property) f ∈ X,

f (λ) = 0 ⇒

f ∈ X. z−λ

(3)

Definition 5. A z-invariant subspace I ⊂ X is said to be of index 1 if dim(I /zI ) = 1. Sometimes this property of I is called the codimension-1 property. As shown in [24, Lemma 3.1], under condition (3) the index-1 property of I is equivalent to the property similar to (3): f ∈ I,

f (λ) = 0 ⇒

f ∈I z−λ

for some λ such that vI (λ) = 0. Obviously, this property holds for any zero-based subspace. Moreover, it is stable under passage to the lower limit. Indeed, suppose that I = lim n→∞ In , where In are z-invariant subspaces with dim(In /zIn ) = 1. If vI (λ) = 0, then there exists g ∈ I with g(λ) = 0 and gn ∈ In such that g = limn→∞ gn . If f ∈ I and f (λ) = 0, we choose fn ∈ In such that f = limn→∞ fn ; we have fn − fn (λ)/gn (λ) gn f = lim ∈ lim In = I. z − λ n→∞ z−λ n→∞ (The continuity of division by z − λ follows from the closed graph theorem.) Therefore, under condition (3), any z-invariant subspace in X admitting weak approximate spectral cosynthesis is of index 1. It is well known (see [3]) that in all radial weighted Bergman spaces L2a (D, w dm2 ) (consisting of functions analytic and square area-integrable with a radial weight w in the unit disk), there exist z-invariant subspaces I for which dim(I /zI ) > 1. (For any n ∈ N ∪ {∞}, it is possible to find a z-invariant subspace I with dim(I /zI ) = n.) Therefore, weak approximate spectral cosynthesis (and, a fortiori, strong approximate spectral synthesis and cosynthesis) is not always possible in Bergman spaces.


7

However, it turns out that in the unweighted Bergman space L2a , the index-1 property is the only obstruction for approximate spectral cosynthesis (weak and even strong), and for weak approximate spectral synthesis there are no obstructions. These facts comprise the following two theorems, which are the principal results of this paper. Theorem 1. Suppose that I ⊂ L2a is a z-invariant subspace of index 1. Then I admits strong approximate spectral cosynthesis. Theorem 2. Any z∗ -invariant subspace J ⊂ (L2a )∗ admits weak approximate spectral synthesis. We see that Theorem 2 provides another example of a Banach space X for which Conjecture 1 holds. The space dual to L2a with respect to the Cauchy pairing is the Dirichlet space D, consisting of functions analytic in D and having finite Dirichlet integral, supplied with the norm fˆ(n)2 (n + 1). f 2D := n≥0

The discussion after Conjecture 2 and Theorem 2 results in the following consequence that describes S ∗ -noncyclic vectors in D in terms of rational approximation. Theorem 3. A function f ∈ D is noncyclic for the operator of backward shift if and only if f = limn→∞ rn and supn≥0 CapD vn < +∞, where rn are rational functions with poles in D− and vn is the divisor of the poles of rn . The hard-analysis basis for the proofs of Theorems 1 and 2 consists of, first, an estimate of weighted Bergman reproducing kernels away from the diagonal, given by Proposition 2 in §3, and, second, an approximation theorem for extremal (Bergmaninner) functions in terms of finitely zero-based extremal functions (finite zero divisors), given by Theorem 1B in §2. In §2 we also show that Theorem 2 is an easy consequence of Theorem 1. The main difficulty lies in the proof of Theorem 1. This proof is based on the concept of an extremal function introduced by Hedenmalm in [12]. A function ϕ ∈ L2a (D, dm2 ) is said to be extremal (or Bergman-inner) if ϕ = 1 and ϕ ⊥ zn ϕ for any n ≥ 1. The extremal functions may be regarded as analogs of the inner functions for the Bergman spaces. They turned out to be very important for the inner-outer factorization (in the sense of Bergman spaces) and for the study of the structure of z-invariant subspaces (see [1], [7], [8], [12], and [26]). For any z-invariant subspace I ⊂ L2a , the elements of I zI having unit norm are extremal functions; in addition, if dim I zI = 1, (k) then there exists a unique extremal function ϕI ∈ I zI such that ϕI (0) > 0, where k = vI (0). It is easily seen that ϕI is the solution of the following extremal problem: sup Re g (k) (0) : g ∈ I, g ≤ 1 .

8

S. M. SHIMORIN

The function ϕI is called the extremal function for the subspace I . In the case where I is zero-based (I = Iv ), the function ϕI is also called a canonical zero divisor, because it may serve as a canonical factor (responsible for zeros) in the factorization of functions from Iv (see [12]). One of the principal results of [1] states that if I ⊂ L2a is a z-invariant subspace of index 1, then I is generated (and, hence, uniquely determined) by the extremal function ϕI ; that is, I=

z n ϕI : n ≥ 0 .

This fact allows us to prove Theorem 1 with the help of the technique of extremal functions and to split the proof into two independent steps (as done in the case of the theorem by Douglas, Shapiro, and Shields for the Hardy space; see [22, Chapter 2] for such an approach). In the first step, we show that pointwise convergence of extremal functions implies the lower limit relations for z-invariant subspaces generated by these functions and for their annihilators. This is done in §3, where we also calculate the lower limits of z-invariant subspaces and their annihilators in the general situation. An important point in justifying the passage from convergence of extremal functions to the approximation of corresponding z-invariant subspaces is an estimate of weighted Bergman reproducing kernels, given by Proposition 2. The second step consists in approximating an arbitrary extremal function by extremal functions generating finitely zero-based z-invariant subspaces (i.e., by Bergman spaces analogs of finite Blaschke products or by canonical finite zero divisors in the case of the unweighted Bergman space). This is done in §4, where we completely describe the class of functions that can be approximated by such finitely zero-based extremal functions in the radial weighted Bergman spaces. In some sense, we obtain a Bergman spaces analog of the Carathéodory-Schur theorem stating that any function in the unit ball of H ∞ (D) is a pointwise limit of finite Blaschke products. Finally, §5 is devoted to some more questions and observations concerning approximate spectral synthesis and cosynthesis in Bergman spaces. p For a special class of z-invariant subspaces in La , weak approximate spectral cosynthesis was studied by Korenblum in [20]. He proved that κ-Beurling-type zp invariant subspaces in La generated by the singular measures supported by Carleson sets admit weak approximate spectral cosynthesis (see [15] and [20] for precise definitions). However, the examples of functions invertible but noncyclic in the Bergman spaces due to Borichev and Hedenmalm (see [5]) show that κ-Beurling-type invariant subspaces do not exhaust all invariant subspaces of index 1, so that the results of Korenblum (for the space L2a ) do not cover Theorem 1 in full generality. 2. Preliminary observations. Suppose that X and Y are as above. For any subset M ⊂ X, we denote by [M] the smallest z-invariant subspace containing M; that is, [M] =

zn f : n ≥ 0, f ∈ M .

9


If M consists of a single element f , we simply write [f ] to denote the cyclic zinvariant subspace generated by f . It is easy to see that every cyclic z-invariant subspace is of index 1 (see, e.g., [24, Corollary 3.3]). The following proposition shows that Theorem 2 is a consequence of Theorem 1. Proposition 1. Suppose that Y is separable as a Banach space. If every cyclic z-invariant subspace in X admits strong approximate spectral cosynthesis, then every weak ∗ -closed z∗ -invariant subspace in Y admits weak approximate spectral synthesis. Proof. Let J ⊂ Y be a weak∗ -closed z∗ -invariant subspace. Then I = J⊥ ⊂ X is a z-invariant subspace in X, and J = I ⊥ . Since the space X is separable (because so is its dual Y ), we can choose a sequence (fn )n≥1 that is contained and dense in I . If In = [fn ], we have I= In , n≥1

whence J=

n≥1

In⊥ .

By assumption, for each n ≥ 1, we have In⊥ = lim Jnk k→∞

for some sequence (Jnk )k≥1 , with dim Jnk < +∞. It remains to use the following general lemma. Lemma 1. Suppose that Y is a separable Banach space and Jnk n, k ≥ 1 are some subspaces of Y . If Jn = lim Jnk , k→∞

then there exist sequences of indices (nj )j ≥1 and (kj )j ≥1 such that

k

Jn = lim Jnjj .

n≥1

j →∞

Proof. We choose a sequence (y m )m≥1 contained and dense in any m, n ≥ 1 we have

lim dist ym , Jnk = 0,

k→∞

for any n ≥ 1 there exists a number kn ∈ N such that ∀k ≥ kn , ∀m ≤ n,

1 dist ym , Jnk ≤ n . 2

n≥1 Jn . Since for

10

S. M. SHIMORIN

kn +lj Now, let j → nj , lj be some enumeration of N × N. Putting J (j ) := Jnj j , we show that Jn = lim J (j ) . j →∞

n≥1

Let m ∈ N and N ≥ m. By the choice of kn , the inequality 1 dist ym , Jnkn +l ≤ N 2

(4)

holds for any n ≥ N and l ≥ 1. For each n = 1, . . . , N − 1, the same inequality holds for all indices l ∈ N except for a finite number of them, because lim dist ym , Inkn +l = 0.

l→∞

Hence, the inequality (4) holds for all pairs (n, l) but a finite number of them, whence lim dist ym , J (j ) = 0.

j →∞

This means that ym ∈ lim J (j ) . j →∞

So, we have the inclusion

Jn ⊂ lim J (j ) .

n≥1

j →∞

On the other hand, if y ∈ lim j →∞ J (j ) , then y ∈ lim Jnkn +l = Jn , l→∞

which proves the converse inclusion and accomplishes the proof of Proposition 1. Before proving Theorem 1, we note that if X is a Hilbert space, then for a z-invariant subspace I , strong approximate spectral cosynthesis is equivalent to approximation of the orthogonal projection to I by orthogonal projections to finitely zero-based subspaces In . We have the following lemma. Lemma 2. Suppose that X is a Hilbert space and E and (E n )n≥1 are closed subspaces of X. The following statements are equivalent: (i) PEn → PE in weak operator topology, where PEn and PE are the operators of orthogonal projections to En and E, respectively; (ii) E = lim n→∞ En and E ⊥ = lim n→∞ En⊥ .


11

Proof. (1) (i) ⇒ (ii). If f ∈ E, then f = w-lim n→∞ PEn f (the limit in the weak topology of X). Since PEn f ≤ f , we have f = limn→∞ PEn f in the norm and, therefore, E ⊂ lim n→∞ En . Similarly, E ⊥ ⊂ lim n→∞ En⊥ , because PE ⊥ = I − PE = limn→∞ PEn⊥ . Since ⊥ ⊥ lim En ⊂ lim En n→∞

n→∞

for arbitrary subspaces En ⊂ X, we arrive at (ii). (2) (ii) ⇒ (i). For any f ∈ H , we have PE f − PEn f = PE f − PEn PE f + PE ⊥ f = PE f − PEn PE f − PEn PE ⊥ f. As n → ∞, the first summand tends to zero because E = lim n→∞ En , and so does the second because E ⊥ = lim n→∞ En⊥ . Now, let I ⊂ L2a be a z-invariant subspace of index 1, and let ϕI be the extremal function for I . We recall that, by the theorem of Aleman, Richter, and Sundberg [1], we have I = [ϕI ]. Theorem 1 is a compilation of the following two independent theorems. Theorem 1A. If ϕn and ϕ are extremal functions in the Bergman space L2a and ϕ = limn→∞ ϕn pointwise in D, then lim P[ϕn ] = P[ϕ]

n→∞

in the weak operator topology. Theorem 1B. If ϕ is an extremal function in L2a , then there exists a sequence of extremal functions (ϕ n )n≥1 such that ϕ = limn→∞ ϕn pointwise in D and [ϕn ] are finitely zero-based invariant subspaces. The proofs of Theorems 1A and 1B are presented in §3 and §4, respectively. Theorems 1A and 1B are analogs for the Bergman space of some results leading to the strong approximate spectral synthesis in the Hardy space H 2 . The Hardy space version of Theorem 1A follows easily from the results of [10]. The approximation of the inner functions by finite Blaschke products is a consequence of the CarathéodorySchur theorem mentioned above (or of the Frostman theorem stating that if θ is an inner function, then (θ − µ)/(1 − µθ) ¯ is a Blaschke product for all µ ∈ D \ E, where E is an exceptional set of zero logarithmic capacity). 3. Lower limits of z- and z∗ -invariant subspaces. For a function f ∈ L2a , we denote by A2f the closure of the polynomials in the space L2 (D, |f |2 dm2 ); · f stands for the norm in this space. (The notation · without indices denotes the norm in L2a .) It is easy to check that [f ] = f · A2f

12

S. M. SHIMORIN

for any function f ∈ L2a (i.e., g ∈ [f ] ⇔ g = f · h, where h ∈ A2f and g = hf ). A description of the spaces A2ϕ for the extremal functions ϕ was obtained in [1]. Fixing an extremal function ϕ ∈ L2a , consider a function . defined as follows: 2 ϕ (λ) /(z, λ) dm2 (λ), .(z) = D

where /(z, λ) is the Green function for the bilaplacian in D, z − λ 2 + 1 − |z|2 1 − |λ|2 . /(z, λ) = |z − λ|2 log 1 − λ¯ z It is well known that /(z, λ) is nonnegative everywhere; thus, . is also nonnegative. The function . may be viewed as the solution of the following boundary value problem: .|T = 0, (5) 0.(z) = |ϕ(z)|2 − 1. Here T = ∂D = {ζ ∈ C : |ζ | = 1}, and 0 is the operator ∂ 2 /(∂z ∂ z¯ ), that is, one quarter of the usual Laplacian. For more information about boundary value problems of type (5), see [1], [7], [8], [12], and [19]. Proposition 4.5 in [1] states that A2ϕ consists of the functions f analytic in D for which 2 f (z) .(z) dm2 (z) < +∞. D

The space

A2ϕ

is contained in L2a , and the norm in A2ϕ is given by the formula 2 2 2 f ϕ = f L2 + f (z) .(z) dm2 (z) a

or, equivalently,

f ϕ = f + 2

2

D D

D

2 2 f (z) ϕ (λ) /(z, λ) dm2 (z) dm2 (λ).

(6)

From this identity we see that the extremal functions in the Bergman space are expansive multipliers, that is, ϕp2 ≥ p2 for any polynomial p. For the first time, this property of extremal functions was discovered by Hedenmalm in [12]. For an extremal function ϕ ∈ L2a , let Kϕ denote the reproducing kernel of the space A2ϕ . From the definition of an extremal function it follows that for any polynomial p, p(z)|ϕ(z)|2 dm2 (z) = p(0). D

This means that (p, 1)A2ϕ = p(0), whence, Kϕ (z, 0) ≡ 1. The following estimate is crucial for further considerations.


13

Proposition 2. There exists a constant B(λ)continuously depending only on λ∈D such that for any extremal function ϕ, the following inequality holds for any z, λ ∈ D: Kϕ (z, λ) ≤ B(λ). (7) Proof. For any n ∈ Z+ , we put In := [zn ϕ]. Clearly, In ⊃ In+1 and n≥1 In = {0}. Hence, we have the decomposition [ϕ] = I0 = In In+1 . n≥0

Obviously, In+1 = zIn and, since In is cyclic, dim(In In+1 ) = 1. If ϕn is the extremal function for In , then In In+1 is the linear span of ϕn ; therefore, the system {ϕn }n≥0 is an orthonormal basis for [ϕ]. Since multiplication by ϕ is a unitary map from A2ϕ to [ϕ], we see that the system {ϕn /ϕ}n≥0 is an orthonormal basis for A2ϕ . Hence, Kϕ (z, λ) =

ϕn (z) ϕn (λ) · . ϕ(z) ϕ(λ)

(8)

n≥0

The desired estimate (7) follows from this decomposition and the two following independent estimates of ϕn /ϕ. Estimate 1. We have

n ϕn (λ) √ ≤ n + 1 |λ| . ϕ(λ) 1/2 1 − |λ|2

(9)

Proof. We note that

n z g ≥ √ 1 g n+1 2 for any function g ∈ La . (This can be checked, e.g., by using the expression for the norm in L2a in terms of the Taylor coefficients.) Furthermore, we have ϕn ∈ [zn ϕ] = zn [ϕ], whence ϕn /ϕ = zn gn for some gn ∈ A2ϕ ⊂ L2a . We show that√the function gn is a multiplier from H 2 to L2a with multiplier norm not exceeding n + 1. For any polynomial p, we have √ √ ϕn p √ √ ≤ n + 1 ϕn p ≤ n + 1 ϕn p = n + 1 ϕn p ≤ n + 1 p 2 . H zn ϕ ϕ ϕ ϕ The last inequality holds, since any extremal function in L2a is a contractive multiplier from H 2 to L2a (see [12]). From estimates of multipliers between Bergman spaces proved in [28], it follows that √ n+1 |gn (λ)| ≤ 1/2 , 1 − |λ|2 which implies (9).

14

S. M. SHIMORIN

Estimate 2. We have

ϕn (z) ϕ(z) ≤ (n + 1)(n + 2).

(10)

Proof. Since ϕ is an extremal function, we have ϕ ⊥ [zn ϕ], whence ϕn (z) ϕ(z) zm dm2 (z) = 0 D

for all m ≥ 0. Since ϕn is the extremal function for [zn ϕ], we have ϕn ⊥ [zn+1 ϕ], so that ϕn (z) ϕ(z) zn+l dm2 (z) = 0 D

for all l ≥ 1. For k = 1, . . . , n, we put γk := ϕn (z) ϕ(z) zk dm2 (z), D

and r(z) :=

n

(k + 1)γk zk .

k=1

Obviously, |γk | ≤ 1, which yields rH ∞ ≤

(n + 1)(n + 2) . 2

Taking the above identities and the choice of r into account, we see that for any harmonic polynomial q, ϕn (z) ϕ(z) − r(z) q(z) dm2 (z) = 0. D

Now, we can perform the standard calculation used in different forms in [7], [8], [12], and [19] for deducing the expansive multiplier property of the extremal functions in the Bergman spaces. In brief, we consider the following boundary value problem in D: 7 |T = 0, 07 = ϕn ϕ − r. Since 07 is orthogonal to the harmonic polynomials in D, we have ∂7/∂n|T = 0 (in an appropriate sense), and 7 can be expressed by the integral formula 7(λ) = 02 7(z) /(z, λ) dm2 (z) = ϕn (z) ϕ (z) /(z, λ) dm2 (z). D

D

15


Now, the application of the Green formula leads to the identity ϕn (z) ϕ (z)0h(λ)/(z, λ) dm2 (z) dm2 (λ), (ϕn ϕ − r) h dm2 = D

(11)

D D

which holds for any h ∈ C (2) (D). (For a detailed account of such calculations, see [8, proof of identity (4)].) Now, let p and p1 be some analytic polynomials. Then, substituting h = pp1 in (12), we get (ϕn ϕ − r)p p1 dm2 = ϕn (z) ϕ (z)p (λ) p1 (λ) /(z, λ) dm2 (z) dm2 (λ) D

D D

≤

1/2

D D

|ϕn (z)|2 |p (λ)|2 /(z, λ) dm2 (z) dm2 (λ)

·

D D

1/2 |ϕ (z)|2 |p1 (λ)|2 /(z, λ) dm2 (z) dm2 (λ) .

Observe that, when applying the Cauchy-Schwarz-Bunyakovskii inequality, we used the positivity of the Green function /(z, λ). Taking identity (6) into account, we can rewrite the last inequality as follows: 1/2 1/2 |(ϕn p, ϕp1 ) − (rp, p1 )| ≤ ϕn p2 − p2 . ϕp1 2 − p1 2 By continuity, this inequality holds for any p1 ∈ A2ϕ (p remains a polynomial). Substituting p1 = (ϕn /ϕ)p, we obtain 2 1/2 ϕ ϕ 1/2 n n ϕn p2 − rp, p ≤ ϕn p2 − p2 ϕn p2 − p . ϕ ϕ Here the left-hand side is at least

ϕn , ϕn p − rp · p ϕ 2

while the right-hand side is at most 2 1/2 (ϕn /ϕ)p2 ϕn 1 2 2 ≤ ϕn p − p . ϕn p 1 − 2 ϕ ϕn p2 Therefore,

2 ϕn ϕn 1 2 , ϕn p − rp · p ≤ ϕn p − p ϕ 2 ϕ 2

whence

ϕn 2 p ≤ 2 · rH ∞ p · ϕn p , ϕ ϕ

16

S. M. SHIMORIN

and

ϕn p ≤ (n + 1)(n + 2) p. ϕ

This shows that the function ϕn /ϕ is a multiplier of L2a with multiplier norm not exceeding (n+1)(n+2). But it is well known (see [28]) that the space of multipliers of L2a coincides with H ∞ (D) with the equality of norms, which yields the desired estimate. Substituting estimates (9) and (10) in (8), we arrive at estimate (7) from Proposition 2 with the constant −1 B(λ) = 1 − |λ|2 (n + 1)3/2 (n + 2)|λ|n . n≥0

Remark. Estimate 2 answers [13, problem 4.2] in the affirmative. Using the modified arguments of the proof of Estimate 2, Hedenmalm (see [14]) recently obtained a better off-diagonal estimate for weighted reproducing kernels: |Kϕ (z, λ)| ≤

2 . ¯ 2 |1 − λz|

We need the reproducing kernels Kϕ for the study of orthogonal projections to the cyclic z-invariant subspaces [ϕ], as the following lemma shows. Lemma 3. If ϕ is an extremal function in L2a , then the orthogonal projection P[ϕ] is given by the integral formula P[ϕ] f (λ) = ϕ(λ) Kϕ (z, λ) ϕ(z)f (z) dm2 (z). (12) D

Proof. For any λ ∈ D, the kernel Kϕ (·, λ) belongs to A2ϕ . Therefore ϕKϕ (·, λ) ∈ [ϕ] ⊂ L2a , and the integral (12) is well defined. If f is orthogonal to [ϕ], then (f, ϕKϕ (·, λ)) = 0, and the integral in the right-hand side of (12) vanishes. If f ∈ [ϕ], then f = ϕg, g ∈ A2ϕ , and the reproducing property of Kϕ implies that ϕ(λ)Kϕ (z, λ)ϕ(z)f (z)dm2 (z) = ϕ(λ) g(z)Kϕ (z, λ) |ϕ(z)|2 dm2 (z) D

D

= ϕ(λ)g(λ) = f (λ). Proof of Theorem 1A. Suppose that ϕn and ϕ are extremal functions in L2a such that ϕn → ϕ weakly. Then ϕn → ϕ in the norm, since ϕn = ϕ = 1. Show that limn→∞ Kϕn (µ, λ) = Kϕ (µ, λ) pointwise. In [12], it was proved that H 2 (D) ⊂ A2ϕ provided that ϕ is an extremal function, so that Kϕn (·, µ) ∈ A2ϕ and Kϕ (·, λ) ∈ A2ϕn by Proposition 2. The reproducing properties of Kϕn and Kϕ lead to relations Kϕ (z, λ)Kϕn (z, µ) |ϕn (z)|2 dm2 (z) = Kϕ (µ, λ) D

17


and

D

Kϕ (z, λ)Kϕn (z, µ) |ϕ(z)|2 dm2 (z) = Kϕn (µ, λ).

Hence, Kϕ (µ, λ) − Kϕn (µ, λ) =

D

Kϕ (z, λ)Kϕn (z, µ) |ϕn (z)|2 − |ϕ(z)|2 dm2 (z),

and, by Proposition 2, Kϕ (µ, λ) − Kϕ (µ, λ) ≤ B(µ)B(λ) |ϕn (z)|2 − |ϕ(z)|2 dm2 (z) n D

n→∞

/ 0.

The pointwise convergence of the reproducing kernels Kϕn to Kϕ implies the limit relation ϕn Kϕn (·, λ) −→ ϕ Kϕ (·, λ)

(13)

in the weak topology of L2a for any fixed λ ∈ D. Indeed, we have pointwise convergence in (13), and it suffices to check uniform boundedness of norms. We have ϕn Kϕn (·, λ) = Kϕn (·, λ)ϕn , and the norm Kϕn (·, λ)ϕn is the norm of the functional f ' → f (λ) in the space A2ϕn . But for any f ∈ A2ϕn (⊂ L2a ), |f (λ)| ≤

f ϕn f ≤ , 1 − |λ|2 1 − |λ|2

whence ϕn Kϕn (·, λ) ≤

1 . 1 − |λ|2

From (12) and (13), it follows immediately that lim P[ϕn ] f (λ) = P[ϕ] f (λ)

n→∞

for any fixed f ∈ L2a and any λ ∈ D, which is equivalent to the convergence P[ϕn ] → P[ϕ] in the weak operator topology. This completes the proof of Theorem 1A. The following proposition establishes a fact that, in view of Lemma 2, is converse to Theorem 1A. Proposition 3. Suppose that ϕn and ϕ are extremal functions in L2a such that ϕn (0) > 0 for all n ≥ 1 and ϕ(0) > 0. If lim n→∞ [ϕn ] = [ϕ] and lim n→∞ [ϕn ]⊥ = [ϕ]⊥ , then limn→∞ ϕn = ϕ in the norm.

18

S. M. SHIMORIN

Proof. The proof is a slight modification of that given in [20] (in a less general context). Suppose that some subsequence (ϕnk )k≥1 weakly converges to an element f ∈ L2a . If g ∈ z[ϕ], then g = limn→∞ gn for some gn ∈ z[ϕn ], and (g, f ) = lim gnk , ϕnk = 0, k→∞

which means that f ⊥ z[ϕ]. Similarly, f ⊥ [ϕ]⊥ , whence f ∈ [ϕ]z[ϕ], and f = γ ϕ for some γ ∈ [0, 1]. On the other hand, since f ∈ [ϕ] = lim n→∞ [ϕn ], we have f = limn→∞ ϕn gn , where gn ∈ A2ϕn , and f (0) ≤ lim ϕnk (0) · gnk ϕ = lim ϕnk (0) · ϕnk gnk = f (0) · f . nk

k→∞

k→∞

The relation f (0)= 0 is impossible (since ϕ(0)> 0 and one can write ϕ= limn→∞ ϕn hn with hn ∈A2ϕn , hn ϕn→1); thus, f = 1 and f = ϕ. It follows that w-lim n→∞ ϕn = ϕ, so that limn→∞ ϕn = ϕ in the norm. Now we pass to calculations of lower limits of cyclic z-invariant subspaces [ϕn ] and their annihilators in the case where the behaviour of the extremal functions ϕn is arbitrary. First, we consider two particular cases, namely, w-lim n→∞ ϕn = 0 and w-lim n→∞ ϕn = ψ = 0. Proposition 4. Suppose that ϕn , n ≥ 1, are extremal functions in L2a such that w-lim n→∞ ϕn = 0. Then (1) lim n→∞ [ϕn ]⊥ = (L2a )∗ . (2) lim n→∞ [ϕn ] = {0}. Proof. To prove the relation lim n→∞ [ϕn ]⊥ = (L2a )∗ , it suffices to prove that limn→∞ P[ϕn ] = 0 in the strong operator topology. To this end, it suffices to verify that limn→∞ P[ϕn ] (zm ) = 0 in the norm for any m ≥ 0. As in the proof of Proposition 2, we (k) (k) consider the z-invariant subspaces In := [zk ϕn ]. Let ϕn be the extremal function (k) for In . We have the decomposition (see (8)) ϕn(k) (z) ϕn(k) (λ) · . Kϕn (z, λ) = ϕn (z) ϕn (λ) k≥0

By Lemma 3, we can write

P[ϕn ] z

m

(k) (k) ϕn (λ) ϕn (z) · · ϕn (z) zm dm2 (z) (λ) = ϕn (λ) ϕn (λ) ϕn (z) D

k≥0

=

m k=0

ϕn(k) (λ) ·

(k)

D

ϕn (z) · zm dm2 (z).


19

(k)

It suffices to show that w-lim n→∞ ϕn = 0 for any k ≥ 0, or, equivalently, limn→∞ (k) ϕn (z) = 0 pointwise in D. But limn→∞ ϕn (z) = 0 pointwise by assumption, and (k) |ϕn (z)| ≤ (k + 1)(k + 2)|ϕn (z)| by Estimate 2 from the proof of Proposition 2. The relation lim n→∞ [ϕn ]⊥ = (L2a )∗ is proved. Now, the second limit relation claimed in the proposition follows immediately. Suppose that a sequence (ϕ n )n≥1 of extremal functions in L2a weakly converges to a function ψ ∈ L2a , where ψ ≡ 0 and ψ is not extremal. The extremal functions in the Bergman space can be characterized by the identity |ϕ|2 h dm2 = h(0), D

which is valid for any bounded harmonic function h. Fatou’s lemma shows that if ψ(z) = limn→∞ ϕn (z) pointwise in D and ϕn are extremal, then |ψ|2 h dm2 ≤ h(0) (14) D

for any positive harmonic function h. The functions ψ satisfying (14) for any positive harmonic h are called subextremal functions. In §4 it is shown that any subextremal function is a pointwise limit of some sequence of extremal functions. Let Pz (ζ ) = (1 − |z|2 )/(|1 − ζ¯ z|2 ) be the Poisson kernel for the disc D, and let ᏼ denote the operator of harmonic extension of functions from the boundary T = ∂D inside the disc D, ᏼf (z) = f (ζ )Pz (ζ ) dm1 (ζ ) T

(dm1 is the normalized Lebesgue measure on T). We consider ᏼ as an operator acting from C(T) to C(D). For the conjugate operator ᏼ∗ : M(D) → M(T) (M denotes the space of finite Borel measures), it is easy to verify the following properties: (i) ᏼ∗ M+ (D) = M+ (T) (where M+ is the cone of nonnegative Borel measures); (ii) ᏼ∗ L1 (D) = L1 (T) (here L1 is identified with a subspace of M). Indeed, in order to verify (i), we take a nonnegative Borel measure µ ∈ M+ (D) and an arbitrary nonnegative function f ∈ C(T); then ∗ f d(ᏼ µ) = ᏼf (z) dµ(z) ≥ 0, T

D

whence ᏼ∗ M+ (D) ⊂ M+ (T). On the other hand, any ν ∈ M(T) may be viewed as an element of M(D), and we have ᏼ∗ ν = ν, which shows that ᏼ∗ M+ (D) = M+ (T). To prove (ii), we consider the rotation operators Rθ defined on the Borel measures µ ∈ M(D) (or M(T)) by the formula (Rθ µ)(E) := µ e−iθ E .

20

S. M. SHIMORIN

(E is a Borel subset of D, and e−iθ E = {e−iθ z : z ∈ E}.) It is well known that a measure ν ∈ M(T) is absolutely continuous with respect to the Lebesgue measure dm1 if and only if limθ →0 ν −Rθ ν = 0. (The norm is taken in M(T).) On the other hand, a simple calculation shows that the operators ᏼ∗ and Rθ commute: ᏼ∗ Rθ = Rθ ᏼ∗ . Since, obviously, limθ →0 µ − Rθ µ = 0 for any µ ∈ L1 (D), the inclusion ᏼ∗ L1 (D) ⊂ L1 (T) follows. The converse inclusion is a consequence of the relations ᏼ∗ (zn )(ζ ) = ζ n /(n + 1) and ᏼ∗ (zn )(ζ ) = ζ n /(n + 1). A function ϕ ∈ L2a is extremal if and only if ᏼ∗ (|ϕ|2 dm2 ) = dm1 , and ψ is subextremal if and only if ᏼ∗ (|ψ|2 dm2 ) = h dm1 , where h ∈ L1 (T) and 0 ≤ h ≤ 1. We need the following technical lemma. Lemma 4. Suppose that ϕn ∈ L2a are extremal functions and limn→∞ ϕn (z) = ψ(z) pointwise in D. Consider the nonnegative measure dν ∈ M+ (T) defined as follows: dν = dm1 − ᏼ∗ |ψ|2 dm2 . Then the following hold: (i) The measures |ϕn |2 dm2 converge to the measure |ψ|2 dm2 + dν in the weak∗ topology of M(D). (ii) For any l ≥ 0, lim ᏼ∗ |z|2l |ϕn |2 dm2 = ᏼ∗ |z|2l |ψ|2 dm2 + dν n→∞

in the norm of M(T). (iii) If the functions x, xn ∈ H ∞ (D) are such that x = w* -limn→∞ xn (the limit is taken in the weak∗ topology of H ∞ ), then zl xn |ϕn |2 dm2 = zl x |ψ|2 dm2 + zl x dν, l = 0, 1, . . . . lim n→∞ D

D

T

(In the second term, x means the boundary function; the integral is well defined because dν is absolutely continuous with respect to dm1 .) Proof. (i) Suppose that w* -lim |ϕnk |2 dm2 = dλ ∈ M(D) k→∞

for some sequence (nk )k≥1 . We show that dλ = |ψ|2 dm2 + dν. Since ϕn (z) → ψ(z) uniformly on the compact subsets of D, for any Borel subset E of D we have λ(E) = |ψ|2 dm2 . E

Hence, it suffices to show that λ(F ) = ν(F )

21


for any Borel subset F ⊂ T. Indeed, for any h ∈ C(T) we can write h dλ = ᏼh dλ − ᏼh · |ψ|2 dm2 T D D 2 = lim ᏼh · |ϕnk | dm2 − ᏼh · |ψ|2 dm2 k→∞ D D ∗ 2 = ᏼh(0) − h · d ᏼ |ψ| dm2 = h dν. T

(ii) The

weak∗

T

convergence ᏼ∗ |z|2l |ϕn |2 dm2 −→ ᏼ∗ |z|2l |ψ|2 dm2 + dν

follows immediately from (i). To see that we have norm convergence, it suffices to observe that both sides of the above limit relation are positive elements of L1 (T) and to use the following simple fact from the integration theory: If (ᐄ, µ) is a space with measure, xn and x are nonnegative integrable functions on ᐄ such that xn → x in measure, and xn dµ −→ x dµ, ᐄ 1 L (ᐄ, dµ)

ᐄ

then xn → x in the norm of (see [11, §26]). (iii) Consider the following operators on the space H ∞ (D): (Ul x) (z) :=

l−1

k x(k)z ˆ +

k=0

l

x(k ˆ + l)

k=0

l − k k+l z l

(the operator of taking the de la Vallée–Poussin means), and (Vl x) (z) :=

1 (x − Ul x). zl

Then, by properties of the de la Vallée–Poussin means, Ul ≤ 3, Vl ≤ 4, and x = Ul x + zl Vl x for any x ∈ H ∞ . By (i), we have l 2 l 2 z Ul xn |ϕn | dm2 = z Ul x |ψ| dm2 + zl Ul x dν, lim n→∞ D

D

T

and by (ii), we have |z|2l Vl xn |ϕn |2 dm2 = |z|2l Vl x |ψ|2 dm2 + Vl x dν, lim n→∞ D

D

T

because we can convert the integrals over D into integrals over T by the use of ᏼ∗ .

22

S. M. SHIMORIN

Now we are ready to describe the lower limits of the subspaces [ϕn ] and [ϕn ]⊥ in the case where w-lim n→∞ ϕn = ψ. Proposition 5. Suppose that ϕn are extremal functions in L2a , and w-lim n→∞ ϕn = ψ ≡ 0. Then lim n→∞ [ϕn ]⊥ = [ψ]⊥ . If ψ is an extremal function (this is equivalent to ψ = 1), then lim n→∞ [ϕn ] = [ψ]; otherwise, lim n→∞ [ϕ] = {0}. Proof. By Lemma 4(i), w* -limn→∞ (|ϕn |2 dm2 ) = |ψ|2 dm2 +dνψ =: dµψ , where dνψ = dm1 − ᏼ∗ (|ψ|2 dm2 ) ∈ M+ (T). Consider the Hilbert space Pψ2 , which is the closure of the analytic polynomials in

L2 (D, dµψ ). Since ψ ≡ 0, the point evaluation functionals f → f (λ) are bounded in Pψ2 for λ ∈ D, which allows us to consider the reproducing kernel Kψ (z, λ) of Pψ2 (defined on D × D). Let us denote Kϕn , simply, by Kn . Show that limn→∞ Kn (z, λ) = Kψ (z, λ) pointwise in D × D. Fix λ ∈ D. By Proposition 2, we have |Kn (z, λ)| ≤ B(λ); hence, it suffices to check that if limj →∞ Knj (·, λ) = k(·, λ) pointwise in D for some sequence (nj )j ≥1 , then the function k(·, λ) possesses the reproducing property zl k(z, λ) |ψ(z)|2 dm2 (z) + zl k(z, λ) dνψ (z), l = 0, 1, . . . . λl = D

T

Indeed, the reproducing property of Kn shows that zl Kn (z, λ) |ϕn (z)|2 dm2 (z), λl = D

and it remains to apply Lemma 4(iii). By Proposition 2, the convergence Kn → Kψ is uniform on the compact subsets of D × D, and |Kψ (z, λ)| ≤ B(λ) for any λ ∈ D. The operator Mz is bounded away from zero in the space Pψ2 ; hence, zm Pψ2 is the closed subspace of Pψ2 given as zl : l ≥ m (the closure is taken in the norm of L2 (D, dµψ )). Clearly, dim(zm Pψ2 zm+1 Pψ2 ) = 1 and m≥0 zm Pψ2 = {0}. Let e(m) be an element of zm Pψ2 zm+1 Pψ2 such that e(m) P 2 = 1 and (d/dzm )e(m) (z)|z=0 > 0. Then we have the decomposition ψ

Kψ (z, λ) =

e(m) (z) e(m) (λ).

m≥0

As in the proofs of Propositions 2 and 4, we can write the decomposition ϕn(m) (z) ϕn(m) (λ) · , Kn (z, λ) = ϕn (z) ϕn (λ) m≥0

23

APPROXIMATE SPECTRAL SYNTHESIS (m)

where ϕn is the extremal function for [zm ϕn ]. Show that (m)

ϕn (z) = e(m) (z) n→∞ ϕn (z) lim

(15)

pointwise in D (and, by Estimate 2 from the proof of Proposition 2, in the weak∗ topology of H ∞ ). For m = 0, we have (0) Kψ (z, 0) Kn (z, 0) ϕn (z) = lim = lim 1/2 n→∞ 1/2 n→∞ ϕn (z) Kψ (0, 0) Kn (0, 0)

e(0) (z) =

(≡ 1).

Suppose that we have already proved (15) for m = 0, . . . , m0 − 1. Then, uniformly on the compact subsets of D × D, (m0 )

lim Kn(m0 ) (z, λ) = Kψ

n→∞

where

(z, λ),

ϕn(m) (z) ϕn(m) (λ) Kn(m0 ) (z, λ) := · ϕn (z) ϕn (λ) m≥m 0

and

(m0 )

Kψ

(z, λ) :=

e(m) (z) e(m) (λ).

m≥m0

We have e

(m0 )

(z) =

∂ 2m0 ∂zm0 ∂λ

(m0 ) m0 Kψ (z, λ) z=λ=0

−1/2 ·

∂ m0 ∂λ

m0

(m0 )

Kψ

(m )

(z, λ)

λ=0

;

(m )

the analogous formula expresses (ϕn 0 (z)/ϕn (z)) by the kernel Kn 0 , and the con(m ) (m ) (m ) vergence ϕn 0 /ϕn → e(m0 ) follows from the convergence of kernels Kn 0 → Kψ 0 . As a consequence of (15), we deduce that w-lim ϕn(m) = e(m) ψ n→∞

in the weak topology of L2a . Consider the operators Tn , T˜ψ , and Tψ defined as follows: (Tn g)(λ) = Kn (z, λ) ϕn (z)g(z) dm2 (z), D T˜ψ g (λ) = Kψ (z, λ) ψ(z)g(z) dm2 (z), D

(16)

24 and

S. M. SHIMORIN

(Tψ g)(λ) = ψ(λ) T˜ψ g (λ) = ψ(λ)

D

Kψ (z, λ) ψ(z) g(z) dm2 (z).

By Lemma 3, we have ϕn Tn g = P[ϕn ] g for any g ∈ L2a ; thus, Tn are contractions acting from L2a to A2ϕn . We are going to show that T˜ψ is a contraction from L2a to Pψ2 and that, for any f ∈ L2a , we have T˜ψ f 2 = lim Tn f 2 . (17) Aϕ P n→∞

ψ

n

It suffices to check that T˜ψ f ∈ Pψ2 and to prove that (17) is true for any polynomial f . Using the above decomposition of kernels Kn and Kψ , we obtain l (m) ϕn (m) · ϕn (z) zl dm2 (z) Tn z = ϕn D l

m=0

and T˜ψ zl =

l

e

(m)

m=0

·

D

e(m) (z) ψ(z) zl dm2 (z).

These formulae and the limit relation (16) imply that lim Tn zl , Tn zp A2 = T˜ψ zl , T˜ψ zp P 2 n→∞

ϕn

ψ

for any l, p ∈ Z+ , which leads immediately to (17). Relation (17) implies that ker Tψ = [ψ]⊥ . The operator Tψ : L2a → L2a is the limit of the projections P[ϕn ] in the weak operator topology. This can be shown by using the same arguments as in the proof of Theorem 1A. If g ∈ [ψ]⊥ , then Tψ g = 0; and so from (17) and Lemma 3, we obtain limn→∞ P[ϕn ] g = limn→∞ Tn gA2ϕ = 0. Therefore, g ∈ lim n→∞ [ϕn ]⊥ , and we have the n

inclusion [ψ]⊥ ⊂ lim n→∞ [ϕn ]⊥ . The converse inclusion is obvious. In the case where ψ is an extremal function, the relation lim n→∞ [ϕn ] = [ϕ] follows from Theorem 1A. If ψ is not extremal, then ψ < 1 and dν > 0. If limn→∞ P[ϕn ] f = f for some nonzero f ∈ L2a , then f = Tψ f and T˜ψ f 2 |ψ|2 dm2 < T˜ψ f 2 |ψ|2 dm2 + dν = T˜ψ f 2 2 ≤ f 2 , f 2 = P D

D

ψ

which is a contradiction. This means that in this case we have lim n→∞ [ϕn ] = {0}. Proposition 5 is proved. The following theorem describes the lower limits of cyclic z-invariant subspaces and of their annihilators in the general case where the behaviour of extremal functions ϕn is arbitrary.


25

Theorem 4. Suppose that (ϕ n )n≥1 is a sequence of extremal functions in L2a . Consider the set A of weak limits of subsequences of this sequence, that is, A = ψ ∈ L2a : ∃ nj j ≥1 such that ψ = w-lim ϕnj . j →∞

Then the following hold: (i) lim n→∞ [ϕn ]⊥ = ψ∈A [ψ]⊥ . (ii) If A consists only of extremal functions, then [ψ]; lim [ϕn ] = n→∞

ψ∈A

otherwise, lim n→∞ [ϕn ] = {0}. Proof. The proof is based on the following general fact proved in [10]. Lemma 5. Suppose that Qn , n ≥ 1, are orthogonal projections in a Hilbert space H . Consider the set {Qn } , consisting of the limit points of the sequence (Qn )n≥1 in the weak operator topology. Then ker Q. lim ker Qn = n→∞

Q∈{Qn }

In our case, we take Qn = P[ϕn ] . The set {Qn } coincides with the set {Tψ }ψ∈A . (Here Tψ is defined as in the proof of Proposition 5.) Indeed, for ψ ∈ A, the inclusion Tψ ∈ {Qn } follows from the proof of Proposition 5. On the other hand, if Q = limj →∞ P[ϕnj ] , then w-lim k→∞ ϕnjk =ψ∈A for some (j k )k≥1 , and we have Q=Tψ . Since ker Tψ = [ψ]⊥ (see the proof of Proposition 5), Lemma 5 readily shows that [ψ]⊥ , lim [ϕn ]⊥ = n→∞

ψ∈A

and (i) is proved. To prove (ii), we observe that if there exists some ψ = w-lim j →∞ ϕnj such that ψ is not extremal, then lim j →∞ [ϕnj ] = {0} by Proposition 5, whence lim n→∞ [ϕn ] = 0. In the case where A consists only of extremal functions, the relation lim n→∞ [ϕn ] = ψ∈A [ψ] follows from Theorem 1A and Lemma 5. For the Hardy space H 2 , a statement analogous to Theorem 4(i) was proved in [10] (see also [22, Chapter 2]). 4. Approximation of subextremal functions. First, we recall the following classical theorem. Carathéodory-Schur theorem. If ψ ∈ H ∞ (D) is such that ψH ∞ ≤ 1, then there exists a sequence of finite Blaschke products Bn such that ψ = lim Bn n→∞

26

S. M. SHIMORIN

pointwise in D. For the proof see, for example, [22, Chapter 2]. Theorem 1B is a particular case of a more general fact, namely, Theorem 5, which is a Bergman spaces version of the Carathéodory-Schur theorem. A positive finite measure µ ∈ M+ (D) is said to be radial if Rθ µ = µ for any θ ∈ R. Obviously, any radial measure can be written in the form dµ0 (r 2 ) × (dϕ/2π) (r and ϕ are the polar coordinates) for some µ0 ∈ M+ ([0, 1)). In what follows, we deal with a fixed radial measure µ ∈ M+ (D), dµ = dµ0 (r 2 ) × (dϕ/2π), satisfying the following two conditions: µ(D) = 1 and µ0 (1 − ε, 1) > 0.

∀ε > 0,

(18)

The radial weighted Bergman space L2a (D, dµ) consists of functions analytic in D for which 2 |f (z)|2 dµ(z) < +∞. f := D

This space is supplied with the natural norm induced from L2 (D, dµ). The condition (18) immediately implies the continuity of the point evaluation functionals for the points in D and the completeness of L2a (D, dµ). If we introduce the moments 1 wn := r n dµ0 (r) = |z|2n dµ(z) D

0

of the measure µ0 , then the norm in L2a (D, dµ) is given by the formula fˆ(n)2 wn . f 2 = n≥0

In what follows, we need the following properties of the moment sequence (w n )n≥0 : (i) w0 = 1; (ii) the sequence (w n )n≥0 is logarithmically convex; √ (iii) limn→∞ n wn = 1; (iv) limn→∞ (wn+1 /wn ) = 1; (v) limn→∞ (wn2 /w2n ) = 0. Property (i) is equivalent to the fact that µ(D) = 1; property (ii) is well known; properties (iii) and (iv) are consequences of (18); and (v) follows from (ii), (iv), and the fact that wk → 0 as k → ∞. Indeed, for k ≤ n, (ii) implies that w n wk wn2 ≤ . w2n wn+k


27

Fixing k, letting n → ∞, and using (iv), we obtain wn2 ≤ wk , n→∞ w2n lim

and it remains to let k → ∞. The notions of an extremal and a subextremal function in the spaces L2a (D, dµ) are analogous to those in the case of the classical (unweighted) Bergman space L2a . Definition 6. A function ϕ ∈ L2a (D, dµ) is said to be extremal if ϕ = 1 and (ϕ, zn ϕ) = 0 for any n ≥ 1. It is easy to check that ϕ is extremal if and only if |ϕ|2 h dµ = h(0) D

for any bounded harmonic function h, or, in other words, ᏼ∗ (|ϕ|2 dµ) = dm1 . For any z-invariant subspace I ⊂ L2a (D, dµ), the extremal function for I is defined as the solution of the following extremal problem: sup Re g (k) (0) : g ∈ I, g ≤ 1 , where k = vI (0). An extremal function ϕ ∈ L2a (D, dµ) is said to be finitely zerobased if [ϕ] is a finitely zero-based z-invariant subspace. The finitely zero-based extremal functions are Bergman spaces analogs of the finite Blaschke products. We avoid the term zero divisor in this general situation, because, in general, zero-based extremal functions fail to be good canonical factors in factorization of functions in radial weighted Bergman spaces (see [16]). Definition 7. A function ψ ∈ L2a (D, dµ) is said to be subextremal if |ψ|2 h dµ ≤ h(0) D

for any positive harmonic function h. Clearly, this condition is equivalent to the identity ᏼ∗ (|ψ|2 dµ) = gdm1 , where g ∈ L1 (T) and 0 ≤ g ≤ 1 on T. For a function ψ ∈ H ∞ , the property ψH ∞ ≤ 1 is equivalent to ˆ |ψ|2 h dm1 ≤ h(0) T

L1 (T).

Therefore, the subextremal functions may be viewed as for any positive h ∈ Bergman spaces analogs of the functions from the unit ball of H ∞ . Theorem 5 (Bergman spaces analog of the Carathéodory-Schur theorem). A function ψ ∈ L2a (D, dµ) is subextremal if and only if there exist finitely zero-based extremal functions ϕn ∈ L2a (D, dµ) such that ψ = limn→∞ ϕn pointwise in D.

28

S. M. SHIMORIN

Obviously, Theorem 1B is a particular case of this theorem. Another Bergman spaces version of the Carathéodory-Schur theorem was suggested by Hedenmalm (see [13, Conjecture 5.2]). He conjectured that any function ϕ analytic in D and satisfying ϕf L2a ≤ f H 2 for any f ∈ H 2 (i.e., any contractive multiplier from H 2 to L2a (D)) is the normal limit of finitely zero-based extremal functions in L2a (D). Any subextremal function ϕ in L2a (D, dµ) is a contractive multiplier from H 2 to L2a (D, dµ). Indeed, for any polynomial p, we have |ϕ(z)p(z)|2 dµ(z) ≤ |ϕ(z)|2 ᏼ|p|2 (z) dµ(z) ≤ ᏼ|p|2 (0) D D = |p(ζ )|2 dm1 (ζ ), T

ϕp2

which means that ≤ p2H 2 . The question as to whether or not any contractive 2 2 multiplier from H to La (D, dµ) is subextremal in L2a (D, dµ) remains open (even in the unweighted case dµ = dm2 ). Proof of Theorem 5. The “if” part of the theorem follows immediately from Fatou’s lemma. The “only if” part is less trivial. First, from Definition 7, we see that the set of subextremal functions is convex, rotation invariant, and closed in the weak topology of L2a (D, dµ). Therefore, for any subextremal function ψ, the Fejér integral ψN (z) := ψ ζ z .N (ζ ) dm1 (ζ ), z ∈ D, T

is also a subextremal function. Here

.N e

iθ

1 sin2 (N + 1)/2 θ = N +1 sin2 (θ/2)

is the Fejér kernel. Since ψN → ψ pointwise in D, it suffices to prove the “only if” implication of Theorem 5 in the case where ψ is a polynomial. Moreover, without loss of generality, we may assume that there exists a positive δ such that |ψ|2 h dµ ≤ (1 − δ)h(0) (19) D

for any positive harmonic polynomial h. Also, we assume that ψ(0) > 0. So, we have a polynomial ψ satisfying (19) and such that ψ(0) > 0. Let d = deg ψ. For the trigonometric polynomial g(ζ ) :=

−1

d n ψ, z|n| ψ ζ |n| + (zn ψ, ψ)ζ ,

n=−d

n=0


it is easy to verify that

29

ᏼ∗ |ψ|2 dµ = g dm1 .

Inequality (19) implies that 0 ≤ g(ζ ) ≤ 1 − δ on T. The function 1 − g is a positive trigonometric polynomial of degree d; therefore, by the Fejér–F. Riesz theorem (see [9, Chapter 1]), there exists an analytic polynomial G of degree d such that G has no zeros in D and |G(ζ )|2 = 1 − g(ζ )

(20)

on T. Moreover, since 1 − g(ζ ) ≥ δ on T, we have |G(z)|2 ≥ δ everywhere in D. Now, we put

(21)

zn pn (z) := ψ(z) + G(z) √ , wn

and In = [pn ]. Let ϕn be the extremal function for In . If we show that ϕn are finitely zero-based and that ψ = limn→∞ ϕn pointwise in D, the theorem will be proved. The crucial limit relation ψ = limn→∞ ϕn is based on the choice of G and pn . The polynomials pn turn out to be “almost extremal,” which means that the sequences (zl pn , pn )l≥0 are “almost” δ-sequences. In a sense, the larger is the parameter n, the √ more “independent” are the terms ψ and G(z)(zn / wn ). Combined with the choice of G, this implies that ᏼ∗ |pn |2 is close to 1. Therefore, ϕn is close to pn and, hence, to ψ. Making these arguments precise requires some technical work. It follows from the next lemma that ϕn are finitely zero-based extremal functions. Lemma 6. If p is a polynomial, then [p] ⊂ L2a (D, dµ) is the finitely zero-based z-invariant subspace determined by the divisor of zeros, lying in D, of p. Proof. It suffices to write p = p1 · p2 , where p1 is a polynomial with zeros in D and where p2 is a polynomial with zeros in C \ D. Observe that p2 is cyclic in the weak∗ topology of H ∞ (D). Lemma 7. There exists a number N0 (depending on ψ and G) such that, for any n ≥ N0 and any f ∈ L2a (D, dµ), we have pn f 2 ≥

δ ˆ 2 wn+k . f (k) 8 wn k≥0

Proof. For each n ≥ 1, let rn be a number in (0, 1) such that 1 |z|2n dµ(z) ≤ wn 2 |z| 0, for all x = 0), then P is called nondegenerate.If dim Pk < ∞, for all k ≥ 0, then we say that P is finite-dimensional. If P = k≥0 Pk is a finite-dimensional, nondegenerate, connected, planar ∗-algebra with positive partition function, we can make each Pk into a C ∗ -algebra. P is then called a C ∗ -planar algebra. A (connected) planar algebra is called spherical if its partition function is an invariant of planar networks on the 2-sphere S 2 . Note that in this case tr L = tr R def

and δ1 = δ2 = δ. Spherical C ∗ -planar algebras (or subfactor planar algebras) are precisely the standard invariants of extremal subfactors with finite index (see [Po3] and [J2]). We next describe how the diagram formalism applies to the analysis of the system of higher relative commutants associated to a subfactor. Let N ⊂ M be an (extremal) e1

e2

e3

inclusion of II1 factors with finite index, and denote by N ⊂ M ⊂ M1 ⊂ M2 ⊂ M3 ⊂ · · · the associated tower of II1 factors constructed from N ⊂ M by iterating the basic construction (see [J1]). We denote as usual by J : L2 (M) → L2 (M) the modular conjugation and by tr or tr M the normalized, faithful trace on M. From the tower of II1 factors associated to N ⊂ M one obtains a sequence of finite-dimensional commuting squares C = N ∩ N ⊂ N ∩ M ⊂ N ∩ M 1 ⊂ N ∩ M2 ⊂ . . . ∪ ∪ ∪ C = M ∩ M ⊂ M ∩ M1 ⊂ M ∩ M2 ⊂ . . . which is called the system of higher relative commutants, or the standard lattice (see [Po3]), or the standard invariant (see [Po2]), or the Popa system (see [BJ1]), associated to the subfactor N ⊂ M (see also [GHaJ]). Since spherical C ∗ -planar algebras are precisely the standard invariants of extremal subfactors with finite index (see [J2]), we can think of an element R ∈ N ∩ Mk as being depicted by a labelled (k+1)-box as in Figure 1 (with k+1 vertical strings on the top and bottom of the box and orientations alternating up-down-up-down). We usually

45

SINGLY GENERATED PLANAR ALGEBRAS

omit the orientation of the vertical strings and simply write R for R ∈ N ∩ Mk . The (unital) embedding R ∈ N ∩Mk → N ∩Mk+1 is given by adding to a (k +1)-box R a vertical string at the right end of the box (with the appropriate orientation) so that we get R . Note that it is not hard to see that a spherical C ∗ -planar algebra gives rise to a Popa system (see [J2]), and hence, by a theorem of Popa, is the system of higher relative commutants of an extremal subfactor (which is not necessarily hyperfinite); see [Po3]. Conversely, it can be shown that a system of higher relative commutants of an extremal subfactor gives rise to a spherical C ∗ -planar algebra (see [J2]). This requires a bit more work, and we do not need to use the result in this paper. R

If Q, R ∈ N ∩ Mk are given by Q R , then Q · R is depicted as Q . (Caution: We follow the convention of multiplying diagrams from bottom to top, which is also the convention used in [BJ1].) As is well known, the special (k +1)-boxes Ej ∈ N ∩Mk , 1 ≤ j ≤ k, given by Figure 2, play an important role in the theory. (The diagram in Figure 2 has k + 1 vertical strings, and the arcs connect hook j and j + 1 on the top and bottom lines of the box.)

Figure 2

We denote as above the parameter associated to a closed loop by δ, and we recall that δ = [M : N ]1/2 (see, for instance, [J1], [BJ1], and [GHaJ]). The projections ej = (1/δ)Ej , 1 ≤ j ≤ k, generate the Temperley-Lieb algebra, if δ > 2 (see, for instance, [GHaJ], [BJ1]). The projection ek+1 implements the trace-preserving conditional expectation EMk−1 : Mk → Mk−1 , that is, ek+1 xek+1 = EMk−1 (x)ek+1 , for all x ∈ Mk . When restricted to the relative commutant N ∩ Mk , EMk−1 is equal to EN ∩Mk−1 , the trace-preserving conditional expectation N ∩ Mk → N ∩ Mk−1 . Sime0 ilarly, if N ⊂ M is extremal and we let N1 ⊂ N ⊂ M be one step in the downward basic construction (see [J1]), then e0 implements the tr M -preserving conditional expectation EM ∩Mk : N ∩Mk → M ∩Mk . If N ⊂ M is not extremal, e0 implements the tr N -preserving conditional expectation (see also [B2, Proposition 2.7]). These conditional expectations can be written diagrammatically as follows: Let R ∈ N ∩ Mk be an arbitrary element depicted as R . Then EMk−1 (R) = EN ∩Mk−1 (R) is computed diagrammatically in Figure 3 (up to scalars).

46

BISCH AND JONES

R

=

R

Figure 3

Thus, taking scalars into account, we get that δEN ∩Mk−1 (R) is given by R . Similarly, we get that δEM ∩Mk (R) is given by the diagram R . Observe that the orientations of the vertical strings of the k-box R are alternating down-up-down-up. Thus elements in M ∩Mk are depicted as k-boxes. However, the orientations are reversed; that is, the k vertical strings on the top and bottom of the box have orientations alternating down-up-down-up from left to right. The unital embedding M ∩ Mk → N ∩ Mk is obtained by taking a k-box representing an element in M ∩ Mk and adding R . a vertical string to the left of the box, which is oriented upward, that is, The result is a (k + 1)-box oriented correctly as in Figure 1. Note that the ∗-operation on N ∩ Mk is obtained by replacing the label R in a (k + 1)-box by the label R ∗ . Other important operations on the higher relative commutants are presented in Section 3. We use in Section 4 a natural (linear) basis of N ∩ M1 and of a certain direct summand in N ∩ M2 . Recall that (N ∩ Mk )ek+1 (N ∩ Mk ) is a two-sided ideal in N ∩ Mk+1 , which we call the basic construction ideal in N ∩ Mk+1 . It is the ideal generated by ek+1 and is isomorphic to the basic construction for N ∩ Mk−1 ⊂ N ∩ Mk (see [GHaJ]). If dim N ∩ M1 = 3, then dim(N ∩ M1 )e2 (N ∩ M1 ) = 9. Let us write N ∩ M1 = Ce1 ⊕ Cq ⊕ C(1 − e1 − q), where e1 , q, and 1 − e1 − q are the minimal (central) or briefly by as projections in N ∩ M1 . We depict q by Q and E1 = δe1 by

usual (where δ = [M : N]1/2 ). Later, we also use the projection p = e1 + q, which is depicted by P . The identity in N ∩ M1 is depicted by or briefly by . Clearly, , , and Q or , , and P form a (linear) basis of N ∩ M1 . Thus the nine diagrams in Figure 4 are a basis of (N ∩ M1 )e2 (N ∩ M1 ). These diagrams, of course, are just the elements E1 , E2 , E1 E2 , E2 E1 , qE2 , E2 q, qE2 E1 , E1 E2 q, and qE2 q in N ∩ M2 (from left to right), where ei = (1/δ)Ei as above.

47


Q

Q

Q

Q

Q Q

Figure 4

Later, we also extensively use the elements δ 3 EM ∩M2 (qe2 e1 ) and δ 3 EM ∩M2(e1 e2 q) depicted in Figure 5.

Q

Q Figure 5

Note that these are elements in M ∩ M2 , so that the orientation of the vertical strings is down-up (from left to right). 2. The main theorems. We describe in this section the main results of this paper. Let us start with the main theorem. Theorem 2.1. Let N ⊂ M be an inclusion of II1 factors with 3 < [M : N] < ∞. Suppose that dim N ∩ M1 = 3 and that N ∩ M2 is abelian modulo the basic construction ideal (N ∩ M1 )e2 (N ∩ M1 ). Then there is an intermediate subfactor P of N ⊂ M, P = N , M. In particular, J x ∗ J = x, for all x ∈ N ∩ M1 . Note that if [M : N] = 3 and dim N ∩ M1 = 3, then the subfactor is given as the fixed point algebra for an outer Z3 action; that is, it is of the form N = M Z3 , and the associated planar algebra (or equivalently the associated Popa system) is determined by the group Z3 and its representation theory. Furthermore, observe that the condition dim N ∩ M1 = 3 implies that N ⊂ M is irreducible and, hence, extremal. We give a proof of Theorem 2.1 in Section 4. Let us point out that the conditions in Theorem 2.1 are very simple conditions on the shape of the principal graph for the tower of inclusions C = N ∩ N ⊂ N ∩ M ⊂ N ∩ M1 ⊂ · · · . Theorem 2.1 says that if the principal graph of N ⊂ M is of the form shown in Figure 6, then N ⊂ M must have an intermediate subfactor. Theorem 2.1 allows us to classify all spherical C ∗ -planar algebras V = ∞ k=0 Vk , which are generated (as spherical C ∗ -planar algebras) by a single 2-box, subject to the conditions dim V2 = 3 (which implies dim V0 = dim V1 = 1) and dim V3 ≤ 12. Note that it is shown in [J2], using a theorem of Popa (see [Po3]), that every spherical C ∗ planar algebra gives rise to an extremal subfactor N ⊂ M such that N ∩ Mk+1 = Vk . In fact, every Popa system coming from an extremal subfactor is a spherical C ∗ -planar algebra (and therefore called a subfactor planar algebra), so that one can go freely between subfactor and planar algebra language (see [J2]).

48

BISCH AND JONES

∗

Figure 6

We call the system of higher relative commutants associated to a subfactor N = M Z3 ⊂ M the Z3 -planar algebra. Note that it is completely determined by the commuting square u u∗ ⊂ M3 (C) ∪ ∪ ⊂ , C where is the algebra of diagonal matrices in√M3 (C) and u = (uij )0 ≤ i, j ≤ 2 is the unitary 3 × 3 matrix with entries ur, s = (1/ 3)(σ r (a s ))0 ≤ r, s ≤ 2 , where Z3 = {1, a, a 2 }, with dual group Zˆ 3 = {1, σ, σ 2 }. Its description in terms of planar diagrams can be found in [J2]. Similarly, we call the system of higher relative commutants of a subfactor N ⊂ M, with principal graphs (D∞ , D∞ ), the D∞ -planar algebra. (Recall that both graphs are automatically D∞ if one is.) It is again uniquely determined by a commuting square (this time an infinite one); see, for instance, [Po3] and [H]. Recall that this system can be obtained as the Popa system of a free composition (see [BH] and [BJ1]) of the form N = P Z2 ⊂ P ⊂ P Z2 , where the group G = Z2 , Z2 , generated by two copies of Z2 in the outer automorphism group of the II1 factor P , is the infinite dihedral group Z2 ∗ Z2 . This system is therefore a special case of the Fuss-Catalan (FC) planar algebras, discovered in [BJ1], which can be viewed as the Popa system of a subfactor obtained from the free composition of the Popa system of a subfactor with principal graphs (An , An ) and that of a subfactor with principal graphs (Am , Am ), n, m = 3, 4, . . . , ∞. The FC planar algebras can be described as colored generalizations of the Temperley-Lieb algebras, and an explicit description as planar algebras can be found in [BJ1] (see also [J2]). These colored generalizations of the Temperley-Lieb algebras turn out to be the minimal system of algebras appearing whenever an intermediate subfactor is present. We recall that their tower of inclusions is given by the Fibonacci


49

graph F (see Figure 7) in the generic case (i.e., the case of free composition of two subfactors with principal graphs A∞ ) and certain subgraphs of F , of which D∞ is a special case, in the nongeneric case. We refer the reader to [BJ1] for details.

Figure 7

Theorem 2.1 implies the following classification result. Theorem 2.2. If V is a spherical C ∗ -planar algebra generated by a threedimensional V2 , subject to the condition dim V3 ≤ 12, then it must be one of the following: (a) if dim V3 = 9, then it is the Z3 -planar algebra; (b) if dim V3 = 10, then it is the D∞ -planar algebra (a special case of (c) in fact); (c) if dim V3 = 11 or 12, it is one of the FC planar algebras in [BJ1]. Proof. Observe that if dim V2 = 3, then dim V0 = dim V1 = 1 and we are investigating spherical C ∗ -planar algebras generated by a single (nontrivial) 2-box such that dim V3 ≤ 12. Let N ⊂ M be a subfactor whose system of higher relative commutants is given by V . If dim V3 = 9, then [M : N] = 3 and we are in case (a). If 10 ≤ dim V3 ≤ 12, Theorem 2.1 implies that N ⊂ M must have an intermediate subfactor. On the other hand, it was shown in [BJ1] that the FC planar algebra (i.e., the system of algebras FC n (a, b) in the notation of [BJ1]) must be contained in the system of higher relative commutants of any subfactor that has an intermediate subfactor. Since by assumption no other conditions on the 2-box are assumed to hold, the spherical C ∗ -planar algebras generated by a single 2-box must be precisely the ones listed in the theorem. 3. Some useful results. We prove in this section several results that are needed to prove the main theorem. We also include some results about Popa systems and intermediate subfactors that are of interest in their own right. Furthermore, we show the reader how to use the diagrammatic description of elements in the higher

50

BISCH AND JONES

relative commutants, and we explain how complicated formulas become simple when rewritten by using diagrams. We start with a lemma. Lemma 3.1. (i) Let N ⊂ M be an inclusion of II1 factors with finite index. Then we have [M : N]EMk (xek+1 )ek+1 = xek+1 , for all x ∈ Mk+1 . (ii) Let N ⊂ M be an extremal inclusion of II1 factors with finite index, and let x ∈ N ∩ Mk . Then [M : N]EM ∩Mk (xe1 )e1 = xe1 . Proof. Let us first show (i). Since xek+1 = yek+1 for a unique y ∈ Mk (see [PiPo1]), namely, y = [M : N]EMk (xek+1 ), the result follows. To prove (ii), we let {mi }i∈I be an orthonormal basis of M over N with m1 = 1 (see [PiPo1]). Then, by [B2, Proposition 2.7], we have EM ∩Mk (xe1 ) = [M : N]−1 i∈I mi xe1 m∗i . Since e1 m∗i e1 = EN (m∗i )e1 = 0 if i = 1 and = 1 if i = 1, we are done. The following proposition establishes that certain higher relative commutants are isometric, although not isomorphic in general (see [B2] and [Oc2]). Proposition 3.2. Let N ⊂ M be an extremal inclusion of II1 factors with finite index. Consider the maps ψk : N ∩ Mk → M ∩ Mk+1 , k ≥ 0 (M0 = M), ψk (x) = [M : N](k+2)/2 EM ∩Mk+1 (xek+1 ek · · · e2 e1 ), x ∈ N ∩ Mk and φk : M ∩ Mk+1 → N ∩ Mk , k ≥ 0, φk (x) = [M : N](k+2)/2 EMk (xe1 e2 · · · ek ek+1 ). Then ψk and φk are isometries (in the norm · 2 ), inverse to each other. Proof. Fix a k ≥ 0, and let ψ = ψk and φ = φk . Set vk = ek ek−1 · · · e2 e1 , k ≥ 1, and note that vk vk∗ = [M : N]−(k−1) ek and vk∗ vk = [M : N]−(k−1) e1 . Let x, y ∈ M ∩ Mk , and compute the inner product ∗ φ(x), φ(y) L2 (M ) = tr φ(y)∗ φ(x) = [M : N]k+2 tr EMk (vk+1 y ∗ )EMk xvk+1 k+1 = [M : N]k+2 tr EMk (ek+1 vk y ∗ )xvk∗ ek+1 = [M : N]k+1 tr ek+1 vk y ∗ xvk∗ = [M : N ]k tr vk∗ vk y ∗ x = [M : N] tr e1 y ∗ x = [M : N ] tr EM ∩M1 (e1 )y ∗ x = tr y ∗ x , where we used Lemma 3.1(i) in the third equality and extremality in the last one. Thus φ is indeed an isometry. The calculation for ψ is similar. Let x, y ∈ N ∩ Mk ; then ∗ ψ(x), ψ(y) L2 (M ) = [M : N]k+2 tr EM ∩Mk+1 vk+1 y ∗ xvk+1 k ∗ = [M : N]k+1 tr vk+1 y ∗ xvk+1 = [M : N] tr ek+1 y ∗ x = [M : N] tr EMk (ek+1 )y ∗ x = tr y ∗ x , where we used Lemma 3.1(ii) in the second equality. Thus ψ is an isometry as well.


51

Let us finally show that the two maps are inverses of each other. Let x ∈ M ∩Mk+1 ; then ∗ vk+1 ψ φ(x) = [M : N]k+2 EM ∩Mk+1 EMk xvk+1 ∗ = [M : N]k+1 EM ∩Mk+1 xvk+1 vk = [M : N]EM ∩Mk+1 xe1 = x. ∗ ) = [M : N]k+1 Similarly, if x ∈ N ∩Mk , then φ(ψ(x)) = EMk (EM ∩Mk+1 (xvk+1 )vk+1 ∗ EMk (xvk+1 vk+1 ) = x. Note that again we used Lemma 3.1 in both calculations.

The maps φk and ψk are usually referred to as Fourier transforms (see [B2, Section 2], [Oc1] and [Oc2]). Let us describe how φ1 and ψ1 act on 2-boxes. If an arbitrary element R ∈ N ∩M1 is represented by R , then ψ1 (R) = δEM ∩M2 (RE2 E1 ) is given by

=

R

R

=

Q

as an element in N ∩ M2 . Similarly, if an arbitrary element Q ∈ M ∩ M2 ⊂ N ∩ M2 is represented by Q , then φ1 (Q) = δEM1 (QE1 E2 ) = δEN ∩M1 (QE1 E2 ) is given by

Q

Using the diagrams, it is now easy to verify that the two maps are inverse to each other. We next give a diagrammatic description of the map Jk · Jk : N ∩ Mk → N ∩ Mk , where Jk : L2 (Mk ) → L2 (Mk ) is the modular conjugation. Recall first that the Jones ek2k+1

projection for N ⊂ Mk ⊂ M2k+1 is given by ek2k+1 = [M : N ]k(k+1)/2 ek+1 ek · · · e1 × ek+2 ek+1 · · · e2 · · · e2k · · · ek e2k+1 · · · ek+1

52

BISCH AND JONES

(see [PiPo2]). Thus, if Ek2k+1 is given by the diagram in Figure 8

Figure 8

((k + 1) arcs at the top and bottom), then ek2k+1 = δ −(k+1) Ek2k+1 = [M : N]−(k+1)/2 Ek2k+1 . To simplify notation, set fk = ek2k+1 . It is shown in [B2, Theorem 2.6] that M for x ∈ N ∩ M2k+1 , we have Jk x ∗ Jk = [Mk : N] i∈I EMk2k+1 (fk mi x)fk m∗i , where {mi }i∈I is any finite basis of Mk over N with y = i∈I mi EN (m∗i y), for all y ∈ Mk . fk

f2k+1

f2k+1 Jk x ∗ Jk = Thus, if N ⊂ Mk ⊂ M2k+1 ⊂ M3k+2 is the basic construction, then ∗ k+1 k+1 [M : N] fk mi = [M : N ] f2k+1 fk ( i∈I mi xf2k+1 i∈I f2k+1 fk mi xf2k+1 ∗ ∗ −(k+1) 2 fk mi ). But (y) = [M : N ] i∈I mi ymi , y ∈ N = N ∩ B(L (Mk )), im plements the tr N -preserving conditional expectation N → Mk (see, for instance, [B2, Proposition 2.7]). Hence, if N ⊂ M is extremal, we have the following diagrams. Suppose an arbitrary element R ∈ N ∩ M2k+1 is depicted as a (2k + 2)-box R with 2k +2 vertical strings on top and bottom. Then Rf2k+1 fk is given by Figure 9 (up to the scalar δ 2k+2 ).

}

}

2k + 2 k + 1

R }

2k + 2 k + 1 Figure 9

Hence

i∈I

mi Rf2k+1 fk m∗i is depicted as


k +1

53

k +1k +1

R k +1

k +1 k +1 Figure 10

and therefore f2k+1 fk multiple):

i∈I

mi Rf2k+1 fk m∗i is given by Figure 11 (up to a scalar 2k + 2

k +1

R k +1 k +1 k +1 Figure 11

Since Jk x ∗ Jk = [M : N]k+1 EM2k+1 (f2k+1 Jk x ∗ Jk ), x ∈ N ∩ M2k+1 ; keeping track of the scalars, we get that Jk R ∗ Jk is depicted as

R Figure 12

We have therefore proved the following proposition.

54

BISCH AND JONES

Proposition 3.3. If the element R ∈ N ∩ M2k+1 is depicted as R , then Jk R ∗ Jk is depicted as . R

Note that Proposition 3.3 can, in fact, be deduced from a more general map on N ∩ M2k+1 , the rotation of order 2k + 2. (See Theorem 3.8 for the case k = 2 and [J2] for the general case.) The map Jk · Jk : N ∩ M2k+1 → N ∩ M2k+1 is then just a rotation of a (2k + 2)-box by 180 degrees. Let us prove another useful formula for the maps Jk · Jk on N ∩ M2k+1 , related to the Fourier transforms in Proposition 3.2. Proposition 3.4. Let A ⊂ B be an extremal inclusion of II1 factors with finite index, and let A ⊂ B ⊂ B1 ⊂ B2 be the beginning of the tower associated to A ⊂ B. Then J x ∗ J = [B : A]3 EB1 (EB ∩B2 (e1 e2 x)e1 e2 ), for all x ∈ A ∩ B1 . Furthermore, we have J x ∗ J = [B : A]3 EB1 (e2 e1 EB ∩B2 (xe2 e1 )). Proof. Let {mi }i∈I be an orthonormal basis of B over A. Then we have, for x ∈ A ∩ B1 , 3 2 ∗ [B : A] EB1 EB ∩B2 e1 e2 x e1 e2 = [B : A] EB1 mi e1 e2 xmi e1 e2 = [B : A]

2

= [B : A]

2

i∈I i∈I

= [B : A]

i∈I

i∈I

mi e1 EB1 e2 xm∗i e1 e2 mi e1 EB (xm∗i e1 )EB1 (e2 )

mi e1 EB xm∗i e1

= (J xJ )∗ = J x ∗ J, where we used extremality in the first equation and [B2, Theorem 2.6] in the last line. The second formula follows by a similar computation. We changed the notation in Proposition 3.4 slightly, denoting an inclusion of II1 factors by A ⊂ B. Clearly, if we let A = N and B = Mk , we get a statement about Jk · Jk : N ∩ M2k+1 → N ∩ M2k+1 . Note that the right-hand side of the formulas for J x ∗ J in Proposition 3.4 make sense even for a nonstandard system of finite-dimensional algebras, as in [Po3]. Hence multiplicativity of the formula is an obstruction for the system to be standard, that is, to come from a subfactor. More precisely, we must have [B : A]3 (EB1 (EB ∩B2 (e1 e2 x) e1 e2 ))2 = EB1 (EB ∩B2 (e1 e2 x 2 )e1 e2 ), x ∈ A ∩ B1 , in this case. We next mention a useful result regarding the normalizer of a subfactor, which can be found in [PiPo1]. Recall that if N ⊂ M is a subfactor, then the normalizer ᏺ(N ) is defined as ᏺ(N ) = {u ∈ ᐁ(M) | uNu∗ = N}. Clearly, all unitaries u ∈ N are contained in ᏺ(N ), and we call ᏺ(N) nontrivial if it contains a unitary in M not contained in N .


55

Proposition 3.5. Let N ⊂ M be an irreducible inclusion of II1 factors with finite index, and suppose there is a projection p ∈ N ∩M1 that is equivalent to e1 (in M1 ), p = e1 . Then N has a nontrivial normalizer. Proof. The proof is very simple. By assumption, there is a unitary u ∈ M1 such that ue1 u∗ = p. By [PiPo1], there is a unique v ∈ M with ue1 = ve1 . Thus ve1 v ∗ = ue1 u∗ = p, and hence [M : N ]−1 vv ∗ = EM (p) = [M : N]−1 · 1 (since N ∩ M = C). Hence v ∈ M is a unitary. Clearly v ∈ / N , since otherwise p = ve1 v ∗ = e1 vv ∗ = e1 , which is a contradiction. Moreover, v normalizes N: Let x ∈ N, then v ∗ ve1 v ∗ xv = v ∗ (ve1 v ∗ )xv = v ∗ pxv = v ∗ xpv = v ∗ xvv ∗ pv = v ∗ xve1 . Thus v ∗ xv ∈ {e1 } ∩ M = N. Observe that the von Neumann algebra generated by ᏺ(N) is an intermediate subfactor of N ⊂ M (which is assumed to be irreducible). Proposition 3.5 can therefore be useful to show the existence of intermediate subfactors. We continue our collection of useful facts with a diagrammatic way of deciding whether a projection in N ∩ M1 comes from an intermediate subfactor. The next proposition is a reformulation of [B1, Theorem 3.2]. (See also [La] for applications of similar types of relations as the ones in Proposition 3.6(iii) to planar algebras.) Proposition 3.6. Let N ⊂ M be an irreducible inclusion of II1 factors with finite index. Let p ∈ N ∩ M1 be an operator depicted by P . Then p is the orthogonal projection onto an intermediate subfactor if and only if (i) p = p2 , p = p ∗ , or diagrammatically 

P =

P ,

∗

   P  =

P

Figure 13

(ii) e1 ≤ p, or diagrammatically

P

=

P

Figure 14

(iii) the following exchange relation holds

=

P

56

BISCH AND JONES

P

P P

=

P

Figure 15

Proof. We use the abstract characterization of intermediate subfactors in [B1]. Conditions (i) and (ii) are obviously necessary and are satisfied if p is the orthogonal projection onto an intermediate subfactor. p1 e1 Suppose N ⊂ M has an intermediate subfactor P , and let N ⊂ P ⊂ M ⊂ P1 ⊂ p2

e2

M1 ⊂ P2 ⊂ M2 be the first few steps in the tower of II1 factors associated to N ⊂ P ⊂ M (see [B1] and [BJ1]). In particular, p2 is the projection obtained from the basic construction for P1 ⊂ M1 , and p = p1 is the orthogonal projection ePM from M onto P . Let α = [P : N]−1 = tr(p2 ), β = [M : P ]−1 = tr(p1 ). Then p1 e2 e1 = βp2 e1 (see [B1]), and hence p2 = α −1 β −2 EM ∩M2 (p1 e2 e1 ) and p1 = α −2 β −1 EM1 (p2 e1 e2 ). Thus p2 is depicted as

P Figure 16

(up to a scalar, i.e., the diagram in Figure 16 equals (β/α)1/2 p2 ), and (iii) is true if and only if p1 p2 = p2 p1 , which holds since p2 actually commutes with the II1 factor P1 . Conversely, if (iii) holds, then p1 EM ∩M2 (p1 e2 e1 ) = EM ∩M2 (p1 e2 e1 )p1 , and hence p1 EM ∩M2 (p1 e2 e1 )e2 = EM ∩M2 (p1 e2 e1 )p1 e2 , which implies EM ∩M2 (p1 e2 e1 )e2 = αβEM ∩M2 (p1 e2 ) = αβ 2 e2 (since EM ∩M2 (p1 ) = β · 1). Thus we get 2 = EM ∩M2 EM ∩M2 p1 e2 e1 p1 e2 e1 EM ∩M2 p1 e2 e1 = EM ∩M2 p1 EM ∩M2 p1 e2 e1 e2 e1 = αβ 2 EM ∩M2 p1 e2 e1 . We next show that EM ∩M2 (p1 e2 e1 ) is selfadjoint. Since e2 p1 EM ∩M2 (p1 e2 e1 ) = e2 EM ∩M2 (p1 e2 e1 )p1 and e2 p1 e2 = EM (p1 )e2 = βe2 , we get e2 p1 EM ∩M2 (p1 e2 e1 ) = βe2 EM ∩M2 (e1 )p1 = αβ 2 e2 p1 . Hence, multiplying the previous equation with e1 from the left and expecting onto M ∩ M2 , we get EM ∩M2 (e1 e2 p1 )EM ∩M2 (p1 e2 e1 )


57

= αβ 2 EM ∩M2 (e1 e2 p1 ). Note that the left-hand side is selfadjoint. Thus EM ∩M2 (p1 e2 e1 ) is a multiple of a projection, which shows by [B1, Corollary 3.3] that N ⊂ M has an intermediate subfactor. Let us show how the proof proceeds, using diagrams. We multiply the equation in Proposition 3.6(iii) with the diagram E2 and get

P

= P

P

P Since P = β , P = β , we get condition (iii ) after expecting onto M1 ,

c P = P

P

Figure 17

for a nonzero scalar c. Thus, by applying the Fourier transform (Proposition 3.2), we get

P P

=c P Figure 18 P

which says that is a scalar multiple of an idempotent. We leave it to the reader to give the diagrammatic argument for selfadjointness of . Observe that we have actually shown that condition (iii) in Proposition 3.6 and condition (iii ) depicted in Figure 17 for some nonzero scalar c plus selfadjointness of are equivalent. Indeed, we showed that (iii) implies (iii ). Conversely, if (iii ) P

P

58

BISCH AND JONES P

holds, then is a multiple of a projection and hence, by [B1], is (up to a scalar) the projection p2 coming from an intermediate subfactor (notation is as in the first part of the proof of Proposition 3.6). But then p1 p2 = p2 p1 ; that is, condition (iii) holds. If a projection p ∈ N ∩M1 satisfies (i), (ii), and (iii) (or (iii )), we call it a biprojection. Thus the biprojections are precisely those orthogonal projections in N ∩M1 that project onto an intermediate subfactor N ⊂ P ⊂ M (assuming N ⊂ M is irreducible). The next part is probably the most important result in this section. Consider the maps ψk : N ∩ Mk → M ∩ Mk+1 , ψk (x) = [M : N](k+2)/2 EM ∩Mk+1 (xek+1 ek · · · e2 e1 ), x ∈ N ∩ Mk , and φk : M ∩ Mk+1 → N ∩ Mk , φk (x) = [M : N](k+2)/2 EMk (xe1 e2 · · · ek ek+1 ), as in Proposition 3.2 (k ≥ 1). Let φ˜ k : M ∩ Mk+1 → N ∩ Mk be defined by φ˜ k (x) = φk (x ∗ )∗ , x ∈ M ∩ Mk+1 . Definition 3.7. The map rk : N ∩Mk → N ∩Mk defined by rk = φ˜ k ◦ψk is called the rotation of period k + 1 (or the rotation by 360/(k + 1) degrees) on the higher relative commutant N ∩ Mk . We would like to point out that the fact that the map rk acts on the higher relative commutants of a subfactor is one of the key features of such a system of finitedimensional algebras. It is to a large extent “responsible” for the planar structure of the higher relative commutants (see [J2]). The next theorem justifies Definition 3.7. Theorem 3.8. Let N ⊂ M be an extremal inclusion of II1 factors with finite index, and let rk : N ∩ Mk → N ∩ Mk be defined by rk (x) = [M : N]k+2 EMk ek+1 ek · · · e2 e1 EM ∩Mk+1 xek+1 ek · · · e2 e1 , x ∈ N ∩ Mk , as in Definition 3.7. Then rkk+1 = id. In this paper, we only use the maps r1 and r2 . Observe that the case k = 1 has already been shown in Proposition 3.3, and we give a proof in the case k = 2 below. For the general case, see [J2], where a different proof is presented. Observe that since Mk+1 ⊂ Mk ∪ ∪ ∩M ⊂ N N ∩ Mk k+1 is a commuting square, we have rk (x) = [M : N]k+2 EN ∩Mk (ek+1 ek · · · e2 e1 EM ∩Mk+1 (xek+1 ek · · · e2 e1 )). We proceed with the proof of Theorem 3.8 for k = 2. ˜ Proof. Let ψ = ψ2 , φ = φ2 , φ(x) = φ(x ∗ )∗ , x ∈ M ∩ M2 , and r2 = φ˜ ◦ ψ. Note ˜ that φ and ψ are invertible with inverses ψ −1 = φ and φ˜ −1 (x) = ψ(x ∗ )∗ , x ∈ N ∩M2 (see Proposition 3.2). Thus r23 = id if and only if ψ ◦ φ˜ ◦ ψ = φ˜ −1 ◦ ψ −1 ◦ φ˜ −1 and if and only if ∗ ∗ ∗ ψ φ ψ(x)∗ = ψ φ ψ(x ∗ )∗ , (∗)

59


for all x ∈ N ∩ M2 . The equality (∗) holds if and only if EM ∩M3 EM2 e3 e2 e1 EM ∩M3 xe3 e2 e1 e3 e2 e1 = EM ∩M3 e1 e2 e3 EM2 EM ∩M3 (e1 e2 e3 x)e1 e2 e3 ,

(∗∗) e13

for all x ∈ N ∩ M2 . Consider next the 2-step basic construction N ⊂ M1 ⊂ M3 , where e13 = [M : N ]e2 e1 e3 e2 (see [PiPo2]). Furthermore, let {mi }i∈I ⊂ M be an orthonormal basis of M over N. Then {[M : N]1/2 mi e1 mj }i, j ∈I is a finite basis of M1 (mj∗ e1 m∗i x), for all x ∈ M1 M1 over N such that x = [M : N] i, j ∈I mi e1 mj EN (compare with [B2, Proposition 2.10]). It is shown in [B2, Theorem 2.6] that one then has M J1 x ∗ J1 = [M : N ]3 EM13 e13 mi e1 mj x e13 mj∗ e1 m∗i i, j ∈I

= [M : N ]

5

= [M : N ]

5

i, j ∈I

i, j ∈I

M EM13 e2 e1 e3 e2 mi e1 mj x e2 e1 e3 e2 mj∗ e1 m∗i M EM13 e2 e1 e3 mi e2 e1 mj x e2 e1 mj∗ e3 e2 e1 m∗i ,

where J1 : L2 (M1 ) → L2 (M1 ) denotes, as usual, the modular conjugation. We compare this result the left-hand side of (∗∗) equals with (∗∗). Since N ⊂ M is extremal, [M : N]−2 i,j ∈I mi EM2 e3 e2 e1 mj xe3 e2 e1 mj∗ e3 e2 e1 m∗i (see [B2, Proposition 2.7]). This in turn is equal to EM2 e3 mi e2 e1 mj xe3 e2 e1 mj∗ e3 e2 e1 m∗i [M : N]−2 i, j ∈I

= [M : N]−2

i, j ∈I

= [M : N]

−3

i, j ∈I

M M2 EM23 EM (mi e2 e1 mj x)e3 e2 e1 mj∗ e3 e2 e1 m∗i 1 EM1 (mi e2 e1 mj x)e2 e1 mj∗ e3 e2 e1 m∗i . M

Let y ∈ M1 be an arbitrary operator, then tr(yEM13 (e2 e1 e3 e2 mi e1 mj x)) = tr(ye2 e1 e3 e2 mi e1 mj x) = [M : N]−1 tr(ye2 e1 e2 mi e1 mj x) = [M : N]−2 tr(ymi e2 e1 M mj x). Thus EM13 (e2 e1 e3 e2 mi e1 mj x) = [M : N]−2 EM1 (mi e2 e1 mj x). This implies that the left-hand side of (∗∗) is equal to [M : N]−6 J1 x ∗ J1 . Since (J1 xJ1 )∗ = J1 x ∗ J1 , we have proved that the equality (∗∗) holds. We now discuss the diagrammatic interpretation of the rotations r1 and r2 . The map r1 is the rotation by 90 degrees, and the diagram for r1 (R), where R ∈ N ∩ M1 is depicted by a 2-box, is given in Proposition 3.3 (case k = 0, see Figure 12). If an element R ∈ N ∩ M2 is given by the 3-box R , then r2 (R) is given by

60

BISCH AND JONES

R

=

R

Figure 19

Note that all orientations of the vertical strings are up-down-up (from left to right), so that the right-hand side of Figure 19 is indeed an element of N ∩M2 . It is now very easy to verify that r23 (R) = R, using the diagram for the rotation given by Figure 19. Let us also mention that the identity (∗∗) in the proof of Theorem 3.8 is depicted as

R

? =

R

Figure 20 R

This identity obviously holds true, since both sides are equal to (which is precisely J1 R ∗ J1 ). Note that the diagrams in Figure 20 depict elements in M ∩ M3 ; the orientation of the vertical strings is down-up-down (from left to right). We next discuss another natural operation on the higher relative commutants, which is inspired by Proposition 3.6 and its reformulation given in Figure 19. Namely, given two elements of two higher relative commutants, there is a natural “comultiplication,” which in the depth-2 case is the actual comultiplication of Hopf algebras (using the canonical duality induced by the trace). For instance, given two 2-boxes R and Q in N ∩ M1 , we can form another 2-box, the coproduct of R and Q as depicted in Figure 21.


R

61

Q

Figure 21

If R, Q ∈ N ∩ M1 are represented by 2-boxes in the usual way, then the element in Figure 21 is given by the formula δ 5 EM1 (e2 REM ∩M2 (e1 e2 Q)) = δ 5 EN ∩M1 (e2 REM ∩M2 (e1 e2 Q)). Observe that comultiplication is associative. To see this, we need to check that EM1 e2 EM1 e2 REM ∩M2 e1 e2 Q EM ∩M2 (e1 e2 L) , = EM1 e2 REM ∩M2 e1 e2 EM1 e2 QEM ∩M2 e1 e2 L

(∗ ∗ ∗)

for all R, Q, L ∈ N ∩ M1 . By Lemma 3.1(i), the left-hand side of equation (∗ ∗ ∗) equals EM1 (e2 REM ∩M2 (e1 e2 Q)EM ∩M2 (e1 e2 L))=EM1 (e2 REM ∩M2 (e1 e2 QEM ∩M2 (e1 e2 L))), which is equal to the right-hand side of (∗ ∗ ∗), again by Lemma 3.1(i). Diagrammatically, associativity of the comultiplication is obvious; both sides of the identity (∗ ∗ ∗) are given by Figure 22.

R

Q

L

Figure 22

Clearly, the operation “comultiplication” generalizes in several ways to other higher relative commutants as indicated in Figure 23.

R Q

R Figure 23

Q

62

BISCH AND JONES

We will discuss these maps in another paper. 4. Proofs of the main results. We give in this section the proof of Theorem 2.1, as the proof of Theorem 4.1, and we present an example of a principal graph satisfying the conditions of the theorem, which is not covered by Theorem 2.2. Let us state again Theorem 2.1 for the convenience of the reader. Theorem 4.1. Let N ⊂ M be II1 factors with 3 < [M : N ] < ∞, dim N ∩M1 = 3, and such that N ∩ M2 is abelian modulo the basic construction ideal. Then there is an intermediate subfactor P of N ⊂ M, and hence we have J x ∗ J = x, for all x ∈ N ∩ M1 . We first prove that under the above conditions, J · J has to be trivial on N ∩ M1 . Since dim N ∩M1 = 3, we can write N ∩M1 = Ce1 ⊕Cq ⊕C(1−e1 −q), where e1 , q, and 1−e1 −q are the minimal projections of N ∩M1 . We then have the following obvious lemma. Lemma 4.2. J x ∗ J = x, for all x ∈ N ∩ M1 , if and only if J qJ = q. Thus J · J is nontrivial on N ∩ M1 if and only if J qJ = 1 − e1 − q. Let us represent the minimal projection q by the 2-box Q . Nontriviality of J ·J is Q

=

Q

Q

then depicted by

Q

= 0 (see Proposition 3.3 for the diagram of J qJ ).

Proposition 4.3. Let N ⊂ M be II1 factors with 3 < [M : N ] < ∞, dim N ∩ M1 = 3, and N ∩ M2 abelian modulo the ideal generated by e2 . Then J · J is trivial on N ∩ M1 ; that is, J x ∗ J = x, for all x ∈ N ∩ M1 . The following lemma determines the orbits under the rotation r2 of special elements R

, which turn out to play an important role in the proof of

R

in N ∩M2 of the form R

Proposition 4.3. A formula for this element is, for instance, δ 3 J R ∗ J EM ∩M2 (e1 e2 R)R. Lemma 4.4. Let R ∈ N ∩ M1 be an arbitrary element, depicted by R . We have the following orbits under the rotation r2 :

R

, R

,

R

R

R R

(1)

R

R

;

R

63


R R

,

,

R

R

;

R

R R

(2)

R

R

R

(3)

;

R

R

.

R R (4) R Furthermore, if R = R ∗ , then by taking adjoints, we map the orbit in (1) to the orbit in (2) and the orbit in (3) to the orbit in (4).

R

R

Proof. Apply the rotation given in Figure 20 to the elements in (1), (2), (3), and (4). The calculation of the orbits is then immediate. To get the second statement, ∗ ∗ . Then the result follows easily. = observe that R = R implies Let us now give the proof of Proposition 4.3. Proof. Since N ∩ M2 is abelian modulo the basic construction ideal, we have

Q

=

+α

Q

Q

Q

Q

+A

+B

+C

+D

Q (1)

+E

Q

+F

Q

+G

Figure 24

Q

+H

Q

64

BISCH AND JONES

for some scalars α, A, B, . . . , G, H ∈ C. This equations reads A B C D α qEM ∩M2 qe2 e1 = EM ∩M2 qe2 e1 q + 2 qe2 q + 2 e1 + 2 e2 + e1 e2 + e2 e1 δ δ δ δ δ E F G H + 2 qe2 + 2 e2 q + qe2 e1 + e1 e2 q. (1 ) δ δ δ δ Orthogonality of the projections q and e1 is expressed diagrammatically by Q

=

Q

= 0. Furthermore, let β ∈ R be such that tr(q) = (β/δ 2 ) (tr denotes the

normalized trace as usual). Then e2 qe2 = EN ∩M (q)e2 = (β/δ 2 )e2 , so that diagrammatically

Q

= βδ . Note that q is a projection, so that

Q Q

=

Q

.

Applying the conditional expectation EM1 to both sides of (1), multiplying (1) by the diagram E1 from above (i.e., multiplying (1 ) by the projection e1 from the right) and multiplying (1) by E1 from the bottom (i.e., multiplying (1 ) by e1 from the left), gives the following three equations: 0 = (α + E + F ) Q + (Aδ + C + D)

Q = (Aδ + C)

+ (B + Dδ)

+B

+ (E + Gδ) Q

(2)

(3)

Q Q

0=

+ (Aδ + D)

+ (B + Cδ)

+ (F + H δ) Q

(4)

Figure 25

The above equations, of course, can be rewritten by using formulas; we do this here. Note that EM1 (EM ∩M2 (qe2 e1 )) = tr(qe2 e1 ) = tr(e1 qe2 ) = 0. Thus applying EM1 to equation (1 ) gives B A C D α E F (2 ) 0 = q 4 + 4 + 4 + 4 · 1 + e1 2 + 3 + 3 , δ δ δ δ δ δ δ which is precisely equation (2) after multiplying by δ 4 . (Recall that tr(ei ) = [M : N]−1 = (1/δ 2 ).)

65


Observe that EM ∩M2 (qe2 e1 )e1 = (1/δ 2 )qe2 e1 , so that multiplying (1 ) by e1 from the right yields A C B D E G 1 e2 e1 + 2 + qe2 e1 , qe2 e1 = 2 + 3 e1 + 2 + (3 ) δ δ δ2 δ δ δ δ which is precisely equation (3) after multiplying by δ 4 . Similarly, one obtains equation (4) from (1 ). Since the diagrams occurring in equations (2) and (3) are linearly independent, we obtain A = B = C = D = 0. It follows from (4) that F +H δ = 0 if J ·J is nontrivial and F + H δ + 1 = 0 otherwise. Thus

Q

Q

Q

=

Q

Q

+α

+E Q

+F

Q

Q

+G Q +H

Q

(5)

We proceed by contradiction. Let us suppose that J · J is nontrivial, so that Q

= 0. We work with diagrams, leaving it to the reader to translate the dia-

Q

grams into formulas. Multiply (5) with This gives

Q

=

Q

Q

from below (i.e., with J qJ from the left).

Q

Q

0=

Q +F Q

Q

from above gives

Q

Q

Similarly, multiplying (5) with

Q

Q Q

(6)

=E

(7) Q

We have seen in Section 3, Figure 23, that we can comultiply a 3-box R in N ∩M2 and a 2-box L in M ∩ M2 to get a new 3-box R L in N ∩ M2 . (Observe that all orientations of the vertical strings at the top and bottom of the boxes match.) Thus

66

BISCH AND JONES Q

and equation (7) with

Q

gives the following

(8) Q

F

Q

comultiplying equation (6) with two equations:

=0

Q Q

Q

(E − 1)

(9) =0

Q Q

= 0. By taking adjoints,

Q

Q

Q

this implies

Q

We have to consider two cases. First, suppose that = 0 (see Lemma 4.4).

Q

Thus we get F = 0 and E = 1, which in turn implies G = 0 by (3), H = 0 by (4), and α = −1 by (2). Therefore equation (5) reduces to

Q

−

Q

Q

= Q

Q

+

Q

Q

(10)

Applying the conditional expectation EM ∩M2 to equation (10) and observing that β

Q = δ we obtain

β = δ

Q

Q

Q

β δ

− Q

Thus

Q Q

+

=

β δ

β δ

(11)

67


which implies Q =β Q But Q = β , so that the left-hand side of the above equation equals βδ 2 δ

hence we obtain β = q = 1.

δ2 .

Thus tr(q) =

(β/δ 2 )

and

= 1, which is a contradiction since Q

Q

Q

Q

Q

Q

of the rotation, as in Lemma 4.4, are zero. We next show that

= 0, which is

Q

Q

Q

equivalent to

= 0. Then, by

= 0, and hence all elements in the first two orbits

Q

taking adjoints, we must have

Q

Let us now deal with the second case, that is, the case in which

= 0 (by taking adjoints). Suppose that this is not the case. Then,

Q

Q

multiplying equation (5) by

from below, we obtain Q 0=F Q

which implies Q 0=F

=

β F δ Q

Q

so that F = 0. Then equation (7) implies E = 0, and hence α = 0 by (2), G = (1/δ) by (3), and H = 0 by (4). Thus equation (5) becomes

Q

+

Q

Q

= Q

1 δ

Q

(12)

68

BISCH AND JONES

Applying again the conditional expectation EM ∩M2 to both sides of equation (12), we get +

1 δ

Q

β δ

Q

=

Q

β δ

Q

which is impossible since = 0. Thus we have shown that the two above diagrams are indeed nonzero elements in the higher relative commutant N ∩ M2 . Comultiplying equation (6) by gives Q

Q

Q

(F + 1)

=0

Q Q

which implies F = −1 by what we have just proved. Comultiplying equation (7) by implies E = 0. Therefore we obtain α = 1, G = (1/δ), and H = (1/δ). Thus equation (5) becomes

Q

Q

+

−

Q

+

1 δ Q

Q

Q =

Q

Q

+

Q

+

1 δ

Q

(13)

β δ

−

= β

β δ

+

1 δ

1 δ

(14)

from above gives

−

1 δ

−

Q

Q

Q Q

Multiplying equation (14) with Q

+

1 δ

(15) Q

=

Q

Q

β δ

Q

Applying again EM ∩M2 to equation (13) leads to

so that (β/δ)2 = β − (2β/δ 2 ), that is, β(β + 2 − δ 2 ) = 0, and hence β = 0 or

69


β = δ 2 − 2. Since 0 = tr(q) = (β/δ 2 ), we get tr(q) = 1 − (2/δ 2 ). Since J qJ = 1−e1 −q by assumption (nontriviality of J ·J ), we√get 1−(2/δ 2 ) = tr(q) = tr(J qJ ) = 1 − (1/δ 2 ) − (1 − (2/δ 2 )) = (1/δ 2 ). Hence δ = 3, that is, [M : N] = 3, a contradiction. Thus, unless [M : N] = 3, we must have J x ∗ J = x, for all x ∈ N ∩ M1 , as claimed. We can now proceed with the proof of the main theorem. Proof of Theorem 4.1. We use the same notation as in the proof of the previous proposition. We have shown in Proposition 4.3 that the map J ·J is necessarily trivial on N ∩ M1 . Therefore Q = , and we just write for this minimal projection in Q

Q

=

Q

Q

= , and N ∩ M1 (reducing the box Q to a point). Similarly, we have for this operator in N ∩ M2 . In this notation, we then have, for we write briefly instance,

=

Q

=

+ 1δ

. The projection

=

where µ = δ tr(q), and hence

=

Q

P

We denote by p ∈ N ∩ M1 the projection p = q + e1 , and we depict it as or, in short form, = + 1δ = + 1δ . Furthermore, we have satisfies

=

=0

= µ

satisfies

=

where ν = δ tr(p). Furthermore, we have

=0

=ν

70

BISCH AND JONES

=

=

and =ν

=µ

=

=0

In this simplified notation, a basis of the basic construction ideal (N ∩M1 )e1 (N ∩ M1 ) is given by

. To show that there is an interor

= −

is a biprojection,

that is, satisfies the conditions of Proposition 3.6. Since (N ∩ M1 )e1 (N ∩ M1 ) by hypothesis, we have

N ∩ M2

is abelian modulo

mediate subfactor N ⊂ P ⊂ M, we show that either

=

+α

+A

+B

+C (16)

+D

+E

+F

+G

+H

As in the proof of Proposition 4.3, we apply the conditional expectation EM1 to both sides of equation (16) and multiply (16) with the diagram E1 from above and from below. We obtain the following three equations: =

+(α + E + F )

+(Aδ + C + D)

=

+(Aδ + C)

+(B + Dδ)

=

+(Aδ + D)

+(B + Cδ)

+B

+(E + Gδ)

+(F + H δ)

(17)

(18)

(19)

This implies A = B = C = D = 0, E + Gδ = 0, F + H δ = 0, and α + E + F = 0

71


by linear independence. Thus α = (G + H )δ = −(E + F ). We next apply the conditional expectation EM ∩M2 to equation (16) and obtain

0=α

=ν

since

=

+(Eµ + F µ)

(G + H )

(20)

. (Note that equation (20) is written in N ∩M2 ⊃ M ∩M2 .)

Equation (20) is the same as −αµ

+

α δ

(21)

Q

0=α

Note that equation (21) is written in M ∩ M2 . Suppose now that α = 0, so that −µ

+

1 δ

(22)

Q

0=

Multiplying equation (22) with the diagram E2 from above yields 0 = µ2

−µδ

+

µ δ

(23)

is µ/δ; that is, tr(q) = so that µ = δ − (1/δ). But the normalized trace of Q = 1 − (1/δ 2 ) = tr(1 − e1 ). But q(1 − e1 ) = q, and hence q = 1 − e1 by nondegeneracy of the (positive) trace. This is a contradiction to dim N ∩ M1 = 3. Thus we must have α = 0, and therefore G = −H and E = −F . Since G = −(E/δ), equation (16) simplifies to 1 − = +E − . + δ (24)

= =

. Furthermore, since + 1δ

, and thus

=

+ 1δ

P

We want to comultiply equation (24) with

=

. Observe that

, we have

=

+ 1δ

and , and

72

BISCH AND JONES

=

Q

P

Q

Since

=

−

1 δ

(25)

=

−

1 δ

(26)

, we have

=

=

P

Thus comultiplying equation (24) with

(27)

gives



 1 − δ

+E 

=

+ 1δ

. Thus

=

1 δ

+

−

1 δ

+ 1δ

and

+

1 δ

1 + δ

=

= −

Q

=

so that

P

Recall that P = Q + 1δ

−

1 + δ

+ 1δ 1 δ

1 − δ + 1δ



(28)

or, in short,

, which implies

=0

(29)

Thus =

+E

−

(30)

Q

If E = 1, then = , so that P = is a biprojection, and hence N ⊂ M has an intermediate subfactor by Proposition 3.6. If E = 1, we show that e1 +(1−e1 −q) = 1−q is necessarily a biprojection. Recall that we depict the projection 1 − q by = − . Hence = − recall that = . We compute

73


=

−

=

−

=

−

−

+

=

−

−

(31)

+

(32)

Thus −

=

−

+

−

(33)

Next, we calculate 1 δ

=

+

1 δ

=

+

=

+

=

+

1 δ

+

1 δ

+

1 δ

+

1 δ

+

1 δ2

(34)

1 δ2

(35)

and

Therefore −

=

−

+

1 δ

−

1 δ

(36)

Hence, using equation (24) with E = 1, we obtain

−

=

−

(37)

. The last identity says that = 1−q is a biprojection, and hence = we are done by again applying Proposition 3.6. and thus

Examples of subfactors whose higher relative commutants satisfy the conditions of Theorem 4.1 can be constructed via free composition of two subfactors (see [BJ1] and [BJ2]; see also [Gn]). For instance, one can construct a spherical C ∗ -planar algebra (or a Popa system) as a free composition of two Popa systems associated to subfactors with principal graph E6 , and using Popa’s theorem (see [Po3]), one obtains in this way a subfactor N ⊂ M with principal graph given by Figure 26.

74

BISCH AND JONES

∗

Figure 26

Observe that dim N ∩ M2 = 14 > 12 and that N ∩ M2 modulo the basic construction ideal is (isomorphic to) C⊕C⊕C⊕C⊕C. Free compositions (or free products) of planar algebras will be discussed in [BJ2]; see also [BJ1]. References [BiWe] [B1] [B2]

[BH] [BJ1] [BJ2] [EK] [FRS]

[Gn] [GHaJ] [H] [Iz] [J1] [J2] [JSu]

J. Birman and H. Wenzl, Braids, link polynomials and a new algebra, Trans. Amer. Math. Soc. 313 (1989), 249–273. D. Bisch, A note on intermediate subfactors, Pacific J. Math. 163 (1994), 201–216. , “Bimodules, higher relative commutants and the fusion algebra associated to a subfactor” in Operator Algebras and Their Applications (Waterloo, Ontario, 1994/1995), Fields Inst. Commun. 13, Amer. Math. Soc., Providence, 1997, 13–63. D. Bisch and U. Haagerup, Composition of subfactors: New examples of infinite depth subfactors, Ann. Sci. École Norm. Sup. (4) 29 (1996), 329–383. D. Bisch and V. Jones, Algebras associated to intermediate subfactors, Invent. Math. 128 (1997), 89–157. , Planar algebras, II, in preparation. D. Evans and Y. Kawahigashi, Quantum Symmetries on Operator Algebras, Oxford Math. Monogr., Oxford Univ. Press, New York, 1998. K. Fredenhagen, K.-H. Rehren, and B. Schroer, Superselection sectors with braid group statistics and exchange algebras, I: General theory, Comm. Math. Phys. 125 (1989), 201–226. S. Gnerre, Free composition of paragroups, Ph.D. thesis, University of California at Berkeley, 1997. F. Goodman, P. de la Harpe, and V. Jones, Coxeter Graphs and Towers of Algebras, Math. Sci. Res. Inst. Publ. 14, Springer-Verlag, New York, 1989. √ U. Haagerup, “Principal graphs of subfactors in the index range 4 < [M : N] < 3 + 2” in Subfactors (Kyuzeso, 1993), World Scientific, River Edge, N.J., 1994, 1–38. M. Izumi, Application of fusion rules to classification of subfactors, Publ. Res. Inst. Math. Sci. 27 (1991), 953–994. V. Jones, Index for subfactors, Invent. Math. 72 (1983), 1–25. , Planar algebras, I, preprint, 1998. V. Jones and V. Sunder, Introduction to Subfactors, London Math. Soc. Lecture Note Ser. 234, Cambridge Univ. Press, Cambridge, 1997.

SINGLY GENERATED PLANAR ALGEBRAS [La] [Lo1] [Lo2] [Mu] [Oc1]

[Oc2]

[PiPo1] [PiPo2] [Po1] [Po2] [Po3] [Po4] [Sc] [Wa]

[We1] [We2]

75

Z. Landau, Ph.D. thesis, University of California at Berkeley, 1998. R. Longo, Index of subfactors and statistics of quantum fields, I, Comm. Math. Phys. 126 (1989), 217–247. , Index of subfactors and statistics of quantum fields, II: Correspondences, braid group statistics and Jones polynomial, Comm. Math. Phys. 130 (1990), 285–309. J. Murakami, The Kauffman polynomial of links and representation theory, Osaka J. Math. 24 (1987), 745–758. A. Ocneanu, “Quantized groups, string algebras and Galois theory for algebras” in Operator Algebras and Applications, Vol. 2, London Math. Soc. Lecture Note Ser. 136, Cambridge Univ. Press, Cambridge, 1988, 119–172. , Quantum symmetry, differential geometry of finite graphs and classification of subfactors, University of Tokyo Seminary Notes (notes recorded by Y. Kawahigashi), no. 45, 1991. M. Pimsner and S. Popa, Entropy and index for subfactors, Ann. Sci. École Norm. Sup. (4) 19 (1986), 57–106. , Iterating the basic construction, Trans. Amer. Math. Soc. 310 (1988), 127–133. S. Popa, Classification of subfactors: The reduction to commuting squares, Invent. Math. 101 (1990), 19–43. , Classification of amenable subfactors of type II, Acta Math. 172 (1994), 163–255. , An axiomatizaton of the lattice of higher relative commutants of a subfactor, Invent. Math. 120 (1995), 427–445. , Classification of Subfactors and Their Endomorphisms, CBMS Regional Conf. Ser. in Math. 86, Amer. Math. Soc., Providence, 1995. J. K. Schou, Commuting squares and index for subfactors, Ph.D. thesis, Odense Universitet, 1991. A. Wassermann, Operator algebras and conformal field theory, III: Fusion of positive energy representations of LSU(N) using bounded operators, Invent. Math. 133 (1998), 467–538. H. Wenzl, Hecke algebras of type An and subfactors, Invent. Math. 92 (1988), 349–383. , Quantum groups and subfactors of type B, C, and D, Comm. Math. Phys. 133 (1990), 383–432.

Bisch: Department of Mathematics, University of California at Santa Barbara, Santa Barbara, California 93106, U.S.A.; [email protected] Jones: Department of Mathematics, University of California at Berkeley, Berkeley, California 94720, U.S.A.; [email protected]

Vol. 101, No. 1


© 2000

A NEW FORMULA FOR WEIGHT MULTIPLICITIES AND CHARACTERS SIDDHARTHA SAHI

1. Introduction. The weight multiplicities of a representation of a simple Lie algebra g are the dimensions of eigenspaces with respect to a Cartan subalgebra h. In this paper, we give a new formula for these multiplicities. Our formula expresses the multiplicities as sums of positive rational numbers. Thus it is very different from the classical formulas of Freudenthal [F] and Kostant [Ks], which express them as sums of positive and negative integers. It is also quite different from recent formulas due to Lusztig [L1] and Littelmann [Li]. For example, for the multiplicity of the next-to-highest weight in the n-dimensional representation of sl2 , we get the following expression (which sums to 1): 1 1 1 1 + +···+ + . (1)(2) (2)(3) (n − 1)(n) n The key role in our formula is played by the dual affine Weyl group. Let V0 , ( , ) be the real Euclidean space spanned by the root system R0 of g, and let V be the space of affine linear functions on V0 . We identify V with Rδ ⊕ V0 via the pairing (rδ + x, y) = r + (x, y) for r ∈ R, x, y ∈ V0 . The dual affine root system is R = {mδ + α ∨ | m ∈ Z, α ∈ R0 } ⊆ V , where α ∨ means 2α/(α, α) as usual. Fix a positive subsystem R0+ ⊆ R0 with base {α1 , . . . , αn }, and let β be the highest short root. Then a base for R is given by a0 = δ − β ∨ , a1 = α1∨ , . . . , an = αn∨ , and we write si for the (affine) reflection about the hyperplane {x | (ai , x) = 0} ⊆ V0 . The dual affine Weyl group is the Coxeter group W generated by s0 , . . . , sn , and the finite Weyl group is the subgroup W0 generated by s1 , . . . , sn . For w ∈ W , its length is the length of a reduced (i.e., shortest) expression of w in terms of the si . The group W acts on the weight lattice P of g, and each orbit contains a unique (minuscule) weight from the set ᏻ := λ ∈ P | α ∨ , λ = 0 or 1, ∀α ∈ R0+ . Definition. For each λ in P , we define (1) λ := λ + (1/2) α∈R + ε(α ∨ ,λ) α, where, for t ∈ R, εt is 1 if t > 0 and −1 if 0 t ≤ 0; Received 1 March 1999. 1991 Mathematics Subject Classification. Primary 17B10, 33C50; Secondary 20F55, 20E46. Author’s work supported by a National Science Foundation grant. 77

78

SIDDHARTHA SAHI

(2) wλ := unique shortest element in W such that λ := wλ · λ ∈ ᏻ. We fix a reduced expression si1 · · · sim for wλ , and, for each J ⊆ {1, . . . , m}, we define (3) wJ := the element of W obtained by deleting sij , j ∈ J , from the product si1 · · · sim ; −1 and λ (4) cJ := j ∈J cj , where cj := (aij , λ (j ) ) (j ) := sij −1 · · · si1 · λ. + Let P ⊂ P be the cone of dominant weights; and, for λ ∈ P + , let Vλ be the irreducible representation of g with highest weight λ. Theorem 1.1. For λ in P + and µ in P , the multiplicity mλ (µ) of µ in Vλ is given by mλ (µ) := (|W0 · λ|/|W0 · µ|) J cJ , where the summation is over all J such that wJ−1 · λ is in W0 · µ. (We prove in Corollary 6.2 that the cJ ’s are positive.) For µ in P , let eµ denote the function x → e(µ,x) on V0 . Then W acts on the eµ ’s si ·µ , and Theorem 1.1 is equivalent to by virtue of its action on P , that is, si eµ = e the following formula for the character χλ := µ mλ (µ)eµ of Vλ . Theorem 1.2. We have χλ = (|W0 · λ|/|W0 |) w∈W0 w(sim + cm ) · · · (si1 + c1 )eλ . We obtain Theorem 1.2 as a consequence of a more general result, namely, an analogous formula for the generalized Jacobi polynomial Pλ of Heckman and Opdam. For the definition and properties of Pλ , we refer the reader to [HSc] and [O]. We recall here that Pλ depends on certain parameters kα , α ∈ R0 , such that kw·α = kα for all w ∈ W0 . For special values of kα , Pλ can be interpreted as a spherical function on a compact symmetric space. In particular, in the limit as all kα → 1, we have Pλ → χλ . Definition. In the context of the previous definition, for λ in P , we redefine λ := λ + (1/2) α∈R + kα ε(α ∨ ,λ) α; (1 ) 0 −1 (4 ) cj = kij (aij , λ (j ) ) , where k0 = kβ and ki = kαi for i ≥ 1. Theorem 1.3. For λ in P + and for cj as above, the Heckman-Opdam polynomial Pλ is given by the same formula as in Theorem 1.2. For λ in P + , define cλ := (|W0 |/|W0 ·λ|) j (aij , λ (j ) ), and let ᏼ := Z+ [kα ] be the set of polynomials in the parameters kα with nonnegative integral coefficients. Then we prove the following theorem. Theorem 1.4. We have that cλ is in ᏼ, as are all coefficients of cλ Pλ . Theorem 1.4 is a generalization of the main result of [KS] to arbitrary root systems. Our proof depends on three fundamental ideas in the “new” theory of special functions. The first idea, due to Macdonald, Heckman, Opdam, and others, is that one can treat root multiplicities on a symmetric space as parameters. The second idea, due to Dunkl and Cherednik, is that radial parts of invariant

79

A NEW FORMULA FOR WEIGHT MULTIPLICITIES

differential operators on symmetric spaces can be written as polynomials in certain commuting first-order differential-reflection operators, namely, the Cherednik operators. The third idea is the method of intertwiners for Cherednik operators. This was developed in [KS], [K], [S1], and [C2], and it can be regarded as the double affine analog of Lusztig’s fundamental relation [L2] in the affine Hecke algebra. Using the intertwiners of [C2] and [S2], our results can be extended to the context of Macdonald polynomials and to the 6-parameter Koornwinder polynomials. These intertwiners correspond to the affine Weyl group (rather than the dual affine Weyl group) and hence are not appropriate for the present context. We shall discuss them elsewhere in [S3]. 2. Preliminaries. The results of this section are due to Cherednik [C1], Heckman, and Opdam [O]. Let F = R(kα ) be the field of rational functions in the parameters kα , and let ᏾ be the F-span of {eλ | λ ∈ P } regarded as a W -module. Definition. For y ∈ V0 , the Cherednik operator Dy is defined by Dy = ∂y +

(y, α)kα

α∈R0+

1 (1 − sα ) − (y, ρ), 1 − e−α

where ρ :=

1 kα α. 2 + α∈R0

Here are some basic facts about Cherednik operators from [O, Section 2]. Proposition 2.1. We have the following. (1) The operators Dy act on ᏾ and commute pairwise. (2) For i = 1, . . . , n, we have si Dy − Dsi y si = −ki (y, αi ). (3) There is a basis {Eλ | λ ∈ P } of ᏾, characterized uniquely as follows: (a) the coefficient of eλ in Eλ is 1; (b) Dy Eλ = (y, λ)Eλ , where λ is as in Definition (1’) of the introduction. (4) For λ in P + , the Heckman-Opdam polynomial Pλ equals (|W0 · λ|/|W0 |) w∈W0 wEλ . λ. (5) For i = 1, . . . , n, if si · λ = λ, then s i · λ = si · 3. The affine reflection. In this section, we prove some basic properties of the affine reflection s0 . Lemma 3.1. If α is a positive root different from β, then (α ∨ , β) equals zero or 1. Proof. Since β is in P + , (α ∨ , β) is a nonnegative integer. Also, since β is a short root, we have (α, α) ≥ (β, β). So, by the Cauchy-Schwartz inequality, we get

(α, β) (α, β) α∨, β = 2 ≤2 ≤ 2. (α, α) (α, α)1/2 (β, β)1/2

80

SIDDHARTHA SAHI

If α = β, then α is not proportional to β and the last inequality is strict. For i = 0, 1, 2, define R0i = {α ∈ R0+ | (α ∨ , β) = i}, and, for α in R0+ , put

if α ∈ R00 , sβ · α α = −sβ · α if α ∈ R01 ∪ R02 . Lemma 3.2. The involution α → α acts trivially on R00 and R02 , and permutes R01 .

Proof. For α in R01 , we have (α ∨ , β) = (α ∨ , −sβ · β) = (α ∨ , β) = 1, which implies that α is a (positive) root in R01 . The assertions about R00 and R02 = {β} are obvious. Lemma 3.3. For λ in P , if s0 · λ = λ, then s λ. 0 · λ = s0 · λ = β + sβ λ using Lemma 3.2 and kα = kα . This gives Proof. We compute s0 · s0 · λ = β + sβ · λ +

1 1 kα ε(α ∨ ,λ) α − 2 2 0 α∈R0

kα ε(α ∨ ,λ) α.

α∈R01 ∪R02

Comparing this to the expression for µ with µ = s0 · λ, it suffices to show that

if α ∈ R00 , ε(α ∨ ,λ) ε(α ∨ ,µ) = −ε(α ∨ ,λ) if α ∈ R01 ∪ R02 . For α in R00 , we easily compute that (α ∨ , µ) = (α ∨ , λ). For α in R01 , we get (α ∨ , µ) = (α ∨ , β + sβ · λ) = 1 − (α ∨ , λ). Being an integer, ∨ (α , λ) is either less than or equal to zero or greater than or equal to 1. In either case, we get ε(α ∨ ,µ) = −ε(α ∨ ,λ) . Finally, for α in R02 , we have α = α = β and (β ∨ , µ) = 2 − (β ∨ , λ). Now s0 λ = λ implies that (β ∨ , λ) = 1; thus we have either (β ∨ , λ) ≥ 2 or (β ∨ , λ) ≤ 0. In either case, we get ε(β ∨ ,λ) = ε(β ∨ ,λ) = −ε(β ∨ ,µ) . 4. The intertwining relation. Dualizing the action y → w · y of W on V0 , we get a representation v → wv of W on V satisfying (wv, y) = (v, w−1 · y). For y in V0 and w in W0 , we have wy = w · y. The affine reflection s0 acts on V by s0 (rδ + y) = (y, β)δ + rδ + sβ y. For v = rδ + y in V , we define the affine Cherednik operator simply by putting Dv = Dy +rI , where I is the identity operator. From Proposition 2.1(2), we know the intertwining relations between the (affine) Cherednik operators and s1 , . . . , sn . In this section, we prove the following intertwining relation between these operators and s0 . Proposition 4.1. For v = rδ + y in V , we have Dv s0 − s0 Ds0 v = kβ (y, β).

81


Proof. Let us write Nα for 1/(1 − e−α )(1 − sα ), so that Dv = ∂y + kα (y, α)Nα − (y, ρ) + r. Since sβ Nα = Nsβ ·α sβ and sβ ∂y = ∂sβ y sβ , we get sβ Dv sβ = ∂sβ y + kα (y, α)Nsβ ·α − (y, ρ) + r. α∈R0+

Now, partitioning R0+ = R00 ∪ R01 ∪ R02 and using Lemma 3.2, we get sβ Dv sβ = ∂sβ y + kα sβ y, α Nα − kα sβ y, α N−α − (y, ρ) + r. α∈R00

α∈R01 ∪R02

The following identities are easy to check: (1) eβ ∂sβ y e−β = ∂sβ y + (y, β); (2) eβ Nα e−β = Nα for α ∈ R00 ; (3) eβ N−α e−β = 1 − Nα for α ∈ R01 ; (4) eβ N−β e−β = 1 − Nβ + s0 . Using these, we get the following formula for s0 Dv s0 = eβ (sβ Dv sβ )e−β : kα sβ y, α Nα − kα sβ y, α − kβ sβ y, β s0 − (y, ρ) + r. ∂sβ y + (y, β) + Since

α∈R0+

α∈R01 ∪R02

α∈R01 ∪R02 kα (sβ y, α) = (sβ y, ρ − sβ · ρ) = (sβ y, ρ) − (y, ρ),

we get

s0 Dv s0 = Dsβ y + (y, β) − kβ (sβ y, β)s0 + r = Ds0 v + kβ (y, β)s0 . The result follows. 5. The Heckman-Opdam polynomials. Let Eλ be as in Proposition 2.1. Proposition 5.1. The polynomials Eλ satisfy the following recursions: (1) Eλ = eλ for λ ∈ ᏻ; (2) if si · λ = λ, then (si + (ki /(ai , λ)))Eλ is a multiple of Esi ·λ . Proof. For (1), we check simply that Dy eλ = (y, λ)eλ , using the identity

eλ if α ∨ , λ = 1, λ Nα e = 0 if α ∨ , λ = 0. For (2), we write F for (si + (ki /(ai , λ)))Eλ and first consider i = 0. Then, for y in V0 , using Proposition 2.1(2), we get y, λ ki Dy F = si Dsi y − ki (y, αi ) + Dy Eλ = si y, λ si + ki − ki (y, αi ) Eλ . ai , λ λ ai ,

82

SIDDHARTHA SAHI

Since (y, λ) − (y, αi )(ai , λ) = (si y, λ), using Proposition 2.1(5), we get λ F = y, si · λ F = y, s Dy F = si y, i · λ F. This proves (2) for i = 0. For i = 0, we use Proposition 4.1 to get y, λ kβ Dy F = s0 Ds0 y + kβ (y, β) + Dy Eλ = s0 y, λ s0 + kβ + kβ (y, β) Eλ . λ λ a0 , a0 , This time, using (y, λ) + (a0 , λ)(y, β) = (s0 y, λ) and Lemma 3.3, we get Dy F = s0 y, λ F = y, s0 · λ F = y, s 0 · λ F. This completes the proof of (2) for i = 0. Corollary 5.2. For λ in P , and ci as in Definition (4 ) of the introduction, we have Eλ = sim + cm · · · si1 + c1 eλ . Proof. By the minimality of wλ , if w is a proper subexpression of wλ−1 = sim · · · si1 , then w · λ = λ. This means that the coefficient of eλ in (sim + cm ) · · · (si1 + c1 )eλ is 1. The result now follows from Proposition 5.1. Proof of Theorem 1.3. This follows from Corollary 5.2 and Proposition 2.1(4). 6. Positivity. Let ᏼ1 ⊂ ᏼ be the set of polynomials of degree less than or equal to 1, with nonnegative integral coefficients and a positive constant term. For λ in P , let aij and λ (j ) be as in Definition (4 ) of the introduction. Proposition 6.1. For each j = 1, . . . , m, (aij , λ (j ) ) belongs to ᏼ1 . Proof. Fix j and write µ = λ(j ) , i = ij , and w = si1 · · · sij −1 . We need to show that (ai , µ) has a positive constant term and nonnegative integral coefficients. The lengths of w and wsi must be j − 1 and j , respectively, since otherwise we could shorten the expression si1 · · · sim for wλ . By a standard argument (see [Hu, Chapter 5]), this implies that w(ai ) is a positive (affine) coroot in R + . Since λ = µ is minuscule, we conclude that 0 ≤ w(ai ), µ = ai , w −1 · µ = (ai , µ). If (ai , µ) were zero, then λ(j +1) = si · µ = µ = λ(j ) and we could shorten the expression for wλ by dropping sij . This shows that (ai , µ), which is the constant term of (ai , µ), is positive. If i = 0, the nonconstant part of (a0 , µ) is 1 − kα ε(α ∨ ,µ) β ∨ , α , 2 + α∈R0


83

and we consider separately the contributions of R00 , R01 , and R02 . For α in R00 , the contribution is zero. For α = β in R02 , we get the term −ε(β ∨ ,µ) kβ . By the first part, (a0 , µ) is a positive integer. Hence (β ∨ , µ) = 1 − (a0 , µ) ≤ 0, which implies that −ε(β ∨ ,µ) = 1. The roots in R01 can be grouped in pairs {α, −sβ · α}, and the contribution of such a pair is −kα

ε(α ∨ ,µ) + ε(−sβ α ∨ ,µ) 2

β ∨, α .

Now (β ∨ , α) is positive, so the coefficient above is a nonnegative integer, unless (α ∨ , µ) and (−sβ α ∨ , µ) are both greater than zero. But in this case, we would get 0 < α ∨ , µ − sβ α ∨ , µ = α ∨ , µ − sβ · µ = β ∨ , µ α ∨ , β ≤ 0, which is a contradiction. µ) is The argument is similar if i > 0. The nonconstant part of (ai , 1 kα ε(α ∨ ,µ) (ai , α). 2 + α∈R0

To compute this, we divide R0+ into three disjoint sets consisting of {αi }, {the roots orthogonal to αi }, and {the remaining positive roots}. For α = αi , we get the coefficient ε(ai ,µ) , which is 1 since (ai , µ) > 0 by the first part. If α is orthogonal to αi , then the coefficient is zero. Finally, the remaining positive roots can be grouped into pairs {α, si · α}, where we may assume that (α ∨ , αi ) > 0. The contribution of each such pair is kα

ε(α ∨ ,µ) − ε(si α ∨ ,µ) (ai , α). 2

Now (α ∨ , αi ) > 0 implies (ai , α) > 0. Therefore, this coefficient is a nonnegative integer, unless (α ∨ , µ) ≤ 0 and (si α ∨ , µ) > 0. But if this were the case, then we would have 0 > α ∨ , µ − si α ∨ , µ = α ∨ , µ − si · µ = (ai , µ) α ∨ , αi > 0, which is a contradiction. Proof of Theorem 1.4. This follows from Theorem 1.3 and Proposition 6.1. Setting all the kα ’s equal to 1 in Proposition 6.1, we deduce the following corollary. Corollary 6.2. The constants cj and cJ in Theorems 1.1 and 1.2 are positive.

84

SIDDHARTHA SAHI

References [C1] [C2] [F] [HSc] [Hu] [K] [KS] [Ko]

[Ks] [Li] [L1]

[L2] [M] [O] [S1] [S2] [S3]

I. Cherednik, Double affine Hecke algebras and Macdonald’s conjectures, Ann. of Math. (2) 141 (1997), 191–216. , Intertwining operators of double affine Hecke algebras, Selecta Math. (N.S.) 3 (1997), 459–495. H. Freudenthal, Zur Berechnung der Charaktere der halbeinfachen Lieschen Gruppen, I, Indag. Math. 16 (1954), 369–376. G. Heckman and H. Schlichtkrull, Harmonic Analysis and Special Functions on Symmetric Spaces, Perspect. Math. 16, Academic Press, San Diego, 1994. J. Humphreys, Reflection Groups and Coxeter Groups, Cambridge Stud. Adv. Math. 29, Cambridge Univ. Press, Cambridge, 1990. F. Knop, Integrality of two variable Kostka functions, J. Reine Angew. Math. 482 (1997), 177–189. F. Knop and S. Sahi, A recursion and a combinatorial formula for Jack polynomials, Invent. Math. 128 (1997), 9–22. T. Koornwinder, “Askey-Wilson polynomials for root systems of type BC” in Hypergeometric Functions on Domains of Positivity, Jack Polynomials, and Applications (Tampa, Fla., 1991), Contemp. Math. 138, Amer. Math. Soc., Providence, 1992, 189–204. B. Kostant, A formula for the multiplicity of a weight, Trans. Amer. Math. Soc. 93 (1959), 53–73. P. Littelmann, Paths and root operators in representation theory, Ann. of Math. (2) 142 (1995), 499–525. G. Lusztig, “Singularities, character formulas, and a q-analog of weight multiplicities” in Analysis and Topology on Singular Spaces, II, III (Luminy, 1981), Astérisque 101– 102, Soc. Math. France, Montrouge, 1983, 208–299. , Affine Hecke algebras and their graded version, J. Amer. Math. Soc. 2 (1989), 599–635. I. Macdonald, Affine Hecke algebras and orthogonal polynomials, Astérisque 237 (1996), 189–207, Séminaire Bourbaki 1994/95, exp. no. 797. E. Opdam, Harmonic analysis for certain representations of graded Hecke algebras, Acta Math. 175 (1995), 75–121. S. Sahi, Interpolation, integrality, and a generalization of Macdonald’s polynomials, Internat. Math. Res. Notices 1996, 457–471. , Nonsymmetric Koornwinder polynomials and duality, to appear in Ann. of Math. (2). , Some properties of Koornwinder polynomials, to appear in Contemp. Math.

Department of Mathematics, Rutgers University, New Brunswick, New Jersey 08903, USA; [email protected]

Vol. 101, No. 1


© 2000

SEMIAMPLE HYPERSURFACES IN TORIC VARIETIES ANVAR R. MAVLYUTOV

0. Introduction. While the geometry and cohomology of ample hypersurfaces in toric varieties have been studied (see [BC]), not much attention has been paid to semiample (i.e., “big” and “nef”) hypersurfaces defined by sections of line bundles generated by global sections with a positive self-intersection number. It turns out that mirror symmetric hypersurfaces in the Batyrev mirror construction [B2] are semiample, but often not ample. In this paper, we study semiample hypersurfaces. Such hypersurfaces bring a geometric construction that generalizes the way of construction in [B2]. The purpose of this paper is to present some approaches to studying the cohomology ring of semiample hypersurfaces in complete simplicial toric varieties. In particular, we explicitly describe the ring structure on the middle cohomology of regular semiample hypersurfaces, when the dimension of the ambient space is 4. Let us explain the main ideas of computing the topological cup product. The first step is to naturally relate the middle cohomology of the hypersurfaces to some graded ring; in our situation this is done using a Gysin spectral sequence. The origin of this idea is in [CarG] and [BC]. The second step is to use the multiplicative structure on the graded ring in order to compute the topological cup product on the middle cohomology. We remark that the cup product was computed on the middle cohomology of smooth hypersurfaces in a projective space (see [CarG]), and this paper generalizes some of the results in [CarG]. The following is a brief summary of the paper. In Section 1, we establish notation and then introduce a geometric construction associated with semiample divisors in complete toric varieties. At the end, we give a criterion for a divisor to be ample (generated by global sections) in terms of intersection numbers. This was known for simplicial toric varieties (the toric Nakai criterion), and we prove it for arbitrary complete toric varieties. Section 2 studies regular semiample hypersurfaces and describes a nice stratification of such hypersurfaces. These hypersurfaces generalize those in the Batyrev construction (see [B2]). Section 3 generalizes the results of [CarG] on an algebraic cup product formula for residues of rational differential forms (from here on, the toric variety is usually simplicial). It shows that there is a natural map from a graded ring (the Jacobian ring Received 15 December 1998. 1991 Mathematics Subject Classification. Primary 14M25. Author’s work partially supported by I. Mirkovic from National Science Foundation grant number DMS-9622863. 85

86

ANVAR R. MAVLYUTOV

R(f ) (see [BC])) to the middle cohomology of a quasismooth hypersurface such that the multiplicative structure on the ring is compatible with the topological cup product. In Section 4, we partially describe the middle cohomology of a regular semiample hypersurface X; in particular, we show that some graded pieces of the ring R1 (f ), considered in [BC], are imbedded into the middle cohomology of X. We explicitly compute the cup product on the part coming from the ring. We should point out that, when X is ample, the graded pieces of R1 (f ) fill up the middle cohomology of the hypersurface, but not so in the semiample case. Section 5 computes the middle cohomology and the cup product on it for regular semiample hypersurfaces in a 4-dimensional toric variety. This is the most interesting case for physics. We describe the whole middle cohomology in algebraic terms, even though R1 (f ) might fill up only part of the middle cohomology. In fact, the complement to the R1 (f ) part is a direct sum of the middle cohomologies of regular ample hypersurfaces in 2-dimensional toric varieties. Hence, this part can also be described in terms of rings similar to R1 (f ). In Section 6, we compute the Hodge numbers hp, 2 of a regular semiample hypersurface, and then apply the obtained formulas to the hypersurfaces in the Batyrev mirror construction (see [B2]) to verify that, in general, the duality predicted by physicists does not occur for the Hodge numbers of such hypersurfaces. Basic references on the theory of toric varieties are [F1], [Od], [D], and [C3]. Acknowledgments. I would like to thank David Cox for his advice and useful comments. I am grateful to David Cox and David Morrison for allowing me to use their unpublished notes for Theorems 3.3 and 3.5. I also thank the referee for pointing out that our notion of “semiample” is a little bit different from the common one (see Remark 1.1). 1. Semiample divisors. In this section, we first establish notation, review some basic facts from the toric geometry, and then discuss a geometric construction associated with semiample divisors in complete toric varieties. At the end of this section, we prove a generalization of the toric Nakai criterion for arbitrary complete toric varieties. As a consequence, we obtain a criterion for semiample divisors in terms of intersection numbers. In notation, we follow [BC] and [C3]. Let M be a lattice of rank d, and let N = Hom(M, Z) be the dual lattice; MR (resp., NR ) denotes the R-scalar extension of M (resp., of N). The symbol P stands for a complete toric variety associated with a finite complete fan in NR . Denote by (k) the set of all k-dimensional cones in ; in particular, (1) = {ρ1 , . . . , ρn } is the set of 1-dimensional cones in with the minimal integral generators e1 , . . . , en , respectively. Each 1-dimensional cone ρi corresponds to a torus-invariant divisor Di in P . A torus-invariant Weil divisor D = ni=1 ai Di determines a convex polyhedron When D =

n

D = {m ∈ MR : m, ei ≥ −ai for all i} ⊂ MR .

i=1 ai Di

is Cartier, there is a support function ψD : NR → R that is

SEMIAMPLE HYPERSURFACES IN TORIC VARIETIES

87

linear on each cone σ ∈ and determined by some mσ ∈ M: ψD (ei ) = mσ , ei = −ai

for all ei ∈ σ.

Since P is complete, a general fact is that a Cartier divisor D (i.e., the corresponding line bundle ᏻP (D)) is generated by global sections (resp., ample) if and only if ψD is convex (resp., strictly convex). A Cartier divisor D on P is called semiample if D is generated by global sections and the intersection number (D d ) > 0. In complete toric varieties, all ample divisors are semiample. From [F1, Section 5.3], it follows that (D d ) = d! vold (D ), where vold is the d-dimensional volume normalized with respect to the lattice M. So, the semiample torus-invariant divisors in complete toric varieties can be characterized by two conditions: The support function ψD is convex and the polyhedron D has maximal dimension d. Remark 1.1. We should mention here that our notion of “semiample” is a little bit different from the common one. In [EV], it is not assumed that semiample sheaves ᏸ have the additional property ᏸd > 0 (the Iitaka dimension is maximal). We believe that the results in this section can be easily generalized for all Cartier divisors generated by global sections. However, for the purpose of studying mirror symmetric hypersurfaces (see [BC]), we simply assume that semiample sheaves have the additional property. The same definition was used in the recent book [CKa]. Let us show how to construct a semiample (but not ample) divisor from an ample one. Consider a proper birational morphism π : P 1 → P 2 between two complete toric varieties corresponding to a subdivision 1 of a fan 2 with an ample torusinvariant divisor Y on P 2 . Then the pullback π ∗ (Y ) is a torus-invariant Cartier divisor with the same support function as the one for Y . Hence, π ∗ (Y ) is semiample, and it is not ample if 1 is different from 2 . We now show that all semiample divisors arise uniquely this way, constructing a complete fan D for a semiample Cartier divisor D = ni=1 ai Di using our fan and the convex support function ψD . The value of the support function ψD on each d-dimensional cone σ ∈ is determined by a unique mσ ∈ M. We glue together those maximal dimensional cones in that have the same mσ . The glued set is again a convex rational polyhedral cone, and one can show that this cone is strongly convex using the fact that D has maximal dimension d. The set of these strongly convex cones with its faces comprise a new complete fan D in NR . This construction is independent of the equivalence relation on the divisors: If we change the divisor D to a linearly equivalent one, the fan D will remain the same. The fan D is exactly the normal fan of D . Indeed, by construction, ψD is strictly convex with respect to D . On the other hand, since D is generated by global sections, the support function ψD coincides with the function of D (see [F1, Section 3.4]): ψD (n) = min m, n. m∈D

88

ANVAR R. MAVLYUTOV

Theorem 2.22 in [Od] implies that D is the normal fan of D . Notice that is a refinement of D . So, the sets of 1-dimensional cones of the fans are related by D (1) ⊂ (1), and we have a proper birational morphism π : P → P D between the two toric varieties. Any proper morphism determines a pushforward map π∗ : Ad−1 (P ) → Ad−1 (P D ) on the Chow group, which takes the class of an irreducible divisor V to the class deg(V /π(V ))[π(V )] if π(V ) has the same dimension as V and to zero otherwise. Now apply the pushforward map to our semiample divisor:   n π∗ [D] = ai π∗ [Di ] =  ai π(Di ), i=1

ρi ∈ D (1)

because Di maps birationally onto its image when ρi ∈ D , and dim π(Di ) < dim Di in all other cases. The divisors π(Di ) for ρi ∈ D (1) are torus-invariant corresponding to the 1-dimensional cones in D . The support function of the Weil divisor π∗ (D) := ρi ∈ D (1) ai π(Di ) coincides with ψD , which is strictly convex with respect to the fan D . Hence, the divisor class π∗ [D] is ample. For the birational map π : P → P D , we also have a commutative diagram (see [F1, Section 3.4] and [F2]): Ad−1 (P ) O Pic(P ) o

π∗

π∗

/ Ad−1 P D O Pic P D ,

where the vertical maps are inclusions. Since the support functions for the Cartier divisors D and π∗ (D) coincide, the pullback π ∗ π∗ [D] is exactly the divisor class [D]. Thus, we have the following useful result. Proposition 1.2. Let P be a complete toric variety with a semiample divisor class [D] ∈ Ad−1 (P ). There exists a unique complete toric variety P D with a toric birational map π : P → P D , such that is a subdivision of D , π∗ [D] is ample, n ∗ and π π∗ [D] = [D]. Moreover, if D = i=1 ai Di is torus-invariant, then D is the normal fan of D . Remark 1.3. Since the fan D is the normal fan of D , there is a one-to-one correspondence between the k-dimensional cones of D and (d − k)-dimensional faces of D . Note, however, that while D is canonical with respect to the equivalence relation on the divisors, the polyhedron D is only canonical up to translation. We next study the intersection theory for the semiample divisors in complete toric varieties. Any toric variety P is a disjoint union of its orbits by the action of the torus T = N ⊗ C∗ that sits naturally inside P . Each orbit Tσ is a torus corresponding to a cone σ ∈ . The closure of each orbit Tσ is again a toric variety denoted by V (σ ).


89

Lemma 1.4. If D is a semiample divisor on a complete toric variety P , then the intersection number (D k · V (σ )) > 0 for any σ ∈ (d − k) contained in a cone of D (d − k) and (D k · V (σ )) = 0 for all other σ ∈ (d − k). Proof. We can assume that D = ni=1 ai Di , which gives a support function ψD determined on each cone by some mσ : ψD (n) = mσ , n for all n ∈ σ . Since D is generated by global sections, for a fixed σ ∈ (d −k) we have (see [F1, Section 5.3]):

⊥

volk D ∩ σ + mσ

=

Dk · V (σ ) . k!

(1)

By Remark 1.3, there is a one-to-one correspondence between the cones of D and the faces of D . Let σ be the minimal cone in D , corresponding to a face of D and containing σ . We claim that = D ∩ σ ⊥ + mσ . Indeed, since ψD is strictly convex with respect to D , from [Od, Lemma 2.12] we have = m ∈ D : m, n = ψD (n) for all n ∈ σ , whence m ∈ implies m − mσ , n = 0 for all n ∈ σ . Conversely, suppose m ∈ D and (m − mσ ) ∈ σ ⊥ . The first condition implies m, n ≥ ψD (n) for all n from the strongly convex cone σ , while the second one gives a point in the interior of σ (by the minimal choice of this cone) for which m and ψD have the same values. Hence, m and ψD have the same values on σ , and the claimed equality of the sets follows. Now, the lemma follows from the fact that volk ( ) > 0 if and only if dim σ = d −k. Remark 1.5. Lemma 1.4 provides another way of constructing the fan D , by gluing the d-dimensional cones in along those facets τ for which (D · V (τ )) = 0. We now give necessary and sufficient conditions for a Cartier divisor on a complete toric variety to be ample, generated by global sections or semiample. This is a generalization of the toric Nakai criterion proved for nonsingular toric varieties in [Od, Theorem 2.18]. Theorem 1.6. Let P be a d-dimensional complete toric variety, and let D be a Cartier divisor on P . Then (i) D is generated by global sections if and only if (D · V (τ )) ≥ 0 for any τ ∈ (d − 1); (ii) D is ample if and only if (D · V (τ )) > 0 for any τ ∈ (d − 1). Proof. Without loss of generality, we can assume that D is torus-invariant. (i) If D is generated by global sections, then the required condition follows from

90

ANVAR R. MAVLYUTOV

equation (1). Conversely, the torus-invariant divisor D has the support function ψD , and it suffices to show that ψD is convex. Here, we use a trick. Consider a nonsingular subdivision of the fan and the corresponding toric morphism f : P → P . Then the support function of the pullback divisor f ∗ (D) coincides with ψD . So, we just need to show that f ∗ (D) is generated by global sections. By [F2, Example 2.4.3], we have (f ∗ (D) · V (τ )) = (D · f∗ (V (τ ))), where V (τ ) is the closure of the 1-dimensional orbit corresponding to τ ∈ (d − 1). If the smallest cone in containing τ is d-dimensional, then the image of V (τ ) is a point, implying that the above intersection number vanishes. Otherwise, τ is contained in some τ ∈ (d −1), in which case f∗ (V (τ )) = V (τ ). So, in either case, by the given condition in (i), the intersection number (D ·f∗ (V (τ ))) is nonnegative. Following the proof of [Od, Theorem 2.18], we get that (f ∗ (D) · V (τ )) ≥ 0 for any τ ∈ (d − 1) implies f ∗ (D) is generated by global sections. (ii) If D is ample, then the required condition follows from Lemma 1.4 or, more generally, from the Nakai criterion for arbitrary complete varieties (see [H, Chapter I, Theorem 5.1] and [K]). Conversely, by part (i), the divisor D is generated by global sections. We show that D is semiample and the fan is exactly the fan D associated with the semiample divisor. Then, by Proposition 1.2, the desired result follows. From equation (1) and the given condition, it follows that the polyhedron D intersects different lines, corresponding to τ ∈ (d − 1), in more than one point. These lines cannot lie in a hyperplane of MR , because is complete. Therefore, D is maximal dimensional, implying that D is semiample. By Remark 1.5 and the given condition, the fan coincides with D . Thus, Proposition 1.2 implies that D is ample. Corollary 1.7. Let P be a complete toric variety. Then a Cartier divisor D on P is semiample if and only if (D d ) > 0 and (D · V (τ )) ≥ 0 for any τ ∈ (d − 1). Remark 1.8. In Mori’s theory, Theorem 1.6(ii) and [R, Proposition 1.6] imply that D is ample if and only if (D ·(NE(P )\{0})) > 0, where NE(P ) is the cone coming from effective 1-cycles. Also, by Theorem 1.6(i), the pseudoample cone PA(P ) is spanned by the divisors generated by global sections. For details see [R] and [Od, Section 2.5]. 2. Regular semiample hypersurfaces. Next we apply results from the previous section to describe a stratification of regular semiample hypersurfaces in a complete toric variety P . The following definition has appeared in [B2]. Definition 2.1. A hypersurface X ⊂ P is called -regular if X ∩ Tσ is empty or a smooth subvariety of codimension 1 in Tσ for any σ ∈ . Remark 2.2. Proposition 6.8 in [D] says that a hypersurface X ⊂ P defined by a general section of a line bundle generated by global sections is -regular. When it is clear from the context, we simply say that a hypersurface is regular.


91

Lemma 2.3. Let X be a semiample hypersurface in a complete toric variety P , such that dim P ≥ 2. Then (i) X is connected, and (ii) X is irreducible if X is -regular. Proof. (i) Consider an effective torus-invariant divisor D equivalent to the divisor X. Since ᏻP (D) is generated by global sections, choosing a basis of the space H 0 (P , ᏻP (D)) gives a mapping ϕD : P −→ Pr−1 , where r = h0 (P , ᏻP (D)) = Card(D ∩M). By [F1, Section 3.4, exercise, page 73], the image of ϕD has dimension equal to dim D . Since D is semiample, we get that dim D = dim P ≥ 2. From [FLa, Theorem 2.1], it follows that every divisor in the linear system |D| is connected. In particular, X is connected. (ii) To prove that X is irreducible, we argue as follows. Consider a nonsingular subdivision of the fan and the corresponding morphism p : P → P . It follows from [B2, Proposition 3.2.1] that p −1 (X) is a -regular hypersurface that supports a semiample divisor p ∗ (X). By the previous part, p−1 (X) is a smooth connected hypersurface that must be irreducible. Therefore, X is irreducible. Proposition 2.4. If X ⊂ P is a -regular semiample hypersurface with the associated morphism π : P → P X for the divisor class [X] ∈ Ad−1 (P ) from Proposition 1.2, then Y := π(X) is a X -regular ample hypersurface, and X = π −1 (Y ). Proof. From Lemma 2.3(ii), we know that X is irreducible. Since X is -regular, it maps birationally onto its image, implying π∗ [X] = [π(X)]. Therefore, by Proposition 1.2, the hypersurface Y = π(X) is ample. Let us now show that Y misses the zero-dimensional orbits in P X . Consider the 1-dimensional orbit closure V (τ0 ) ⊂ P X corresponding to a cone τ0 ∈ X (d − 1), and take a cone τ ∈ (d − 1) that lies in τ0 . Since X is -regular, Card X ∩ Tτ = Card X ∩ V (τ ) = X · V (τ ) . We also know that the orbit Tτ maps onto Tτ0 ; hence, Y · V (τ0 ) ≥ Card Y ∩ V (τ0 ) ≥ Card Y ∩ Tτ0 ≥ Card X ∩ Tτ = X · V (τ ) . By [F2, Example 2.4.3], we have (Y ·V (τ0 )) = (X ·V (τ )), whence the above inequalities are equalities. Therefore, the hypersurface Y intersects the orbit Tτ0 transversally and does not intersect the points in the complement V (τ0 )\Tτ0 , corresponding to the d-dimensional cones in X that contain τ0 . Thus, we have shown that Y misses all zero-dimensional orbits in P X . One can easily show X = π −1 (Y ) from the facts that X and Y = π(X) are irreducible and that Y misses the zero-dimensional orbits. Finally, for arbitrary σ0 ∈ X

92

ANVAR R. MAVLYUTOV

we take σ ∈ , contained in σ0 , of the same dimension. Then we have an isomorphism Tσ ∼ = Tσ0 inducing another isomorphism X ∩ Tσ = π −1 (Y ) ∩ Tσ ∼ = Y ∩ Tσ0 . So, the -regularity of X implies that π(X) is X -regular. Remark 2.5. By construction in [B2], the MPCP-desingularizations (maximal pro of regular projective hypersurfaces Z in jective crepant partial desingularizations) Z a toric Fano variety P are regular semiample hypersurfaces. Proposition 2.4 shows that if we start with an arbitrary regular semiample hypersurface, then we come up with a similar picture. Let us note that a regular ample hypersurface Y in a complete toric variety P intersects all orbits transversally, except for zero-dimensional orbits. Such a hypersurface is called nondegenerate in [BC] and [DKh]. Also, in this case, a hypersurface in the torus T isomorphic to the affine hypersurface Y ∩ T in T is called nondegenerate. Such a hypersurface satisfies the following property. Lemma 2.6 [DKh]. Let Z be a nondegenerate affine hypersurface in the torus T; then the natural map H i (T) → H i (Z), induced by the inclusion, is an isomorphism for i < dim(T) − 1 and an injection for i = dim(T) − 1. Like in [B2], by Proposition 2.4, we get a nice stratification of a semiample regular hypersurface X ⊂ P in terms of nondegenerate affine hypersurfaces. Let Y = π(X); then X = π −1 (Y ). Using the standard description of a toric blow-up, we obtain (2) X ∩ Tσ ∼ = Y ∩ Tσ0 × (C∗ )dim σ0 −dim σ , where σ0 ∈ X is the smallest cone containing σ ∈ . 3. A cup product formula for quasismooth hypersurfaces. The purpose of this section is to give a generalization of the algebraic cup product formula for the residues of rational forms presented in [CarG]. In this section, we assume that P is a complete simplicial toric variety. Such a toric variety has a homogeneous coordinate ring S = C[x1 , . . . , xn ], with variables x1 , . . . , xn corresponding to the irreducible torusinvariant divisors (see [C1]). This ring is graded by the Chow group nD1 , .a.i . , Dn Ad−1 (P) : deg( i=1 xi ) = [ ni=1 ai Di ]. Furthermore, if ᏸ is a line bundle on P, then for β = [ᏸ] ∈ Ad−1 (P) one has an isomorphism H 0 (P, ᏸ) ∼ = Sβ . So, the homogeneous polynomials in Sβ identified with the global sections of ᏸ determine hypersurfaces in the toric variety P. Definition 3.1 [BC]. A hypersurface X ⊂ P defined by a homogeneous polynomial f ∈ Sβ is called quasismooth if ∂f/∂xi , 1 ≤ i ≤ n, do not vanish simultaneously on P. Definition 3.2 [BC]. Fix an integer basis m1 , . . . , md for the lattice M. Then, given subset I = {i1 , . . . , id } ⊂ {1, . . . , n}, denote det(eI ) = det(mj , eik 1≤j, k≤d ),


dxI = dxi1 ∧ · · · ∧ dxid , and xÎ =

i ∈I / xi .

,=

93

Define the n-form , by the formula

det(eI )xÎ dxI ,

|I |=d

where the sum is over all d element subsets I ⊂ {1, . . . , n}. Let X ⊂ P be a quasismooth hypersurface defined by f ∈ Sβ . For A ∈ S(a+1)β−β0 (here, β0 := ni=1 deg(xi )), consider a rational d-form ωA :=

A, 0 d ∈ H P, , (a + 1)X . P f a+1

This form gives a class in H d (P \ X), and by the residue map Res : H d (P \ X) −→ H d−1 (X) we get Res(ωA ) ∈ H d−1 (X). We need an explicit algebraic formula for the Hodge ˇ cohomology. component Res(ωA )d−1−a, a in Cech Denote fi = ∂f/∂xi , and let Ui = {x ∈ P : fi (x) = 0} for i = 1, . . . , n. If X is a quasismooth hypersurface, then ᐁ = {Ui }ni=1 is an open cover of P. The next two theorems with their proofs are corrected and generalized versions of unpublished results of D. Cox and D. Morrison. Theorem 3.3. Let X ⊂ P be a quasismooth hypersurface defined by f ∈ Sβ and A ∈ S(a+1)β−β0 , β0 = ni=1 deg(xi ). Then, under the natural map d−1−a d−1−a ∼ d−1−a, a Hˇ a ᐁ|X , ,X (X), −→ H a X, ,X =H ˇ cocycle ca {AKia · · · Ki0 ,/ the component Res(ωA )d−1−a, a corresponds to the Cech d−1+(a(a+1)/2) , and Ki is the contraction opfi0 · · · fia }i0 ...ia , where ca = (1/a!)(−1) erator (∂/∂xi ). Proof. The residue map can be calculated in hypercohomology using the commutative diagram Res / H d−1 (X) H d (P \ X) O O Hd ,•P (log X)

Res

/ Hd−1 ,• , X

ˇ where the vertical maps are isomorphisms. As in [CarG], we can work in the Cech–de Rham complex C • (ᐁ, ,• (∗X)) with arbitrary algebraic singularities along X, where ᐁ = {Ui }ni=1 . Then we can apply the arguments of [CarG, pages 58–62] almost without any change. We only need to check that df ∧ , ≡ 0

modulo multiples of f

(3)

94

ANVAR R. MAVLYUTOV

for part (i) of the lemma on page 60 in [CarG]. But df ∧ (dx1 ∧ · · · ∧ dxn ) = 0, and, by [C2, Lemma 6.2], , = θ1 · · ·θn−d (dx1 ∧ · · · ∧ dxn ) for some Euler vector fields θi . The equivalence (3) can be obtained by repeatedly applying the following argument. If df ∧ ω ≡ 0 (mod f ) for some form ω, and θ is a Euler vector field, then 0 ≡ θ (df ∧ ω) = (θ df )ω − df ∧ (θ ω). Since θ df = θ(f ) = θ(β)f (see [C2, proof of Proposition 5.3]), we get df ∧ (θ ω) ≡ 0. Thus, the lemma on page 60 of [CarG] is true in our situation. The rest of the arguments apply without change, and the theorem is proved. Let us remark that the constructive proof of [CarG] implies ˇ cocycle. that ca {(AKia · · · Ki0 ,)/(fi0 · · · fia )}i0 ...ia is actually a Cech n Definition 3.4. For β = [ i=1 bi Di ] ∈ Ad−1 (P) and a multi-index I = (i0 , . . . , id ) β viewed as an ordered subset of {1, . . . , n}, we introduce a constant cI that is the determinant of the (d + 1) × (d + 1) matrix obtained from (mj , eik 1≤j ≤d, ik ∈I ) by adding the first row (bi0 , . . . , bid ), where m1 , . . . , md is the fixed integer basis of the β lattice M as in Definition 3.2. One can easily check that cI is well defined. Theorem 3.5. Let X ⊂ P be a quasismooth hypersurface defined by f ∈ Sβ , and suppose a + b = d − 1, ωA = A,/f a+1 , and ωB = B,/f b+1 for A ∈ S(a+1)β−β0 and B ∈ S(b+1)β−β0 . Then, under the composition ∪ d−1 δ H a X, ,bX ⊗ H b X, ,aX −−→ H d−1 X, ,X −→ H d P, ,dP (here, δ is the coboundary map in the Poincaré residue sequence), we have that ˇ δ(Res(ωA )ba ∪ Res(ωB )ab ) is represented by the Cech cocycle β ABcI xÎ , cab ∈ Hˇ d ᐁ, ,dP , fi0 · · · fid I

where I = (i0 , . . . , id ) and cab = (−1)(a(a+1)/2)+(b(b+1)/2)+a

2 +d−1

/a! b!.

Proof. As in [CarG, page 63], by Theorem 3.3 we see that the residue product Res(ωA )ba ∪ Res(ωB )ab is represented by the cocycle AKia · · · Ki0 , BKid−1 · · · Kia , d−1 ψ = cãb ∧ ∈ C d−1 ᐁ|X , ,X , fi0 · · · fia fia · · · fid−1 i0 ...id−1 where cãb = (−1)a ca cb = (−1)(a(a+1)/2)+(b(b+1)/2)+a /a!b!. To calculate the coboundary of this cocycle, we use the following commutative diagram: 2

2

0

/ C d ᐁ , ,d P O

/ C d ᐁ, ,d (log X) PO

0

/ C d−1 ᐁ, ,d P

/ C d−1 ᐁ, ,d (log X) P

Res

Res

/ C d ᐁ|X , ,d−1 X O / C d−1 ᐁ|X , ,d−1 . X


95

Lift the cocycle ψ to AKia · · · Ki0 , BKid−1 · · · Kia , df ˜ ψ = cãb ∧ ∧ ∈ C d−1 ᐁ, ,dP (log X) . fi0 · · · fia fia · · · fid−1 f i0 ...id−1 From the diagram, we can see that changing of the numerator by a multiple of f will not affect the image of ψ in Hˇ d (ᐁ, ,dP ). Hence, we need to compute Kia · · · Ki0 , ∧ Kid−1 · · · Kia , ∧ df modulo multiples of f . First we show Kia · · · Ki0 , ∧ Kid−1 · · · Kia , ∧ df ≡ (some function) · , (mod f ).

(4)

As in the proof of Theorem 3.3, we can write , = E dx, where E is a wedge of some Euler vector fields and dx = dx1 ∧ · · · ∧ dxn . Denote du = dxi0 ∧ · · · ∧ dxia−1 , dv = dxia+1 ∧ · · · ∧ dxid−1 , and dw = ∧i ∈I / 0 dxi , where I0 = (i0 , . . . , id−1 ). Then dx = ± du ∧ dxa ∧ dv ∧ dw. Now compute: Kia · · · Ki0 , = Kia · · · Ki0 (E dx) = ±E (Kia · · · Ki0 du ∧ dxa ∧ dv ∧ dw) = ±E (dv ∧ dw) = ± (E dv) ∧ dw + (−1)(d−a−1)(n−d) dv(E dw) . Similarly, Kid−1 · · · Kia , = ± (E du) ∧ dw + (−1)a(n−d) du(E dw) . Since dw ∧ dw = 0, we get Kia · · · Ki0 , ∧ Kid−1 · · · Kia , = ±(E dw) (E dv) ∧ dw ∧ (−1)a(n−d) du + (−1)(d−a−1)(n−d) dv ∧ (E du) ∧ dw + (−1)(d−1)(n−d) dv ∧ du ∧ (E dw) = ±(E dw) E (dv ∧ du ∧ dw) = ±(E dw) E Kia dx = ±(E dw) Kia (E dx) = ±(E dw) Kia , . From equation (3), we know that ,∧df ≡ 0 modulo multiples of f . Applying the contraction operator Kia to this identity, we obtain (Kia ,) ∧ df ≡ ±fia ,, whence equation (4) follows: Kia · · · Ki0 , ∧ Kid−1 · · · Kia , ∧ df = ±(E dw) Kia , ∧ df ≡ ±(E dw)fia ,. We next claim that Kia · · · Ki0 , ∧ Kid−1 · · · Kia , ∧ df ≡ (−1)d−1 det eI0 xÎ0 fia , (mod f ).

(5)

96

ANVAR R. MAVLYUTOV

Examine the coefficient of dxI0 = dxi0 ∧ · · · ∧ dxid−1 in the left-hand side. The only place dxI0 can occur is in Kia · · ·Ki0 det eI0 xÎ0 dxI0 ∧ Kid−1 · · · Kia det eI0 xÎ0 dxI0 ∧ fia dxia 2 = det eI0 xÎ20 dxia+1 ∧ · · · ∧ dxid−1 ∧ (−1)a(d−a) dxi0 ∧ · · · ∧ dxia−1 ∧ fia dxia = (−1)d−1 det eI0 xÎ0 fia det eI0 xÎ0 dxI0 . From here, equation (5) follows, because , = |I |=d det(eI )xÎ dxI , and the left-hand side of (5) is (some function) · , modulo multiples of f . Returning to the calculation of the coboundary δ(ψ), by equation (5) we have (−1)d−1 cãb AB det eI0 xÎ0 , ψ˜ ≡ , f fi0 · · · fid−1 i0 ...id−1

so that d det eI \{ik } xik fik xÎ , (−1)d−1 cãb k δ(ψ) ≡ (−1) AB , f fi0 · · · fid k=0

I

d

where I = (i0 , . . . , id ). But the identity k=0 (−1)k det(eI \{ik } )eik = 0 holds and gives β an Euler formula cI f = dk=0 (−1)k det(eI \{ik } )xik fik (see [BC]). Thus, β ABcI xÎ , δ(ψ) ≡ cab , fi0 · · · fid I

where cab = (−1)d−1 cãb . As in the classic case (see [PS]), we go further to relate the multiplicative structure on some quotient of the homogeneous ring S to the cup product on the middle cohomology of the quasismooth hypersurface X given by a homogeneous polynomial f ∈ Sβ . Definition 3.6 [BC]. For f ∈ Sβ , the Jacobian ideal J (f ) ⊂ S is the ideal generated by the partial derivatives ∂f/∂x1 , . . . , ∂f/∂xn . Also, the Jacobian ring R(f ) is the quotient ring S/J (f ) graded by the Chow group Ad−1 (P). To show a relation between the cup product and multiplication in R(f ), we need two lemmas. We have the natural map S(a+1)β−β0 → H d−1−a, a (X) that sends A to the corresponding component of Res(ωA ). The map Res( ω_ )d−1−a, a : R(f )(a+1)β−β0 −→ H d−1−a, a (X), induced by the above one is well defined because of the following statement.

97


Lemma 3.7. If A ∈ J (f )(a+1)β−β0 , then Res(ωA )d−1−a, a = 0. Proof. In the case a = 0, the statement is trivial because J (f )β−β0 = 0. Assume that a > 0. By Theorem 3.3, since A ∈ J (f ), it suffices to show that {(fj Kia · · · Ki0 ,)/ d−1−a ˇ (fi0 · · · fia )}i0 ...ia in C a (ᐁ|X , ,X ) is a Cech coboundary for one of the partial derivatives fj = ∂f/∂xj . We have Kj Kia · · · Ki0 (df ∧ ,) = (−1)a+2 df ∧ Kj Kia · · · Ki0 , + (−1)a+1 fj Kia · · · Ki0 , +

a (−1)k fik Kj Kia · · · K ik · · · Ki0 ,. k=0

But df ∧ , ≡ 0 (mod f ) by equation (3), and df = 0 on the hypersurface X. Therefore, on X we have the identity a fj Kia · · · Ki0 , = (−1)a+k fik Kj Kia · · · K ik · · · Ki0 ,. k=0

Hence, {(fj Kia · · · Ki0 ,)/(fi0 · · ·fia )}i0 ...ia is the image of (−1)a {(Kj Kja−1 · · · Kj0 ,)/ d−1−a ˇ (fj0 · · · fja−1 )}j0 ...ja−1 under the Cech coboundary map C a−1 (ᐁ|X , ,X ) → d−1−a a C (ᐁ|X , ,X ). Consider the map S(d+1)β−2β0 → H d, d (P) that sends a polynomial h to the class in H d, d (P) represented by the cocycle β hcI xÎ , ∈ Hˇ d ᐁ, ,dP fi0 · · · fid I

as in Theorem 3.5. This induces the map λ : R(f )(d+1)β−2β0 → H d,d (P), well defined by the following statement. β

ˇ Lemma 3.8. If h ∈ J (f ), then {hcI xÎ ,/(fi0 · · · fid )}I is a Cech coboundary. Proof. We can assume that h is one of the partial derivatives fj = ∂f/∂xj . Let I be the ordered subset {i0 , . . . , id } ⊂ {1, . . . , n}. Then the equality β

cI ej +

d k=0

β

(−1)k+1 c{j }∪I \{ik } eik = 0

ik , . . . , id }) holds and gives the Euler (here, {j } ∪ I \ {ik } is the ordered set {j, i0 , . . . , formula (see [BC]) d d β β β k+1 β cI bj + (−1) c{j }∪I \{ik } bik f = cI xj fj + (−1)k+1 c{j }∪I \{ik } xik fik , k=0

n

k=0

β

where the numbers bi are determined by β = [ i=1 bi Di ]. But the number cI bj + d k+1 cβ k=0 (−1) {j }∪I \{ik } bik is the determinant of a matrix with the same two rows

98

ANVAR R. MAVLYUTOV

(bj , bi0 , . . . , bid ), so it vanishes. Using the above Euler formula, we see that under the ˇ Cech coboundary map C d−1 (ᐁ, ,dP ) → C d (ᐁ, ,dP ), the cocycle

β

fj cI xÎ , fi0 · · · fid

d

=

k=0 (−1)

k cβ {j }∪I \{ik } fik xˆ {j }∪(I \{ik }) ,

fi0 · · · fid

I

β is the image of {c{j }∪J xˆ{j }∪J ,/(fj0 · · · fjd−1 )}J , where J is the ordered set {j, j0 , . . . , jd−1 }.

I

= {j0 , . . . , jd−1 } and {j }∪J

As a consequence of Theorem 3.5 and Lemmas 3.7 and 3.8, we have proved the following theorem. Theorem 3.9. Let X ⊂ P be a quasismooth hypersurface defined by f ∈ Sβ , and suppose a + b = d − 1. Then the diagram R(f )(a+1)β−β0 × R(f )(b+1)β−β0

cab ·multiplication

/ R(f )(d+1)β−2β0

Res( ω_ )ba ×Res( ω_ )ab

H b, a (X) × H a, b (X)

λ

∪

/ H d−1, d−1 (X)

δ

/ H d, d (P)

commutes, where λ is as defined above and δ is the Gysin map. 4. Cohomology of regular hypersurfaces. In this section, we present an application of the Gysin spectral sequence for computing the cohomology of regular semiample hypersurfaces in a complete simplicial toric variety P. We obtain an explicit description of the cup product on some part of the middle cohomology of such hypersurfaces. Section 3 studied the relation between multiplication in R(f ) and the cup product, whereas this section studies such a relation of a smaller ring R1 (f ) and the cup product. The rings R(f ) and R1 (f ) were previously used in [BC] for studying the cohomology of ample hypersurfaces. Let D = P \ T = ni=1 Di , and let X ⊂ P be a regular hypersurface. Then (X, X ∩ D) is a toroidal pair (see [D, Section 15]), and also X ∩ D consists of quasismooth components that intersect quasitransversally. Therefore, by the results from [D, Section 15] we have • ∼ Gr W ,•−k k ,X log(X ∩ D) = X∩V (σ ) , dim σ =k

and the (Gysin) spectral sequence of this filtered complex (see [De, Section 3.2]) pq • E1 = Hp+q X, Gr W −p ,X log(X ∩ D) ∼ H 2p+q X ∩ V (σ ) ⇒ H p+q X \ (X ∩ D) = dim σ =−p

99


degenerates at E2 and converges to the weight filtration W• on H p+q (X \ (X ∩ D)): pq p+q E2 = Gr W X \ (X ∩ D) . q H In particular, note that 1 H 1 (X) ∼ = Gr W 1 H (X ∩ T).

(6)

Now assume that X ⊂ P is a regular semiample hypersurface. In this case X \ (X ∩ D) = X ∩ T is a nondegenerate affine hypersurface in T. Hence, by Lemma pq p+q (X ∩ T) vanishes unless p + q = d − 1 and q ≥ d − 1, or 2.6, E2 = Gr W q H p + q < d − 1 and q = −2p. Therefore, from the Gysin spectral sequence we obtain the following exact sequences. First, for s odd, s < d − 1, we have 0 −→ H 1 X ∩ V (σ ) dim σ =(s−1)/2

−→ · · · −→

H s−2k X ∩ V (σ ) −→ · · · −→ H s (X) −→ 0,

dim σ =k

where the maps are alternating sums of the Gysin morphisms. Next, for s even, s < d − 1, we get s/2 0 −→ Gr W (X ∩ T) −→ H 0 X ∩ V (σ ) −→ · · · −→ H s (X) −→ 0. s H dim σ =s/2

Finally, for s = d − 1, · · · −→ H d−1−2k X ∩ V (σ ) dim σ =k d−1 (X ∩ T) −→ 0. (7) −→ · · · −→ H d−1 (X) −→ Gr W d−1 H

Similar sequences exist for s > d −1 that are exact except for one term. We are mainly concerned with the last exact sequence, which determines the middle cohomology group of X. The following fact, contained in [C2, Proposition 5.3], characterizes regular hypersurfaces. Lemma 4.1 [C2]. Let X ⊂ P be a hypersurface defined by a homogeneous polynomial f . Then X is regular if and only if xi (∂f /∂xi ), i = 1, . . . , n, do not vanish simultaneously on P. In this case, we call f nondegenerate. This lemma shows that, in complete simplicial toric varieties, regular hypersurfaces are quasismooth. In Theorem 4.5, we prove a stronger analog of Theorem 3.9 for regular semiample hypersurfaces. i = {x ∈ P : xi fi (x) = 0} cover the In the case f is nondegenerate, the open sets U i }n is a refinement of ᐁ defined toric variety P. In particular, the open cover ᐁ = {U i=1 in Section 3.

100

ANVAR R. MAVLYUTOV

Definition 4.2 [BC]. Given f ∈ Sβ , we get the ideal quotient x1 ∂f xn ∂f ,..., : x 1 · · · xn J1 (f ) = ∂x1 ∂xn (see [CLO, page 193]) and the ring R1 (f ) = S/J1 (f ) graded by the Chow group Ad−1 (P). To show the relation between multiplication in R1 (f ) and the cup product on the hypersurface defined by f , we need some results similar to those in Section 3. Lemma 4.3. Let f ∈ Sβ be nondegenerate, and let h ∈ J1 (f ); then the cocycle β , ,d ). {hcI xÎ ,/(fi0 · · · fid )}I vanishes in Hˇ d (ᐁ P Proof. The proof of this is similar to the proof of Lemma 3.8. Theorem 4.4. Let X ⊂ P be a regular semiample hypersurface defined by f ∈ Sβ , and suppose a + b = d − 1. (i) If A ∈ J1 (f )(a+1)β−β0 , then Res(ωA )b, a = 0. (ii) The map Res( ω_ )b, a : R1 (f )(a+1)β−β0 → H b, a (X) is injective, and the natural composition R1 (f )(a+1)β−β0 −→ H b, a (X) −→ H b, a H d−1 (X ∩ T) is an isomorphism, so that we have a natural imbedding Gr F Wd−1 H d−1 (X ∩ T) @→ Gr F H d−1 (X). Moreover, we have an isomorphism n b, a b−1, a−1 ∼ H (X) = R1 (f )(a+1)β−β ϕi ! H (X ∩ Di ) , 0

i=1

where ϕi ! are the Gysin maps for ϕi : X ∩ Di @→ X, and Res(ωA )b, a ∪ ϕi ! H a−1, b−1 (X ∩ Di ) = 0

for all A ∈ R1 (f )(a+1)β−β0 .

Proof. (i) We prove the statement using the Poincaré duality H b, a (X) ⊗ H a, b (X) −→ H d−1, d−1 (X), where b = d − 1 − a. Since the pairing is nondegenerate, it suffices to show for A ∈ J1 (f )(a+1)β−β0 that the cup product of Res(ωA )b, a with all elements in H a, b (X) vanishes. For this we need to find the elements that span the group H a, b (X). Let X be linearly equivalent to a torus-invariant divisor ni=1 ai Di with ai ≥ 0, and let be the corresponding polytope defined by the inequalities m, ei ≥ −ai . As in [B1], S denotes the subring of C[t0 , t1±1 , . . . , td±1 ] spanned over C by all monomials of the form t0k t m = t0k t1m1 · · · t1md , where k ≥ 0 and m ∈ k. We have a

101


natural isomorphism of graded rings (see [BC, proof of Theorem 11.5]) ∞

S ∼ =

Skβ ⊂ S,

k=0

sending t0k t m to

n

kai +m, ei . i=1 xi

Skβ−β0

This isomorphism induces the bijection

n

i=1 xi

(1) / x1 · · · xn kβ ∼ = I , k

(1)

where I ⊂ S is the ideal spanned by all monomials t0k t m such that m is in the interior of k. As a consequence of the exact sequence (7) and [B1, Theorems 7.13 and 8.2], we have the diagram n i=1

H a−1, b−1 (X ∩ Di )

⊕ϕi !

/ H a, b (X) O

/ H a, b H d−1 (X ∩ T) O

Res( ω_ )ab

Res(ω˜ _ )ab

S(b+1)β−β0

/ I (1)

/0

(8) b+1

,

where the top row is exact and the right vertical map is defined by

dt1 dtd tm ∧···∧ ω˜ t b+1 t m = 0 t1 td f˜(t)b+1 (here, f˜(t) is the Laurent polynomial defining the affine hypersurface X ∩ T, so that t0 f˜(t) corresponds to f (x) under the isomorphism (S )1 ∼ = Sβ ) and by Resab induced by the Poincaré residue mapping [B1, Section 5]: Res : H d T \ (X ∩ T) −→ H d−1 (X ∩ T). The diagram commutes because the restriction of the form ωB = B,/f b+1 , with (b+1)ai −1+m, ei , to the torus T coincides with (t m /f˜(t)b+1 )(dt1 /t1 ) ∧ B = ni=1 xi m , e · · · ∧ (dtd /td ). (Use the coordinates tj = ni=1 xi j i on the torus with the fixed integer basis m1 , . . . , md from Definition 3.2.) The first row in (8) is exact and the composition S(b+1)β−β0

/ I (1)

Res( ω˜ _ )ab b+1

/ H a, b H d−1 (X ∩ T)

a, b is surjective by [B1, Theorem 8.2]. Therefore, a−1, b−1the group H (X) is spanned by ab Res(ωB ) for B ∈ S(b+1)β−β0 and ϕi ! H (X ∩ Di ) for i = 1, . . . , n.

102

ANVAR R. MAVLYUTOV

From Theorem 3.5 and Lemma 4.3, it follows that Res(ωA )ba ∪ Res(ωB )ab = 0 for A ∈ J1 (f )(a+1)β−β0 and all B ∈ S(b+1)β−β0 . Also, for any A ∈ S(a+1)β−β0 and h ∈ H a−1, b−1 (X ∩ Di ) we have (9) Res(ωA )ba ∪ ϕi ! h = ϕi ! ϕi∗ Res(ωA )ba ∪ h by the projection formula for Gysin homomorphisms. However, ϕi∗ Res(ωA )ba is repˇ resented by the restriction of the Cech cocycle ca {(AKia · · · Ki0 ,)/(fi0 · · · fia )}i0 ...ia ∈ b a |X , , ) from Theorem 3.3 to X ∩ Di . This restriction vanishes, because if Hˇ (ᐁ X i0 ∩ · · · ∩ U ia ∩ Di is empty, and if i ∈ i ∈ {i0 , . . . , id }, then U / {i0 , . . . , id }, then each term in the form Kia · · · Ki0 , contains xi or dxi . Thus, we have shown that the cup product of Res(ωA )ba for A ∈ J1 (f )(a+1)β−β0 , with all elements in H a, b (X), vanishes, and the result follows. (ii) From the diagram (8) and part (i), we get a natural map R1 (f )(a+1)β−β0 → H d−1−a, a (H d−1 (X ∩ T)). The fact that this map is an isomorphism follows from [BC, proof of Theorem 11.8]. Using the diagram (8), we can now see that the map Res( ω_ )d−1−a, a : R1 (f )(a+1)β−β0 → H d−1−a, a (X) is injective, and we get the desired description of the middle cohomology group H d−1 (X). By equation (9), Res(ωA )d−1−a, a ∪ ϕi ! H a−1, d−2−a (X ∩ Di ) = 0. Combining Theorem 3.9 with Lemma 4.3 and Theorem 4.4(i), we get the following result. Theorem 4.5. Let X ⊂ P be a regular semiample hypersurface defined by f ∈ Sβ , and suppose a + b = d − 1. Then the diagram R1 (f )(a+1)β−β0 × R1 (f )(b+1)β−β0

cab ·multiplication

/ R1 (f )(d+1)β−2β0

Res( ω_ )ba ×Res( ω_ )ab

H b, a (X) × H a, b (X)

λ

∪

/ H d−1, d−1 (X)

commutes, where cab = (−1)(a(a+1)/2)+(b(b+1)/2)+a

2 +d−1

δ

/ H d, d (P)

/a!b!.

We finish this section with an explicit procedure of computing Res(ωA )ba ∪ Res(ωB )ab . X

To have this, we need generalizations of some results in [C2]. Definition 4.6 [C2]. Assume F0 , . . . , Fd ∈ Sβ do not vanish simultaneously on a complete toric variety P. Then the toric residue map ResF :

Sρ −→ C, F0 , . . . , Fd ρ


103

ρ = (d + 1)β − β0 , is given by the formula ResF (H ) = Tr P ([ϕF (H )]), where Tr P : H d (P, ,dP ) → C is the trace map, and [ϕF (H )] is the class represented by the d-form ˇ H ,/F0 · · · Fd in Cech cohomology with respect to the open cover {x ∈ P : Fi (x) = 0}. Proposition 4.7. If F0 , . . . , Fd ∈ Sβ , then there is JF ∈ S(d+1)β−β0 such that d

j ∧ · · · ∧ dFd = JF ,. (−1)j Fj dF0 ∧ · · · ∧ dF

j =0 β

Furthermore, if I = {i0 , . . . , id } ⊂ {1, . . . , n} such that cI = 0 (if β = 0, there is at β least one such I ), then JF = det(∂Fj /∂xik )/cI xÎ . The polynomial JF is called the toric Jacobian of F0 , . . . , Fd . Proof. This is essentially Proposition 4.1 in [C2]. To show that JF coincides with the toric Jacobian in [C2], use the Euler formula β

cI g =

d ∂g (−1)k det eI \{ik } xik ∂xik

for g ∈ Sβ

k=0

from the proof of Theorem 3.5. Theorem 4.8. Let P be a complete toric variety, and let β ∈ Ad−1 (P) be semiample. If F0 , . . . , Fd ∈ Sβ do not vanish simultaneously on P, then: (i) The toric residue map ResF : Sρ /F0 , . . . , Fd ρ → C, ρ = (d + 1)β − β0 , is an isomorphism. (ii) If JF ∈ S(d+1)β−β0 is the toric Jacobian of F0 , . . . , Fd , then ResF (JF ) = d! vol() = deg(F ), where is the polyhedron associated to a torus-invariant divisor in the equivalence class of β and F : P → Pd is the map defined by F (x) = (F0 (x), . . . , Fd (x)). Proof. This statement was proved for ample β in [C2, Theorem 5.1], but the proof can be applied in our case almost without change. Indeed, consider the map F = (F0 , . . . , Fd ) : P → Pd given by the sections of a semiample line bundle ᏻP (D). Since (D d ) > 0 and F0 , . . . , Fd do not vanish simultaneously on P, it follows that F0 , . . . , Fd are linearly independent. We can extend F0 , . . . , Fd to a basis of H 0 (P, ᏻP (D)), which gives the associated map φ : P → PN , where N = h0 (P, ᏻP (D)) − 1. Then the map F factors through the map φ and a projection p : PN \ L −→ Pd ,

(y0 , . . . , yN ) ' −→ (y0 , . . . , yd ),

where L ⊂ PN is a projective subspace defined by y0 = · · · = yd = 0. By [F1, Section 3.4, exercise, page 73], the dimension of the image of φ is d. Using a dimension

104

ANVAR R. MAVLYUTOV

argument, one can show that p −1 (y0 , . . . , yd ) ∩ im(φ) is nonempty. Hence, F is surjective and, consequently, generically finite. Propositions 3.1, 3.2, and 3.3 in [C2] are still valid in the case when β is semiample, because the isomorphism S ∼ = S∗β holds. The rest of the arguments in [C2] apply without change. Definition 4.9 [BC]. Given f ∈ Sβ , let J0 (f ) ⊂ S denote the ideal generated by xi ∂f/∂xi , 1 ≤ i ≤ n, and put R0 (f ) = S/J0 (f ). β

Lemma 4.10. If I = {i0 , . . . , id } ⊂ {1, . . . , n} such that cI = 0, then xi ∂f/∂xi , i ∈ I , do not vanish simultaneously on P, and J0 (f ) = xi0 ∂f/∂xi0 , . . . , xid ∂f/∂xid . β

β

Proof. If cI = 0, then ei0 , . . . , eid span MR . From the Euler formula cI f = d k k=0 (−1) det(eI \{ik } )xik (∂f /∂xik ) and [C2, Proposition 5.3] the lemma follows. We now return to the calculation of X Res(ωA )ba ∪Res(ωB )ab , when X is a regular semiample hypersurface. Let Fj = xj (∂f /∂xj ), and let I = {i0 , . . . , id } ⊂ {1, . . . , n} β β be such that cI = 0. Then denote J = det(∂Fj /∂xi )i, j ∈I /(cI )2 xÎ . One can show that J does not depend on the choice of I . By Lemma 4.10, the polynomials Fi , i ∈ I , do not vanish simultaneously on P and determine the toric residue map ResFI . From the definitions of λ, ResFI , and [C2, Proposition A.1], we obtain a commutative diagram R1 (f )(d+1)β−2β0 λ

H d, d (P)

n

i=1 xi

/ R0 (f )(d+1)β−β0 β

√

−1/2π −1

c ResFI

d I

P

(10)

/ C,

where the arrow on the top is just the multiplication. Using Theorem 4.8, we get the following procedure. For given A ∈ R1 (f )(a+1)β−β0 and B ∈ R1 (f )(b+1)β−β0 , there is a unique constant c such that

x1 ∂f xn ∂f A · Bx1 · · · xn − cJ ∈ ,..., . ∂x1 ∂xn Then

X

where D =

√ d Res(ωA )ba ∪ Res(ωB )ab = c − 2π −1 cab d! vol(D ),

n

i=1 ai Di

such that [D] = β.

5. Cup product on regular semiample 3-folds. In this section, we completely describe the middle cohomology and the cup product on it for a regular semiample hypersurface X ⊂ P , when dim P = 4. It follows from (7) that the map

105


⊕ni=1 H 1 (X ∩ Di ) H

b, a

⊕ϕi !

/ H 3 (X) is injective. Hence, by Theorem 4.4,

(X) ∼ = R1 (f )(a+1)β−β0

n

ϕi ! H

b−1, a−1

X ∩ Di

,

(11)

i=1

where a + b = 3. We first determine which of the groups H b−1, a−1 (X ∩ Di ) vanish. Lemma 5.1. Let X ⊂ P , dim P = 4, be a -regular semiample hypersurface, and let π : P → P X be the morphism associated with X. Then: (i) H 1 (X ∩ Di ) = 0 unless ρi ⊂ σ for some 2-dimensional cone σ ∈ X , and ρi ∈ / X (1) (so ρi \ {0} lies in the relative interior of σ ). (ii) For ρi ⊂ σ , such that σ ∈ X (2) and ρi ∈ / X (1), we have πi∗ : H 1 Y ∩ V (σ ) ∼ = H 1 X ∩ Di , where V (σ ) = π(Di ) is the orbit closure corresponding to σ ∈ X , Y := π(X), and πi : X ∩ Di → Y ∩ V (σ ) is the map induced by π. Proof. (i) Applying (6) to the regular hypersurface X ∩ Di in the toric variety Di , we have 1 H 1 X ∩ Di ∼ (12) = Gr W 1 H X ∩ Tρi . If ρi ∈ X (1), then X ∩ Tρi is a nondegenerate affine hypersurface in Tρi because of (2). Hence, 1 ∼ W 1 Gr W 1 H X ∩ Tρi = Gr 1 H Tρi = 0. If ρi does not lie in a cone σ ∈ X (2), then X ∩Tρi is empty or a disjoint finite union 1 of (C∗ )2 , by equation (2). In this case, Gr W 1 H (X ∩ Tρi ) also vanishes, and part (i) follows. (ii) Suppose ρi ∈ / X (1) is contained in a cone σ ∈ X (2), and let σ ∈ (2) be the cone such that ρi ⊂ σ ⊂ σ . Then we get a composition H 1 Y ∩ V (σ )

πi∗

/ H 1 X ∩ Di

ϕi,∗ σ

/ H1 X∩V σ ,

where ϕi,σ : X ∩ V (σ ) @→ X ∩ Di is the inclusion. To prove part (ii), it suffices to show that this composition is an isomorphism and all spaces in the composition are of the same dimension. Applying (6) to the regular hypersurfaces X ∩V (σ ) in V (σ ) and Y ∩ V (σ ) in V (σ ), we get a commutative diagram 1 ∼ H 1 Y ∩ V (σ ) = Gr W 1 H (Y ∩ Tσ ) H1 X∩V σ

∼ =

1 X∩T , H Gr W σ 1

106

ANVAR R. MAVLYUTOV

where the vertical arrow on the right is induced by the isomorphism Tσ ∼ = Tσ . From the diagram, we see that the natural map H 1 (Y ∩ V (σ )) → H 1 (X ∩ V (σ )) is an isomorphism. On the other hand, since X ∩ Tρi ∼ = (Y ∩ Tσ ) × C∗ because of (2), it follows from (12) that 1 1 H 1 X ∩ Di ∼ Y ∩ Tσ × C∗ ∼ = Gr W = Gr W 1 H 1 H Y ∩ Tσ by the Künneth isomorphism. Thus, the dimensions of spaces H 1 (X∩Di ) and H 1 (Y ∩ V (σ )) coincide. This finishes the proof of part (ii). Lemma 5.1 relates the nonvanishing groups H 1 (X ∩ Di ) to the middle cohomologies of regular ample hypersurfaces in 2-dimensional toric varieties. Using (11) and Theorem 4.4, we can now give a complete algebraic description of the middle cohomology group H 3 (X). Let S(V (σ )) = C[yγ : σ ⊂ γ ∈ X (3)] be the coordinate ring of the 2-dimensional complete toric variety V (σ ) ⊂ P X , and let fσ ∈ S(V (σ ))β σ denote the polynomial defining the hypersurface Y ∩V (σ ) in V (σ ). Then, as in Definition 4.2, we have the ideal J1 (fσ ) in S(V (σ )) and the quotient ring R1 (fσ ) = S(V (σ ))/J1 (fσ ). By Theorem 4.4(ii), we have an isomorphism H 2−a, a−1 Y ∩ V (σ ) ∼ = R1 fσ aβ σ −β σ , where β0σ = deg( section.

γ

0

yγ ) ∈ A1 (V (σ )). We can now state our first main result of this

Theorem 5.2. Let X ⊂ P , dim P = 4, be a regular semiample hypersurface defined by f ∈ Sβ . Then there is a natural isomorphism n(σ ) H 3−a, a (X) ∼ , R 1 fσ σ σ = R1 (f )(a+1)β−β 0

σ ∈ X (2)

aβ −β0

where n(σ ) is the number of cones ρi such that ρi ⊂ σ and ρi ∈ / X (1). Remark 5.3. As we mentioned in the introduction and Remark 2.5, in the Batyrev

of an ample Calabi-Yau mirror construction (see [B2]), an MPCP-desingularization Z hypersurface of a toric Fano variety P , corresponding to a reflexive polytope , is a regular semiample hypersurface. In [B2, Corollary 4.5.1], Batyrev calculated the Hodge number

= l() − 5 − h2, 1 (Z) l ∗ (θ) + l ∗ (θ)l ∗ (θ ∗ ), codimθ=1

θ∗

codimθ=2

where θ is a face of , is the corresponding dual face of the dual reflexive ∗ polyhedron , and l( ) (resp., l ∗ ( )) denotes the number of integer (resp., interior integer) points in . We can compare this number with the algebraic description

in Theorem 5.2. From Theorem 4.4, we know that dim R1 (f )2β−β0 = of H 2, 1 (Z)


107

∩ T), which is equal to l() − 5 − codimθ=1 l ∗ (θ) by [B2, Theorem 4.3.1]. h2, 1 (Z The number l ∗ (θ ∗ ) is equal to n(σ ) of Theorem 5.2 for the cone σ , corresponding to the face θ of . And finally, one can verify that dim R1 (fσ )β σ −β0σ corresponds to

is related to l ∗ (θ ). We can now see how the formula for the Hodge number h2, 1 (Z) our algebraic description. The next thing we want to do is to compute the cup product on H 3 (X) in terms of the algebraic description in Theorem 5.2. To compute this cup product, we need one topological result. Lemma 5.4. Let K and L be subvarieties of a compact V-manifold M, which intersect quasitransversally, and suppose that K, L, and K ∩ L are compact Vmanifolds. Then the diagram H • (K) i∗

H • (K ∩ L)

i!

/ H • (M) j∗

α·j !

/ H • (L)

commutes, where i, j, i , and j are inclusions, and the constant α satisfies [K]∪[L] = α[K ∩ L] for fundamental cohomology classes of K, L, and K ∩ L in M. Proof. The arguments are the same as in [Do, VIII, proof of Proposition 10.9]. The constant α in the above diagram is caused by the difference between [K] ∪ [L] and [K ∩ L]. (In the smooth case we do not see this difference.) Example 5.5. A simple nontrivial example of Lemma 5.4 occurs when M is a 2-dimensional toric variety and K and L are irreducible torus-invariant divisors, intersecting in a point. In this case, we have to compare the composition of maps i!

i∗

j∗

j !

H 0 (K) −→ H 2 (M) −→ H 2 (L) with H 0 (K) −→ H 0 (K ∩ L) −→ H 2 (L). Since i∗

j!

∼ = H 0 (K) and H 2 (L) ∼ = H 4 (M) are isomorphisms, it suffices to compare ∗ ∗ j! j i! i = [K] ∪ [L] ∪ _ with j! j ! i ∗ i ∗ = [K ∩ L] ∪ _ on H 0 (M). The difference between [K] ∪ [L] and [K ∩ L] can be easily determined by means of the ring isomorphism A• (M)⊗C ∼ = H 2• (M) (see [D, Section 10]), which sends a cycle class of a subvariety V to its fundamental cohomology class [V ] in M. H 0 (M)

Equation (11) provides a description of the middle cohomology group H 3 (X). We first show where the cup product on H 3 (X) vanishes. Lemma 5.6. Let ϕi ! H 1 (X ∩ Di ) ∪ ϕj ! H 1 (X ∩ Dj ) = 0, i = j , unless ρi and ρj span a cone σ ∈ contained in a 2-dimensional cone of X . Proof. By the projection formula for Gysin homomorphisms, we know that ϕi ! _ ∪ ϕj ! _ = ϕj ! ϕj∗ ϕi ! _ ∪ _ .

108

ANVAR R. MAVLYUTOV

By Lemma 5.4, for i = j we have a commutative diagram H 1 (X ∩ Di )

ϕi !

∗ ϕij

H 1 (X ∩ Di ∩ Dj )

/ H 3 (X) ϕj∗

αij ϕj i !

/ H 3 (X ∩ Dj ),

(13)

where ϕij : X ∩ Di ∩ Dj @→ X ∩ Di is the inclusion map and αij is an appropriate constant. Hence, it suffices to show that H 1 (X ∩ Di ∩ Dj ) = 0. This is so, if ρi and ρj do not span a cone in , because Di ∩Dj is an empty set in this case. If ρi and ρj span a cone σ ∈ , then Di ∩ Dj = V (σ ). Applying (6) to the regular hypersurface X ∩ V (σ ) in V (σ ), we see 1 H1 X∩V σ ∼ = Gr W 1 H X ∩ Tσ . On the other hand, if σ is not contained in a 2-dimensional cone of X , then X ∩Tσ 1 is empty or a disjoint finite union of C∗ , by equation (2). In this case, Gr W 1 H (X ∩ Tσ ) = 0, and the result follows. From the above result and Lemma 5.1, we can see that the cup product of two different spaces ϕi ! H 1 (X ∩ Di ) and ϕj ! H 1 (X ∩ Dj ) vanishes unless we assume that ρi \ {0} and ρj \ {0} lie in the relative interior of a 2-dimensional cone σ ∈ X and that ρi and ρj span a cone σ ∈ : ✏ ✏✏ ✏✏ ✏ ✏ ✥✥ ✥✥✥ ✏✏ ✥ ✏ ρi ✥ ❤ ❤ P ❍P ❤❤❤σ ❍❤ P ❤ ❤ ρ ❍P ❍PPP j ❍❍ σ ❍ ❍❍ .

In this case, by Lemma 5.1(ii) we have natural isomorphisms ϕ i ! H 1 X ∩ Di ∼ = H 1 Y ∩ V (σ ) ∼ = ϕ j ! H 1 X ∩ Dj , which provide a natural way to compute the cup product on different spaces in the following lemma. Lemma 5.7. If ρi = ρj , not belonging to X , span a cone σ ∈ contained in a cone σ ∈ X (2), then ϕi ! πi∗ l1 ∪ ϕj ! πj∗ l2 = mult σ ϕσ ! πσ∗ (l1 ∪ l2 ) for l1 , l2 ∈ H 1 (Y ∩ V (σ )), where πσ : X ∩ V (σ ) → Y ∩ V (σ ) is the projection and ϕσ : X ∩ V (σ ) @→ X is the inclusion.

109


Proof. Suppose that ρi and ρj span a cone σ ∈ contained in σ ∈ X (2). Then, using (13) and the projection formula, for l1 , l2 ∈ H 1 (Y ∩ V (σ )) we compute ϕi ! πi∗ l1 ∪ ϕj ! πj∗ l2 = ϕj ! ϕj∗ ϕi ! πi∗ l1 ∪ πj∗ l2 = ϕj ! αij ϕj i ! ϕij∗ πi∗ l1 ∪ πj∗ l2 = ϕj ! ϕj i ! αij ϕij∗ πi∗ l1 ∪ ϕj∗i πj∗ l2 = αij ϕj ! ϕj i ! ϕij∗ πi∗ l1 ∪ l2 = ϕj ! ϕj∗ ϕi ! πi∗ l1 ∪ l2 . We want to compare the map ϕj ! ϕj∗ ϕi ! πi∗ : H 2 Y ∩ V (σ ) −→ H 6 (X) with the map

ϕσ ! πσ∗ : H 2 Y ∩ V (σ ) −→ H 6 (X).

These are the linear maps between 1-dimensional spaces, so they differ by a multiple of a constant. We determine this constant using the two commutative diagrams: H 2 P X H 2 V (σ ) H 2 Y ∩ V (σ )

π∗

/ H 2 (P ) ϕi∗

/ H 2 (D ) i

πi∗

πi∗

/ H 2 (X ∩ D ) i

ϕi !

/ H 4 (P )

ϕi !

/ H 4 (X)

ϕj∗

ϕj∗

ϕj !

/ H 4 (Dj ) / H 4 (X ∩ Dj )

/ H 6 (P ) i∗

ϕj !

/ H 6 (X) i!

H 8 (P ),

H 2 P X

π∗

H 2 V (σ )

πσ∗

/ H2 V σ

H 2 Y ∩ V (σ )

πσ∗

/ H2 X∩V σ

/ H 2 (P ) ϕσ∗

ϕσ !

/ H 6 (P ) i∗

ϕσ !

/ H 6 (X)

i!

/ H 8 (P ),

where the vertical maps are induced by the inclusions. By Lemma 5.4, we had to have some multiplicities in the above diagrams. These multiplicities are all 1 because for any γ ∈ we have X · V (γ ) = X ∩ V (γ ) in A• (P ). Indeed, consider a resolution p : P → P , corresponding to a nonsingular subdivision of . Then, by the

110

ANVAR R. MAVLYUTOV

proof of Lemma 2.3, p −1 (X) ⊂ P is a regular semiample hypersurface. By the projection formula for cycles, for γ ∈ (dim γ ), contained in γ , we have X · V (γ ) = p∗ p ∗ (X) · V γ = p∗ p −1 (X) · V γ = p p −1 (X) ∩ V γ = X ∩ V (γ ). We know a nonzero class [Y ] ∈ H 2 (P X ), the fundamental cohomology class of Y in P X . Mapping this class to H 8 (P ) in the above two diagrams, we get [X] ∪ [Di ] ∪ [Dj ] ∪ [X] and [X] ∪ [V (σ )] ∪ [X], respectively. Using the ring isomorphism A• (P ) ⊗ C ∼ = H 2• (P ), from Lemma 1.4 we find that [X] ∪ [V (σ )] ∪ [X] does not vanish, and, since Di · Dj = (1/mult(σ ))V (σ ) (see [F1, Section 5.1]), it follows that ϕj ! ϕj∗ ϕi ! πi∗ = mult(σ )ϕσ ! πσ∗ on H 2 (Y ∩ V (σ )). We have computed the cup product of any two different spaces in (11). Now we compute the cup product on ϕi ! H 1 (X ∩Di ), which does not vanish when ρi \{0} lies in the relative interior of a 2-dimensional cone σ ∈ X . In this case, there are exactly two cones in , contained in σ and containing ρi : ✏ ✏✏ ✏✏ ✏ ✏ ✥✥ ✥✥✥ σ ρ ✏✏ ✥ ✏ ✥ i ❤ ❤ P ❍P ❤❤❤σ ❍❤ P ❤ ❤ ❍P ❍PPP ❍❍ σ ❍❍ ❍.

In terms of this, we have the following lemma. / X be in some σ ∈ X (2), and let σ , σ ∈ (2) be the two Lemma 5.8. Let ρi ∈ cones containing ρi and contained in σ . Then mult σ + σ ∗ ∗ ϕσ ! πσ∗ l1 ∪ l2 ϕi ! πi l1 ∪ ϕi ! πi l2 = − mult σ mult σ for l1 , l2 ∈ H 1 (Y ∩ V (σ )). Proof. By the projection formula, we have ϕi ! πi∗ _ ∪ ϕi ! πi∗ _ = ϕi ! ϕi∗ ϕi ! πi∗ _ ∪ πi∗ _ = ϕi ! ϕi∗ ϕi ! πi∗ ( _ ∪ _ ). As in the proof of Lemma 5.7, we compare the maps ϕi ! ϕi∗ ϕi ! πi∗ and ϕσ ! πσ∗ . Also, using the arguments of Lemma 5.7 we get 2 X · V σ ϕi ! ϕi∗ ϕi ! πi∗ = X 2 · Di2 ϕσ ! πσ∗ (14) number (X 2 ·Di2 ). Take on H 2 (Y ∩V (σ )). All we need is to compute the intersection n any m ∈ M, such that m, ei = 0. The Weil divisor j =1 m, ej Dj is equivalent to


zero, whence 2 2 X · Di =

111

1 X 2 · Di · −m, ej Dj . m, ei j =i

However, Di ·Dj = (1/mult(γ ))V (γ ), if ρi and ρj span a cone γ ∈ , or Di ·Dj = 0 otherwise. On the other hand, by Lemma 1.4, (X 2 · V (γ )) = 0 unless γ is contained in σ . There are exactly two such cones σ and σ , contained in σ and containing ρi . Suppose that e and e are the primitive generators of the cones σ and σ , not lying in ρi . Then

X2 · Di2 = −

m, e m, e X2 · V σ − X 2 · V σ . m, ei mult σ m, ei mult σ

Since σ ⊥ = σ ⊥ , equation (1) shows that (X 2 ·V (σ )) = (X 2 ·V (σ )). Also, from [D, Section 8.2] it follows that mult(σ + σ )ei = mult(σ )e + mult(σ )e . Therefore, 2 2 mult σ + σ X2 · V σ , X · Di = − mult σ mult σ and the result follows from equation (14). We have finished the calculation of the cup product on H 3 (X). To state a theorem in a nice form we need to define a couple of maps. The map η : R1 (f ) → C is defined as P λ on R1 (f )5β−2β0 (different by a multiple from the map in (10)), and zero in all other degrees. Similarly, replacing P with V (σ ) and f with fσ , we have the map ησ : R1 (fσ ) → C equal to zero in all degrees except for 3β σ − 2β0σ . Recall also that Theorem 5.2 gives the isomorphism n(σ ) H 3−a, a (X) ∼ R 1 fσ σ σ . = R1 (f )(a+1)β−β aβ −β0

0

σ ∈ X (2)

The following is the description of the cup product on the middle cohomology of the hypersurface. Theorem 5.9. Let X ⊂ P , dim P = 4, be a regular semiample hypersurface defined by f ∈ Sβ . If A ∈ R1 (f )(a+1)β−β0 and B ∈ R1 (f )(b+1)β−β0 are identified with elements of Gr F H 3 (X) by means of the isomorphism in Theorem 5.2, then (a(a+1)/2)+(b(b+1)/2)+a 2 +3 /a!b!. If we write X A∪B = cab η(A·B), where cab = (−1) n(σ ) i = Lσ, R1 (f )aβ σ −β0σ a , σ ⊃ρi ∈ / X

i where Lσ, a = R1 (fσ )aβ σ −β0σ correspond to the cones ρi lying in a 2-dimensional σ, j

i σ, i cone σ ∈ , then for li ∈ Lσ, a , li ∈ Lb , and lj ∈ Lb

(identified with elements of

112

ANVAR R. MAVLYUTOV

Gr F H 3 (X)) we have

X

li ∪ lj = mult σ (−1)a−1 ησ li · lj

in the case when ρi and ρj span a cone σ ∈ (2), and mult σ + σ (−1)a−1 ησ li · li , li ∪ l i = − mult σ mult σ X where σ , σ ∈ (2) are the two cones contained in σ and containing ρi . The cup product vanishes in all other cases.

of an ample Calabi-Yau hyRemark 5.10. If X is an MPCP-desingularization Z persurface as in [B2], then the multiplicity mult(σ ) is 1 for all 2-dimensional cones σ , by the properties of a reflexive polytope. Also, in this case mult(σ + σ ) = 2 in Theorem 5.9. 6. Hodge numbers and a “counterexample” in mirror symmetry. In this section, we discuss what kind of mirror symmetry has to be studied. Mirror symmetry proposes that if two smooth m-dimensional Calabi-Yau varieties V and V ∗ form a mirror pair, then their Hodge numbers must satisfy the relations hp, q (V ) = hm−p, q (V ∗ ),

0 ≤ p, q ≤ m.

(15)

A construction in [B2], associated with a pair of reflexive polytopes, satisfies the above equalities for q = 0, 1 (see [BDa]), even if V and V ∗ are compact orbifolds (i.e., V -manifolds). We compute the Hodge numbers hp, 2 of a regular semiample hypersurface in a complete simplicial toric variety P . Then we apply our formula to the Batyrev mirror construction [B2], and we check that there is no symmetry for the

of ample Calabi-Yau hypersurfaces Z Hodge numbers of MPCP-desingularizations Z coming from a pair of reflexive polytopes and ∗ . However, [BBo, Theorem 4.15]

are smooth, and [BDa, Theorem 6.9] show that if these MPCP-desingularizations Z then the duality (15) holds. On the other hand, [BBo, Theorem 4.15] shows that (15) p, q holds for the string-theoretic Hodge numbers hst of the singular ample Calabi-Yau hypersurfaces Z. This confirms the idea that mirror symmetry has to be studied for smooth varieties with usual Hodge numbers or for singular varieties with stringtheoretic Hodge numbers. In order to compute the Hodge numbers, we use the ep, q numbers introduced in [DKh]: (−1)k hp, q Hck (V ) , ep, q (V ) = k

defined for arbitrary algebraic variety V . These numbers satisfy the property ep, q(V )= (−1)p+q hp, q (V ) if V is a compact orbifold.


113

Let X ⊂ P , dim P = d, be a -regular semiample hypersurface with the associated map π : P → P X , Y = π(X), as in Proposition 2.4. Using the properties of ep, q numbers (see [DKh]) and equation (2), for p + q > d − 1, p = q, we compute hp, q (X) = (−1)p+q ep, q (X) = (−1)p+q = (−1)

p+q

e

p−i, q−i

ep, q (X ∩ Tσ )

σ ∈

Y ∩ Tγ · ei, i (C∗ )dim γ −dim σ ,

γ ∈ X γ ⊃σ ∈

where the sum is by all σ ∈ such that γ ∈ X is the smallest cone containing σ . Hence, we get h

p, q

(X) = (−1)

p+q

ak (γ )(−1)

dim(γ )−k+i

γ ∈ X 0 2, p = d − 3, we have hp, d−3 (X) = (−1)

p+d−3

ak (γ )(−1)

l−k+i

γ ∈ X (l) 0 0, there exists a set of primes WM of M of Dirichlet density greater than ((m−1)/m)−ε such that OM has a Diophantine definition over OM,WM . Furthermore, if WF is the set of F -primes below WM , then the Dirichlet density of F can be arranged to be greater than 1 − φ(m)/m − ε. As a corollary of this theorem, we obtain the following result. Corollary 4.7. For every δ > 0, there exist a number field M and a set of Mprimes W such that the following statements are true. • OM is definable over OM,W . • The density of W is greater than 1 − δ. • The density of the set of rational primes below W is greater than 1 − δ. Before we proceed with the details of the proof, we need to discuss some of the known facts concerning Diophantine definability over number fields. First of all, we have the following theorem, whose proof can be found in [17].

INTEGRALITY AT PRIME SETS OF HIGH DENSITY

119

Theorem 1.3. Let M be a number field. Let Q be a non-Archimedean prime of M. Then the set of elements of M integral at Q is Diophantine over M. Second, we have the following proposition, whose proof can also be found in [17]. Proposition 1.4. Let M be a number field. Let WM be a set of non-Archimedean primes of M. Then the set of nonzero elements of OM,WM is Diophantine over OM,WM . This proposition implies the following corollary. Corollary 1.5. Let M be a number field and let A ⊂ M have a Diophantine definition over M. Let WM be any set of non-Archimedean primes of M. Then A ∩ OM,WM has a Diophantine definition over OM,WM . In particular, we can conclude that for any prime p ∈ WM , the set of elements of OM,WM integral at p has a Diophantine definition over OM,WM . Finally, we need to state a few simple “rewriting” results, that is, propositions dealing with replacing polynomial equations over bigger fields with equivalent polynomial equations over smaller fields. Proposition 1.6. Let M/G be a finite extension of number fields. Let OG,WG be a ring of WG -integers of G. Let P (x1 , . . . , xs ) ∈ M[x1 , . . . , xs ]. Then there exists Q(x1 , . . . , xs ) ∈ G[x1 , . . . , xs ] such that for all (a1 , . . . , as ) ∈ (OG,WG )s , P (a1 , . . . , as ) = 0 ⇐⇒ Q(a1 , . . . , as ) = 0. Proposition 1.7. Let M/G be a finite extension of number fields of degree l. Let OG,WG be a ring of WG -integers of G and let OM,WM be the integral closure of OG,WG in M. Let P (t1 , . . . , tr , u1 , . . . , uw , X1 , . . . , Xs ) ∈ M[t1 , . . . , tr , u1 , . . . , uw , X1 , . . . , Xs ]. Then there exists Q(t1 , . . . , tr , u1 , . . . , uw , x1,1 , . . . , xs,l ) ∈ G[t1 , . . . , tr , u1 , . . . , uw , x1,1 , . . . , xs,l ] such that ∀t1 , . . . , tr ∈ OG,WG , ∃ u1 , . . . , uw ∈ OG,WG , ∃ X1 , . . . , Xs ∈ OM,WM , P (t1 , . . . , tr , u1 , . . . , uw , X1 , . . . , Xs)= 0 ⇐⇒ ∃ u1 , . . . , uw , x1,1 , . . . , xs,l ∈ OG,WG , Q(t1 , . . . , tr , u1 , . . . , uw , x1,1 , . . . , xs,l ) = 0. Proof. Let = {ω1 , . . . , ωl } ⊂ OM be a basis of M/G. Let D = DM/G be the discriminant of the basis. Then it is well known that for all X ∈ OM,WM , X = (xi /D)ωi , where xi ∈ OG,WG . Let A1 (x1 /D, . . . , xl /D) = T rM/G (X), . . . , Al (x1 /D, . . . , xl /D) = NM/G (X) be all the coefficients the monic irreducible

120

ALEXANDRA SHLAPENTOKH

polynomial of X over G. Note that all these coefficients are fixed polynomials in x1 , . . . , xl whose coefficients depend on the choice of only. Then for any x1 , . . . , xl ∈ OG,WG , (xi /D)ωi ∈ OM,WM if and only if x x xl xl 1 1 A1 ,..., ∈ OG,WG , . . . , Al ,..., ∈ OG,WG . D D D D Then ∀t1 , . . . , tr ∈ OG,WG , ∃ u1 , . . . , uw ∈OG,WG , ∃ X1 , . . . , Xs ∈OM,WM , P (t1 , . . . , tr , u1 , . . . , uw , X1 , . . . , Xs ) = 0 ⇐⇒ ∃ u1 , . . . , uw , x1,1 , . . . , xs,l , z1,1 , . . . , zs,l ∈ OG,WG , 

l l  x1,i xs,i    P t 1 , . . . , t r , u1 , . . . , u w , ωi , . . . , ωi = 0,   D D   i=1 i=1 x   x1,l 1,1   , . . . , = A , z  1,1 1  D D      x···  x1,l 1,1 z1,l = Al ,..., , D D       x· · ·  xs,l s,1   , . . . , , = A z  s,1 1   D D      x···   zs,l = Al s,1 , . . . , xs,l . D D By Proposition 1.6, we know that this system of equations can be replaced by an equivalent system of equations with coefficients in G. The system over G can, in turn, be replaced by a single equivalent polynomial equation over G. Corollary 1.8. Let M/G be a finite extension of number fields of degree l. Let OG,WG be a ring of WG -integers of G and let OM,WM be the integral closure of OG,WG in M. Let p1 , . . . , ps ∈ WM,WM . Let UM = WM \ {p1 , . . . , ps }. Let P (t1 , . . . , tr , u1 , . . . , uw , X1 , . . . , Xs ) ∈ M[t1 , . . . , tr , u1 , . . . , uw , X1 , . . . , Xs ]. Then there exists Q(t1 , . . . , tr , u1 , . . . , uw , x1,1 , . . . , xs,l ) ∈ G[t1 , . . . , tr , u1 , . . . , uw , x1,1 , . . . , xs,l ] such that ∀t1 , . . . , tr ∈ OG,WG , ∃ u1 , . . . , uw ∈ OG,WG , X1 , . . . , Xs ∈ OM,UM , P (t1 , . . . , tr , u1 , . . . , uw , X1 , . . . , Xs ) = 0 ⇐⇒ ∃ u1 , . . . , uw , x1,1 , . . . , xs,l ∈OG,WG , Q(t1 , . . . , tr , u1 , . . . , uw , x1,1 , . . . , xs,l ) = 0. Note that OM,UM is not necessarily the integral closure in M of any ring of WG integers.


121

Proof. By Theorem 1.3 and by Corollary 1.5, there exists a polynomial S(X, Y1 , . . . , Yl ) ∈ M[X, Y1 , . . . , Yl ] such that for any X ∈ OM,WM , X ∈ OM,UM ⇐⇒ ∃ Y1 , . . . , Yl ∈ OM,WM ,

S(X, Y1 , . . . , Yl ) = 0.

Thus, ∀t1 , . . . , tr ∈ OG,WG , ∃ u1 , . . . , uw ∈OG,WG , ∃ X1 , . . . , Xs ∈OM,UM , P (t1 , . . . , tr , u1 , . . . , uw , X1 , . . . , Xs ) = 0 if and only if ∃ u1 , . . . , uw ∈ OG,WG , ∃ X1 , . . . , Xs , Y1,1 , . . . , Ys,l ∈ OM,WM ,  P (t1 , . . . , tr , u1 , . . . , uw , X1 , . . . , Xs ) = 0,     S(X1 , Y1,1 , . . . , Y1,l ) = 0,  ...    S(Xs , Ys,1 , . . . , Ys,l ) = 0. Now the rest of the proof follows by Proposition 1.7. 2. Notation and assumptions. In this section, we describe the notation that is used for the remainder of the paper. We also prove some technical results concerning units in number fields. The properties of units described below are the foundations of our Diophantine definitions. Notation 2.1. • Let M/F denote a nontrivial finite extension of totally real fields of degree m. • Let M G denote the Galois closure of M over F . • Let [F : Q] = n. • Let PF denote a prime of F that does not split in M. Let PM denote the sole factor of PF in M. • Let L denote a totally complex extension of F of degree 2. • Let K denote a totally real cyclic extension of F of degree p > max(n+1, m+1). • The product of any two fields from the triple of fields (M G , K, L) is linearly disjoint from the third one over F and over M. • PF splits completely in the extension KL/F . • Let δ ∈ OKL denote a generator of MKL over M and a generator of KL over F . • Let φ ∈ OM denote a generator of M over F . • Let VM denote the set of primes of M dividing the discriminant of φ, and let D denote a natural number divisible by all the primes of VM . • Let δG denote an integral generator of M G K over M such that NM G K/M (δG ) ∼ =0 modulo PM . • Let VKM G denote the set of all the primes of M either dividing the discriminant of δG or one of the coefficients of the monic irreducible polynomial of δG over M.

122 •

• • • • •


Let H G (T ) ∈ OM [T ] denote the monic irreducible polynomial of δG over M. K denote a set of primes in M not splitting in the extension MK/M and Let WM K does not contain any primes not contained in VKM G . Furthermore, assume WM with F -conjugates in VKM G . Let WFK denote the set of primes of F lying below K. the primes of WM Let k ∈ N denote a constant defined as in Lemma 2.4. Let hM , hF denote the class numbers of M and F , respectively. Let c ∈ N denote the constant defined in Lemma 3.1. Let aPF denote a fixed element of N, such that aPF ∼ = 0 modulo PF . Let l0 = 0, l1 , . . . , l[KM G :Q] denote distinct natural numbers equivalent to zero modulo PM .

Proposition 2.2. There exists α ∈ KL satisfying the following requirements. (1) NKL/K (α) = 1. (2) NKL/L (α) = 1. (3) The divisor of α is composed of the factors of PF only. (4) α is not a root of unity. Proof. Let UKL,PF be the multiplicative group of PF -units of KL, that is, the multiplicative group of elements of KL whose divisors consist of factors of PF in KL only. By generalized Dirichlet’s unit theorem (see [12, p. 77]), the rank of this group is equal to np +2p −1 = (n+2)p −1, where np is the number of independent Archimedean valuations on KL, and 2p is the number of non-Archimedean valuations on KL restricting to the PF -adic valuation on F . (We remind the reader that, by assumption, PF splits completely in KL.) Let UK,PF be the group of PF -units of K defined analogously to UKL,PF . The rank of that unit group is np + p − 1 = (n + 1)p − 1. Since the norm map from KL to K sends UKL,PF to UK,PF , we must conclude that the rank of the kernel is equal to p > n + 1. Thus there exists β ∈ KL whose divisor consists of factors of PF only such that NKL/K (β) = 1 and β is not a root of unity. Let β1 = β, . . . , βp be all the conjugates of β over L. If for all i = 2, . . . , p, β = ξi βi , where ξi is a root of unity, then some power of β is in UL,PF , where UL,PF is the multiplicative group of PF units in L. The rank of UL,PF is equal to n + 2 − 1 = n + 1 < p. Thus, there exists β ∈ ULK,PF such that NKL/K (β) = 1 and for some 2 ≤ i ≤ p, β/βi is not a root √ of unity. Further, for all i = 1, . . . , p, NKL/K (βi ) = 1. Indeed, assume L = F ( d), where d is a totally negative element √ of F . (In other words, all conjugates √ of d over Q are negative.) Then β = x − dy, where x, y ∈ K, and βi = xi − dyi , where xi and yi are conjugates of x and y, respectively, over L and over F . (This is so because K and L are linearly disjoint over F .) Furthermore, NKL/K (β) = 1 translates into x 2 − dy 2 = 1. The last equation implies xi2 − dyi2 = 1. Let α = β/βi . Then NKL/K (α) = (NKL/K (β))/(NKL/K (βi )) = 1/1 = 1. At the same time, NKL/L (α) = (NKL/L (β))/(NKL/L (βi )) = 1 because the numerator and the denominator are equal.


123

Lemma 2.3. Let ᏿M be the set of all the primes of M splitting completely in the extension MKL/M, and let ᏿MK , ᏿ML , and ᏿MKL be the sets of primes above ᏿M in MK, ML, and MKL, respectively. Let x ∈ MKL be such that NMKL/ML (x) = 1,

(2.1)

NMKL/MK (x) = 1.

(2.2)

Then x is an ᏿MKL -unit; that is, the divisor of x is composed of primes from ᏿MKL only. Proof. Let x be as described in the statement of the lemma. Let QMKL be a prime factor of the divisor of x. Then x must have a nonzero order at a conjugate of QMKL over MK and ML distinct from QMKL . Denote by QMK and QML the primes below QMKL in MK and ML, respectively. Since extensions MKL/MK and MKL/ML are cyclic of prime degrees, we must conclude that QMK and QML split completely in the respective extensions. Let QM be the prime below QMKL in M. Then the number of factors of QM is divisible by 2 and p. Hence this prime will split completely in the extension MKL/M. ns ⊂ ᏿ contain primes of ᏿ without other conjugates over Lemma 2.4. Let ᏿M M M ns ns . Assume further that all the primes F . Let ᏿F be the set of F -primes below ᏿M ns in ᏿Fns split completely in the extension KL/F . Let x ∈ MKL be an ᏿MKL unit, ns ns where ᏿MKL is the set of MKL primes above ᏿M primes, such that x satisfies (2.1) and (2.2). Then there exists a positive integer k depending on M, K, L only such that x k ∈ KL. ns , let PKL be the KL-prime below PMKL , and let PF Proof. Let PMKL ∈ ᏿MKL be the F -prime below PMKL . In MKL, PF has [MLK : M] = [LK : F ] factors. Therefore, PKL does not split in the extension MKL/KL. Let y = NMLK/LK (x). Then the divisors of y and x [MLK:LK] are equal as divisors in MKL. On the other hand, consider the following:

NMLK/KM (y) = NLK/K (y) = NLK/K NMLK/LK (x)

= NMLK/K (x) = NMK/K NMLK/MK (x) = 1.

Therefore,

NMLK/KM

y x [MLK:LK]

= 1,

and y/x [MLK:LK] is an integral unit. But the only integral units of MLK with KMnorm 1 are roots of unity. Thus, for some positive natural number k depending on M, L, and K only, x k ∈ LK. Corollary 2.5. Let x be as in Lemma 2.4, and let x k = [MLK:M]−1 ai δ i , a i ∈ i=0 M. Then ai ∈ F .

124


3. Bounds. This section is devoted to establishing polynomial ways to bound the heights. Lemma 3.1. Let (α/β) ∈ M, with α, β ∈ OM and relatively prime to each other. Let y ∈ OF be such that y G , i = 0, . . . , KM : Q . (3.1) ∈ O M H G (α/β − li ) Assume also that for every embedding σ : M → C and for every root γ of H G (T ), σ α > |σ (γ )| + 1. (3.2) β Let NM/Q (β)α = d0 + d1 φ + · · · + dm−1 φ m−1 ,

d0 , . . . , dm−1 ∈ F.

(3.3)

Then there exists a constant c > 0 depending on D, l0 , . . . , l[KM G :Q] , M, H G (T ), and F only such that NF /Q (Ddi ) < NF /Q (y)c . (3.4) Proof. In KM G , G

yβ deg(H ) y = ∈ OKM G , γ (α/β − li − γ ) γ (α − li β − γβ)

(3.5)

where the product is taken over all roots γ of H G . Since (α, β) = 1 in OKM G , (β, α − li β − γβ) = 1. Thus, for each γ , (y/(α − li β − γβ)) ∈ OKM G . Therefore, N G (α − li β − γβ) ≤ NF /Q (y)[KM G :F ] , KM /Q α N G (β) N G ≤ NF /Q (y)[KM G :F ] . − l − γ KM /Q KM /Q β i Using the fact that |NKM G /Q (β)| ≥ 1, we can conclude that α [KM G :F ] N G . KM /Q β − li − γ ≤ NF /Q (y) On the other hand, using i = 0 and the assumption (3.2), we can conclude that N G (β) ≤ NF /Q (y)[KM G :F ] , KM /Q and also NM/Q (β) ≤ NF /Q (y)[M:F ] ≤ NF /Q (y)[KM G :F ] .

(3.6)


125

From (3.6) and an argument similar to the one used to prove Lemma 3.2 of [18], we can conclude that there exists a positive constant C depending on l0 , . . . , l[KM G :Q] , M, F , and H G only such that σ α ≤ C NF /Q (y)[KM G :F ] (3.7) β for all σ -embeddings of M into C. Using (3.7) and Lemma A.1, we can now conclude that m−1 NM/Q (β) α = di φ i , β

(3.8)

i=0

where di ∈ F , and for all embeddings σ of F into C, G |σ (di )| ≤ C¯ NF /Q (y)2[KM :F ] ,

(3.9)

where C¯ is a positive constant depending on l0 , . . . , l[KM G :Q] , M, F , and H G only. Thus, for some positive constant c depending on D, l0 , . . . , l[KM G :Q] , M, F, H G (T ) and D only, NF /Q (Ddi ) < NF /Q (y)c . (3.10) 4. Main theorems Lemma 4.1. Let QM be a prime of M not splitting in the extension MK/M. Let TM be an F -conjugate of QM . Then TM does not split completely in the extension M G K/M. Proof. Let QF = TF be the F -prime below QM and TM . Let QM G K and TM G K be factors of QM and TM , respectively, in M G K. Since M G K/F is Galois,

f Q M G K / Q F = f T M G K /T F = f T M G K /T M f ( T M / T F ) . Since QM does not split in the extension MK/M, f (QM G K /QF ) ∼ = 0 modulo p > m + 1 = [M : F ] + 1 > f (TM /TF ). Thus, p | f (TM G K /TM ). Lemma 4.2. Suppose the following equations and conditions are satisfied in variables a0,j , . . . , a2p−1,j , b0,j , . . . , b2p−1,j , x, xr , wr,0 , . . . , wr,[KM G :Q] , U0,r , U2p−1,r , v0,r , . . . , v2p−1,r over OM,W K ∪{PM } for all r = 1, . . . , hM , j = 1, . . . , hM + 2: M

αj =

2p−1 i=0

ai,j δ i ,

βj =

2p−1 i=0

bi,j δ i ;

(4.1)

126


NMKL/KM βj = 1;

NMKL/LM βj = 1;

(4.2) (4.3)

αj = βjk ; xr = aPF (x + sr )

hM

(4.4)

, x is integral at all the primes of M dividing aPF ,

(4.5)

where s1 = 0, . . . , shM are distinct natural numbers; wr,i =

hF a1,1

H G (x

r − li )

,

i = 0, . . . , KM G : Q ;

(4.6)

αr+2 − 1 ; α2 − 1

(4.7)

U0,r + U1,r δ + · · · + U2p−1,r δ 2p−1 =

xr − U0,r + U1,r δ + · · · + U2p−1,r δ 2p−1 chF = a1,1 v0,r + v1,r δ + · · · + v2p−1,r δ 2p−1 .

(4.8)

Assume also that for every embedding σ : KM G → C and for every root γ of H G (T ), |σ (xr )| > |σ (γ )| + 1.

(4.9)

Then x ∈ F . Proof. First of all, we observe that ordPM H G (xr − li ) = 0. Since the M-norm of δG is not divisible by PM , the free term of H G (T ) is not zero modulo PM . From (4.5) we deduce that for all r, xr ∼ = 0 modulo PM , and by assumption, for K all i, li is equivalent to zero modulo PM . Thus, our assertion is true. Since WM contains only primes not splitting in the extension MK/M and does not contain any primes dividing the discriminant of H G (T ) or its coefficients or any primes whose F conjugates divide the discriminant of H G (T ) or its coefficients, by Lemmas A.2 and K or their conjugates 4.1, H G (xr −li ) does not have positive order at the primes of WM G over F . Hence, we can conclude that H (xr −li ) does not have positive order at any primes where elements of the ring OM,W K ∪{PM } are allowed negative orders or any M

hF /H G (xr − li )) can have negative order only at primes primes above WFK . Thus, (a1,1 where a1,1 has negative order. By Lemma 2.4, αj ∈ KL for all j = 1, . . . , hM + 2. Since δ generates KL over F , we have a1,1 ∈ F ∩ OM,W K ∪{PM } ⊂ OF,W K ∪{PF } . Let A be the divisor of a1,1 . M F Then A = BC, where B has a product of primes from WFK and powers of PF in the denominator and numerator, while C is an integral divisor composed of primes hF = zy, z ∈ F, y ∈ OF , such that outside WFK ∪ {PF }. Thus, a1,1

y ∈ OM , HG (xr − li )

i = 0, . . . , M G K : Q .


127

Therefore, by Lemma 3.1 we can conclude the following: xr =

αr , βr

αr , βr ∈ O M ; (4.10) NF /Q (Ddj,r ) < NF /Q (y c ). NM/Q (βr )xr = d0,r + d1,r φ + · · · + dm−1,r φ m−1 ; (4.11) chF By (4.7), xr − U0,r = a1,1 v0,r , where U0,r ∈ F . Further, chF NF /Q (βr )xr − NF /Q (βr )U0,r = NF /Q (βr )a1,1 v0,r .

Hence, chF Cr , NF /Q (βr )xr − Br = a1,1

where a1,1 , Br ∈ OF,W K ∪{PF } , NF /Q (βr )xr ∈ OM . From the discussion above, F chF = y c y, ¯ where y c ∈ OF , y¯ ∈ OF,W K ∪{PF } and the divisor of y c has no faca1,1 F tors at which elements of the ring OM,W K ∪{PM } are allowed to have negative orders. M Thus, NF /Q (βr )xr − Br = y c Zr , where Zr ∈ OM,W K ∪{PM } . Further, by the strong approximation theorem, there exists M Ar ∈ OF such that ((Br − Ar )/y c ) ∈ OF,W K ∪{PF } . Thus, we have F

NF /Q (βr )xr − Ar = y c Er ,

(4.12)

where Er ∈ OM . The left-hand side of (4.12) can be rewritten as d0,r − Ar + d1,r φ + · · · + dm−1,r φ m−1 = y c Er . Hence, we can conclude that d0,r − Ar d1,r dm−1,r m−1 + c φ +···+ φ c y y yc is an algebraic integer. Then, by a well-known number-theoretic result, for each s > 0, Dds,r /y c is an algebraic integer. However, this implies that NF /Q (Dds,r ) = 0,

s = 1, . . . , m − 1,

or NF /Q (Dds,r ) ≥ NF /Q (y c ),

s = 1, . . . , m − 1.

The latter case contradicts (4.11). Thus, for all s > 0, ds,r = 0, and consequently, xr ∈ F . By [18, Lemma 5.2], having xr ∈ F for r = 1, . . . , hM implies x ∈ F .

128


Lemma 4.3. Let x ∈ N be large enough in absolute value so that (4.9) is satisfied. Then all the equations (4.1)–(4.8) can be satisfied in all the other variables over OM,W K ∪{PM } . M

Proof. By Lemma 2.2, there exist α, β in the integral closure of OM,{PM } in MKL satisfying equations (4.2)–(4.4). Furthermore, by Lemma 2.3, the divisors of α and β are composed of factors of PM = PF . Using an argument similar to the one used in [18, Lemma 2.6 and Corollary 2.7], one can show that for any integral divisor A of F relatively prime to PF , there exists a natural number s such that the following conditions are satisfied. • βs ∼ = 1 mod A in the integral closure of OM,PM in MKL. • The coordinates of β s with respect to the power basis of δ are in OM,PM . • The first coordinate of β s with respect to the power basis of δ is equivalent to 1 modulo A in OM,PM . • All the other coordinates of β s are equivalent to 0 modulo A in OM,PM . If x ∈ Z, then (4.5) is satisfied. Let α1 = α s and β1 = β s where s corresponds to A equal to the product of numerators of the divisors of H G (xr − li ) for all i and r that we know by Lemma 4.2 to be prime to PM . Then (4.1) and (4.6) can also be satisfied. Similarly, let α2 be a sufficiently high power of α1 so that chF | α2 − 1 a1,1

in OM,PM [δ]. To satisfy (4.7) and (4.8), set αr+2 = α2xr , βr+2 = β2xr . Observe that αr+2 − 1 ∼ = xr α2 − 1 in Z[α2 ] ⊂ OM,PM [δ]. Thus,

modulo α2 − 1

αr+2 − 1 ∼ ch = xr modulo a1,1F α2 − 1 in OM,PM [δ]. Note that α2 , . . . , αhM +2 , β2 , . . . , βhM +2 ∈ OM,PM [δ], chosen in such a manner, also satisfy (4.1)–(4.4). Now we are ready to state the first of our main theorems. Theorem 4.4. Let F, OM,W K ∪{PM } be as in Notation 2.1. Then F ∩OM,W K ∪{PM } M M has a Diophantine definition over OM,W K ∪{PM } . M

Proof. We only need to note two things. First, every element of F ∩OM,W K ∪{PM } M :Q]−1 can be written in the form [F ±(ui /v)ψ i , where ui , v ∈ N and ψ is an integral i=0 generator of F over Q. Second, (4.9) can be rewritten as a polynomial equation using an argument similar to the one used in [18, Corollary 4.4]. Theorem 4.5. Let G be any totally real nontrivial extension of Q. Then there exists an infinite set WG of primes of G such that OG has a Diophantine definition over OG,WG .


129

Proof. Let M be the Galois closure of G over Q. Let F be a cyclic subextension of M such that G = G ∩ F . Such an F always exists. Since G = Q, there exists σ ∈ Gal(M/Q) such that G is not fixed by σ . Then let F be the fixed field of σ . Clearly G ⊂ F , and thus R = G∩F is a proper subfield of G. Let LQ , KQ be cyclic, prime-degree extensions of Q defined so that the fields L = F LQ and K = F KQ satisfy the requirements of Notation 2.1 with respect to M and F , and KQ R is linearly disjoint from M over R. (It is clear that such a pair of fields LQ , KQ always exists for any totally real pair of fields M and F .) By Lemma A.3, there exist infinitely many primes of R splitting completely in the extension M/R, such that their factors remain prime in the extension MK/M. Let WGK be an infinite collection of factors of these primes in G such that every prime in WGK has a conjugate over R that is not in K be the set of all the factors of primes in W K in M with possibly finitely WGK . Let WM G K . (The many primes removed to satisfy the assumptions listed in Notation 2.1 for WM G reader should note here that, in this case, M = M.) Applying Lemma A.3 again, we conclude that there exist infinitely many primes of F splitting completely in KL and not splitting in the extension M/F . (Note that factors of these primes in M split completely in the extensions MKL/M.) Let PF be one of these primes. Let PM be its sole factor in M and let PG be the G-prime below PM . By Theorem 4.4, there exists a polynomial P (t, X1 , . . . , Xr ) ∈ M[t, X1 , . . . , Xr ] such that ∀t ∈ OM,W K ∪{PM } , M

∃ X1 , . . . , Xr ∈ OM,W K ∪{PM } , M

P (t, X1 , . . . , Xr ) = 0

if and only if t ∈ OM,W K ∪{PM } ∩F . Therefore, there exists a polynomial P (t, X1 , . . . , M Xr ) ∈ M[t, X1 , . . . , xr ] such that ∀t ∈ OM,W K ∪{PM } ∩ G, M

∃ X1 , . . . , Xr ∈ OM,W K ∪{PM } , M

P (t, X1 , . . . , Xr ) = 0

if and only if t ∈ OM,W K ∪{PM } ∩ F ∩ G = OM,W K ∪{PM } ∩ R. M

M

Let us examine OM,W K ∪{PM } ∩G more closely. If PG has only one factor in M, then M OM,W K ∪{PM } ∩ G = OG,W K ∪{PG } . However, if PG has more than one factor in M, M G then OM,W K ∪{PM } ∩ G = OG,W K . Without loss of generality, assume we are in the M G second case. Then by Theorem 1.3, there exists a polynomial S(t, z1 , . . . , zs ) such that for every t ∈ OG,W K ∪{PG } , ∃ z1 , . . . , zs ∈ OG,W K ∪{PG } , S(t, z1 , . . . , zs ) = 0 ⇔ G G t ∈ OG,W K . Thus, for every t ∈ OG,W K ∪{PG } , G

G

∃ z1 , . . . , zs ∈ OG,W K ∪{PG } , G

X1 , . . . , Xr ∈ OM,W K ∪{PM } , M

(4.13)

130


S(t, z1 , . . . , zs ) = 0, P (t, X1 , . . . , Xr ) = 0

if and only if t ∈ OM,W K ∪{PM } ∩ R. M

By Corollary 1.8, we can find Q(t, z1 , . . . , zs , x1,1 , . . . , xr,[M:G] ) ∈ G[t, z1 , . . . , zs , x1,1 , . . . , xr,[M:G] ] such that for every t ∈ OG,W K ∪{PG } , G

∃ z1 , . . . , zs , x1,1 , . . . , xr,[M:G] ∈ OG,W K ∪{PG } , Q t, z1 , . . . , zs , x1,1 , . . . , xr,[M:G] = 0 G

if and only if t ∈ OM,W K ∪{PM } ∩ R. M

Next consider the following: Under the assumption that either PG had more than one factor in M or more than one conjugate over R,

OM,W K ∪{PM } ∩ R = OM,W K ∪{PM } ∩ G ∩ R = OG,W K ∩ R = OR . M

M

G

In the case PG has just one factor in M and has no other conjugates over R, OM,W K ∪{PM } ∩ R = OR,{PR } , M

where PR is the R prime below PG . Since we know how to define integrality at finitely many primes using polynomial equations, again, without loss of generality, we can assume that the intersection is equal to OR . Thus we conclude that OR has a Diophantine definition over OG,W K ∪{PG } . Next we note that rational integers have a G Diophantine definition over OR , and we can use that fact, together with an integral basis of OG over Z, to complete a Diophantine definition of OG over OG,W K ∪{PG } . G

Theorem 4.6. Let M be a totally real number field with a cyclic subextension F of degree m. Then for any ε > 0, there exists a set of primes WM of M of Dirichlet density greater than ((m−1)/m)−ε such that OM has a Diophantine definition over OM,WM . Furthermore, if WF is the set of F -primes below the primes of WM , then the Dirichlet density of WF can be arranged to be greater than 1 − (φ(m)/m) − ε, where φ is the Euler function. Proof. First of all, note the existence of extensions L and K of F satisfying the conditions of Notation 2.1. Next let WFK be the set of all the primes in F splitting in K be the largest possible M with factors not splitting in the extension MK/M. Let WM K set of M-primes above the primes of WF satisfying the following condition. Every K has exactly one F -conjugate not in W K . (As above, we might have to prime in WM M


131

K and W K to make sure all the requirements remove finitely many primes from WM F described in Notation 2.1 are satisfied. Here again, M G = M.) Using Lemma A.3, again we ascertain existence of an M-prime PM such that PM splits completely in MKL/M but has no other conjugates over F . By Theorem 4.4, we conclude that F ∩ OM,W M ∪{PM } = OF,{PF } has a Diophantine definition over OM,W M ∪{PM } . Since we K K can define integrality at one prime, we can now conclude that OF has a Diophantine definition over OM,W M ∪{PM } . Using the same argument as in Theorem 4.5, we deduce K that OM has a Diophantine definition over OM,W M ∪{PM } . It remains to calculate the K

K and W K . We start with the latter. density of WM F By the Chebotarev density theorem (see [8, Theorem 10.4]), the density of WFK is ((p −1)/pm)(m−φ(m)). Indeed, consider Gal(MK/F ) = Gal(M/F )×Gal(K/F ). If p ∈ WFK , then, except for finitely many primes, either p does not split in M or some factors of p split in K. Let σM , σK be generators of Gal(M/F ), Gal(K/F ), respectively. Then the Frobenius automorphism of factors of p in Gal(MK/F ) must i , σ j ), where either (i, m) = 1 and j = 0, 1, . . . , p −1, or (i, m) > 1 be of the form (σM K and j = 0. Thus, the total number of elements of the Galois group of MK over F that can be Frobenius automorphisms of the primes not above WFK is φ(m)p+(m−φ(m)). Therefore, the number of elements in the Galois group of MK over F that can be Frobenius automorphisms of primes above WFK is mp − φ(m)p − m + φ(m) = (m−φ(m))(p−1). Hence, it follows by Chebotarev’s density theorem that the density of WFK is ((p − 1)/pm)(m − φ(m)). By letting p → ∞, we get the desired result for the density of the prime set in F . K , we first note that we need to consider primes of To calculate the density of WM K WM with m conjugates over F only, since the density of the set of M-primes lying above totally splitting primes of F is 1. (See [8, Theorem 4.6].) By assumption, K except possibly for finitely many primes, for every totally splitting prime of F , WM contains m − 1 of its factors if they do not split in K. Let WM be a set of M primes containing m−1 factors of every totally splitting prime of F . Then by the Chebotarev K and W is less than 1/p and thus density theorem, the difference in densities of WM M can be made arbitrarily small by letting p be arbitrarily large. Thus it is sufficient to compute the density of WM . Let UM be the set of all the M-primes above totally splitting primes of F . Then the density of UM is 1. We can obtain the density of WM from the density of UM by subtracting the density of the set consisting of single factors of every totally splitting prime of F . The density of that set is exactly the same as the density of the set of all the primes in F totally splitting in M, since their Q-norms are the same. This density is equal to 1/m. Therefore, the density of WM is 1 − (1/m) as required.

Corollary 4.7. For every δ > 0, there exist a number field M and a set of Mprimes W such that the following statements are true. • OM is definable over OM,W . • The density of W is greater than 1 − δ.

132


• The density of the set of rational primes below W is greater than 1 − δ. Proof. Let M be a totally real cyclic extension of Q of degree m = p1 · · · pk , where pi is the ith rational prime under the standard enumeration of primes. Then K of M primes such that the by Theorem 4.6, for every ε > 0, there exists a set WM following are true: K is greater than ((m − 1)/m) − ε; (1) the density of WM K is greater than (2) the density of the set of rational primes below WM k

1− Since limk→∞ true.

k

pi − 1 φ(m) − ε. −ε = 1− m pi i=1

i=1 ((pi

− 1)/pi ) = 0 (see [14, (9.1.14)]), the corollary is clearly Appendix

Lemma A.1. Let M/F be a finite extension of number fields of degree m. Let δ be i a generator of M/F . Let x ∈ M, x = m−1 i=0 ai δ , ai ∈ F . Assume for some positive constant C, for every σ -embedding of M into C, |σ (x)| < C. Then for every σ ˜ where C˜ depends embedding of M into C, and every i = 0, . . . , m−1, |σ (ai )| < CC, on δ only. Proof. Consider the following linear system: m−1

aj σj δ i = σj (x),

j = 1, . . . , m,

i=0

where σ1 = identity, . . . , σm are all the embeddings of M into C leaving F fixed. Solving this system via Cramer’s rule, we can deduce that for each i = 0, . . . , m − 1, ai = Pi (δ, σ1 (x), . . . , σm (x)), where for each i, Pi is a fixed polynomial, linear in σj (x), depending only on m. Next let τ be an embedding of M into C that does not fix F . Repeating the argument above over τ (F ), we obtain a similar bound for all τ (ai ). This way we can obtain a bound as described in the statement of the lemma for all the conjugates of ai over Q. Lemma A.2. Let E/M be a finite extension of algebraic number fields. Let γ ∈ OE generate E over M and let H (T ) be the monic irreducible polynomial of γ over M. Let WM be a set of primes of M without relative degree 1 factors in E. Then for every Q ∈ WM such that Q does not divide the discriminant of γ and no coefficient of H (T ) has a positive order Q, for every x ∈ M, ordQ H (x) ≤ 0. Proof. Let γ and Q be as in the statement of the lemma. Then powers of γ constitute a local integral basis of E over M with respect to Q. Thus the factorization of the minimal polynomial of γ modulo Q corresponds to the factorization of Q in


133

E (see [9, Proposition 25].) Let x ∈ M and assume x has a negative order at Q. Then H (x) has a negative order at Q. On the other hand, suppose x is integral at Q and H (x) has positive order at Q. Then H (T ) has a root modulo Q and thus a linear factor modulo Q. This implies Q has a factor of relative degree 1 in E in contradiction of our assumption. Lemma A.3. Let F be a number field. Let N1 be a cyclic extension of F , and let N2 be a Galois extension of F linearly disjoint from N1 over F . Then there are infinitely many primes of p of F such that p does not split in N1 ; its unique factor in N1 splits completely in the extension N1 N2 /N1 , and p splits completely in N2 . If N2 is also cyclic, then there are infinitely many primes p of F such that p splits completely in N1 but none of its factors split or ramify in the extension N1 N2 /N1 . Proof. For the first and the second assertions of the lemma, apply parts 2 and 3 of [18, Lemma 2.1], respectively, with L = F , K = N1 , and E = N2 . References [1] [2] [3]

[4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]

J.-L. Colliot-Thélène, A. N. Skorobogatov, and P. Swinnerton-Dyer, Double fibres and double covers: Paucity of rational points, Acta. Arith. 79 (1997), 113–135. M. Davis, Hilbert’s tenth problem is unsolvable, Amer. Math. Monthly 80 (1973), 233–269. M. Davis, Y. Matijasevich, and J. Robinson, “Hilbert’s tenth problem: Diophantine equations: Positive aspects of a negative solution” in Mathematical Developments Arising from Hilbert Problems (Northern Illinois University, Dekalb, Ill., 1974), Proc. Sympos. Pure Math. 28, Amer. Math. Soc., Providence, 1976, 323–378. J. Denef, Hilbert’s tenth problem for quadratic rings, Proc. Amer. Math. Soc. 48 (1975), 214–220. , Diophantine sets over algebraic integer rings, II, Trans. Amer. Math. Soc. 257 (1980), 227–236. J. Denef and L. Lipschitz, Diophantine sets over some rings of algebraic integers, J. London Math. Soc. (2) 18 (1978), 385–391. M. Fried and M. Jarden, Field Arithmetic, Ergeb. Math. Grenzgeb. (3) 11, Springer-Verlag, Berlin, 1986. G. Janusz, Algebraic Number Fields, Pure Appl. Math. 55, Academic Press, New York, 1973. S. Lang, Algebraic Number Theory, Addison-Wesley, Reading, Mass., 1970. B. Mazur, The topology of rational points, Experiment. Math. 1 (1992), 35–45. , Questions of decidability and undecidability in number theory, J. Symbolic Logic 59 (1994), 353–371. O. T. O’Meara, Introduction to Quadratic Forms, Grundlehren Math. Wiss. 117, SpringerVerlag, Berlin, 1963. T. Pheidas, Hilbert’s tenth problem for a class of rings of algebraic integers, Proc. Amer. Math. Soc. 104 (1988), 611–620. H. N. Shapiro, Introduction to the Theory of Numbers, Pure Appl. Math., Wiley, New York, 1983. H. N. Shapiro and A. Shlapentokh, Diophantine relationships between algebraic number fields, Comm. Pure Appl. Math. 42 (1989), 1113–1122. A. Shlapentokh, Extension of Hilbert’s tenth problem to some algebraic number fields, Comm. Pure Appl. Math. 42 (1989), 939–962.

134 [17] [18]

ALEXANDRA SHLAPENTOKH , Diophantine classes of holomorphy rings of global fields, J. Algebra 169 (1994), 139–175. , Diophantine definability over some rings of algebraic numbers with infinite number of primes allowed in the denominator, Invent. Math. 129 (1997), 489–507.

Department of Mathematics, East Carolina University, Greenville, North Carolina 27858, USA; [email protected]

Vol. 101, No. 1


© 2000

ALMOST COMPLEX STRUCTURES ON S 2 × S 2 DUSA MCDUFF

Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 2. Main ideas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 2.1. The effect of increasing λ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 2.2. Stable maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 2.3. Gluing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 2.4. Moduli spaces and the stratification of ᏶ . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 3. The link ᏸ2,0 of ᏶2 in ᏶0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 3.1. Some topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 3.2. Structure of the pair (ᐂJ , ᐆJ ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 3.3. The projection ᐂJ → ᏶ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 4. Analytic arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 4.1. Regularity in dimension 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 4.2. Gluing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 1. Introduction. It is well known that every symplectic form on X = S 2 × S 2 is, after multiplication by a suitable constant, symplectomorphic to a product form ωλ = (1 + λ)σ1 + σ2 for some λ ≥ 0, where the 2-form σi has total area 1 on the ith factor. We are interested in the structure of the space ᏶λ of all C ∞ ωλ -compatible, almost complex structures on X. Observe that ᏶λ itself is always contractible. However, it has a natural stratification that changes as λ passes each integer. The reason for this is that as λ grows, the set of homology classes that can be represented by an ωλ -symplectically embedded 2-sphere changes. Since each such 2-sphere can be parametrized to be J -holomorphic for some J ∈ ᏶λ , there is a corresponding change in the structure of ᏶λ . To explain this in more detail, let A ∈ H2 (X, Z) be the homology class [S 2 × pt] and let F = [pt ×S 2 ]. (The reason for this notation is that we are thinking of X as a fibered space over the first S 2 -factor, so that the smaller sphere F is the fiber.) When − 1 < λ ≤ , ωλ (A − kF ) > 0 for 0 ≤ k ≤ . Moreover, it is not hard to see that for each such k, there is a map ρk : S 2 → S 2 of Received 13 October 1998. 1991 Mathematics Subject Classification. Primary 53C15. Author partially supported by National Science Foundation grant number DMS 9704825. 135

136

DUSA MCDUFF

degree −k whose graph

z −→ z, ρk (z)

is an ωλ -symplectically embedded sphere in X. It follows easily that the space ᏶λk = J ∈ ᏶λ : there is a J -hol curve in class A − kF is nonempty whenever k < λ + 1. Let λ

᏶k = ∪m≥k ᏶λm .

Because (A−kF )·(A−mF ) < 0 when k = m > 0, positivity of intersections implies that there is exactly one J -holomorphic curve in class A − kF for each J ∈ ᏶k . We denote this curve by J . λ

Lemma 1.1. The spaces ᏶λk , 0 ≤ k ≤ , are disjoint, and ᏶k is the closure of ᏶λk λ

in ᏶λ . Further, ᏶λ = ᏶0 .

Proof. It is well known that for every J ∈ ᏶, the set of J -holomorphic curves in class F form the fibers of a fibration πJ : X → S 2 . Moreover, the class A is represented by either a curve or a cusp-curve (i.e., a stable map).1 Since the class F is always represented and (mA+pF )·F = m, it follows from positivity of intersections that m ≥ 0 whenever mA + pF is represented by a curve. Hence any cusp-curve in class A has one component in some class A−kF for k ≥ 0, and all others represent a multiple of F . In particular, each J ∈ ᏶λ belongs to some set ᏶λk . Moreover, because (A − kF ) · (A − mF ) < 0 when k = m and k, m ≥ 0, the different ᏶λk are disjoint. The second statement holds because if Jn is a sequence of elements in ᏶λk , then the corresponding sequence of Jn -holomorphic curves in class A − kF has a convergent subsequence whose limit is a cusp-curve in class A − kF . This limit has to have a component in some class A − mF , for m ≥ k, and so J ∈ ᏶λm for some m ≥ k. For further details, see Lalonde and McDuff [LM]. Here is our main result. Throughout we work with C ∞ -maps and almost complex structures, and so by manifold we mean a Fréchet manifold. By a stratified space ᐄ we mean a topological space that is a union of a finite number of disjoint manifolds that are called strata. Each stratum ᏿ has a neighborhood ᏺ᏿ that projects to ᏿ by a map ᏺ᏿ → ᏿. When ᏺ᏿ is given the induced stratification, this map is a locally trivial fiber bundle whose fiber has the form of a cone C(ᏸ) over a finite-dimensional stratified space ᏸ that is called the link of ᏿ in ᐄ. Moreover, ᏿ sits inside ᏺ᏿ as the set of vertices of all these cones. Theorem 1.2. (i) For each 1 ≤ k ≤ , ᏶λk is a submanifold of ᏶λ of codimension 4k − 2. λ λ (ii) For each m > k ≥ 1, the normal link ᏸm,k of ᏶λm in ᏶k is a stratified space 1 We

follow the convention of [LM] by defining a “curve” to be the image of a single sphere, while a “cusp-curve” is either multiply covered or has domain equal to a union of two or more spheres.

ALMOST COMPLEX STRUCTURES ON S 2 × S 2

137 λ

of dimension 4(m − k) − 1. Thus, there is a neighborhood of ᏶λm in ᏶k that is fibered λ . over ᏶λm with fiber equal to the cone on ᏸm,k λ is independent of λ (provided that λ > m − 1). (iii) The structure of the link ᏸm,k The first part of this theorem was proved by Abreu in [A], at least in the C s -case where s < ∞. (Details are given in §4.1 below.) The second and third parts follow by globalizing recent work by Fukaya and Ono [FO], Li and Tian [LiT], Liu and Tian [LiuT1], Ruan [R], and others on the structure of the compactification of moduli spaces of J -holomorphic spheres via stable maps. We extend current gluing methods by showing that it is possible to deal with obstruction bundles whose elements do not vanish at the gluing point (see §4.2.3). Another essential point is that we use Fukaya and Ono’s method of dealing with the ambiguity in the parametrization of a stable map, since this involves the least number of choices and allows us to globalize by constructing a gluing map that is equivariant with respect to suitable local torus actions (see §4.2.4 and §4.2.5). The above theorem is the main tool used in [AM] to calculate the rational cohomology ring of the group Gλ of symplectomorphisms of (X, ωλ ). The methods of proof may also prove useful in situations in which one wants to understand properties of families of almost complex structures. These arise when one is considering family Gromov-Witten invariants, as in the work of Bryan and Leung [BL] on the Yau-Zaslow conjecture [YZ], and that of Li and Liu [LL1], [LL2] on symplectic 4-manifolds with torsion canonical class. Observe that part (iii) states that the normal structure of the stratum ᏶λk does not change with λ. On the other hand, it follows from the results of [AM] that the cohomology of ᏶λk definitely does change as λ passes each integer. Obviously, it would be interesting to know if the topology of ᏶λk is otherwise fixed. For example, one could try to construct maps ᏶λ → ᏶µ for λ < µ that preserve the stratification, µ and then try to prove that they induce homotopy equivalences ᏶λk → ᏶k whenever − 1 < λ ≤ µ ≤ . So far, the most we have managed to do in this direction is to prove the following lemma, which, in essence, constructs maps ᏶λ → ᏶µ for λ < µ. It is not clear whether these are homotopy equivalences for λ, µ ∈ ( − 1, ]. It is convenient to fix a fiber F0 = pt ×S 2 and define ᏶λk ᏺ(F0 ) = J ∈ ᏶λk : J = Jsplit near F0 , where Jsplit is the standard product almost complex structure. Lemma 1.3. (i) The inclusion ᏶λ (ᏺ(F0 )) → ᏶λ induces a homotopy equivalence for all k < λ + 1. (ii) Given any compact subset C ⊂ ᏶λ (ᏺ(F0 )) and any µ > λ, there is a map ιµ,λ : C −→ ᏶µ ᏺ(F0 )

᏶λk (ᏺ(F0 )) −−→ ᏶λk

µ

that takes C ∩ ᏶λk (ᏺ(F0 )) into ᏶k (ᏺ(F0 )) for all k.

138

DUSA MCDUFF

λ . So far this This lemma is proved in §2. The next task is to calculate the links ᏸm,k has been done for the easiest case as follows. λ Proposition 1.4. For each k ≥ 1 and λ, the link ᏸk+1, k is the 3-dimensional lens space L(2k, 1).

Finally, we illustrate our methods by using the stable map approach to confirm that the link of ᏶λ2 in ᏶λ is S 5 , as predicted by part (i) of Theorem 1.2. Our method first calculates an auxiliary link ᏸᐆ from which the desired link is obtained by collapsing certain strata. The S 5 appears in a surprisingly interesting way, which can be briefly described as follows. Let ᏻ(k) denote the complex line bundle over S 2 with Euler number k, where we write C instead of ᏻ(0). Given a vector bundle E → B, we write S(E) → B for its unit sphere bundle. Note that the unit 3-sphere bundle S ᏻ(k) ⊕ ᏻ(m) −→ S 2 decomposes as the composite S LP (k,m) −→ ᏼ ᏻ(k) ⊕ ᏻ(m) −→ S 2 , where LP (k,m) → ᏼ(ᏻ(k) ⊕ ᏻ(m)) is the canonical line bundle over the projectivization of ᏻ(k) ⊕ ᏻ(m). In particular, the space S(ᏻ(−1) ⊕ C) can be identified with S(LP (−1,0) ). But ᏼ(ᏻ(−1) ⊕ C) is simply the blow-up CP 2 #CP 2 , and its canonical bundle is the pullback of the canonical bundle over CP 2 . We are interested in the singular line bundle (or orbibundle) LY → Y whose associated unit sphere bundle has total space S(LY ) = S 5 and fibers equal to the orbits of the following S 1 -action on S 5 : θ · (x, y, z) = eiθ x, eiθ y, e2iθ z , x, y, z ∈ C. The plumbing described in (i) below can be considered as an orbifold analog of the process of blowing up a point lying on the zero section of a bundle (see §3.1 below and [G]). Theorem 1.5. (i) The space ᏸᐆ obtained by plumbing the unit sphere bundle of ᏻ(−3) ⊕ ᏻ(−1) with the singular circle bundle S(LY ) → Y may be identified with the unit circle bundle of the canonical bundle over ᏼ(ᏻ(−1) ⊕ C) = CP 2 #CP 2 . λ is obtained from ᏸ by collapsing the fibers over the exceptional (ii) The link ᏸ2,0 ᐆ divisor to a single fiber, and hence may be identified with S 5 . Under this identification, λ = RP 3 corresponds to the inverse image of a conic in CP 2 . the link ᏸ2,1 In his recent paper [K], Kronheimer shows that the universal deformation of the quotient singularity C2 /(Z/mZ) is transverse to all the submanifolds ᏶k and so is an explicit model for the normal slice of ᏶m in ᏶. Hence one can investigate the structure λ using tools from algebraic geometry. It is very possible of the intermediate links ᏸm,k


139

that it would be easier to calculate these links this way. However, it is still interesting to try to understand these links from the point of view of stable maps, since this is more closely connected to the symplectic geometry of the manifold X. Another point is that throughout we consider ωλ -compatible, almost complex structures rather than ωλ -tame ones. However, it is easy to see that all our results hold in the latter case. Other ruled surfaces. All the above results have analogs for other ruled surfaces Y → ). If Y is diffeomorphic to the product ) × S 2 , we can define ωλ , ᏶λk as above, though now we should allow λ to be any number greater than −1 since there is no symmetry between the class A = [) ×pt] and F = [pt ×S 2 ]. In this case, Theorem 1.2 still holds. The reason for this is that if u : ) → Y is an injective J -holomorphic map in class A−kF where k ≥ 1, then the normal bundle E to the image u()) has negative first Chern class so that the linearization Du of u has kernel and cokernel of constant dimension. (In fact, the normal part of Du with image in E is injective in this case; see [HLS, Theorem 1 ].) However, Lemma 1.1 fails unless ) is a torus since there are tame, almost complex structures on Y with no curve in class A. One might think to remedy this by adding other strata ᏶λ−k consisting of all J such that the class A + kF is represented by a J -holomorphic curve u : (), j ) → Y for some complex structure j on ). However, although the universal moduli space ᏹ(A + kF, ᏶λ ) of all such pairs (u, J ) is a manifold, the map (u, J ) → J is no longer injective: even if one cuts down the dimension by fixing a suitable number of points, each J , in general, admits several curves through these points. Moreover, as (uJ ) varies over ᏹ(A + kF, ᏶λ ), the dimension of the kernel and cokernel of Du can jump. Hence the argument given in §4.1 that the strata ᏶λk are submanifolds of ᏶λ fails on several counts. In the case of the torus, ᏶λ0 is open, and so Lemma 1.1 does hold. However, it is not clear whether this is enough for the main application, which is to further our understanding of the groups Gλ of symplectomorphisms of (Y, ωλ ). One crucial ingredient of the argument in [AM] is that the action of this group on each stratum ᏶λk is essentially transitive. More precisely, we show that the action of Gλ on ᏶λk induces a homotopy equivalence Gλ / Aut(Jk ) → ᏶λk , where Jk is an integrable element of ᏶λk and Aut(Jk ) is its stabilizer. It is not clear whether this would hold for the stratum ᏶λ0 when ) = T 2 . One might have to take into account the finer stratification considered by Lorek in [Lo]. He points out that the space ᏶λ0 of all J that admit a curve in class A is not homogeneous. A generic element admits a finite number of such curves that are regular (that is, curves u with Du surjective), but since this number can vary, the set of regular elements in ᏶λ0 has an infinite number of components. Lorek also characterises the other strata that occur. For example, the codimension 1 stratum consists of J such that all J -holomorphic A curves are isolated, but there is at least one where the kernel of Du has dimension 3 instead of 2. (Note that these two dimensions correspond to the reparametrization group, since Du is the full linearization, not just the normal component.)

140

DUSA MCDUFF

Similar remarks can be made about the case when Y → ) is a nontrivial bundle. In this case we can label the strata ᏶λk so that the J ∈ ᏶λk admit sections with selfintersection −2k + 1. Again, Theorem 1.2 holds, but Lemma 1.1 may not. When ) = S 2 , the homology class of the exceptional divisor is always represented so that λ ᏶λ = ᏶1 . When ) = T 2 , the homology class of the section of self-intersection +1 is λ always represented. Thus ᏶λ = ᏶−1 . Hence the analog of Lemma 1.1 holds in these two cases. Moreover, all embedded tori of self-intersection +1 are regular (by the same result in [HLS]), which may help in the application to Symp(Y ). We now state in detail the result for the nontrivial bundle Y → S 2 since this is used in [AM]. Here Y = CP 2 #CP 2 , and so every symplectic form on Y can be obtained from an annulus Ar,s = {z ∈ C2 : r ≤ |z| ≤ s} by collapsing the boundary spheres to S 2 along the characteristic orbits. This gives rise to a form ωr,s that takes the value π s 2 on the class L of a line and πr 2 on the exceptional divisor E. Let us write ωλ for the form ωr,s where π s 2 = 1 + λ, πr 2 = λ > 0. Then the class F = L − E of the fiber has size 1 as before, and ᏶λk , k ≥ 1 is the set of ωλ -compatible J for which the class E − (k − 1)F is represented. Theorem 1.6. When Y = CP 2 #CP 2 , the spaces ᏶λk are Fréchet submanifolds of of codimension 4k, and form the strata of a stratification of ᏶λ whose normal structure is independent of λ. Moreover, the normal link of ᏶λk+1 in ᏶λk is the lens space L(4k + 1, 1), k ≥ 1.

᏶λ

This paper is organized as follows. §2 describes the main ideas in the proof of Theorem 1.2. This relies heavily on the theory of stable maps, and for the convenience of the reader, we outline its main points. References for the basic theory are, for example, [FO], [LiT], and [LiuT1]. §3 contains a detailed calculation of the link of ᏶λ2 in ᏶λ . In particular, we discuss the topological structure of the space of degree 2 holomorphic self-maps of S 2 with up to two marked points, and of the canonical line bundle that it carries. Plumbing with the orbibundle LY → Y turns out to be a kind of orbifold blowing-up process (see §3.1). Finally, in §4 we work out the technical details of gluing that are needed to establish that the submanifolds ᏶λk do have a good normal structure. The basic method here is taken from McDuff and Salamon [MS] and Fukaya and Ono [FO]. Acknowledgements. I wish to thank Dan Freed, Eleni Ionel, and, particularly, John Milnor for useful discussions on various aspects of the calculation in §3, and Fukaya and Ono for explaining to me various details of their arguments. 2. Main ideas. We begin by proving Lemma 1.3 since this is elementary; then we describe the main points in the proof of Proposition 1.2. 2.1. The effect of increasing λ Proof of Lemma 1.3. Recall that F0 is a fixed fiber pt ×S 2 and that


141

᏶λk ᏺ(F0 ) = J ∈ ᏶λk : J = Jsplit near F0 .

We also use the space

᏶λk (F0 ) = J ∈ ᏶λk : J = Jsplit on T F0 .

Let Ᏺλ be the space of ωλ -symplectically embedded curves in the class F through a fixed point x0 . Because there is a unique J -holomorphic F -curve through x0 for each J ∈ ᏶ (see Lemma 1.1), there is a fibration ᏶λ (F0 ) −→ ᏶λ −→ Ᏺλ .

Since the elements of ᏶λ (F0 ) are sections of a bundle with contractible fibers, ᏶λ (F0 ) is contractible. Hence Ᏺλ is also contractible. By using the methods of Abreu [A], it is not hard to show that the symplectomorphism group Ᏻλ = Symp0 (X, ωλ ) of (X, ωλ ) acts transitively on Ᏺλ . Since the action of Ᏻλ on ᏶λ preserves the strata ᏶λk , it follows that the projection ᏶λk → Ᏺλ is surjective. Hence there are induced fibrations ᏶λk (F0 ) −→ ᏶λk −→ Ᏺλ .

This implies that the inclusion ᏶λk (F0 ) → ᏶λk is a weak homotopy equivalence. We now claim that the inclusion ᏶λk (ᏺ(F0 )) → ᏶λk (F0 ) is also a weak homotopy equivalence. To prove this, we need to show that the elements of any compact set ᏷ ⊂ ᏶λk (F0 ) can be altered near F0 by a homotopy so as to make them coincide with Jsplit . Since the set of tame, almost complex structures at a point is contractible, this is always possible in ᏶λ ; the difficulty here is to ensure that ᏷ remains in ᏶λk throughout the homotopy. To deal with this, we argue as follows. For each J ∈ ᏶λk (F0 ), let J denote the unique J -holomorphic curve in class A − kF . Then J meets F0 transversally at one point; call it qJ . For each J ∈ ᏷, move the curve J by a symplectic isotopy that fixes qJ to make it coincide in a small neighborhood of qJ with the flat section S 2 ×pt that contains qJ . (Details of a very similar construction can be found in [MP, Proposition 4.1.C].) Now lift this isotopy to ᏶λk . Finally, adjust the family of almost complex structures near F0 , keeping J holomorphic throughout. This proves (i). Statement (ii) is now easy. For any compact subset C of ᏶λ (ᏺ(F0 )), there is ε > 0 such that J = Jsplit on the ε-neighborhood ᏺε (F0 ) of F0 . Let ρ be a nonnegative 2-form supported inside the 2-disc of radius ε that vanishes near 0, and let π ∗ (ρ) denote its pullback to ᏺε (F0 ) by the obvious projection. Then every J that equals Jsplit on ᏺε (F0 ) is compatible with the form ωλ + κπ ∗ (ρ) for all κ > 0. Since ωλ + κπ ∗ (ρ) is isotopic to ωµ for some µ, there is a diffeomorphism φ of X that is isotopic to the identity and is such that φ ∗ (ωλ + κπ ∗ (ρ)) = ωµ . Moreover, because, by construction, π ∗ (ρ) = 0 near F0 , we can choose φ = Id near F0 . Hence the map J → φ ∗ (J ) takes ᏶λ (ᏺ(F0 )) to ᏶µ (ᏺ(F0 )). Clearly it preserves the strata ᏶k . 2.2. Stable maps. From now on, we drop λ from the notation, assuming that k < λ+1 as before. We study the spaces ᏶k and ᏶k by exploiting their relation to the

142

DUSA MCDUFF

corresponding moduli spaces of J -holomorphic curves in X. Definition 2.1. When k ≥ 1, ᏹk = ᏹ(A − kF, ᏶) is the universal moduli space of all unparametrized J -holomorphic curves in class A − kF . Thus its elements are equivalence classes [h, J ] of pairs (h, J ), where J ∈ ᏶ = ᏶λ , h is a J -holomorphic map S 2 → X in class A − kF , and where (h, J ) ≡ (h ◦ γ , J ) when γ : S 2 → S 2 is a holomorphic reparametrization of S 2 . Similarly, we write ᏹ0 = ᏹ(A, x0 , ᏶) for the universal moduli space of all unparametrized J -holomorphic curves in class A that go through a fixed point x0 ∈ X. Thus its elements are equivalence classes of triples [h, z, J ] with z ∈ S 2 , (h, J ) as before, h(z) = x0 , and where (h, z, J ) ∼ (h ◦ γ , γ −1 (z), J ) when γ : S 2 → S 2 is a holomorphic reparametrization of S 2 . The next lemma restates part (i) of Theorem 1.2. The proof uses standard Fredholm theory for J -holomorphic curves and is given in §4.1. The only noteworthy point is that when k > 0, the almost complex structures in ᏶k are not regular. In fact, the index of the relevant Fredholm operator is −(4k − 2). However, because we are in 4 dimensions, the Fredholm operator has no kernel, which is the basic reason why the space of J for which it has a solution is a submanifold of codimension 4k − 2. Lemma 2.2. For all k ≥ 0, the projection πk : ᏹk −→ ᏶k : [h, J ] −→ J is a diffeomorphism of the Fréchet manifold ᏹk onto the submanifold ᏶k of ᏶. This submanifold is an open subset of ᏶ when k = 0 and has codimension 4k −2 otherwise. Our tool for understanding the stratification of ᏶ by the ᏶k is the compactification ᏹ(A − kF, ᏶) of ᏹ(A − kF, ᏶) that is formed by J -holomorphic stable maps. For the convenience of the reader, we recall the definition of stable maps with p marked points. We always assume the domain ) to have genus 0. Therefore it is a connected union ∪m i=0 )i of Riemann surfaces, each of which has a given identification with the standard sphere (S 2 , j0 ). (Note that we consider ) to be a topological space: the labelling of its components is a convenience and not part of the data.) The intersection pattern of the components can be described by a tree graph with m+1 vertices (one for each component of )) that are connected by an edge if and only if the corresponding components intersect. No more than two components meet at any point. Also, there are p-marked points z1 , . . . , zp placed anywhere on ) except at an intersection point of two components. (Such pairs (), z1 , . . . , zp ) = (), z) are called semistable curves.) Now consider a triple (), h, z) where h : ) → X is such that h∗ ([)]) = B and where the following stability condition is satisfied: the restriction hi of the map h to )i is nonconstant unless )i contains at least three special points. (By definition, special points are either points of intersection with other components or marked points.) A stable map σ = [), h, z] in class B ∈ H2 (X, Z) is an equivalence class of such triples, where (), h, z ) ≡ (), h ◦ γ , z) if there is an element γ of the group Aut()) of all holomorphic self-maps of ) such that γ (zi ) = zi for all i. For example, if ) has


143

only one component and there are no marked points, then (), h) ≡ (), h ◦ γ ) for all γ ∈ Aut(S 2 ) = PSL(2, C). Thus stable maps are unparametrized. We may think of the triple (), h, z) as a parametrized stable map. We almost always consider only stable maps that are J -holomorphic for some J . If necessary, we include J in the notation, writing elements as σ = [), h, z, J ], but often J is understood. Note that some stable maps σ = [), h, z, J ] have a nontrivial reparametrization group 5σ . Given a representative (), h, z, J ) of σ , this group may be defined as 5σ = γ ∈ Aut()) : h ◦ γ = h, γ (zi ) = zi , 1 ≤ i ≤ p . It is finite because of the stability condition. The points where this reparametrization group 5σ is nontrivial are singular or orbifold points of the moduli space. Here is an example where it is nontrivial. Example 2.3. Let ) have three components, with )2 and )3 both intersecting )1 , and let z1 be a marked point on )1 . Then we can allow h1 to be constant without violating stability. If, in addition, h2 and h3 have the same image curve, there is an automorphism that interchanges )2 and )3 . Since nearby stable maps do not have this extra symmetry, [), h, z1 ] is a singular point in its moduli space. However, because marked points are labelled, there is no such automorphism if we put one marked point z2 on )2 and another z3 at the corresponding point on )3 , that is, so that h2 (z2 ) = h3 (z3 ). One can also destroy this automorphism by adding just one marked point z0 to [), h, z1 ] anywhere on )2 or )3 . Definition 2.4. For k ≥ 0, we define ᏹ(A − kF, J ) to be the space of all J holomorphic stable maps σ = [), h, J ] in class A − kF . Further, given any subset ᏷ of ᏶, we write ᏹ(A − kF, ᏷) = ∪J ∈᏷ ᏹ(A − kF, J ). p

It follows from the proof of Lemma 1.1 that the domain ) = ∪i=0 )i of σ ∈ ᏹ(A − kF, ᏶) contains a unique component that is mapped to a curve in some class A−mF , where m ≥ k. We call this component the stem of ) and label it )0 . Thus ᏹ(A − kF, ᏶m ) is the moduli space of all curves whose stems lie in class A − mF . Note that ) − )0 has a finite number of connected components called branches. If h0 is parametrized as a section, a branch Bw that is attached to )0 = S 2 at the point w is mapped into the fiber πJ−1 (w). In particular, distinct branches are mapped to distinct fibers. The moduli spaces ᏹ(A − kF, J ) and ᏹ(A − kF, ᏶) have natural stratifications in which each stratum is defined by fixing the topological type of the pair (), z) and the homology classes [h∗ ()i )] of the components. Observe that the class A − mF of the stem is fixed on each stratum ᏿ in ᏹ(A − kF, ᏶). Hence there is a projection ᏿ −→ ᏶m ,

whose fiber at J ∈ ᏶m is some stratum of ᏹ(A − kF, J ). Usually, in order to have a moduli space with a nice structure, one needs to consider perturbed J -holomorphic

144

DUSA MCDUFF

curves. But, because we are working with genus 0 curves in dimension 4, the work of Hofer-Lizan-Sikorav [HLS] shows that all J -holomorphic curves are essentially regular. In particular, all curves representing some multiple mF of the fiber class are regular. Therefore, each stratum of ᏹ(A − kF, J ) is a (finite-dimensional) manifold. The following result is an immediate consequence of Lemma 2.2. Lemma 2.5. Each stratum ᏿ of ᏹ(A − kF, ᏶) is a manifold, and the projection ᏿ → ᏶m is a locally trivial fibration. Definition 2.6. When k ≥ 1, we set ᏹk = ᏹ(A−kF, ᏶). Further, ᏹ0 = ᏹ(A, x0 , ᏶) is the space of all stable maps [), h, z, J ] where [), h, z] is a J -holomorphic stable map in class A with one marked point z such that h(z) = x0 . In the next section we show how to fit the strata of ᏹk together by gluing to form an orbifold structure on ᏹk itself. 2.3. Gluing. In this section we describe the structure of a neighborhood ᏺ(σ ) ⊂ ᏹk of a single point σ ∈ ᏹ(A − kF, ᏶m ). Suppose that σ = [), h, J ], and order the

components )i of ) so that )0 is the stem and so that the union ∪i≤ )i is connected for all . Then each )i , i > 0, is attached to a unique component )ji , ji < i by identifying some point wi ∈ )i with a point zi ∈ )ji . At each such intersection point, consider the gluing parameter: ai ∈ Twi )i ⊗C Tzi )ji . The basic process of gluing allows one to resolve the singularity of ) at the node wi = zi by replacing the component )i by a disc attached to )ji − nbhd(zi ) and suitably altering the map h. As we now explain, there is a two-dimensional family of ways of doing this that is parametrized by (small) ai . Proposition 2.7. Each σ ∈ ᏹ(A−kF, ᏶m ) has a neighborhood ᏺ(σ ) in ᏹk that is a product ᐁ᏿ (σ )×(ᏺ(Vσ )/ 5σ ) where ᐁ᏿ (σ ) ⊂ ᏹ(A−kF, ᏶m ) is a small neighborhood of σ in its stratum ᏿ and where ᏺ(Vσ ) is a small 5σ -invariant neighborhood of 0 in the space of gluing parameters, Vσ = Twi )i ⊗C Tzi )ji . i>0

Proof. The proof is an adaptation of standard arguments in the theory of stable maps. The only new point is that the stem components are not regular, so that when one does any gluing that involves this component, one has to allow J to vary in a normal slice ᏷J to the submanifold ᏶m at J . This analytic detail is explained in §4.2. What we do here is describe the topological aspect of the proof. First of all, let us describe the process of gluing. Given a ∈ Vσ , the idea is first to construct a (parametrized) approximately J -holomorphic stable map ()a , ha , J ) on a glued domain )a and then to perturb ha and J , using a Newton process, to a Ja holomorphic map ha : )a → X in ᏹ(A−kF, ᏷J ). We describe the first step in some


145

detail here since it is used in §3. The analytic arguments needed for the second step are postponed to §4. It is important to note that one must choose a parametrization of the stable map before one can glue. We denote gluing on the parametrized level by Ᏻ˜ , and we write Ᏻ for the appropriate induced map on the unparametrized level. Thus ˜ (), h, J ; a) = )a , ha , Ja , Ᏻ Ᏻ(), h, J ; a) = )a , ha , Ja . The glued domain )a is constructed as follows. For each i such that ai = 0, cut out a small open disc IntDwi (ri ) in )i centered at wi and a similar disc IntDzi (ri ) in Bji where ri2 = |ai |, and then glue the boundaries of these discs together with a twist prescribed by the argument of ai . The Riemann surface )a is the result of performing this operation for each i with ai = 0. (When ai = 0, one simply leaves the component )i alone.) To be more precise, consider gluing w ∈ )0 to z ∈ )1 . Take a Kähler metric on )0 that is flat near w and identify the disc Dw (r) isometrically with the disc of radius r in the tangent space Tw = Tw ()0 ) via the exponential map. Take a similar metric on ()1 , z). Then the gluing ∂Dw (r) → ∂Dz (r) may be considered as the restriction of the map :a : Tw − {0} −→ Tz − {0} that is defined for x ∈ Tw by the requirement x ⊗ :a (x) = a,

x ∈ Tw .

Tz with C, :a is given by the Thus, with respect to chosen identifications of Tw and√ formula x → a/x and so takes the circle of radius r = |a| into itself. This describes the glued domain )a as a point set. It remains to put a metric on )a in order to make it a Riemann surface. By hypothesis, the original metrics on )0 , )1 are flat near w and z and so may be identified with the flat metric |dx|2 on C. Since |a|2 :a∗ |dx|2 = 4 |dx|2 , |x| :a (|dx|2 ) = |dx|2 on the circle |x| = r. Hence we may choose a function χr : (0, ∞) → (0, ∞) so that the metric χr (|x|)|dx|2 is invariant by :a and so that χr (s) = 1 when s > (1 + ε)r. Then patch together the given metrics on )0 − Dz (2r) and )1 − Dz (2r) via χr (|x|)|dx|2 . In §3 we need to understand what happens as a rotates around the origin. It is not hard to check that if we write aθ = eiθ aw ⊗ az , where aw ∈ Tw , az ∈ Tz are fixed, then :aθ eiθ pw = pz for all θ, where pw ∈ ∂Dw (r), pz ∈ ∂Dz (r). The next step is to define the approximately holomorphic map (or pregluing) ha : )a → X for sufficiently small |a|. The map ha equals h away from the discs

146

DUSA MCDUFF

Dwi (ri ), Dzi (ri ), and elsewhere it is defined by using cutoff functions that depend only on |a|. To describe the deformation of ha to a holomorphic map, one needs to use analytical arguments. Hence further details are postponed until §4. We are now in a position to describe a neighborhood of σ . It is convenient to think of Vσ as the direct sum Vσ ⊕Vσ where Vσ consists of the summands Twi )i ⊗C Tzi )ji with ji = 0 and Vσ of the rest. Note that the obvious action of 5σ on Vσ preserves this splitting. (It is tempting to think that the induced action on Vσ is trivial since the elements of 5σ act trivially on the stem. However, this need not be so since they may rotate branch components that are attached to the stem.) If we glue at points parametrized by a ∈ Vσ , then the corresponding curves lie in some branch and are regular. Hence the result of gluing is a J -holomorphic curve (i.e., there is no need to perturb J ). Further, because the gluing map Ᏻ˜ is 5σ -equivariant, a normal slice in ᏹ(A − kF, ᏶m ) to the stratum ᏿ at σ has the form ᏺ Vσ ᐁ (σ ) = ᐁ᏿ (σ ) × , 5σ where ᏺ(V ) denotes a neighborhood of 0 in the vector space V . When we glue with elements from Vσ , the homology class of the stem changes, and so the result cannot be J -holomorphic since J ∈ ᏶m . We show in Proposition 4.4 that if ᏷J is a normal slice to the submanifold ᏶m at J , then for sufficiently small a ∈ Vσ , the approximately holomorphic map ha : )a → X deforms to a unique Ja -holomorphic map Ᏻ˜ (hσ , a) with Ja ∈ ᏷J . Therefore, for each element σ1 = [), h1 , J1 ] ∈ ᐁ (σ ), there is a homeomorphism from some neighborhood ᏺ(Vσ1 ) onto a slice in ᏹ(A−kF, ᏷J1 ) that meets ᏹ(A−kF, ᏶m ) transversally at σ1 . Moreover, if ᐁ (σ ) is sufficiently small, the spaces Vσ1 can all be identified with Vσ , and it follows from the proof of Proposition 4.4 that the neighborhoods ᏺ(Vσ1 ) can be taken to have uniform size and so may all be identified. Hence the neighborhood ᏺ(σ ) projects to ᐁ (σ ) with fiber at σ1 equal to ᏺ(Vσ )/ 5σ1 . In general, the groups 5σ1 are subgroups of 5σ that vary with σ1 ; in fact, they equal the stabilizer of the corresponding gluing parameter a ∈ Vσ . However, since elements of ᐁ᏿ (σ ) lie in the same stratum, the group 5σ itself does not vary as σ varies in ᐁ᏿ (σ ). It is now easy to check that the composite map ᏺ(σ ) −→ ᐁ (σ ) −→ ᐁ᏿ (σ )

has fiber ᏺ(Vσ )/ 5σ as claimed. 2.4. Moduli spaces and the stratification of ᏶. Since each stable J -curve in class A−kF has exactly one component in some class A−mF with m ≥ k, the projection πk : ᏹ(A − kF, ᏶) → ᏶ has image ᏶k . Consider the inverse image ᏹ A − kF, ᏶m = πk−1 ᏶m . The next result shows that we can get a handle on the structure of ᏶k by looking at


147

the spaces ᏹ(A − kF, ᏶m ). Proposition 2.8. When k > 0, the projection πk : ᏹ A − kF, ᏶m −→ ᏶m is a locally trivial fibration whose fiber ᏲJ (m − k) at J is the space of all stable J curves [), h] in class A−kF that have as one component the unique J -holomorphic curve J in class A − mF . In particular, ᏲJ (m − k) is a stratified space with strata that are manifolds of (real) dimension less than or equal to 4(m − k). Its diffeomorphism type depends only on m − k. Proof. Let us look at the structure of ᏲJ (m − k) = πk−1 (J ). The stem of each element [), h, J ] ∈ ᏲJ (m−k) is mapped to the unique J -curve J in class A−mF . Fix this component further by supposing that it is parametrized as a section of the fibration πJ : X → S 2 (where πJ is as in Lemma 1.1). We may divide the fiber ᏲJ (m − k) into disjoint sets ᐆᏰ,J , each parametrized by a fixed decomposition Ᏸ of m − k into a sum d1 + · · · + dp of unordered positive numbers. The elements of ᐆᏰ,J are those with p branches Bw1 , . . . , Bwp where h∗ [Bwi ] = di [F ]. Thus ᐆᏰ,J maps onto the configuration space of p distinct (unordered) points in S 2 labelled by the positive integers d1 , . . . , dp with sum m − k. Moreover, this map is a fibration with fiber equal to the product P

ᏹ0,1 S 2 , q, di , i=1

where ᏹ0,1 (S 2 , q, d) is the space of J -holomorphic stable maps into S 2 of degree d and with one marked point z such that h(z) = q. (This point q is where the branch is attached to J .) According to the general theory, ᏹ0,1 (S 2 , q, d) is an orbifold of real dimension 4(d − 1). It follows easily that ᐆᏰ,J is an orbifold of real dimension 4(m − k) − 2p. It remains to be understood how the different sets ᐆᏰ,J fit together, that is, what happens when two or more of the points wi come together. This may be described by suitable gluing parameters as in Proposition 2.7. The result follows. (For more details, see any reference on stable maps, e.g., [FO], [LiT], and [LiuT1]. An example is worked out in §3.2.4 below.) Note. For an analogous statement when k = 0, see Proposition 3.5. Next, we describe the structure of a neighborhood of ᏹ(A − kF, ᏶m ) in ᏹk = ᏹ(A − kF, ᏶). We write ᐆJ for the fiber ᏲJ (m − k) of πk that was considered above and set ᐆ=

J ∈᏶m

ᐆJ ,

ᐆᏰ =

ᐆᏰ,J .

J ∈᏶m

(The letter ᐆ is used here because ᐆ is the zero section of the space of gluing parameters ᐂ constructed below.) Consider an element σ = [), h, J ] that lies in a

148

DUSA MCDUFF

substratum ᐆ᏿ of ᐆᏰ where Ᏸ = d1 + · · · + dp . Then ) has p branches B1 , . . . , Bp that are attached at the distinct points w1 , . . . , wp ∈ )0 . Let zi be the point in Bi that is identified with wi ∈ )0 , and define Vσ =

p

Tzi Bi ⊗C Twi )0 .

i=1

As explained in Proposition 2.7, the gluing parameters a ∈ Vσ (when quotiented out by 5σ ) parametrize a normal slice to ᐆᏰ at σ . (Note that previously Vσ was called Vσ .) We now want to show how to fit these vector spaces together to form the fibers of an orbibundle2 over ᐆᏰ . Here we must incorporate twisting that arises from the fact that gluing takes place on the space of parametrized stable maps. Since this is an important point, we dwell on it at some length. For the sake of clarity, in the next few paragraphs we denote parametrized stable maps by σ˜ = (), h) and the usual (unparametrized) maps by σ = [), h]. Further, 5σ˜ denotes the corresponding realization of the group 5σ as a subgroup of Aut()). Recall that X is identified with S 2 ×S 2 in such a way that the fibration πJ : X → S 2 , whose fibers are the J -holomorphic F -curves, is simply given by projection onto the first factor. Hence each such fiber has a given identification with S 2 . Further, we assume that the stem hσ,0 : )σ,0 → J is parametrized as a section z → (z, ρ(z)). Hence we only have to choose parametrizations of each branch. Since each branch component has at least one special point, its automorphism group either is trivial or has the homotopy type of S 1 . Let Aut ()) be the subgroup of Aut()) consisting of automorphisms that are the identity on the stem. Then the identity component of Aut ()) is homotopy equivalent to a torus T k(᏿) . (Here ᏿ is the label for the stratum containing σ .) Let g be a 5σ˜ -invariant metric on the domain ) that is also invariant under some action of the torus T k(᏿) . Definition 2.9. The group AutK ()) is defined to be the (compact) subgroup of the isometry group of (), g) generated by 5σ˜ and T k(᏿) . Note that 5σ˜ is the semidirect product of a subgroup 5σ˜ of T k(᏿) with a subgroup 5σ˜ that permutes the components of each branch. Further, Aut K ()) is a deformation retract of the subgroup p−1 (5σ˜ ) of Aut()), where we consider 5σ˜ as a subgroup of π0 (Aut())), and p : Aut()) −→ π0 Aut()) is the projection. For a further discussion, see §4.2.4. Let us first consider a fixed J ∈ ᏶k . It follows from the above discussion that on each stratum ᐆ᏿,J , there is a principal bundle rank k orbibundle π : E → Y over an orbifold Y has the following local structure. Suppose that σ ∈ Y has local chart U ⊂ U˜ / 5σ where the uniformizer U˜ is a subset of Rn . Then π −1 (U ) has the form U˜ × Rk / 5σ where the action of 5σ on Rn × Rk lifts that on Rn and is linear on Rk . There is an obvious compatibility condition between charts: see [FO, §2]. 2A


149

para

ᐆ᏿,J −→ ᐆ᏿,J

with fiber Aut K ()) such that the elements of ᐆ᏿,J are parametrized stable maps σ˜ = (), h). Since the space Vσ˜ of gluing parameters at σ˜ is made from tangent spaces to ), there is a well-defined bundle para

para

para

ᐂ᏿,J −→ ᐆ᏿,J

with fiber Vσ˜ . Further, the action of the reparametrization group Aut K ()) lifts to para para ᐂ᏿,J , and we define ᐂ᏿,J to be the quotient ᐂ᏿,J / Aut K ()). Thus there is a commutative diagram para / ᐂ᏿,J ᐂ ᏿,J

para ᐆ᏿,J

/ ᐆ᏿,J,

where the right-hand vertical map is an orbibundle with fiber Vσ˜ / 5σ˜ . Now consider the space ᐆᏰ,J = ∪᏿⊂Ᏸ ᐆ᏿,J . The local topological structure of ᐆᏰ,J is given by gluing parameters as in Proposition 2.7. Observe that every J is regular for the branch components so that the necessary gluing operations can be performed para para keeping J fixed. The spaces ᐆᏰ,J and ᐂᏰ,J are defined similarly, and clearly there para para is a vector bundle ᐂᏰ,J → ᐆᏰ,J . We want to see that the union ᐂᏰ,J = ᐂ᏿,J ᏿∈Ᏸ

has the structure of an orbibundle over ᐆᏰ,J . The point here is that the groups AutK ()) change dimension as σ˜ moves from stratum to stratum. Hence we need to see that the local gluing construction that fits the different strata in ᐂᏰ,J together is compatible with the group actions. We show in §4.2.4 that the gluing map Ᏻ can be defined at the point σ˜ to be AutK ())-invariant. More precisely, )a , Ᏻ˜ (hσ , a) = )θ·a , Ᏻ˜ hσ ◦ θ −1 , θ · a , where Ᏻ˜ (hσ , a) is the result of gluing the map hσ with parameters a. In the situation considered here, we divide the set of gluing parameters at σ˜ into two, and we write a = (ab , as ) where ab are the gluing parameters at intersections of branch components para and as are those involving the stem component. As hσ moves within ᐆᏰ,J , we glue along ab , considering as to be part of the p-dimensional fiber Vσ . (Recall that p is the number of elements of the decomposition Ᏸ = d1 + · · · + dp .) Moreover, if σ˜ = ()a , Ᏻ˜ (hσ , ab )), Lemma 4.9(ii) shows that the parametrized gluing map Ᏻ˜ b

can be constructed to be compatible with the actions of the groups Aut K ()) and

150

DUSA MCDUFF

AutK () ) on the fibers Vσ and Vσ of ᐂᏰ,J at σ˜ , σ˜ . It follows without difficulty that the quotient ᐂᏰ,J −→ ᐆᏰ,J para

is an orbibundle with generic fiber Cp . Finally, one forms spaces ᐂJ = ᐂᏰ,J ,

ᐂ=

ᐂJ ,

J ∈᏶k

Ᏸ

whose local structure is also described by appropriate gluing parameters as above. Forgetting the gluing parameters gives projections ᐂJ −→ ᐆJ = ᏲJ (m − k),

ᐂ −→ ᐆ = Ᏺ(m − k),

and ᐆJ , ᐆ embed in ᐂJ and ᐂ as the zero sections. The map ᐂJ → ᐆJ preserves the stratifications of both spaces. However, it is no longer an orbibundle since the dimension of the fiber Vσ depends on Ᏸ. In fact, the way that the different sets ᐂᏰ,J are fitted together is best thought of as a kind of plumbing; see §3.2.4. Example 2.10. Everything is greatly simplified when m−k = 1. Here there is only one decomposition Ᏸ and the space ᐆᏰ,J consists of just one stratum diffeomorphic to para S 2 . Moreover, the bundle ᐆᏰ,J → ᐆᏰ,J has a section with the following description. Choose J ∈ ᏶k+1 so that πJ : X → S 2 is the standard projection onto the first factor and so that the graph h0 of the map ρk : S 2 → S 2 of degree −(k +1) is J -holomorphic. para Let )0 , )1 be two copies of S 2 , and for each w ∈ S 2 , define ()w , hw ) ∈ ᐆᏰ,J by )w = )0 ∪w=ρk (w) )1 , hw |)0 = h0 ,

hw |)1 : z −→ (w, z).

Hence, in this case, ᐂJ is a complex line bundle over ᐆJ = S 2 . To calculate its Chern class, observe that ᐂJ can be identified with the space −k−1 Tρk (w) )1 ⊗ Tw ()0 ) = T S 2 ⊗ T S2, w∈S 2

and so ᐂJ has Chern class −2k. The following result is proved in §4. Proposition 2.11. There is a neighborhood ᏺᐂ (ᐆ) of ᐆ in ᐂ and a gluing map Ᏻ : ᏺᐂ (ᐆ) −→ ᏹ(A − kF, ᏶)

that maps ᏺᐂ (ᐆ) homeomorphically onto a neighborhood of ᏹ(A − kF, ᏶m ) in ᏹ(A − kF, ᏶). It follows from the construction of Ᏻ : ᏺᐂ (ᐆ) → ᏹ(A − kF, ᏶m ) outlined in Proposition 2.7 that the stem of the glued map Ᏻ(σ, a) lies in the class


151

A− m− ni F, i

where the indices i label the branches Bi of σ and ni is defined as follows. If ai = 0, then ni = 0. Otherwise, if )ji is the component of Bi that meets )0 , then ni is the multiplicity of hji , that is, [h()ji )] = ni F . Let ᏺp denote the set of all elements (σ, a) ∈ ᏺᐂ (ᐆ) such that the stem of the glued map lies in class A − pF . In other words, −1 ᏺp = πk ◦ Ᏻ ᏶p . Clearly, ᏺp is a union of strata in the stratified space ᏺᐂ (ᐆ). Further, when k > 0, the map πk : Ᏻ(ᏺp ) → ᏶p is a fibration with fiber Ᏺ(p − k). The next proposition follows immediately from Proposition 2.8. Proposition 2.12. The link ᏸm,k is the finite-dimensional stratified space obtained from the link of ᐆJ in ᐂJ by collapsing the fibers of the projections ᐂJ ∩ ᏺp → ᏶p to single points. Proof of Proposition 1.4. We have to show that the link ᏸk+1, k is the lens space L(2k, 1). We saw in Example 2.10 that ᐂJ is a line bundle with Chern class −2k. In this case, there is only one nontrivial stratum in ᏺᐂ (ᐆ), namely, ᏺk , which is the complement of the zero section. Moreover, the map πk ◦ Ᏻ is clearly injective. Hence, by the above lemma, ᏸk+1, k is simply the unit sphere bundle of ᐂJ and so is a lens space as claimed. 3. The link ᏸ2,0 of ᏶2 in ᏶0 . In this section, we illustrate Proposition 2.12 by calculating the link ᏸ2,0 . We know from Lemma 2.2 that ᏸ2,0 = S 5 . The general theory of §2 implies that ᏸ2,0 can be obtained from the link ᏸᐆ of the zero section ᐆJ in the stratified space ᐂJ of gluing data by collapsing certain strata. When looked at from this point of view, the S 5 appears in quite a complicated way, which was described in Theorem 1.5. We begin here by explaining the plumbing construction and then discussing how this relates to ᏸᐆ . 3.1. Some topology. Recall that S(LP ) → ᏼ(ᏻ(k)⊕ ᏻ(m)) is the unit circle bundle of the canonical line bundle LP over the projectivization ᏼ(ᏻ(k) ⊕ ᏻ(m)). Lemma 3.1. The bundle S(LP ) → ᏼ(ᏻ(−1) ⊕ C) can be identified with the pullback of the canonical circle bundle S(Lcan ) → CP 2 over the blow-down map CP 2 #CP 2 → CP 2 . Proof. It is well known that ᏼ(ᏻ(−1) ⊕ C) can be identified with CP 2 #CP 2 → CP 2 . Indeed, the section S− = ᏼ({0} ⊕ C) has self-intersection −1, while S+ = ᏼ(ᏻ(−1) ⊕ {0}) has self-intersection 1. Further, the circle bundle S(LP ) is trivial over S− and has Euler class −1 over S+ and over the fiber class. The result follows.

152

DUSA MCDUFF

The space we are interested in is formed by plumbing a rank 2 bundle E → S 2 to a line bundle L → Y where dim(Y ) = 4. This plumbing E ≡ ! L is the space obtained from the unit disc bundles D(E) → S 2 and D(L) → Y by identifying the inverse images of discs D 2 , D 4 on the two bases in the obvious way: The disc fibers of D(E) → S 2 are identified with flat sections of D(L) over D 4 , and flat sections of D(E) over D 2 are identified with fibers of D(L). There is a corresponding plumbing S(E) ≡ ! S(L) of the two sphere bundles S 3 → S(E) → S 2 and S 1 → S(L) → Y , obtained by cutting out the inverse images of open discs in the two bases and appropriately gluing the boundaries. The resulting space S(E) ≡ ! S(L) is the link of the core S 2 ∪ Y in the plumbed bundle E ≡ ! L. Lemma 3.2. Let Lcan → CP 2 be the canonical line bundle, and let E = ᏻ(k) ⊕ ᏻ(m). Then E ≡ ! Lcan may be identified with the blow-up of ᏻ(k + 1) ⊕ ᏻ(m + 1) at a point on its zero section. Hence S(E) ≡ ! S(Lcan ) = S ᏻ(k + 1) ⊕ ᏻ(m + 1) . Proof. First consider the structure of the blow-up C3 of C3 = C×C2 at the origin. 2 The fibration π : C × C → C induces a fibration π : C3 −→ C. π −1 (0) Clearly, the inverse image π −1 (z) of each point z = 0 is a copy of C2 , while is the union of the exceptional divisor together with the set of lines in the original fiber π −1 (0). Let λ be the line in C × C2 through the origin and the point (1, a, b). Lift λ to the blow-up and consider its intersection with π −1 S 1 = π −1 S 1 ⊂ C × C2 , where S 1 is the unit circle in C. This intersection consists of the points (eit , eit a, eit b); hence it is these circles (rather than the circles (eit , a, b)) that bound discs in the blow-up. Therefore, if we think of the blow-up C3 as the plumbing of the bundle π : 2 C × C → C with Lcan , the original trivialization of π differs from the trivialization (or product structure) near π −1 (0) that is used to construct the plumbing. Now recall that ᏻ(k) = D + × C ∪α D − × C, where D + , D − are 2-discs, with D + oriented positively and D − oriented negatively, and where the gluing map α is given by α : ∂D + × C −→ ∂D − × C : eit , w −→ eit , e− ikt w . It follows easily that the blow-up of D(ᏻ(k + 1) ⊕ ᏻ(m + 1)) at a point on its zero section is obtained by plumbing the disc bundle D(ᏻ(k) ⊕ ᏻ(m)) with D(Lcan ). This proves the first statement. The second statement is then immediate. We are interested in plumbing not with Lcan → CP 2 but with a particular singular


153

line bundle (or orbibundle) LY → Y . This means that the unit circle bundle S(LY ) → Y is a Seifert fibration with a finite number of singular (or multiple) fibers. In our case, there is an S 1 -action on S(LY ) such that the fibers of the map S(LY ) → Y are the S 1 -orbits. In fact, we can identify S(LY ) with S 5 in such a way that the S 1 action is θ · (x, y, z) = eiθ x, eiθ y, e2iθ z , x, y, z ∈ C. Thus there is one singular fiber that goes through the point (0, 0, 1). All other fibers F are regular. For each such F , there is a diffeomorphism of S 5 that takes F to the circle γ0 = (eiθ , eiθ , 0). Identify a neighborhood of γ0 with S 1 × D 4 in such a way that S 5 = S 1 × D4 ∪ D2 × S 3, with the identity map of S 1 × S 3 as gluing map. Then, in these coordinates near γ0 , the fibers of S(LY ) are (diffeomorphic to) the circles iθ e 0 1 4 . γx = θ, Aθ (x) ∈ S × D : Aθ = 0 e2iθ By way of contrast, the fibers of S 5 with the Hopf fibration have neighborhoods fibered by the circles iθ e 0 1 4 γx = θ, Aθ (x) ∈ S × D : Aθ = . 0 eiθ The next result shows that plumbing with S(L) is a kind of twisted blow-up. Proposition 3.3. Let LY → Y be the orbibundle described in the previous paragraph. Then the manifold obtained by plumbing S(ᏻ(k) ⊕ ᏻ(m)) with a regular fiber of S(LY ) is diffeomorphic to S(ᏻ(k + 2) ⊕ ᏻ(m + 1)). Proof. We may think of plumbing as the result of a surgery that matches the flat circles S 1 × pt in the copy of S 1 × S 3 in S(ᏻ(k) ⊕ ᏻ(m)) with the circles γx in the neighborhood of a regular fiber γ0 of S(LY ). We would get the same result if we matched the circles −iθ e 0 δx = θ, Aθ (x) ∈ S 1 × S 3 : Aθ = 0 1 in S 1 × S 3 ⊂ S(ᏻ(k) ⊕ ᏻ(m)) with the circles γx in the standard (Hopf) S 5 . But if we trivialize the boundary of S(ᏻ(k) ⊕ ᏻ(m)) − D 2 × S 3 by the circles δx , we get the same result as if we trivialized the boundary of S(ᏻ(k + 1) ⊕ ᏻ(m)) − D 2 × S 3 in the usual way by flat circles. Thus S ᏻ(k) ⊕ ᏻ(m) ≡ ! S(LY ) = S ᏻ(k + 1) ⊕ ᏻ(m) ≡ ! S(Lcan ) = S ᏻ(k + 2) ⊕ ᏻ(m + 1) .

154

DUSA MCDUFF

There is a question of orientation here: Do we have to add or subtract 1 from k to compensate for the extra twisting in S(LY )? We can check that it is correct to add 1 by using the present approach to give an alternate proof of the previous lemma. For if we completely untwisted the circles in the neighborhood of γ0 (thereby increasing the twisting of the other side by an additional 2), we would be doing the trivial surgery in which the attaching map is the identity. Note also that because the sum ᏻ(k) ⊕ ᏻ(m) depends only on k + m, we could equally well have put the extra twist on the other factor. 3.2. Structure of the pair (ᐂJ , ᐆJ ). Our aim is to prove the following proposition where ᐂJ is the space of gluing parameters for a fixed J ∈ ᏶2 that describes the link of the space of (A − 2F )-curves in the space of (pointed) A-curves. Proposition 3.4. The link ᏸᐆ of the zero section ᐆJ in the stratified space ᐂJ is constructed by plumbing S(ᏻ(−3) ⊕ ᏻ(−1)) to S(LY ). Hence ᏸᐆ = S ᏻ(−1) ⊕ C . We are now not quite in the situation described in Proposition 2.8 because we are including the open stratum ᏶0 of ᏶. This means that we have to replace the space ᏹk = ᏹ(A − kF, ᏶) by a space ᏹ0 of curves of class A that go through the fixed point x0 . Since we are interested in working out the structure of the fiber of the projection ᏹ0 → ᏶ at a point J ∈ ᏶2 , we choose x0 so that it does not lie on the unique J -holomorphic (A − 2F )-curve J , and then we define ᏹ0 to be the space ᏹ(A, x0 , ᏶) in Definition 2.6. Let π0 denote the projection π0 : ᏹ0 −→ ᏶ and set ᏹ0 (᏶m ) = π0−1 (᏶m ) as before. It is not hard to see that the following analog of Proposition 2.8 holds. Proposition 3.5. (i) Let J ∈ ᏶m be any almost complex structure such that the unique J -holomorphic (A−mF )-curve J does not go through x0 . Then the projection πk : ᏹ0 (᏶m ) −→ ᏶m is a locally trivial fibration near J whose fiber ᏲJ (0, m) is the space of all stable J -curves [), u] in class A that have J as one component and go through x0 . In particular, ᏲJ (0, m) is a stratified space whose strata are orbifolds of (real) dimension less than or equal to 4m − 2. (ii) The singular fibers of πk : ᏹ0 (᏶m ) → ᏶m occur at points J for which x0 ∈ J . For such J , πk−1 (J ) can be identified with the space ᏲJ (m) described in Proposition 2.8. As before, we now construct a pair (ᐂJ , ᐆJ ) that describes a neighborhood of ᏹ0 (᏶m ) in ᏹ0 . We concentrate on the case m = 2 and suppose that x0 ∈ / J . We

further normalize J by requiring that the projection πJ along the J -holomorphic


)3

)2 )3

155 )2

z0

z0

a0 b

b

)1

a1

a w )0

σ∗ in ᐆ1,J

)0

σ in ᐆ2,J Figure 1

F -curves is simply the projection onto the first factor S 2 . We write q0 = πJ (x0 ). 3.2.1. The bundle ᐂ2,J → ᐆ2,J . Observe first that ᐆJ has two subsets: ᐆ1,J consisting of all stable A-maps [), z0 , h] that are the union of the (A − 2F )-curve J with a double covering of the fiber F0 through x0 , and ᐆ2,J consisting of all stable A-maps [), z0 , h] that are the union of J with two distinct fibers. We call the ᐆi,J strata. This is accurate as far as ᐆ2,J is concerned, but strictly speaking, ᐆ1,J is a union of strata. (Recall that the strata are determined by the topological type of the marked domain [), z0 ], and the homology class of the images of its components under h.) Let us first consider ᐆ2,J . Since h(z0 ) = x0 always, one of the two fibers has to be F0 , and the other moves. Therefore, the stratum ᐆ2,J maps onto S 2 − {q0 }. It is convenient to compactify ᐆ2,J by adding a point σ∗ that projects to q0 . The domain ) of σ∗ has four components with )0 , )2 , )3 , all meeting )1 and a marked point z0 ∈ )3 (see Figure 1). The map h0 : )0 → J parametrizes J as a section, h1 takes )1 onto the point F0 ∩ J , and h2 , h3 have image F0 with h3 (z0 ) = x0 . The argument of Example 2.10 gives the following result, where ᐂ2,J is extended over σ∗ using the trivialization given by the gluing coordinates a, b of Figure 1. Lemma 3.6. The space ᐂ2,J of gluing parameters over ᐆ2,J ∪ {σ∗ } = S 2 is the bundle ᏻ(−2) ⊕ C. The structure of the bundle ᐂ1,J → ᐆ1,J is more complicated, and we start by looking at its base. 3.2.2. The stratum ᐆ1,J . Let p0 , p1 be two distinct points on F0 ≡ S 2 , with p0 = x0 and p1 = J ∩ F0 . Then ᐆ1,J is the orbifold ᐆ1,J = Y = ᏹ0,2 S 2 , p0 , p1 , 2 of all stable maps to S 2 with two marked points z0 , z1 that are in the class 2[S 2 ] and

156

DUSA MCDUFF

are such that h(z0 ) = p0 , h(z1 ) = p1 . We also need to consider the space ᏹ0,0 (S 2 , 2) of genus 0 stable maps of degree 2 into S 2 that have no marked points, and the space Y˜ = ᏹ0,3 S 2 , p0 , p1 , p2 , 2 of all degree 2 stable maps to S 2 with three marked points z0 , z1 , z2 such that h(z0 ) = p0 , h(z1 ) = p1 , h(z2 ) = p2 . Lemma 3.7. (i) ᏹ0,0 (S 2 , 2) is a smooth manifold diffeomorphic to CP 2 . (ii) Y˜ = ᏹ0,3 (S 2 , p0 , p1 , p2 , 2) is a smooth manifold diffeomorphic to CP 2 . (iii) The forgetful map f : Y˜ → Y may be identified with the 2-fold cover that quotients CP 2 by the involution τ : [x : y : z] → [x : y : −z]. In particular, Y is smooth except at the point σ01 = f ([0 : 0 : 1]) that has the local chart C2 /(x, y) = (−x, −y). This point σ01 is the stable map [S 2 , h, z0 , z1 ] where the critical values of h are at p0 and p1 . Proof. (i) The space ᏹ0,0 (S 2 , 2) has two strata. The first, ᏿1 , consists of selfmaps of S 2 of degree 2, and the second, ᏿2 , consists of maps whose domain has two components, each taken into S 2 by a map of degree 1. The equivalence relation on each stratum is given by precomposition with a holomorphic self-map of the domain. It is not hard to check that each equivalence class of maps in ᏿1 is uniquely determined by its two critical values (or branch points). Since these can be any pair of distinct points, ᏿1 is diffeomorphic to the set of unordered pairs of distinct points in S 2 . On the other hand, there is one element σw of ᏿2 for each point w ∈ S 2 , the correspondence being given by taking w to be the image under h of the point of intersection of the two components. If σ{x,y} denotes the element of ᏿1 with critical values {x, y}, we claim that σ{x,y} → σw when x, y both converge to w. To see this, let h{x,y} : S 2 → S 2 be a representative of σ{x,y} and let α{x,y} be the shortest geodesic from x to y. (We assume that x, y are close to w.) Then h−1 {x,y} (α{x,y} ) is a circle γ{x,y} through the critical points of h{x,y} . This is obvious if h{x,y} is chosen to have critical points at 0, ∞, and if x = 0, y = ∞ since h{x,y} is then a map of the form z → az2 . It follows in the general case because Mobius transformations take circles to circles. Hence h{x,y} takes each component of S 2 − γ{x,y} onto S 2 − α{x,y} . If we now let x, y converge to w, we see that σ{x,y} converges to σw . The above argument shows that ᏹ0,0 (S 2 , 2) is the quotient of S 2 × S 2 by the involution (x, y) → (y, x). This is well known to be CP 2 . In fact, it is easy to check that the map H : [x0 : x1 ], [y0 : y1 ] −→ x0 y0 : x1 y1 : x0 y1 + x1 y0 − x0 y0 − x1 y1 induces a diffeomorphism from the quotient to CP 2 . Under this identification, the stratum ᏿2 = H (diag) is the quadric (u + v + w)2 = 4uv (where we use coordinates


157

[u : v : w] on CP 2 ). Further, if we put p0 = [0 : 1],

p1 = [1 : 0],

p2 = [1, 1],

the set of points in ᏹ0,0 (S 2 , 2) = CP 2 consisting of maps that branch over pi is a line i , the image by H of (S 2 × pi ) ∪ pi × S 2 . Thus 0 = {u = 0},

1 = {v = 0},

2 = {w = 0}.

Note finally that all stable maps in ᏹ0,0 (S 2 , 2) are invariant by an involution; for example, the map z → z2 is invariant under the reparametrization z → −z. Since all elements have the same reparametrization group, ᏹ0,0 (S 2 , 2) is smooth. However, this is no longer the case when we add two marked points. (ii) Now consider the forgetful map φ30 : ᏹ0,3 S 2 , p0 , p1 , p2 , 2 −→ ᏹ0,0 S 2 , 2 . For a general point of ᏹ0,0 (S 2 , 2), that is, a point where neither branching point is at p0 , p1 or p2 , φ30 is 4-to-1. To see this, note that for i = 0, 1, 2, zi can be either of the points that get mapped to pi , which seems to give an 8-fold cover. However, because h has degree 2, h is invariant under an involution γh of S 2 that interchanges the two inverse images of a generic point. Hence the cover is 4-to-1, and the covering group is Z/2Z ⊕ Z/2Z. When just one branching point is at some pi , φ30 is 2-to-1, and when both branching points are at some pi , it is 1-to-1. This determines φ30 . In fact, with the above identification for ᏹ0,0 (S 2 , 2) = CP 2 , φ30 is the map φ30 : CP 2 −→ CP 2 : [x : y : z] −→ x 2 : y 2 : z2 . Note that the inverse image of ᏿2 = {4uv = (u + v + w)2 } consists of the four lines x ± y ± iz = 0. These components correspond to the four different ways of arranging three points on the two components of the stable maps in ᏿2 . Note further that none of the points in ᏹ0,3 (S 2 , p0 , p1 , p2 , 2) is invariant by any reparametrization of their domains. Hence all points of this moduli space are smooth. (iii) Similar reasoning shows that the forgetful map φ20 : Y = ᏹ0,2 S 2 , p0 , p1 , 2 −→ ᏹ0,0 S 2 , 2 is a 2-fold cover branched over 0 ∪ 1 . Hence we may identify Y as Y = [u : v : w : t] ∈ CP 3 : t 2 = uv ,

158

DUSA MCDUFF

where the cover φ20 : Y → CP 2 forgets t. There is one point in Y that is invariant under a reparametrization of its domain, namely, the point σ01 corresponding to the map h : S 2 → S 2 that branches at p0 and p1 . In the above coordinates on Y , σ01 = φ2−1 0 ∩ 1 = [0 : 0 : 1 : 0]. It is also easy to check that φ32 : ᏹ0,3 S 2 , p0 , p1 , p2 , 2 = CP 2 −→ Y has the formula

φ32 [x : y : z] = x 2 : y 2 : z2 : xy .

Since φ32 ◦ τ ([x : y : z]) = φ32 ([x : y : −z]) = φ32 ([x : y : z]), φ32 is equivalent to quotienting out by τ as claimed. 3.2.3. The bundle ᐂ1,J → ᐆ1,J . Now we consider the structure of the orbibundles of gluing parameters over Y˜ = ᏹ0,3 (S 2 , p0 , p1 , p3 , 2) and ᐆ1,J = Y = ˜ → Y˜ and the second LY → Y . In both cases, ᏹ0,2 (S 2 , p0 , p1 , 2). We call the first L the fiber at the stable map [), h, zi ] is the tangent space Tz1 ). Lemma 3.8. (i) The orbibundle L˜ → Y˜ is smooth and may be identified with the canonical line bundle Lcan over Y˜ = CP 2 . (ii) The orbibundle LY → Y is smooth except at the point σ01 . It can be identified with the quotient of Lcan by the obvious lift τ˜ of τ . (iii) The set S(LY ) of unit vectors in LY is smooth and diffeomorphic to S 5 . The orbibundle S(LY ) → Y can be identified with the quotient of S 5 by the circle action θ · (x, y, z) = eiθ x, eiθ y, e2iθ z . Proof. (i) Since Y˜ is smooth, the general theory implies that L˜ is smooth. Therefore, it is a line bundle over CP 2 , and to understand its structure we just have to figure out its restriction to one line. It is easiest to consider one of the lines x ± y ± iz = 0 that lie over ᏿2 . Recall that σw ∈ ᏿2 is the stable map [)w , hw ] with domain )w = S 2 ∪w=w S 2 and where h is the identity map on each component. Suppose we look at the line in Y˜ whose generic point has z1 on one component of )w and z0 , z2 on the other. Then the bundle L˜ has a natural trivialization over the set {w ∈ S 2 : w = z0 , z1 , z2 }. It is not hard to check that this trivialization extends over the points z0 , z2 , but that one negative twist is introduced when z1 is added. The argument is very similar to the proof of Lemma 3.9 below and is left to the reader. (ii) It follows from the general theory that LY → Y is smooth over the smooth points of Y . Moreover, at σ01 = [S 2 , h], the automorphism γ : S 2 → S 2 such that h ◦ γ = h, γ (zi ) = zi acts on Tz1 S 2 by the map v → −v. (To see this, note that we can identify S 2 with C ∪ {∞} in such a way that z0 = p0 = 0, z1 = p1 = ∞. Then h(z) = z2 and γ (z) = −z.) Hence the local structure of L at σ01 is given by


159

quotienting the trivial bundle D 4 × C by the map (x, y) × v → (−x, −y) × −v. This is precisely the structure of the quotient of L˜ by τ˜ at the singular point. Moreover, we can identify S(LY ) with S 5 /τ globally since LY → Y pulls back to L˜ → Y˜ under the map Y˜ → Y . The quotient S 5 /τ is smooth except possibly at the fixed points (x, y, 0) of τ . Since S(LY ) is smooth at these points, S(LY ) is smooth everywhere. It may be identified with S 5 by the map S5 5 2 2 2 S(LY ) ≡ −→ S : (x, y, z) −→ x 1 + |z| , y 1 + |z| , z . τ The last statement may be proved by noting that the formula (x, y, z) −→ x 2 : y 2 : z : xy ∈ CP 3 defines a diffeomorphism from the orbit space of the given circle action to CP 2/τ = Y . 3.2.4. Attaching the strata. The next step is to understand how the two strata ᐂ1,J and ᐂ2,J fit together. The two zero sections ᐆ1,J and ᐆ2,J intersect at the point σ∗ (see Figure 1). Recall that the domain ) of σ∗ has four components with )0 , )2 , )3 , all meeting )1 and a marked point z0 ∈ )3 . The map h0 : )0 → J parametrizes J as a section, h1 takes )1 onto the point x1 = F0 ∩J , and h2 , h3 have image F0 with h3 (z0 ) = x0 . The stratum of ᐆ1,J containing σ∗ consists just of this one point. Hence the local coordinates of σ∗ in ᐆ1,J are given by two gluing parameters (a0 , a1 ). If we write zij for the point )i ∩ )j , these are (a0 , a1 ),

where a0 ∈ Tz12 )1 ⊗ Tz12 )2 , a1 ∈ Tz13 )1 ⊗ Tz13 )3 .

Similarly, the local coordinates for a (deleted) neighborhood of σ∗ in ᐂ1,J are (b, a0 , a1 ), where (a0 , a1 ) are as before, and b ∈ Tz01 )0 ⊗ Tz01 )1 is a gluing parameter at the point z01 , where the component )0 mapping to J is attached. On the other hand, the natural coordinates for a neighborhood of σ∗ in ᐂ2,J are triples (w, b, a) where b is a gluing parameter at the point z03 where the component )0 that maps to J is attached to the fixed fiber )3 , w is the point where the moving fiber )2 (the one not containing z0 ) is attached to )0 , and a is a gluing parameter at w. Lemma 3.9. The attaching map α at σ∗ has the form (b, a0 , a1 ) → (wb , ba0 , ba1 ), where b = 0 and |b| is small. Here the map b → wb identifies a small neighborhood of 0 in Tz01 ) with a neighborhood of x1 in J in the obvious way. Proof. The attaching of ᐆ1,J to ᐆ2,J comes from gluing at the point z01 via the parameter b. Thus we are gluing the “ghost component” )1 to the component )0

160

DUSA MCDUFF

that maps to J in the space of stable A-curves that are holomorphic for a fixed J . (It is only when one glues at a0 or a1 that one changes the homology class of the curve J and, hence, has to change J .) In particular, we can forget the components )2 , )3 of the domain ) of σ∗ , retaining only the points z12 , z13 on )1 where they are attached. Therefore, we can consider the domain of the attaching map α to be the 2-dimensional space )0 ∪z01 )1 , z12 , z13 , h ; b : b ∈ C = Tz01 )0 ⊗ Tz01 )1 , and its range to be the space of all elements [)0 , q0 , w, h , J ] where )0 = S 2 and w moves in a small disc about x1 . Here the map h : )0 → J is fixed and parametrizes J as a section. We can encode this by picking two points q1 , q2 in )0 that are different from q0 = πJ (x0 ) and then considering h to be the map that takes these two marked points to two other fixed points on J . Thus the attaching map α is equivalent to the following map α that attaches different strata in the moduli space ᏹ0,4 (S 2 ) of four marked points on S 2 : α : )0 ∪ )1 , q1 , q2 , z12 , z13 ; b : b ∈ C −→ )0 , q1 , q2 , z12 , z13 ∈ ᏹ0,4 S 2 . Here, as before, each )i is a copy of S 2 . On the left, q1 , q2 are two marked points on )0 and z12 , z13 are two marked points on )1 . On the right, we should consider the three points q1 , q2 , z13 to be fixed, while z12 = w moves, since this corresponds to our previous trivialization of the neighborhood of σ∗ in ᐂ2,J . Thus α may be considered as a map taking b to wb = z12 ∈ J . It remains to check that as b moves once (positively) around 0, wb moves once positively around z13 . This follows by examining the identification of the glued domain )b = )0 − D(z01 ) ∪glb )1 − D(z01 ) with )0 = S 2 . Observe that the two points q1 , q2 in )0 −D(z01 ) and the single point z13 in )1 −D(z01 ) must be taken to the corresponding three fixed points on S 2 = )1 . Hence the identification on )0 −D(z01 ) is fixed, while that on )1 −D(z01 ) can rotate about z13 as b moves. Hence when b moves round a complete circle, so does wb . It remains to check the direction of the rotation. Now, as we saw in Proposition 2.7, as b moves once around this circle positively as seen from z01 , the point pb on ∂D(z01 ) ⊂ )1 that is matched with a fixed point p on )0 − D(z01 ) moves once positively around ∂D(z01 ). In order to line up pb with p, )1 must be rotated in the opposite direction, that is, positively as seen from the fixed point z13 (see Figure 2). Hence wb rotates positively around z13 . To complete the proof of the lemma, we must understand how the gluing parameters a0 , a1 fit into this picture. Since nothing is happening in the vertical (i.e., fiberwise) direction, we may consider the ai to be elements of the following tangent spaces: a0 ∈ Tz12 )1 ,

a1 ∈ Tz13 )1 .


z13

z12 = wb )1

Pb

P

q1

161

q2

)0 Figure 2

As b rotates positively, the image of a0 in the glued curve rotates once positively in the tangent space of z12 , and a1 ∈ Twb )1 also rotates once with respect to the standard trivialization of the tangent spaces Twb )1 ⊂ T ()1 )|D(z12 ) , hence the result. Proof of Proposition 3.4. We have identified the orbibundle ᐂ1,J → ᐆ1,J with L → Y and the bundle ᐂ2,J → ᐆ2,J with ᏻ(−2) ⊕ C → S 2 . The previous lemma shows that these are attached by first twisting ᐂ1,J to ᏻ(−3)⊕ ᏻ(−1) and then plumbing it to L. Hence ᏸᐆ = S ᏻ(−3) ⊕ ᏻ(−1) ≡ ! S(LY ) as claimed. The identification of the latter space with S(ᏻ(−1) ⊕ C) follows from Proposition 3.3. 3.3. The projection ᐂJ → ᏶. In order to complete the calculation of the link ᏸ2,0 of ᏶2 in ᏶, it remains to understand the projection ᐂJ → ᏶. This is 1-to-1 except over the points of ᏶1 . In ᐂ2,J , it is clearly the points with zero gluing parameter at the moving fiber that get collapsed. Thus the subbundle R− of the circle bundle S(LP ) → ᏼ(ᏻ(−2)⊕C) that lies over the (rigid) section S− = ᏼ({0}⊕C) must be collapsed to a single circle. The subbundle R+ lying over the other section S+ = ᏼ(ᏻ(−2) ⊕ {0}) maps to a family of distinct elements in ᏶1 . The story on ᐂ1,J is, of course, more complicated. Here the points that concern us are the maps in ᏿2 where the branch points coincide. Thus, if we identify ᏹ0,0 (S 2 , 2) with CP 2 as in Lemma 3.7, these are the points of the quadric Q = {(u + v + w)2 = 4uv}. Note that the attaching point σ∗ ∈ ᏹ0,2 (S 2 , p0 , p1 , 2) sits over [1 : 0 : −1] = 1 ∩ Q ∈ CP 2 = ᏹ0,0 S 2 , 2 . The lift of Q to ᏹ0,2 (S 2 , 2) has two components Q± , given by the intersection Y ∩H± where H± is the hyperplane 2t = ±(u+v +w). Since we can assign these at will, we say that Q− corresponds to elements with the two marked points z0 , z1 on the same component of )w = )0 ∪w=w )1 and that Q+ corresponds to elements with z0 , z1 on different components. Then, when one glues at z1 , the resulting A-curve is the union of an (A − F )-curve with an F curve. It is not hard to check that the points

162

DUSA MCDUFF

on Q− give rise to a (A − F )-curve through x0 , which is independent of w, while those on Q+ give rise to a varying (A − F )-curve that meets the Jw -holomorphic fiber through x0 at the point w. Note that the intersection Q+ ∩ Q− consists of two points, p∗ = [1 : 0 : −1 : 0] (corresponding to σ∗ ) and q∗ = [0 : 1 : −1 : 0]. Moreover, in the coordinates (a0 , a1 ) of a neighborhood of σ∗ used in Lemma 3.9,

(a0 , a1 ) : a0 = 0 ⊂ Q− ,

(a0 , a1 ) : a1 = 0 ⊂ Q+ .

This confirms that when ᐂ2,J ⊗ ᏻ(−1) = ᏻ(−3) ⊕ ᏻ(−1) is plumbed to ᐂ1,J , Q− is plumbed to the subbundle {0} ⊕ ᏻ(−1) corresponding to R− , and Q+ is plumbed to the subbundle ᏻ(−3) ⊕ {0} corresponding to R+ . Let S(Q± ) → Q± denote the restriction of S(LY ) → Y to Q± . Then the plumbing ᏻ(−3) ⊕ ᏻ(−1) ≡ ! S(LY ) contains the plumbings R− ≡ ! S(Q− ) and R+ ≡ ! S(Q+ ). ! S(Q− ) = S(C) = S 2 ×S 1 , and R+ ≡ ! S(Q+ ) = S(ᏻ(−2)). Lemma 3.10. (i) R− ≡ (ii) The subsets R− ≡ ! S(Q− ) and R+ ≡ ! S(Q+ ) of ᏻ(−3)⊕ ᏻ(−1) ≡ ! S(LY ) intersect in a circle. Proof. Since Q− and Q+ do not meet the singular point of Y , both bundles S(Q± ) → Q± have Euler number −1. Hence R− ≡ ! S(Q− ) = S ᏻ(−1) ≡ ! S ᏻ(−1) = S(C) = S 2 × S 1 , and

R+ ≡ ! S(Q+ ) = S ᏻ(−3) ≡ ! S ᏻ(−1) = S ᏻ(−2) .

This proves (i). To prove (ii), note that the inverse image (in S(Q± )) of the intersection point p∗ = [1 : 0 : −1] of Q− with Q+ disappears under the plumbing. But the other one remains. Proof of Theorem 1.5. It follows from part (i) of the preceding lemma that it is possible to collapse the subset R− ≡ ! S(Q− ) of ᏸᐆ to a single circle. Moreover, it is not hard to see that under the identification of ᏸᐆ = S(ᏻ(−3)⊕ ᏻ(−1)) ≡ ! S(LY ) with S(ᏻ(−1) ⊕ C), this collapsing corresponds to collapsing the circle bundle over the exceptional divisor. Since the intersection of R− ≡ ! S(Q− ) with R+ ≡ ! S(Q+ ) is a single circle, this collapsing does not affect R+ ≡ ! S(Q+ ). Note that R+ ≡ ! S(Q+ ) is the 2 ! S(Q+ ) = inverse image of some 2-dimensional submanifold of CP . Because R+ ≡ S(ᏻ(−2)), this submanifold must be a quadric. 4. Analytic arguments. In §4.1 we prove the (easy) Lemma 2.2. §4.2 contains a detailed analysis of gluing. The exposition here is fairly self-contained, though some results are quoted from [MS] and [FO]. 4.1. Regularity in dimension 4. The theory of J -holomorphic spheres in dimension 4 is much simplified by the fact that any injective J -holomorphic map h : S 2 → X


163

that represents a class A with A · A ≥ −1 is regular; that is, the linearized delbar operator 2 ∗ Dh : W 1,p h∗ (T X) −→ Lp J0,1 J S ⊗ h (T X) is surjective. This remains true even if h is a multiple covering. (For a proof, see Hofer, Lizan, and Sikorav [HLS]. The notation is explained in the proof of Lemma 2.2 below.) Therefore, regularity is automatic: one does not have to perturb the equation in order to achieve it. The analogous statement when A · A < −1 is that Coker Dh always has rank equal to 2 + 2A · A. As is shown below, this almost immediately implies that the ᏶k are submanifolds of ᏶. Proof of Lemma 2.2. We begin by proving that ᏶k is a Fréchet manifold. This is obvious when k = 0, since ᏶0 is an open subset of ᏶. For k > 0, let Ꮿk denote the space of all symplectically embedded spheres in the class A−kF , and let Ꮿk (᏶) be the bundle over Ꮿk whose fiber at C is the space of all smooth almost complex structures on C that are compatible with ω|C . Then Ꮿk (᏶) fibers over Ꮿk , and it is easy to check that both spaces are Fréchet manifolds. (Note that Ꮿk is an open submanifold in the space of all embedded spheres in the class A − kF . Because these spheres are not parametrized, the tangent space to Ꮿk at C is the space of all sections of the normal bundle to C.) Further, ᏶k fibers over Ꮿk (᏶) with fiber at (C, J |C ) are equal to all ω-compatible almost complex structures that restrict to J on T C. This proves the claim. To see that πk is bijective when k > 0, note that each J ∈ ᏶k admits a holomorphic curve in class A − kF by definition and that this curve is unique by positivity of intersections. A similar argument works when k = 0 since the curves in ᏹ(A, ᏶) are constrained to go through x0 . Hence ᏹk inherits a Fréchet manifold structure from ᏶k . To show that ᏶k is a submanifold of ᏶ when k > 0, we must use the theory of J holomorphic curves, as explained in Chapter 3 of [MS], for example. Let ᏹks , ᏶sk , ᏶s denote the similar spaces in the C s -category for some large s. These are all Banach manifolds. It is easy to check that the tangent space TJ ᏶s is the space End(T X, ω, J ) of all C s -sections Y of the endomorphism bundle of T X such that J Y + Y J = 0,

ω(Y x, y) = ω(x, Yy).

These conditions imply that ω(Y x, x) = ω(Y x, J x) = 0 for all x. It follows easily that Y is determined by its value on a single nonzero vector x that it has to take to the ω-orthogonal complement to the J -complex line through x. Observe further that there is an exponential map exp : TJ ᏶s −→ ᏶s that preserves smoothness and is a local diffeomorphism near the zero section. Next, note that the tangent space T[h,J ] ᏹks is the quotient of the space of all pairs (ξ, Y ) such that 1 Dh(ξ ) + Y ◦ dh ◦ j = 0 2

(∗)

164

DUSA MCDUFF

by the 6-dimensional tangent space to the reparametrization group PSL(2, C). Here j is the standard almost complex structure on S 2 , and Dh is the linearization of the delbar operator that maps the Sobolev space of W 1,p -smooth sections of h∗ (T X) to anti-J -holomorphic 1-forms, namely, 2 ∗ Dh : W 1,p S 2 , h∗ (T X) −→ Lp J0,1 (1) J S , h (T X) , where the norms are defined using the standard metric on S 2 and a metric on T X. In [HLS], Hofer, Lizan, and Sikorav show how to interpret elements of Ker Dh and of Ker Dh∗ (where Dh∗ is the formal adjoint) as J -holomorphic curves in their own right. Using the fact that the domain is a sphere and that X has dimension 4, they then use positivity of intersections to show that Ker Dh is trivial when k > 0, that is, it consists only of vectors that generate the action of PSL(2, C). Hence Ker Dh∗ is a bundle over ᏹks ∼ = ᏶sk of rank 4k − 2 = −indexDh, and it is not hard to see that it is isomorphic to the normal bundle of ᏶sk in ᏶s . In other words, TJ ᏶s = TJ ᏶sk ⊕ Ker Dh∗ . To see this, observe that the map 1 ι : Y −→ Y ◦ dh ◦ j 2

(2)

2 ∗ maps TJ ᏶s onto the space of C s -sections of J0,1 J (S , h (T X)), and that the kernel of this projection consists of elements Y that vanish on the tangent bundle to the image of h and so lie in TJ ᏶sk whenever [h, J ] ∈ ᏹks . It follows from equation (1) that the image of TJ ᏶sk under this projection is precisely equal to the image of Dh, and so its complement is isomorphic to Ker Dh∗ . (For more details on all this, see the appendix to [A].) It now remains to show that ᏶k is a submanifold of ᏶ whose normal bundle has fibers Ker Dh∗ . This means in particular that the codimension of ᏶k is − ind Dh = 4k − 2. We therefore have to check that each point in ᏶k has a neighborhood U in ᏶ that is diffeomorphic to the product (U ∩ ᏶k )×R4k−2 . It is here that we use the exponential map exp. Clearly, one can use exp to define such local charts for ᏶sk in ᏶s . The point here is that the derivative of the putative chart is the identity along (U ∩ ᏶sk )×{0}, and so by the implicit function theorem for Banach manifolds, it is a diffeomorphism on a neighborhood. Then, because Ker Dh∗ consists of C ∞ sections when J is C ∞ , and because exp respects smoothness, this local diffeomorphism takes (U ∩ ᏶k ) × R4k−2 onto a neighborhood of J in ᏶.

4.2. Gluing. The next task is to complete the proof of Propositions 2.7 and 2.11. The standard gluing methods are local and work in the neighborhood of one stable map, and so our main problem is to globalize the construction. The first step in doing this is to show that one can still glue even when the elements of the obstruction


165

bundle are nonzero at the gluing point. We use the gluing method of McDuff and Salamon [MS] and Fukaya and Ono [FO]. Much of the needed analysis appears in [MS], but the conceptual framework of that work has to be enlarged to include the idea of stable maps as in [HS]. No doubt the other gluing methods can be adapted to give the same results. Our aim is to construct a gluing map Ᏻ : ᏺᐂ (ᐆ) −→ ᏹ(A − kF, ᏶),

where ᐆ = ᏹ(A − kF, ᏶m ) is the space of stable maps in class A − kF with one component in class A − mF , and ᏺᐂ (ᐆ) is a neighborhood of ᐆ in the space ᐂ of gluing parameters. Once and for all, choose a (4m − 2)-dimensional subbundle K of T ᏶|᏶m that is transverse to ᏶m . As explained in §3.1, the exponential map exp maps a neighborhood of the zero section in K diffeomorphically onto a neighborhood of ᏶m in ᏶. For each J ∈ ᏶m , let ᏷J ⊂ ᏶ be the slice through J (i.e., the image under exp of a small neighborhood ᏺJ (K) of 0 in the fiber of K at J ). We prove the following sharper version of Proposition 2.11. Proposition 4.1. Fix J ∈ ᏶m and let ᏺᐂ (ᐆJ ) be the fiber of the map ᏺᐂ (ᐆ) → ᏶m at J . Then, if the neighborhood ᏺᐂ (ᐆ) is sufficiently small, there is a homeomorphism

ᏳJ : ᏺᐂ ᐆJ −→ ᏹ A − kF, ᏷J

onto a neighborhood of ᏹ(A − kF, J ) in ᏹ(A − kF, ᏷J ). Moreover, the union of all the sets Im ᏳJ , J ∈ ᏶m , is a neighborhood of ᏹ(A − kF, ᏶m ) in ᏹ(A − kF, ᏶). Let πJ : ᏺᐂ (ᐆJ ) → ᐆJ denote the projection. We first construct the map ᏳJ in the fiber at one point σ = [)σ , hσ , J ] of ᐆJ , and then show how to fit these maps together to get a global map over ᏺᐂ (ᐆJ ) with the stated properties. For the next paragraphs (until §4.2.4), we fix a particular representative hσ : )σ → X of σ , and we define ˜ as a map into the space of parametrized stable maps. In order to understand a full Ᏻ neighborhood of σ , we have to glue not only at points where the branches meet the stem )0 , but also at points internal to the branches. Therefore, for the moment, we forget the stem-branch structure of our stable maps and consider the general problem of gluing at the points zi ∈ )i0 ∩ )i1 with parameter a = ⊕ i ai ∈ Tzi )i0 ⊗ Tzi )i1 . i

4.2.1. Construction of the pregluing ha . In Proposition 2.11 we showed how to construct the glued domain )a . Since this construction depends on a choice of metric on ), we must assume that the domain ) of each stable map is equipped with a Kähler metric that is flat near all double points and is invariant under the action of

166

DUSA MCDUFF

the isotropy group 5σ . Fukaya and Ono point out in [FO, §9] that it is possible to choose such a metric continuously over the whole moduli space: one just has to start at the strata containing elements σ with the largest number of components, extend the choice of metric near these strata by using the gluing construction (which is invariant by 5σ ), and then continue inductively, strata by strata. In what follows, we assume this has been done. We also suppose that the cutoff functions χr used to define )a have been chosen once and for all. The approximately holomorphic map ha : )a → X is defined √ from hσ by using cutoff functions. As before, we write ri , or simply r, instead of |ai |. Hence, if R is as in [FO] or [MS], r = 1/R. We choose a small δ > 0 once and for all so that r/δ is still small.3 Set xi = hσ (zi ). Then for α = 0, 1, define α 2r ha (z) = hσ (z) for z ∈ )iα − Dzi , δ r = xi for z ∈ Dzαi − Dzαi (r), δ and interpolate on the annulus Dzαi (2r/δ) − Dzαi (r/δ) in )iα by setting δ|z| ha (z) = expxi ρ ξiα (z) , r

where ρ is a smooth cutoff function that equals 1 on the interval [2, ∞) and 0 on [0, 1], and the vectors ξiα (z) ∈ Txi X exponentiate to give hσ (z) on )iα : 2r hσ (z) = expxi ξiα (z) for z ∈ Dzαi . δ The whole expression is defined provided that 2r/δ is small enough for the exponential maps to be injective. Later it will be useful to consider the corresponding map hσ,r with domain ). This map equals ha on ) − ∪i,α Dzαi (ri ) and is set equal to xi on each disc Dzαi (ri ). Note that hσ,r : ) → X converges in the W 1,p -norm to hσ as r → 0. 4.2.2. Construction of the gluing Ᏻ˜ (hσ , a). Let ᏺ0 (Wa ) = ᏺ0 W 1,p )a , h∗a (T X) be a small neighborhood of 0 in W 1,p ()a , h∗a (T X)). Note that if )a has several components )a,j , the elements σ of Wa can be considered as collections ξj of sections in W 1,p ()a,j , (ha,j )∗ (T X)) that agree pairwise at the points zi . (This makes sense since the ξj are continuous.) Further, we may identify ᏺ0 (Wa ) via the exponential logic is that one chooses δ > 0 small enough for certain inequalities to hold and then chooses r ≤ r(δ). See Lemma 4.7. 3 The


167

map with a neighborhood of ha in the space of W 1,p -maps )a → X. We write ha,ξ for the map )a → X given by ha,ξ (z) = expha (z) ξ(z) , z ∈ )a . Recall that ᏺJ (K) is a neighborhood of 0 in the fiber of K. Given Y ∈ ᏺJ (K), we write JY for the almost complex structure exp(Y ) in the slice ᏷J . Now consider the locally trivial bundle Ᏹ = Ᏹa → ᏺ0 (Wa ) × ᏺJ (K) whose fiber at (ξ, Y ) is Ᏹ(ξ,Y ) = Lp J0,1 ()a ) ⊗JY h∗a,ξ (T X) . We wish to convert the pregluing ha to a map that is JY -holomorphic for some Y by using the implicit function theorem for the section Ᏺa of Ᏹa defined by Ᏺa (ξ, Y ) = ∂ JY (ha,ξ ).

Note that Ᏺa (ξ, Y ) = 0 exactly when the map ha,ξ is JY -holomorphic. The linearization ᏸ(Ᏺa ) of Ᏺa at (0, 0) equals ᏸ(Ᏺa ) = D(ha ) ⊕ ιa : W 1,p h∗a (T X) ⊕ K −→ Lp J0,1 ()a ) ⊗J h∗a (T X) , where ιa is defined by ιa (Y ) = (1/2)Y ◦ dha ◦ j as in equation (2) in §4.1. Lemma 4.2. Suppose that there is a continuous family of right inverses Qa to

ᏸ(Ᏺa ) that are uniformly bounded for |a| ≤ r0 . Then there is r1 > 0 such that for all

a satisfying |a| ≤ r1 , there is a unique element (ξa , Ya ) ∈ Im Qa such that Ᏺa (ξa , Ya ) = 0.

Moreover, (ξa , Ya ) depends continuously on the initial data. Proof. This follows from the implicit function theorem as stated in [MS, §3.3.4]. It also uses [MS, Lemma A.4.3]. See also [FO, §11]. We construct the required family Qa in §4.2.3. The above lemma allows us to define the gluing map. Definition 4.3. We set Ᏻ˜ (hσ , a) = ()a , ha,ξa , JYa ) where (ξa , Ya ) is the unique element in Lemma 4.2. Further, Ᏻ(hσ , a) = [)a , ha,ξa , JYa ]. The next proposition states the main local properties of the gluing map Ᏻ. Proposition 4.4. Each σ ∈ ᐆJ has a neighborhood ᏺᐂ (σ ) in ᐂJ such that the map ᏺᐂ (σ ) −→ ᏹ(A − kF, ᏷J ) : (σ , a ) −→ Ᏻ(hσ , a ) takes ᏺᐂ (σ ) bijectively onto an open subset in ᏹ(A − kF, ᏷J ). Moreover, this map depends continuously on J ∈ ᏶m .

168

DUSA MCDUFF

Proof. This is a restatement of [FO, Theorem 12.9]. Note that the stable map

Ᏻ(hσ , a ) depends on the choice of representative () , hσ ) of the equivalence class

σ = [) , hσ ]. However, it is always possible to choose a smooth family of such representatives in a small enough neighborhood of σ in ᐆJ . (This point is discussed further in §4.2.4.) Moreover, if σ is an orbifold point (i.e., if 5σ is nontrivial), then hσ is 5σ -invariant and we can define Ᏻ˜ so that it is equivariant with respect to the natural action of 5σ on the space of gluing parameters a and its action on a neighborhood of σ in the space of parametrized maps. The composite Ᏻ of Ᏻ˜ with the forgetful map is therefore 5σ -invariant. (Cf. the discussion before Lemma 4.9.) This shows that Ᏻ is well defined. We prove that it is a local homeomorphism as in [FO, §§13, 14], and we say no more about this except to observe that our adding of K to the domain of Dhσ is equivalent to their replacement of the range of Dhσ by the quotient Lp /ιa (K). 4.2.3. Construction of the right inverses Qa . This is done essentially as in [MS, A.4] and [FO, §12]. However, there are one or two extra points to take care of, firstly because the stem of the map hσ is not regular, so that the restriction of Dhσ to )0 is not surjective, and secondly because the elements of the normal bundle K → ᏶m do not necessarily vanish near the points xi in X where gluing takes place. For simplicity, let us first consider the case when ) has just two components )0 , )1 intersecting at the point w, and suppose that hσ maps )0 onto the (A−kJ )-curve J and )1 onto a fiber. (For the general case, see Remark 4.8.) Then the linearization of ∂ J at hσ has the form ∗ Dhσ : W 1,p ), h∗σ (T X) −→ Lp J0,1 J ()) ⊗ hσ (T X) . Here the domain consists of pairs (ξ0 , ξ1 ) where ξj is a W 1,p -smooth section of the bundle h∗σj (T X) → )j subject to the condition ξ0 (w) = ξ1 (w), and the range consists of pairs of Lp -smooth (0, 1)-forms over )j with values in h∗σj (T X) and with no condition at w. For short, we denote this map by Dhσ : Wσ → Lσ0 ⊕ Lσ1 . Recall from the discussion before Proposition 4.1 that we chose K so that Dhσ0 ⊕ ι0 : Wσ0 ⊕ K −→ Lσ0 is surjective and ι0 : K → Lσ0 is injective. (All maps ι are defined as in equation (2): it should be clear from the context what the subscripts mean.) Lemma 4.5. There are constants c, r0 > 0 so that the following conditions hold for all r < r0 :


169

(i) ιa is injective for all |a| ≤ r; (ii) the projection pr K : Lσ0 → K that has kernel Im Dσ0 and satisfies pr K ◦ι0 = idK has norm less than or equal to c; (iii) for all Y ∈ K and j = 0, 1, 1/p 1/p 1 ιj (Y )p ιj (Y )p ≤ , j 12c )j Dw (r) j

where Dw (r) is the disc in )j on which gluing takes place and integration is with respect to the area form defined by the chosen Kähler metric on )σ . Proof. There is c so that (ii) holds because Im Dσ0 is closed and Im ι0 is finitedimensional. Then there is r0 = r0 (c) satisfying (i) and (iii) since the elements of K are C ∞ -smooth (as are the elements of ᏶). Lemma 4.6. The operator Dhσ ⊕ (ι0 , ι1 ) : Wσ ⊕ K −→ Lσ0 ⊕ Lσ1 , is surjective and has kernel ker Dhσ . Proof. We know from the proof of Lemma 2.2 that Dhσ0 ⊕ ι0 : W 1,p )0 , h∗σ0 (T X) ⊕ K −→ Lp J0,1 ()0 ) ⊗J h∗σ0 (T X) = Lσ0 is surjective. Similarly, Dhσ1 is surjective. Therefore, to prove surjectivity, we just need to check that the compatibility condition ξ0 (w) = ξ1 (w) for the elements of Wσ causes no problem. However, the pullback bundle h∗σ1 T X splits naturally into the sum of a line bundle with Chern class 2d (where d ≥ 0 is the multiplicity of hσ,1 ) and a trivial line bundle, the pullback of the normal bundle to the fiber Im hσ1 . Hence there is an element ξ1 of ker Dhσ1 with any given value ξ1 (w) at w. The result follows. Note that an appropriate version of this argument applies for all σ , not just those with two components, since there is just one condition to satisfy at each double point z of ), and the maps ker Dhσj → C2 : ξ → ξ(z) are surjective for j > 0. The second statement holds because ι0 is injective. Note that the right inverse Qσ to Dhσ ⊕(ι0 , ι1 ) is completely determined by choosing a complement to the finite-dimensional subspace ker Dhσ in Wσ . Consider the composite pr σ0 : Lσ0 ⊕ Lσ1 −→ Lσ0 −→K, where the second projection is as in Lemma 4.5(ii). The fiber (pr σ0 )−1 (Y ) at Y has the form (Im Dhσ0 + ι0 (Y )) ⊕ Lσ1 , and we write QYσ : Im Dhσ0 ⊕ Lσ1 + ι0 (Y ), ι1 (Y ) −→ Wσ for the restriction of Qσ to this fiber.

170

DUSA MCDUFF

We now use the method of [MS, A.4] to construct an approximate right inverse Qa,app to ᏸ(Ᏺa ) = Dha ⊕ ιa : Wa ⊕ K → La , where

Wa = W 1,p h∗a (T X) ,

La = Lp J0,1 ()a ) ⊗J h∗a (T X) .

It is convenient to use the approximations hσ,r : )σ → X to hσ that are defined at the end of §4.2.1 where r 2 = |a|. We write hσj ,r for the restriction of hσ,r to the component )j . Since hσ,r converges W 1,p to hσ as r → 0, Dhσ,r has a uniformly bounded inverse Qσ,r : Lσ0 ,r ⊕ Lσ1 ,r −→ Wσ,r ⊕ K. (In the notation of [MS], Qσ,r = QuR ,vR .) As above, there is a projection pr σ0 ,r : Lσ0 ⊕ Lσ1 → K, and we write QYσ,r for the restriction of Qσ,r to the fiber over Y . As a guide to defining Qa,app , consider the diagram of spaces Wσ,r ⊕ K o Wa ⊕ K o

Lσ0 ,r ⊕ Lσ1 ,r O Qa,app

La ,

where the maps are given by (ξ0 , ξ1 , Y ) o (ξ, Y ) ∈ Wa ⊕ K o

Qσ,r Qa,app

(η0 ,O η1 ) η ∈ La .

We define the horizontal arrow Qa,app by following the other three arrows. Here ηα , α (r) extended by 0 as in [MS]. Note for α = 0, 1, is the restriction of η to )σ,α − Dw p that the ηα are in L even though they are not continuous. Next decompose η0 = η0 + ισ0 ,r (Y ) ∈ Im Dσ0 ,r + ισ0 ,r (K) = Lσ0 ,r , η1 = η1 + ισ1 ,r (Y ) ∈ Lσ1 ,r . Then (ξ0 , ξ1 ) = QYσ,r (η0 , η1 ). Note that ξ0 (w) = ξ1 (w) = v. We then define the α (r/δ) for α = 0, 1 and then extending section ξ by putting it equal to ξα on )α − Dw it over the neck using cutoff functions so that it equals ξ0 + ξ1 − v on the circle 0 (r) = ∂D 1 (r) ⊂ ) . In the formula below, we think of the gluing map : of ∂Dw a a w Proposition 2.11 as inducing identifications 0 r 0 1 1 − Dw (r) −→ Dw (r) − Dw (rδ), :a : A 0 = D w δ r 0 0 1 1 :a : A 1 = D w − Dw (r) − Dw (rδ) −→ Dw (r). δ


171

Then ξ is given by

 zδ  ξ0 (z) + β ξ1 :a (z) − v if z ∈ A0 , r ξ(z) =  ξ : (z) + 1 − β z ξ (z) − v if z ∈ A , 1 a 0 1 r

where β : C → [0, 1] is a cutoff function that equals 1 if |z| ≤ δ and equals 0 for |z| ≥ 1. The next lemma is the analog of [MS, Lemma A.4.2]. It shows that Qa,app is the approximate inverse that we are seeking. The norms used are the usual Lp -norms with respect to the chosen metric on )σ and the glued metrics on )a . Note that we suppose that the metrics on )a agree√with the standard model χr (|x|)|dx|2 on the annuli Diα (r/δ) − Diα (rδ) (where r = |a|) so that :a is an isometry. Lemma 4.7. For all sufficiently small δ, there is r(δ) > 0 and a cutoff function β such that for all η ∈ La , |a| ≤ r(δ)2 , we have Dha ⊕ ιa Qa,app η − η ≤ 1 !η!. 2 Proof. It follows from the definitions that Dha ⊕ ιa Qa,app η = η j

on each set )j −Dw (r/δ). (Observe that hσj ,a = ha on this domain so that ισj ,a = ιa here.) Therefore, we have only to consider what happens on the subannuli 0 0 r 0 0 A0 = D w − Dw (r), :a (A1 ) = :a Dw (r) − Dw (rδ) δ of )a . In this region the maps hσj ,a , as well as the glued map ha , are constant so that the maps ισj ,a , ιa are constant. Further, the linearizations Dhσj ,a and Dha are all equal and on functions coincide with the usual ∂-operator. We consider what happens in :a (A1 ), leaving the similar case of A0 to the reader. It is not hard to check that for z ∈ A1 , Dha ξ0 (z) = η0 (z) = −ιa (Y ), Dha ξ1 (:a z) = η1 (:a z). Let us write βr for the function βr (z) = β(z/r). Then if r 2 = |a| and (ξ, Y ) = Qa,app η, we have for z ∈ A1 , Dha ξ + ιa Y − η (:a z) = η1 (:a z) + (1 − βr ) − ιa Y − Dha (v) (z) − ∂(βr ) ⊗ (ξ0 − v)(z) + (ιa Y − η)(:a z) = (βr − 1) ιa Y + Dha (v) (z) − ∂(βr ) ⊗ (ξ0 − v)(z).

172

DUSA MCDUFF

Therefore, taking the Lp -norm, Dha ξ + ιa Y − η ◦ :a p L ,A1 ≤ ιa (Y ) Lp ,A + Dha (v)Lp ,A + ∂(βr ) ⊗ (ξ0 − v)Lp ,A . 1

1

1

If r is sufficiently small, by Lemma 4.5 we can suppose that !ιa (Y )!Lp ,A1 ≤ !η!/12. Moreover, because v is a constant section, Dha acts on v just by its zeroth order part, and so there are constants c1 , c2 such that Dha (v) p ≤ c1 !v!(area A1 )1/p ≤ c2 !v!r 2/p . L ,A1

Furthermore, by [MS, Lemma A.1.2], given any ε > 0, we can choose δε > 0 and β so that ∂(βr ) ⊗ (ξ0 − v) p ≤ ε!ξ0 − v! 1,p , W L ,A 1

for all δ ≤ δε . Hence

!η! !Dha ξ + ιa Y − η!Lp ,A1 ≤ c2 r 2/p + ε !v! + !ξ0 − v!W 1,p + 12 2/p !η! ≤ c3 r + ε η0 , η1 Lp + 12 2/p !η! ≤ c4 r + ε (η0 , η1 )Lp + 12 1 2/p = c4 r + ε + !η!, 12

where the second inequality holds because of the uniform estimate for the right inverse QYσ,r , and the third inequality holds because the projection of Lσ0 ,r ⊕ Lσ1 ,r onto the subspace ImDhσ,r ⊕Lσ1 ,r is continuous. Then if we choose δε so small that c4 ε < 1/12 and r # δε so small that c4 r < 1/12, we find Dha ξ + ιa Y − η ◦ :a p ≤ 1 !η!. L ,A1 4 Repeating this for A0 gives the desired result. Finally, we define the right inverse Qa by setting −1 Qa = Qa,app Dha ⊕ ιa Qa,app . It follows easily from the fact that the inverses Qσ,r are uniformly bounded for 0 < r ≤ r0 that the Qa are too. It remains to remark that the above construction can be carried out in such as way as to be 5σ -equivariant. The only choice left unspecified above is that of the right inverse Qσ,r . This in turn is determined by the choice of a subspace Rσ,r of W 1,p ), h∗σ,r (T X) complementary to the kernel of Dhσ,r . But since 5σ is finite, we can arrange that Rσ,r is 5σ -equivariant. For example, since Dhσ,r is a finite-dimensional space consisting of


173

C ∞ sections, we can take Rσ,r to be the L2 -orthogonal complement of Dhσ,r defined with respect to a 5σ -invariant norm on h∗σ,r (T X).4 Note that because hσ,r = hσ,r ◦ γ for γ ∈ 5σ , we can obtain a 5σ -invariant norm on h∗σ,r (T X) by integrating the pullback by hσ,r of any norm on the tangent bundle T X with respect to a 5σ -invariant area form on the domain )σ . We can achieve this uniformly over ᐆJ by choosing a suitable metric on each domain )σ as described at the beginning of §4.2.1. Remark 4.8. If one is gluing two branch components )j , j = 0, 1 of ), then both linearizations Dhj are surjective, and one can construct the inverse Qa to have image in Wa , thus forgetting about the summand K. The general gluing argument combines both of these cases. If one is gluing at N different points, then one needs to choose r so small that one has an inequality of the form Dha ξ + ιa Y − η

Lp ,A

≤

1 !η! 4N

on each of the 2N-annuli A. Note that the number of components of ) is bounded above by some number that depends on m (where J ∈ ᏶m ). Hence there is always r0 > 0 such that gluing at σ is possible for all r < r0 , provided that one is looking at a family of parametrized stable maps σ = (), hσ , J ) that is compact for each J and where J ∈ ᏶m is bounded in C ∞ -norm. This completes the proof of Lemma 4.2 and hence of Proposition 4.4. 4.2.4. Aut K ())-equivariance of Ᏻ˜ . Note that there is an action of S 1 on the pair (hσ , a) that rotates one of the components (say )1 ) of ) = )0 ∪ )1 , fixing the intersection point w = )0 ∩ )1 . We claim that by choosing an invariant metric on ), we can make the whole construction invariant with respect to the action of this compact group, that is, so that as unparametrized stable maps, )a , Ᏻ˜ (hσ , a) = )θ·a , Ᏻ˜ hσ · θ −1 , θ · a . To see this, note that there is an isometry ψ from the glued domain )a to )θ·a such that ha = hθθ·a ◦ ψ, where hθb denotes the pregluing of hθσ = hσ ·θ −1 with parameter b. There is a similar formula for the maps hσ,r . It is not hard to check that the rest of the construction can be made compatible with this S 1 action. It is important here to use the Fukaya-Ono choice of Rσ,r , as described above, instead of cutting down the domain of Dh fixing the images of certain points as in [LiT], [LiuT1], and [Sieb]. More generally, consider a parametrization σ˜ = (), h) and an arbitrary element 4 As

pointed out in [FO], the map ξ −→ ξ −

$ξ, ej %ej ,

ξ ∈ Wσ,a

j

is well defined whenever e1 , . . . , ep is a finite set of C ∞ -smooth sections.

174

DUSA MCDUFF

σ ∈ ᐆJ . Recall that a component of ) is said to be unstable if it contains less than three special points, that is, points where two components of ) meet. Each unstable branch component has at least one special point where it attaches to the rest of ), and so the identity component of its automorphism group has the homotopy type of a circle. Therefore, if there are k such unstable components, the torus group T k is a subgroup of Aut()). It is not hard to see that if the automorphism group 5σ˜ of σ˜ is nonzero, we can choose the action of T k to be 5σ˜ -equivariant so that the groups fit together to form the compact group Aut K ()) of Definition 2.9. Note further that if () , h ) is obtained from σ˜ = (), h) by gluing, then Aut K () ) can be considered as a subgroup of Aut K ()). To see this, suppose for example that ) is obtained by gluing )i to )ji with parameter a, and that both these components have at most one other special point. Then we can choose metrics on )i ∪ )ji that are invariant under an S 1 action in each component and so that the glued metric on )a is invariant under the action of an S 1 in AutK () ). Note that the diagonal subgroup S 1 × S 1 of AutK ()) acts trivially on the gluing parameters at the double point )i ∩ )ji since it rotates in opposite directions in the two tangent spaces. It is now easy to check that if we write θˆ for the image of θ ∈ S 1 in the diagonal subgroup of S 1 × S 1 , then ˆ a) . )a , h ◦ θ = )a , Ᏻ˜ (h, a) ◦ θ = )a , Ᏻ˜ (h ◦ θ, Observe also that if b is a gluing parameter at the intersection of )i with some other component )k of σ˜ , then it can also be considered as a gluing parameter for σ˜ . Moreover, under this correspondence, θˆ · b corresponds to θ · b. These arguments prove the following result. Lemma 4.9. Let σ˜ = (), h) and suppose that the metric on ) is Aut K ())invariant. Then the following statements hold. (i) The composite Ᏻ of Ᏻ˜ with the forgetful map into the space of unparametrized stable maps is Aut K ())-invariant. (ii) Divide the set P of double points of ) into two sets Pb , Ps and correspondingly write the gluing parameter a as ab + as . Suppose that () , h ) = ()ab , Ᏻ˜ (h, ab )) and consider as as a gluing parameter at σ˜ . Then we can choose metrics and choose the groups Aut K ()), Aut K () ) so that there is an inclusion Aut K ) −→ AutK ()) : θ −→ θˆ such that

)ab , h ◦ θ −1 ; θ · as = )ab , Ᏻ˜ h ◦ θˆ −1 , ab ; θˆ · as .

Further, this can be done continuously as ab (and hence h ) varies, and smoothly if )ab varies in a fixed stratum. Remark 4.10. In [LiuT2, §5], Liu and Tian also develop a version of gluing that is invariant with respect to a partially defined torus action.


175

4.2.5. Globalization. The preceding paragraphs construct the gluing map Ᏻ˜ (hσ , a) over a neighborhood ᏺ(σ ) of one point σ ∈ ᐆJ . We now show how to define a gluing map ᏳJ : ᏺᐂ (ᐆJ ) → ᏹ(A − kF, ᏷J ) on a whole neighborhood ᏺᐂ (ᐆJ ) of ᐆJ in the space of gluing parameters ᐂJ . The only difficulty in doing this lies in choosing a suitable parametrized representative s(σ ) = (), hσ ) of the equivalence class σ = [), h] as σ varies over ᐆJ . In other words, in order to define Ᏻ˜ (hσ , a), we need to choose a parametrization hσ : ) → X of the stable map σ , and now we have to choose this consistently as σ varies. We now show that although we may not be able to make a single-valued choice s(σ ) = hσ continuously over ᐆJ , we can find a section that at each point is well defined modulo the action of a suitable subgroup of Aut K ()). More precisely, we claim the following. Lemma 4.11. We may choose a continuous family of metrics gσ on )σ for σ ∈ ᐆJ and a family of parametrizations s(σ ) for each σ ∈ ᐆJ such that (i) s(σ ) consists of a Gσ -orbit of maps hσ : )σ → X, and gσ is Gσ -invariant, where Gσ ⊂ Aut K ()); (ii) the assignment σ → s(σ ) is continuous in the sense that near each σ , there is a (single-valued) continuous map σ → hσ ∈ s(σ ) whose restriction to each stratum is smooth. Moreover, gσ varies smoothly on each stratum. Proof. The strata in ᐆJ can be partially ordered with ᏿ ≤ ᏿ if there is a gluing that takes an element in the stratum ᏿ to one in ᏿ , that is, if the stratum ᏿ is contained in the closure of ᏿ . If ᏿ is maximal under this ordering and σ ∈ ᏿, then each branch component in ) is mapped to a fiber by a map of degree less than or equal to 1. It is easy to check that in this case there is a unique identification of the domain )σ with a union of spheres such that the map hσ is a section on the stem and, on each branch component, is either constant or the identity map (cf. Example 2.10). We assume this and then extend the choice of parametrization to a neighborhood of each of these maximal strata by gluing. We now start extending our choice s(σ ) = {hσ } of parametrization to the whole of ᐆJ by downwards induction over the partially ordered strata. Clearly we can always choose a parametrization modulo the action of AutK ()). In order for the image of the fiber πJ−1 (σ ) = {(σ, a) : |a| < ε} under the gluing map to be independent of this choice, we need the metric gσ on )σ to be AutK ()σ )-invariant. This choice of metric can be assumed to be smooth as σ varies in a stratum. However, it cannot always be chosen continuously as σ goes from one stratum to another. For example, if σ has one component )i with three special points at 0, 1, ∞ and that is glued to some component )ji at 1 with gluing parameter a, then the resulting component )a is unstable if )ji has no other special points. But for small |a|, the metric on )a is determined by the metrics on ) = )i ∪ )ji by the gluing construction and cannot be chosen to be S 1 -invariant. On the other hand, if both )i and )ji have at most one other special point, then the glued metric on )a is S 1 -invariant provided that the original metrics on )i , )ji are also S 1 -invariant.

176

DUSA MCDUFF

The above remarks show that suitable gσ᏿ , s(σ )᏿ and G᏿ σ can be defined over each stratum ᏿ and, in particular, over maximal strata. If gσ , s(σ ) and Gσ are already suitably defined over some union Y of strata, then the above remarks about gluing gl show that they can be extended to a neighborhood ᐁ(Y ) of Y . Let us write gσ , s(σ )gl gl and Gσ for the objects obtained by gluing when σ ∈ ᐁ(Y ). It is not hard to see that gl we can suppose that Gσ ⊂ G᏿ σ . Then if β : ᐁ(Y ) ∪ ᏿ → [0, 1] is a smooth cutoff function that equals 0 near Y and 1 near the boundary of ᐁ(Y ), set gσ = 1 − β(σ ) gσgl + β(σ )gσ᏿ , ᏿

s(σ ) = s(σ ) ,

if β(σ ) = 1,

= s(σ )gl , ᏿

Gσ = Gσ , = Ggl σ,

σ ∈ ᏿,

otherwise,

if β(σ ) = 1, otherwise.

It is easy to check that the required conditions are satisfied. Proof of Proposition 4.1. By Lemmas 4.9 and 4.11, there is a well-defined continuous gluing map ᏳJ : ᏺᐂ (ᐆJ ) −→ ᏹ A − kF, ᏷J that restricts on ᐆJ to the inclusion. Therefore, because ᐆJ is compact, the injectivity of ᏳJ on a small neighborhood ᏺᐂ (ᐆJ ) follows from the local injectivity statement in Proposition 4.4. Similarly, the local surjectivity of Proposition 4.4 implies that the image of ᏳJ is open in ᏹ(A − kF, ᏷J ). Note that all the restrictions made on the size of ᏺᐂ (ᐆJ ) vary smoothly with J (and involve no more than the C 2 norm of J ). Hence ∪J Im ᏳJ is an open subset of ᏹ(A − kF, ᏶). References [A] [AM] [BL] [FO] [G] [HLS] [HS] [K]

M. Abreu, Topology of symplectomorphism groups of S 2 × S 2 , Invent. Math. 131 (1998), 1–23. M. Abreu and D. McDuff, Topology of symplectomorphism groups of rational ruled surfaces, in preparation. J. Bryan and C. Leung, The enumerative geometry of K3 surfaces and modular forms, preprint. K. Fukaya and K. Ono, Arnold conjecture and Gromov-Witten invariant, Topology 38 (1999), 933–1048. L. Godinho, Circle actions on symplectic manifolds, Ph.D. thesis, State Univ. of New York at Stony Brook, 1999. H. Hofer, V. Lizan, and J.-C. Sikorav, On genericity for holomorphic curves in fourdimensional almost-complex manifolds, J. Geom. Anal. 7 (1997), 149–159. H. Hofer and D. Salamon, Gromov compactness and stable maps, preprint, 1997. P. Kronheimer, Some nontrivial families of symplectic structures, preprint, 1998.

ALMOST COMPLEX STRUCTURES ON S 2 × S 2 [LM]

[LL1] [LL2] [LiT]

[LiuT1] [LiuT2] [Lo] [MP] [MS] [R] [S] [YZ]

177

F. Lalonde and D. McDuff, “J -curves and the classification of rational and ruled symplectic 4-manifolds” in Contact and Symplectic Geometry (Cambridge, 1994), Publ. Newton Inst. 8, Cambridge Univ. Press, Cambridge, 1996, 3–42. T. J. Li and A. Liu, Symplectic 4-manifolds with torsion canonical classes, in preparation. , Family Seiberg-Witten invariants and wall crossing formulas, preprint. Jun Li and Gang Tian, “Virtual moduli cycles and Gromov-Witten invariants of general symplectic manifolds” in Topics in Symplectic 4-Manifolds (Irvine, Calif., 1996), First Int. Press Lect. Ser., Internat. Press, Cambridge, Mass., 1998, 47–83. Gang Liu and Gang Tian, Floer homology and Arnold conjecture, J. Differential Geom. 49 (1998), 1–74. , Weinstein conjecture and GW invariants, preprint, 1997. W. Lorek, Generalized Cauchy-Riemann operators in symplectic geometry, Ph.D. thesis, State Univ. of New York at Stony Brook, 1996. D. McDuff and L. Polterovich, Symplectic packings and algebraic geometry, Invent. Math. 115 (1994), 405–434. D. McDuff and D. A. Salamon, J -Holomorphic Curves and Quantum Cohomology, Univ. Lecture Ser. 6, Amer. Math. Soc. Providence, 1994. Yongbin Ruan, Virtual neighborhoods and pseudo-holomorphic curves, preprint, http://xxx.lanl.gov/abs/alg-geom/9611021. B. Siebert, Gromov-Witten invariants for general symplectic manifolds, preprint, 1996. S.-T. Yau and E. Zaslow, BPS states, string duality, and nodal curves on K3, Nuclear Phys. B 471 (1996), 503–512.

Department of Mathematics, State University of New York at Stony Brook, Stony Brook, New York 11794-3651, USA; [email protected]

Vol. 101, No. 1


© 2000

TANGENT VECTORS TO HECKE CURVES ON THE MODULI SPACE OF RANK 2 BUNDLES OVER AN ALGEBRAIC CURVE JUN-MUK HWANG

The moduli space of semistable bundles of a fixed determinant over an algebraic curve has been studied by many authors from various points of view. Especially wellunderstood is the case of rank 2 bundles. In this case, quite a detailed study of the geometry of the moduli space has been done in [Be], [BV], and [DR]. This moduli space is a Fano variety of Picard number 1. In a series of joint works with N. Mok, we have studied the geometry of Fano manifolds of Picard number 1 by investigating the projective geometry of the variety of tangent directions of the minimal rational curves (see [HM3] for a survey). Our aim here is to apply this study to the moduli space Mi of rank 2 bundles of a fixed determinant of degree i = 0, 1 over an algebraic curve of genus g ≥ 2. In this case, the minimal rational curves in the sense of [HM3] turn out to be “Hecke curves,” originally introduced by Narasimhan and Ramanan [NR1], [NR2]. Using the results in [NR2, Section 5], we study the variety of tangent vectors to Hecke curves through a fixed point on Mi . As a consequence of this study and our previous work [H], we get the following result, which seems to have been speculated by experts in this field. Theorem 1. Let M1 be the moduli space of stable bundles of rank 2 with a fixed determinant of odd degree over an algebraic curve of genus g ≥ 2. Then the tangent bundle of M1 is stable. Our next result is on the deformation rigidity of generically finite morphisms over Mi . A result of this type is expected for many Fano manifolds of Picard number 1, and the result here can be regarded as an example (see [HM1] for a general discussion). To streamline the presentation, we assume g ≥ 4 for M0 and g ≥ 5 for M1 in the next theorem. It may be possible to extend our result to some cases of lower genus, but one would need a different idea to cover all the cases, especially the case of g = 2. Theorem 2. Assume g ≥ 4 for i = 0 and g ≥ 5 for i = 1. Let Y be any projective manifold of dimension 3g − 3, and let f : Y → Mi be a surjective holomorphic map. Received 20 April 1998. Revision received 2 March 1999. 1991 Mathematics Subject Classification. Primary 14H60; Secondary 14J45. Author supported by Seoul National University Research Fund and by grant number 98-0701-01-5-L from the Korea Science and Engineering Foundation. 179

180

JUN-MUK HWANG

Then f is locally rigid in the sense that any deformation {ft : Y → Mi , t ∈ C, |t| < , f0 = f } satisfies ft = f for all t. Theorem 2 follows from a detailed study of the variety of tangent vectors to Hecke curves using the deep results of [Be] and [BV]. Theorem 2 can be regarded as a generalization of Narasimhan and Ramanan’s result that Mi does not have any nonzero vector field (see [NR1]). Indeed our proof is well in the spirit of [NR2, Remark 5.17(iii)] concerning the original proof of results in [NR1]. This paper consists of three sections. The first section is a collection of relevant results of Narasimhan and Ramanan from [NR2]. The proofs of Theorems 1 and 2 are given in the second and third sections, respectively. 1. Hecke curves. Throughout this paper, X is a nonsingular complex algebraic curve of genus g ≥ 2. Let Mi be the moduli space of semistable bundles of rank 2 over X with a fixed determinant of degree i, i = 0, 1. They have dimension 3g −3. Let Mis ⊂ Mi be the locus of stable bundles. Then M1s = M1 , and M0s is the smooth part of M0 . We use the notation W ∈ Mi to denote the point corresponding to a semistable bundle W on X. It is well known that Pic(Mi ) ∼ = Z. We use the additive notation for the group multiplication of Pic(Mi ). When ᏸi is the ample generator of Pic(Mi ), −KM0 = 4ᏸ0 and −KM1 = 2ᏸ1 (see [Be], [R]). Hecke curves on Mi were introduced by Narasimhan and Ramanan in [NR1], [NR2]. We recall some basic properties of Hecke curves. Almost all the results in this section can be found in [NR2, Section 5]. Given two nonnegative integers k and l, a vector bundle W of rank 2 and degree i is (k, l)-stable if, for every line subbundle L of W , we have deg(L) + k < (1/2)(i + k − l). The (0, 0)-stability is equivalent to the usual stability. A (k, l)-stable bundle is (k, l −1)-stable for l > 0. The dual bundle of a (k, l)-stable bundle is (l, k)-stable. We need the next two propositions, proved in [NR2], to explain the definition of Hecke curves. Proposition 1 [NR2, 5.4]. A generic W ∈ M1 is (1, 1)-stable if g ≥ 3. A generic W ∈ M0 is (1, 1)-stable if g ≥ 4. A generic W ∈ Mi is (1, 2)-stable if g ≥ 5. Proposition 2 [NR2, 5.5]. Given an exact sequence 0 → V → W → ᏻx → 0, where V , W are vector bundles of rank 2 and ᏻx is the skyscraper sheaf at a point x ∈ X, if W is (k, l)-stable, then V is (k, l − 1)-stable. From now on, we assume that g ≥ 3 for i = 1 and g ≥ 4 for i = 0 so that a generic point of Mi is (1, 1)-stable. For a rank 2 bundle W , its projectivization PW is canonically isomorphic to the projectivization of its dual PW ∗ . Given a point η ∈ PW , we use the same letter η to denote the 1-dimensional spaces in W and W ∗ corresponding to η. Let W ∈ Mi be a (1, 1)-stable bundle. Let π : PW → X be the natural projection. Given a point η ∈ PW with x = π(η) ∈ X, the canonical projection Wx → Wx /η

HECKE CURVES

181

defines a new rank 2 bundle W η by 0 −→ W η −→ W −→ ᏻx ⊗ (Wx /η) −→ 0. Let V be W η∗ , the dual of W η . Then det(V ) = − det(W ) + Lx , where Lx is the line bundle on X corresponding to the divisor x. For each ν ∈ PVx , we have 0 −→ V ν −→ V −→ ᏻx ⊗ (Vx /ν) −→ 0. V ν∗ has the same determinant as W , and V ν is stable from Proposition 2. Thus the family {V ν∗ , ν ∈ PVx } defines a rational curve on Mis , called the Hecke curve associated to η ∈ PW . By a Hecke curve, we mean a rational curve on Mi which is the Hecke curve associated to some η ∈ PW for some (1, 1)-stable bundle W . η At the point x, the map Wx → Wx has 1-dimensional kernel. Let η ∈ PVx be its annihilator. By writing down transition matrices explicitly, one can see easily that the dual of V η is isomorphic to W (see [NR1, 4.11] for a proof). So the Hecke curve associated to η ∈ PW is a rational curve through W ∈ Mi . The next four propositions were proved in [NR2]. Proposition 3 [NR2, 5.16]. A Hecke curve has degree 4 with respect to −KMi . Proposition 4 [NR2, 5.9]. A Hecke curve is a smooth rational curve on Mis . Proposition 5 [NR2, 5.15]. The normal bundle of a Hecke curve in Mis is generated by global sections. Proposition 6 [NR2, 5.13]. For any (1, 1)-stable W ∈ Mi , the Hecke curves associated to two distinct η1 , η2 ∈ PW are distinct rational curves on Mi . For the next proposition, we need some definitions. The family of bundles {V ν , ν ∈ PVx } whose duals define the Hecke curve associated to η ∈ PW can be viewed as a deformation of the bundle W ∗ = V η . Let ζ : Tη (PVx ) = Hom(η , Vx /η ) → η

η

Hom(Vx , Vx /η ) be the map induced by β : Vx → η in the defining exact sequence β 0 −→ V η −→ V −→ ᏻx ⊗ Vx /η −→ 0. η

Let δ : Hom(Vx , Vx /η ) → H 1 (X, End(V η )) be the connecting homomorphism for the long exact sequence coming from 0 −→ End(V η ) −→ Hom(V η , V ) −→ Hom V η , ᏻx ⊗ Vx /η −→ 0. Proposition 7 [NR2, 5.10]. The Kodaira-Spencer map Tη (PVx ) → H 1 (X, ad(V η )) associated to the deformation {V ν , ν ∈ PVx } of V η is equal to δ ◦ ζ up to sign. Since the next two propositions are not given explicitly in [NR2], we provide proofs.

182

JUN-MUK HWANG

Proposition 8. Let W ∈ Mi be a generic point. Then Hecke curves through W have minimal degree among rational curves through W . Proof. For M0 , this is immediate from −KM0 = 4ᏸ0 and Proposition 3. For M1 , we have to show that there exists no rational curve of degree 1 with respect to ᏸ1 through W . By Kodaira’s stability (see [K]), if a rational curve of degree 1 exists at a generic point of M1 for some X, such a curve exists at a generic point of M1 for any X of the same genus. In particular, such a curve exists when X is hyperelliptic. From [DR, Theorem 1], M1 for a hyperelliptic curve is the set of (g −2)-dimensional linear subspaces in the intersection of two quadrics in P2g+1 determined by the hyperelliptic curve. A line on M1 gives a (g − 1)-dimensional linear subspace in the intersection of the two quadrics. If lines exist through generic points of M1 , we have at least a (3g − 3) − (g − 1) = (2g − 2)-dimensional family of (g − 1)-dimensional linear subspaces in the intersection of the two quadrics. But [DR, Theorem 2] says that the set of (g − 1)-dimensional linear subspaces of the intersection of the two quadrics is equivalent to the Jacobian of X, a contradiction to our assumption g ≥ 3. Proposition 9. Let ᏷ be an irreducible component of the Hilbert scheme of Mi containing a Hecke curve. Then ᏷ is smooth at the point corresponding to the Hecke curve, and all Hecke curves belong to ᏷. Let W ∈ Mi be (1, 1)-stable, and let ᏷W be the subscheme of ᏷ consisting of curves passing through W . Then ᏷W is smooth and consists of Hecke curves through W . In particular, ᏷W is naturally biholomorphic to PW . Proof. From [NR2, 5.15], the set of all Hecke curves form an irreducible family. Thus all Hecke curves belong to ᏷. Let C be a Hecke curve through W , and let N be its normal bundle in Mi . From Proposition 5, H 1 (C, N) = 0 and H 1 (C, N(−1)) = 0, which is equivalent to the smoothness of ᏷ and ᏷W at the point C. From Proposition 3, dim(᏷) = h0 (C, N ) = 3g − 2 and dim(᏷W ) = h0 (C, N(−1)) = 2. Thus Hecke curves are dense in ᏷, and ᏷W consists of Hecke curves through W . From Proposition 6, the classifying map PW → ᏷W is biholomorphic. 2. Proof of Theorem 1. In this section, we prove Theorem 1. For g = 2, the stability of the tangent bundle of a Fano 3-fold of Picard number 1 is well known. So we assume g ≥ 3. The proof uses a result of [H], which we briefly recall here. Let Z be any n-dimensional Fano manifold of Picard number 1. Fix an irreducible component ᏷ of the Hilbert scheme of rational curves on Z so that members of ᏷ cover Z and a generic member has minimal degree among rational curves through generic points of Z. For a torsion-free sheaf Ᏺ of rank r > 0 on Z, choose a ᏷-curve C disjoint from the singular loci of Ᏺ and define the slope of Ᏺ to be µ(Ᏺ) := (1/r) det(Ᏺ) · C. A vector bundle E of rank k on Z is stable if, for every subsheaf Ᏺ of rank r with 0 < r < k, the inequality µ(Ᏺ) < µ(E) holds. For a generic point z ∈ Z, let ᏷z be the subscheme of ᏷ corresponding to curves through z and let ᏷zo ⊂ ᏷z be the subscheme corresponding to curves smooth at z.

HECKE CURVES

183

For each member of ᏷zo , we associate its tangent vector at z to define a rational map *z : ᏷z → PTz (Z), which is called the tangent map. Let Ꮿz ⊂ PTz (Z) be the strict image of *z . Let p be the dimension of ᏷z . The following is Proposition 4 of [H]. Proposition 10. In the above situation, if the tangent bundle of Z is not stable, then there exists a subsheaf Ᏺ of rank r in T (Z) such that, for a generic point z ∈ Z and a generic point α of any component of Ꮿz , the intersection of the projective tangent space to Ꮿz at α and PᏲz in PTz (Z) is nonempty and has dimension larger than or equal to (r/n)(p + 2) − 1. In fact, if we choose Ᏺ as the subsheaf of maximal slope, it is not difficult to translate µ(Ᏺ) ≥ µ(T (Z)) into the above form, by examining the splitting type of Ᏺ on a generic ᏷-curve. See [H] for details. We apply the above result to the case Z = M1 , where p = 2 and n = 3g − 3. We choose ᏷ as the component of the Hilbert scheme containing Hecke curves given in Proposition 9. Then z is a general (1, 1)-stable W ∈ M1 . Since all Hecke o = ᏷ ∼ PW . So the tangent map * : curves are smooth (see Proposition 4), ᏷W W = W PW → PTW (Mi ) is a morphism, associating its tangent vector at W to a Hecke curve through W . To use Proposition 10 here, we need an explicit description of *W . For later use, we include the case of M0 in the next proposition. Although M0 is not smooth, we can define the tangent morphism *W : PW → PTW (Mi ) for any (1, 1)-stable W ∈ M0 , just as in the case of M1 . Proposition 11. Assume that g ≥ 3 for i = 1 and g ≥ 4 for i = 0. For a generic point W ∈ Mi , the tangent morphism *W is given by the complete linear system |2π ∗ KX − KPW | and is finite over its image. Proof. We can describe the map φ defined by the complete linear system |2π ∗ KX− KPW | as follows. Note that 2π ∗ KX − KPW = π ∗ KX + .π , where .π is the relative tangent sheaf. From π∗ .π = ad(W ) and R j π∗ .π = 0, j > 0 (see [NR1, 2.3]), H 0 (PW, 2π ∗ KX − KPW ) = H 0 (X, KX ⊗ ad(W )). Given a point η ∈ PWx , we have the corresponding η ∈ PWx∗ , which we denote by η⊥ in this proof to avoid confusion. Consider η⊥ ⊗ η ∈ P ad(Wx ) ⊂ PWx∗ ⊗ Wx . A representative of η⊥ ⊗ η in ad(Wx ) defines a linear functional H 0 (X, KX ⊗ ad(W )) → KX, x by taking the trace of endomorphisms of Wx , which gives the element φ(η) ∈ PH 0 (X, KX ⊗ ad(W ))∗ . On the other hand, from the definition of the Hecke curve associated to η ∈ PWx , *W (η) ∈ PTW (Mi ) = PH 1 (X, ad(W )) is represented by the Kodaira-Spencer class of the family {V ν∗ , ν ∈ PVx } at W = V η ∗ , where η ∈ PVx is the annihilator of the η kernel of the map Wx → Wx . Since we are interested in the projectivization, we may consider the Kodaira-Spencer class of the family {V ν , ν ∈ PVx } at V η , or the image of the map δ ◦ ζ in Proposition 7. We can find a cocycle representing this class as follows. Choose a coordinate covering {U, U1 , . . . , UN } of X so that all vector bundles are

184

JUN-MUK HWANG

trivial on U, Uj and x ∈ U , x ∈ Uj . On Uj , let us identify W ∗ = V η and V once and for all by the map β : V η → V in the exact sequence defining V η . Choose a coordinate z on U centered at x and choose frames {e1 , e2 } of V η and {f1 , f2 } of η V over U so that e1, x ∈ η⊥ ⊂ Wx∗ = Vx , f2,x ∈ η , β(e1 ) = zf1 , and β(e2 ) = f2 . η Given a nonzero vector v ∈ Hom(η , Vx /η ), ζ (v) = v ◦ β ∈ Hom(Vx , Vx /η ) sends e1, x to zero and e2, x to a nonzero element, namely, an element of Vx /η represented by a constant multiple of f1, x . After multiplying v by a suitable nonzero constant, ˜ 1 ) = 0, v(e ˜ 2 ) = f1 . ζ (v) can be extended to v˜ ∈ H 0 (U, Hom(V η , V )) satisfying v(e 0 η η Then δ(ζ (v)) is defined by the cocycle vˆj ∈ H (U ∩ Uj , Hom(V , V )) obtained by composing v˜ with the inverse of the isomorphism β|U ∩Uj : V η |U ∩Uj → V |U ∩Uj . ∗ ∗ Then vˆj (e1 ) = 0, vˆj (e2 ) = (1/z)e1 . Thus when {e1 , e2 } is the dual frame, *W (η) is represented by the cocycle {(1/z)e2∗ ⊗ e1 on U ∩ Uj }. ∗ ∈ η. It follows that the cocycle From the choice of e1 and e2 , e1, x ∈ η⊥ and e2, x ∗ ⊥ 0 {(1/z)e2 ⊗e1 } corresponds to η ⊗η ∈ PH (X, KX ⊗ad(W ))∗ via the residue pairing giving the Serre duality H 1 (X, ad(W )) = H 0 (X, KX ⊗ ad(W ))∗ . Thus *W (η) = φ(η). It remains to show that *W is finite over its image. In the notation of [Ha, V.2], the numerical class of 2π ∗ KX −KPW is 2C0 +(2g −2+e)f with e < 0 for a stable bundle W (see [Ha, Remark 2.16.1, page 379]). Thus it is ample by [Ha, Proposition 2.21, page 382], and *W is finite over its image. We are ready to prove Theorem 1. From Proposition 10, it suffices to have the following proposition. Proposition 12. Let ᏯW ⊂ P3g−4 = PTW (M1 ) be the image of PW under the finite morphism *W defined by 2π ∗ KX − KPW . Given any linear subspace PF ⊂ PTW (M1 ) of dimension r − 1, its intersection with the projective tangent space at a generic point of ᏯW is either empty or has dimension smaller than (4r/(3g −3))−1. Proof. Suppose that the intersection is nonempty and of dimension greater than or equal to (4r/(3g − 3)) − 1. Since the surface ᏯW is nondegenerate in PTW (M1 ), the intersection can have dimension 0 or 1. If the intersection has dimension 1, then the projection from PF sends the tangent space at a generic point of ᏯW to zero. Thus the projection sends ᏯW to a point. This implies that ᏯW is contained in some linear subspace Pr containing PF , a contradiction to the nondegeneracy of ᏯW . It follows that the intersection has dimension 0 and r ≤ (3/4)(g − 1). Moreover, the projection from PF projects ᏯW to a curve l in P3g−4−r . Suppose the *W -image of a generic fiber of π : PW → X is dominant over l. Since the image of this fiber under *W is of degree less than or equal to 2, l must be contained in a plane. This implies that ᏯW is contained in some Pr+2 containing PF , a contradiction to the nondegeneracy of ᏯW again. Thus the projection to P3g−4−r contracts generic fibers of π to a point. It follows that the *W -image of a generic

HECKE CURVES

185

fiber of π is contained in some linear subspace Pr containing PF as a hyperplane, and it intersects PF . Let 5 ⊂ |2π ∗ KX − KPW | be the subsystem of dimension 3g − 4 − r defining the projection of PW to P3g−4−r from PF . Let D ⊂ PW be the fixed component of 5. D corresponds to the intersection of ᏯW with PF . Hence generic fibers of π : PW → X intersect D twice, counting multiplicity. Using the notation of [Ha, V.2], the numerical class of D is of the form 2C0 +df for some integer d. From [Ha, V.2], the numerical class of 2π ∗ KX − KPW is 2C0 + (2g − 2 + e)f . Thus the moving part of the system 5 is just the pullback of a linear system on X of degree 2g − 2 + e − d. By Nagata, 0 < C02 = −e ≤ g (see [Na]). Since C0 is ample (see [Ha, V.2, Proposition 2.21]), 0 < D · C0 and −2e + d > 0. So 5 is the pullback of a linear system of degree less than or equal to 3g − 3. By the Riemann-Roch theorem and Clifford’s theorem (see [Ha, page 343]), dim(5) ≤ max((3/2)(g − 1), 2g − 3) = 2g − 3. Combined with dim(5) = 3g − 4 − r, we get g ≤ r + 1, a contradiction to r ≤ (3/4)(g − 1). 3. Proof of Theorem 2. For the proof of Theorem 2, we need some refinements of Proposition 11. These follow from the results of [Be] and [BV] rather easily. Proposition 13. Assume g ≥ 4. Then *W is an embedding for any (1, 1)-stable W ∈ M0 , except possibly when X is hyperelliptic and W is a fixed point of the involution of M0 induced by the hyperelliptic involution of X. Proof. We use the following result from [Be] and [BV]. Theorem (Beauville, Brivio-Verra). The complete linear system associated to ᏸ0 is base point free, defining a morphism ψ : M0 → PH 0 (M0 , ᏸ0 )∗ . If X is not hyperelliptic, ψ is an embedding on M0s . If X is hyperelliptic, ψ is equivalent to the quotient by the involution of M0 induced by the hyperelliptic involution of X. Since Hecke curves on M0 have degree 1 with respect to ᏸ0 , the tangent morphism *W at a point W ∈ M0 (where ψ is immersive), is just associating to lines through ψ(W ), their tangent vectors. Thus *W is an embedding of PW for any (1, 1)-stable W that is not a fixed point of the hyperelliptic involution. Proposition 14. Assume g ≥ 5. Then *W is birational for any generic W ∈ M1 . Proof. For η ∈ PW , PW η is related to PW by the elementary transformation as follows. The blow-up of PW at η is naturally biholomorphic to the blow-up of PW η at η . The exceptional divisor over η corresponds to the strict transform of the fiber π −1 (π(η )), and the exceptional divisor over η corresponds to the strict transform of the fiber π −1 (π(η)). Namely, the projectivized tangent space at η corresponds to the fiber π −1 (π(η )). Under this correspondence, a section of 2π ∗ KX − KPW η on PW η vanishing at η can be lifted to the blow-up of PW η at η and then pushed to a section of 2π ∗ KX − KPW on PW vanishing at η. For a generic (1, 2)-stable W ∈ M1 and a generic point η ∈ PW , W η ∈ M0 is

186

JUN-MUK HWANG

(1, 1)-stable and is not a fixed point of the hyperelliptic involution. Thus 2π ∗ KX − KPW η is very ample from Proposition 13. For any ζ ∈ PW with π(η) = π(ζ ), we can find a section of 2π ∗ KX − KPW η on PW η vanishing at η and nonvanishing at ζ . Thus η and ζ can be separated by a section of 2π ∗ KX − KPW . If η = ζ ∈ PW with π(η) = π(ζ ), they correspond to two different tangent vectors at η in PW η and can be separated by a section of 2π ∗ KX − KPW . This shows that *W is birational. To prove Theorem 2, we combine Propositions 13 and 14 with the result in [HM2, Section 1]. To explain the latter result, we need some definitions. Let g : S → Z be a regular map between two quasiprojective complex algebraic varieties. We can stratify S and Z into finitely many nonsingular quasiprojective subvarieties. On the other hand, given g : S → Z with both S and Z smooth, we can stratify S into finitely many quasiprojective subvarieties on each of which g has constant rank. Applying these two stratifications repeatedly, we can stratify S naturally into finitely many irreducible quasiprojective nonsingular subvarieties S = S1 ∪· · ·∪Sk , such that for each i, the image g(Si ) is nonsingular and the holomorphic map g|Si : Si → g(Si ) is of constant rank. It is called the g-stratification of S. Let Y be a projective manifold, and let y ∈ Y be a point. We consider the subscheme of the Hilbert scheme of curves on Y passing through y, parametrizing irreducible and reduced curves smooth at y with a fixed geometric genus, and we denote by ᏿y the underlying quasiprojective variety of an irreducible component of that subscheme. For each member l of ᏿y , let *y (l) ∈ PTy (Y ) be the tangent to the curve l at y. This defines the tangent map *y : ᏿y → PTy (Y ). Let {Si } be the *y -stratification of ᏿y . A subvariety of PTy (Y ) is called a variety of distinguished tangents in PTy (Y ), if it is the closure of the image *y (Si ) for some choice of ᏿y and Si . Note that there exist only countably many subvarieties in PTy (Y ) which can serve as varieties of distinguished tangents, because the Hilbert scheme has only countably many components. Let Z be a Fano variety and let ᏷ be as in the beginning of Section 2, namely, it is an irreducible component of the Hilbert scheme of curves so that a generic member of ᏷ is a rational curve of minimal degree through a generic point of Z. When Z has singularity, we assume that, for a generic point z ∈ Z, all members of ᏷z lie on the smooth part of Z. For a generic point z ∈ Z, let Ꮿz ⊂ PTz (Z) be the strict image of the tangent map *z : ᏷z → PTz (Z). The main result of [HM2, Section 1] is the following. The statement is slightly different from [HM2], but the proof works verbatim. Proposition 15 [HM2, Proposition 3]. Let f : Y → Z be a generically finite morphism from a projective manifold Y onto a Fano variety Z of Picard number 1 which has a family of rational curves ᏷ with the above mentioned properties. Let z ∈ Z and y ∈ f −1 (z) so that df : Ty (Y ) → Tz (Z) is an isomorphism. If we choose z and y generically, then each irreducible component of df −1 (Ꮿz ) ⊂ PTy (Y ) is a variety of distinguished tangents. Now we are ready to prove Theorem 2. We apply Proposition 15 to Z = Mi . Let ᏷

HECKE CURVES

187

be the Hilbert component containing Hecke curves. It satisfies the above requirement. Let ᏯW ⊂ PTW (Mi ) be the image of *W for (1,1)-stable W ∈ Mi . For a generic point y ∈ Y , let Wt = ft (y). By Proposition 15, dft,−1 y (ᏯWt ) is a family of varieties of distinguished tangents, and it must be a fixed subvariety of PTy (Y ) from the countability of varieties of distinguished tangents. It follows that ᏯWt is biholomorphic to ᏯW0 for all t. But from Proposition 13 for i = 0, g ≥ 4, and Proposition 14 for i = 1, g ≥ 5, this implies that PWt ∼ = PW0 . It follows that Wt = W0 and ft = f0 . Acknowledgement. We wish to thank Professor Ngaiming Mok for encouragement, Dr. Julee Kim for a special help, and the referee for many helpful suggestions. References [Be] [BV] [DR] [Ha] [H] [HM1] [HM2] [HM3] [K] [Na] [NR1] [NR2] [R]

A. Beauville, Fibrés de rang 2 sur une courbe, fibré déterminant et fonctions thêta, Bull. Soc. Math. France 116 (1988), 431–448. S. Brivio and A. Verra, The theta divisor of SU C (2, 2d)s is very ample if C is not hyperelliptic, Duke. Math. J. 82 (1996), 503–552. U. V. Desale and S. Ramanan, Classification of vector bundles of rank 2 on hyperelliptic curves, Invent. Math. 38 (1976/77), 161–185. R. Hartshorne, Algebraic Geometry, Grad. Texts in Math. 52, Springer-Verlag, New York, 1977. J.-M. Hwang, Stability of tangent bundles of low-dimensional Fano manifolds with Picard number 1, Math. Ann. 312 (1998), 599–606. J.-M. Hwang and N. Mok, Cartan-Fubini type extension of holomorphic maps for Fano manifolds with Picard number 1, preprint, 1998. , Holomorphic maps from rational homogeneous spaces of Picard number 1 onto projective manifolds, Invent. Math. 136 (1999), 209–231. , Varieties of minimal rational tangents on uniruled projective manifolds, to appear in the Proceedings of MSRI Special Year in Several Complex Variables. K. Kodaira, On stability of compact submanifolds of complex manifolds, Amer. J. Math. 85 (1963), 79–94. M. Nagata, On self-intersection number of a section on a ruled surface, Nagoya Math. J. 37 (1970), 191–196. M. S. Narasimhan and S. Ramanan, Deformations of the moduli space of vector bundles over an algebraic curve, Ann. of Math. (2) 101 (1975), 391–417. , “Geometry of Hecke cycles, I” in C. P. Ramanujam—A Tribute, Tata Inst. Fund. Res. Studies in Math. 8, Springer-Verlag, Berlin, 1978, 291–345. S. Ramanan, The moduli spaces of vector bundles over an algebraic curve, Math. Ann. 200 (1973), 69–84.

Department of Mathematics, Seoul National University, Seoul, 151-742, Korea; [email protected]

Vol. 101, No. 2


© 2000

ON THE MORGAN-SHALEN COMPACTIFICATION OF THE SL(2, C) CHARACTER VARIETIES OF SURFACE GROUPS G. DASKALOPOULOS, S. DOSTOGLOU, and R. WENTWORTH 1. Introduction. Let 6 be a closed, compact, oriented surface of genus g ≥ 2 and fundamental group 0. Let X(0) denote the SL(2, C) character variety of 0, and D(0) ⊂ X(0) the closed subset consisting of conjugacy classes of discrete, faithful representations. Then X(0) is an affine algebraic variety admitting a compactification X(0) (due to Morgan and Shalen [MS1]), whose boundary points ∂ X(0) = X(0) \ X(0) correspond to elements of PL(0), the space of projective classes of length functions on 0 with the weak topology. Choose a metric σ on 6, and let MHiggs (σ ) denote the moduli space of semistable rank-2 Higgs pairs on 6 (σ ) with trivial determinant, as constructed by Hitchin [H]. Then MHiggs (σ ) is an algebraic variety, depending on the complex structure defined by σ (cf. [Si]). By the theorem of Donaldson [D], MHiggs (σ ) is homeomorphic to X(0), though not complex-analytically so. Let us denote this map h : X(0) → MHiggs (we henceforth assume the choice of base point σ ). We define a compactification of MHiggs as follows: Let QD (more precisely, QD(σ )) denote the finite-dimensional complex vector space of holomorphic quadratic differentials on 6. Then there is a surjective, holomorphic map MHiggs → QD taking the Higgs field 8 to ϕ = det 8. We compose this with the map ϕ −→ where kϕk =

R

6 |ϕ|,

4ϕ , 1 + 4kϕk

and obtain

f : MHiggs −→ BQD = {ϕ ∈ QD : kϕk < 1} . det Let SQD = {ϕ ∈ QD : kϕk = 1} be the space of normalized holomorphic quadratic differentials. We then define MHiggs = MHiggs ∪ SQD with the topology given via the Received 6 August 1998. 1991 Mathematics Subject Classification. Primary 58E20; Secondary 20E08, 30F30, 32G13. Daskalopoulos’s work partially supported by National Science Foundation grant number DMS9803606. Dostoglou’s work partially supported by the Research Board of the University of Missouri and the Arts and Science Travel Fund of the University of Missouri, Columbia. Wentworth’s work partially supported by National Science Foundation grant number DMS-9971860 and a Sloan Fellowship. 189

190

DASKALOPOULOS, DOSTOGLOU, AND WENTWORTH

f The aim of this paper is to compare the two compactifications X(0) and map det. MHiggs . The points of PL(0) may be regarded as arising from the translation lengths of minimal, nontrivial 0 actions on R-trees. Modulo isometries and scalings, this correspondence is one-to-one, at least in the nonabelian case (cf. [CM] and our Section 2). The boundary ∂ D(0) consists of small actions, that is, those for which the arc-stabilizer subgroups are all cyclic. With our choice of conformal structure σ , we can define a continuous, surjective map (1.1)

H : PL(0) −→ SQD .

When the length function [`] is realized by the translation length function of a tree dual to the lift of a normalized holomorphic quadratic differential ϕ, then H ([`]) = ϕ; the full map is a continuous extension of this (see Theorem 3.9) with the fibers of H corresponding more generally to foldings of dual trees. Let PMF(0) denote the space of projective classes of measured foliations on 6, modulo isotopy and Whitehead equivalence (cf. [FLP, exposé 5]). By the theorem ∼ of Hubbard-Masur [HM] we also have a homeomorphism HM : PMF(0) −→ SQD. It is not clear how to lift H to factor through PMF(0) in a manner independent of σ . However, it follows essentially by Skora’s theorem [Sk] that if H is restricted to PSL(0), the small actions, then it factors through HM by a homeomorphism ∼ PSL(0) −→ PMF(0). With this understood, we define a (set-theoretic) map (1.2)

h¯ : X(0) −→ MHiggs

by extending the map h to H on the boundary. We prove the following. Main Theorem. The map h¯ is continuous and surjective. Restricted to the compactification of the discrete, faithful representations D(0), it is a homeomorphism onto its image. Note that the second statement follows from the first, since ∂ D(0) consists of small actions, and therefore the restricted map is injective by the above-mentioned theorem of Skora. The full map is not bijective: For example, quadratic differentials that are squares of holomorphic 1-forms are images of the length functions of their dual trees, but they also appear as images of the limits of abelian representations (see Section 3). It would be interesting to determine the fibers of h¯ in general; this question will be taken up elsewhere. We also remark that the SL(2, R) version of the above theorem leads to a harmonic-maps description of the Thurston compactification of Teichmüller space and was first proved by Wolf [W1]. Generalizing this result to SL(2, C) is one of the motivations for this paper. This paper is organized as follows: In Section 2 we review the Morgan-Shalen compactification, the definition of the Higgs moduli space, and the notion of a harmonic map to an R-tree. In Section 3, we define the boundary map H . The key point

THE MORGAN-SHALEN COMPACTIFICATION

191

is that the nonuniqueness in the correspondence between abelian length functions and R-trees alluded to above nevertheless leads, via harmonic maps, to a well-defined geometric object on 6, in this case, a quadratic differential. The most important result here is Theorem 3.7. Along the way, we give a criterion, Theorem 3.3, for uniqueness of harmonic maps to trees, using the arguments in [W3]. The main theorem is then proven in Section 4 as a consequence of our previous work [DDW]. In the last section, a somewhat more concrete analysis of the behavior of high energy harmonic maps is outlined, illustrating previous ideas. 2. Definitions. Let 0 be a hyperbolic surface group as in the introduction. We denote by R(0) the set of representations of 0 into SL(2, C), and by X(0) the set of characters of representations. Recall that a representation ρ : 0 → SL(2, C) defines a character χρ : 0 → C by χρ (g) = Tr ρ(g). Two representations ρ and ρ 0 are equivalent if χρ = χρ 0 . It is easily seen (cf. [CS]) that equivalent irreducible representations are conjugate. If ρ is a reducible representation, then we can write a(g) λρ (g) ρ(g) = 0 λρ (g)−1 for a representation λρ : 0 → C∗ . The character χρ determines λρ up to the inversion coming from the action of the Weyl group and is, in turn, completely determined by it. It is shown in [CS] that the set of characters X(0) has the structure of an affine algebraic variety. In [MS1], a (nonalgebraic) compactification X(0) of X(0) is defined as follows: Let C be the set of conjugacy classes of 0, and let P(C) = P(RC ) be the (real) projective space of nonzero, positive functions on C. Define the map ϑ : X(0) → P(C) by ª ¡ ϑ(ρ) = log |χρ (γ )| + 2 : γ ∈ C and let X(0)+ denote the 1-point compactification of X(0) with the inclusion map ı : X(0) → X(0)+ . Finally, X(0) is defined to be the closure of the embedded image of X(0) in X(0)+ × P(C) by the map ı × ϑ. It is proved in [MS1] that X(0) is compact and that the boundary points consist of projective length functions on 0 (see the definition ª below). Note that in its definition, ϑ(ρ) could be replaced by the function `ρ (γ ) γ ∈C , where `ρ denotes the translation length for the action of ρ(γ ) on H3 :

ª `ρ (γ ) = inf distH3 (x, ρ(γ )x) : x ∈ H3

(see [Cp]). Recall that an R-tree is a metric space (T , dT ) such that any two points x, y ∈ T are connected by a segment [x, y] (that is, a rectifiable arc isometric to a compact (possibly degenerate) interval in R whose length realizes dT (x, y)) and that [x, y] is the unique embedded path from x to y. We say that x ∈ T is an edge point (resp., vertex) if T \ {x} has two (resp., more than two) components. A 0-tree is an R-tree

192


with an action of 0 by isometries, and it is called minimal if there is no proper 0invariant subtree. We say that 0 fixes an end of T (or more simply, that T has a fixed end) if there is a ray R ⊂ T such that for every γ ∈ 0, γ (R) ∩ R is a subray. When the action is understood, we often refer to “trees” instead of “0-trees.” Given an R-tree (T , dT ), the associated length function `T : 0 → R+ is defined by `T (γ ) = inf x∈T dT (x, γ x). If `T 6≡ 0, which is equivalent to 0 having no fixed point in T (cf. [MS1, Prop. II.2.15]), then the class of `T in P(C) is called a projective length function. We denote by PL(0) the set of all projective length functions on 0-trees. A length function is called abelian if it is given by |µ(γ )| for some homomorphism µ : 0 → R. We use the following result. Theorem 2.1 [CM, Cor. 2.3 and Thm. 3.7]. Let T be a minimal 0-tree with nontrivial length function `T . Then `T is nonabelian if and only if 0 acts without fixed ends. Moreover, if T 0 is any other minimal 0-tree with the same nonabelian length function, then there is a unique equivariant isometry T ' T 0 . It is a fact that abelian length functions, in general, no longer determine a unique minimal 0-tree up to isometry (e.g., see [CM, Example 3.9]), and this presents one of the main difficulties dealt with in this paper. We now give a quick review of the theory of Higgs bundles on Riemann surfaces and their relationship to representation varieties. Let 6, 0 be as in the introduction. A Higgs pair is a pair (A, 8), where A is an SU(2) connection on a rank-2 smooth vector bundle E over 6; and 8 ∈ 1,0 (6, End0 (E)), where End0 (E) denotes the bundle of traceless endomorphisms of E. The Hitchin equations are (2.1)

FA +[8, 8∗ ] = 0, 00 8 = 0. DA

The group G of (real) gauge transformations acts on the space of Higgs pairs and preserves the set of solutions to (2.1). We denote by MHiggs the set of gauge equivalence classes of these solutions. Then MHiggs is a complex analytic variety of dimension 6g − 6 (the holomorphic structure depending upon the choice σ on 6), which admits a holomorphic map (cf. [H]) (2.2)

det : MHiggs −→ QD = H 0 (6, K6⊗2 ) : (A, 8) 7→ det 8 = − Tr 82 .

By associating to [(A, 8)] ∈ MHiggs the character of the flat SL(2, C) connection A+8+8∗ , one obtains a homeomorphism h : MHiggs → X(0) (cf. [D], [C]). Implicit in the definition of h is a 0-equivariant harmonic map u from the universal cover H2 of 6 to H3 . It is easily verified that the Hopf differential of u, Hopf(u) = ϕ˜ = huz , uz idz2 , descends to a holomorphic quadratic differential ϕ on 6 equal to det 8 (up to a universal nonzero constant). Having introduced harmonic maps, we now give an alternative way to view the


193

Morgan-Shalen compactification. First, it follows by an easy application of the Bochner-Weitzenböck formula that a sequence of representations ρi diverges to the boundary only if the energies E(uρi ) of the associated equivariant harmonic maps uρi are unbounded. Furthermore, given such a sequence, it is shown in [DDW] that if the ρi converge to a boundary point in the sense of Morgan-Shalen, then the harmonic maps uρi converge (perhaps after passing to a subsequence) in the sense of Korevaar-Schoen to a 0-equivariant harmonic map u : H2 → (T , dT ), where (T , dT ) is a minimal 0-tree having the same projective length function as the Morgan-Shalen limit of the ρi . As pointed out before, the tree is not necessarily uniquely defined, and even in the case where the tree is unique, uniqueness of the harmonic map is problematic. Recall that a harmonic map to a tree means, by definition, an energy minimizer for the energy functional defined in [KS1]. Given such a map, its Hopf differential ϕ˜ can be defined almost everywhere, and by [S1, Lemma 1.1], which can be adapted to the singular case, one can show that the harmonicity of u implies that ϕ˜ is a holomorphic quadratic differential. The equivariance of u implies that ϕ˜ is the lift of a differential on 6. Note also that if u : H2 → T is harmonic, then Hopf(u) ≡ 0 if and only if u is constant. In the equivariant case, this in turn is equivalent to `T ≡ 0 (cf. [DDW]). For the rest of the paper, we tacitly assume `T 6≡ 0. A particular example is the following: Consider a nonzero holomorphic quadratic differential ϕ, and denote by ϕ˜ its lift to H2 . Locally away from the zeros, ϕ˜ may be written as dz2 with respect to a local conformal coordinate z = ξ + iη. The lines ξ = const (the vertical leaf space) and transverse measure |dξ | give the structure of a metric space Tϕ˜ , which is independent of the choice of coordinate z and naturally extends past the zeros. According to [MS2] (and using the correspondence between measured foliations and geodesic laminations), Tϕ˜ is an R-tree with an action of 0, and the projection π : H2 → Tϕ˜ is a 0-equivariant continuous map. We note two ˜ important facts: (1) The vertices of Tϕ˜ are precisely the image by π of the zeros of ϕ. (2) Since the action of 0 on Tϕ˜ is small, Tϕ˜ has no fixed ends (cf. [MO]). ˜ Proposition 2.2. The map π : H2 → Tϕ˜ is harmonic with Hopf differential ϕ. Proof. Since Tϕ˜ has no fixed ends, the existence of a harmonic map follows from [KS2, Cor. 2.3.2]. The fact that π is itself an energy minimizer seems to be well known. See, for example, [W2] and the introduction to [GS]: Although the definition of harmonic map in [W2] is a priori different from the notion of an energy minimizer, a proof follows easily. Indeed, for fixed ϕ 6= 0 and positive real numbers ti → ∞, we can find a sequence of hyperbolic metrics σi on 6 such that the unique harmonic maps 6(σ ) → 6(σi ) homotopic to the identity have Hopf differentials ti ϕ (cf. [W1] and [Wan]). Uniformizing the σi , we obtain a sequence ρi of discrete faithful SL(2, R) ⊂ SL(2, C) representations and ρi -equivariant harmonic maps ui : ˜ Let di denote the pullback distance functions H2 → H2 with Hopf differentials ti ϕ. on H2 by the ui , and let d∞ denote the pseudometric obtained by pulling back the

194


metric on Tϕ˜ by the projection π . Extend all of these to pseudometrics, also denoted di and d∞ , on the space H2∞ constructed in [KS2]. Then the natural projection H2 → H2∞ /d∞ ' Tϕ˜ coincides with the map π. On the other hand, by [W2, Section 4.2], di → d∞ pointwise, locally uniformly. Therefore, by [KS2, Thm. 3.9], π is an energy minimizer. Next, we consider 0-trees that are not necessarily of the form Tϕ˜ . We need the following. Definition 2.3. A morphism of R-trees is a map f : T → T 0 such that every nondegenerate segment [x, y] has a nondegenerate subsegment [x, w] such that f restricted to [x, w] is an isometry onto its image. The morphism f is said to fold at a point x ∈ T if there are nondegenerate segments [x, y1 ] and [x, y2 ] with [x, y1 ] ∩ [x, y2 ] = {x} such that f maps each segment [x, yi ] isometrically onto a common segment in T 0 . It is a fact that a morphism f : T → T 0 is an isometric embedding unless it folds at some point (cf. [MO, Lemma I.1.1]). We also note that, in general, foldings T → T 0 may take vertices to edge points. Conversely, vertices in T 0 need not lie in the image of the vertex set of T . Proposition 2.4 (cf. [FW]). Let T be an R-tree with 0 action, and let u : H2 → T be an equivariant harmonic map with Hopf differential ϕ. ˜ Then u factors as u = 2 f ◦π , where π : H → Tϕ˜ is as in Proposition 2.2 and f : Tϕ˜ → T is an equivariant morphism. Proof. Consider f = u ◦ π −1 : Tϕ˜ → T . We first show that f is well defined: Indeed, assume z1 , z2 ∈ π −1 (w). Then z1 and z2 may be connected by a vertical leaf e of the foliation of ϕ. ˜ Now, by the argument in [W3, p. 117], u must collapse e to a point, and so u(z1 ) = u(z2 ). In order to show that f is a morphism, consider a ˜ Moreover, segment [x, z] ∈ Tϕ˜ . We may lift x to a point x˜ away from the zeros of ϕ. we may choose a small horizontal arc e˜ from x˜ to some y˜ projecting to [x, y] ⊂ [x, z], still bounded away from the zeros. The analysis in [W3] again shows that this must map by u isometrically onto a segment in T . Remark. It is easily shown (cf. [DDW]) that images of equivariant harmonic maps to trees are always minimal subtrees; hence, throughout this paper we assume, without loss of generality, that our trees are minimal. Thus, for example, the factorization f : Tϕ˜ → T above either folds at some point or is an equivariant isometry. 3. The map H . The Hopf differential for a harmonic map to a given tree is uniquely determined, as shown by the following statement. Proposition 3.1. Let T be a minimal R-tree with a nontrivial 0 action. If u, v are equivariant harmonic maps H2 → T , then Hopf(u) = Hopf(v).


195

Proof. This is proven in [KS1], where in fact the full pullback “metric tensor” is considered. In our situation, the result can also be seen as a direct consequence of the leaf structure of the Hopf differential. First, by [KS1, p. 633], the function z 7 → dT2 (u(z), v(z)) is subharmonic; hence, by the equivariance it must be equal to a constant c. We assume c 6= 0, since otherwise there is nothing to prove. Set ˜ and let 1 be a small ϕ˜ = Hopf(u), ψ˜ = Hopf(v). Suppose that p ∈ H2 is a zero of ϕ, ˜ except perhaps neighborhood of p containing no other zeros of ϕ˜ and no zeros of ψ, p itself. Then by Proposition 2.4 it follows that u is constant and equal to u(p) on every arc e ⊂ 1 of the vertical foliation of ϕ˜ with endpoint p. On the other hand, v(e) is a connected set satisfying dT (u(p), v(z)) = c for all z ∈ e. Since spheres are discrete in trees, v is constant and equal to v(p) on e as well. Referring again to ˜ In this Proposition 2.4, this implies that e must be contained in a vertical leaf of ψ. way, one sees that the zeros of ϕ˜ and ψ˜ coincide with multiplicity in H2 . Thus, the same is true for ϕ and ψ on 6. Since the quadratic differentials are both normalized, they must be equal. We also need the following restriction on the kinds of foldings that arise from harmonic maps. Lemma 3.2. Let Tϕ˜ → T arise from a harmonic map as in Proposition 2.4. Then folding occurs only at vertices, that is, the images of zeros of ϕ. ˜ At the zeros of ϕ, ˜ adjacent edges may not be folded. In particular, folding cannot occur at simple zeros. Proof. The argument is similar to that in [W2, p. 587]. Suppose p ∈ H2 is a zero at which a folding occurs, and choose a neighborhood 1 of p contained in a fundamental domain and containing no other zeros. We can find distinct segments e, e0 of the horizontal foliation of ϕ˜ with a common endpoint p that map to segments of Tϕ˜ . We may further assume that the folding Tϕ˜ → T carries each of e and e0 isometrically onto a segment e¯ of T . Suppose that e and e0 are adjacent. Then there is a small disk 10 ⊂ H2 that, under the projection π : H2 → Tϕ˜ , maps to π(e) ∪ π(e0 ) and whose center maps to π(p) (see Figure 1). Then the harmonic map u : H2 → T maps 10 onto the segment e¯ with the center mapping to an endpoint. Let q denote the other endpoint of e. ¯ The function z 7→ (dT (u(z), q))2 is subharmonic on 10 with an interior maximum. It therefore must be constant, which contradicts ϕ 6≡ 0. For the last statement, recall that the horizontal foliation is trivalent at a simple zero, so that any two edges are adjacent. Though the following is not important in this paper, we find it interesting that a uniqueness result for equivariant harmonic maps to trees follows from these considerations, in certain cases. Theorem 3.3. Let u : H2 → T be an equivariant harmonic map with ϕ˜ = Hopf(u). Suppose there is some vertex x of Tϕ˜ such that the map f : Tϕ˜ → T from Proposition 2.4 does not fold at x. Then u is the unique equivariant harmonic map to T .

196


e p e0

= vertical foliation = horizontal foliation Figure 1

Proof. Let p be a zero of ϕ˜ projecting via π to x, and let v be another equivariant harmonic map to T . Choose a neighborhood 1 of p as in the proof of Proposition 3.1, and again suppose that the constant c = dT (u(z), v(z)) 6= 0. Recall that x is a vertex of Tϕ˜ . By the assumption of no folding at x, there must be a segment e of the vertical foliation of ϕ˜ in 1, with one endpoint being p, having the following property: For any z 6= p in e there is a neighborhood 10 ⊂ 1 of z such that u(10 ) ∩ [u(p), v(p)] = {u(p)}. By Proposition 3.1 and Lemma 3.2, we see that for such 10 , v(10 ) 6⊂ [u(p), v(p)]. Thus, there is a q ∈ 1 such that u(q) 6∈ [u(p), v(p)] and v(q) 6∈ [u(p), v(p)]. But then dT (u(q), v(q)) > dT (u(p), v(p)) = c, a contradiction. Corollary 3.4. Let ϕ 6 ≡ 0 be a holomorphic quadratic differential on 6. Then the map π : H2 → Tϕ˜ in Proposition 2.2 is the unique equivariant harmonic map to Tϕ˜ . If u : H2 → T is an equivariant harmonic map and Hopf(u) has a zero of odd order, then u is unique. Proof. The first statement is clear from Theorem 3.3. For the second statement, notice that if p is a zero of odd order, we can still find a neighborhood 10 as in the proof of Theorem 3.3. Proposition 3.1 allows us to associate a unique ϕ ∈ SQD to any nonabelian length function. Proposition 3.5. Let [`] ∈ PL(0) be nonabelian. Then there is a unique choice ϕ ∈ SQD with the following property: If T is any minimal R-tree with length function ` in the class [`], and u : H2 → T is a 0-equivariant harmonic map, then Hopf(u) = ϕ. Proof. Let ` ∈ [`]. By Theorem 2.1, there is a unique minimal tree T , up to isometry, with length function ` and no fixed ends. By Proposition 3.1, any two harmonic maps u, v : H2 → T have the same normalized Hopf differential. Furthermore, if T 0 is isometric to T and u0 is a harmonic map to T 0 , then, composing with the


197

isometry, we see that u0 has the same Hopf differential as any harmonic map to T . If the length function ` is scaled, then the normalized Hopf differential remains invariant. Finally, since T has no fixed ends, it follows from [KS2, Cor. 2.3.2] that there exists an equivariant harmonic map u : H2 → T ; so we set ϕ = Hopf(u). We now turn our attention to the abelian length functions. These no longer determine a unique R-tree in general; nevertheless, we see that there is still a uniquely defined quadratic differential associated to them. Proposition 3.6. Let ` be an abelian length function, and let 0 act on R with translation length function equal to `. Then there is an equivariant harmonic function ˜ 2 , where u : H2 → R, unique up to translations of R, with Hopf differential ϕ˜ = (ω) 2 ω˜ is the lift to H of an abelian differential ω on 6. Moreover, ` is determined by the periods of Re(ω). Proof. The uniqueness statement is clear. By harmonic theory, there is a unique holomorphic one-form ω on 6 such that the real parts of its periods correspond to the homomorphism µ : π1 (6) −→ H1 (6, Z) −→ R. harmonic function is Choosing any base point ∗ of H2 , the desired equivariant Rz ˜ The Hopf differential is the real part of the holomorphic function f (z) = ∗ ω. (f 0 (z))2 = (ω) ˜ 2. It is generally true that harmonic maps to trees with abelian length functions have Hopf differentials with even-order vanishing and that the length functions are recovered from the periods of the associated abelian differential, as the next result demonstrates. Theorem 3.7. Let u : H2 → T be an equivariant harmonic map to a minimal R-tree with nontrivial abelian length function `. Then Hopf(u) = (ω) ˜ 2 , where ω˜ is 2 the lift to H of an abelian differential ω on 6. Moreover, ` is determined by the periods of Re(ω). Proof. We first prove that the Hopf differential ϕ˜ = Hopf(u) must be a square. It suffices to prove that the zeros of ϕ˜ are all of even order. Let p be such a zero, and choose a neighborhood 1 of p as above. Since T has an abelian length function, the action of 0 must fix an end E of T . Then, applying the construction of Section 5 of [DDW], we find a continuous family of equivariant harmonic maps uε obtained by “pushing” the image of u a distance ε in the direction of the fixed end. On the other hand, if ϕ˜ had a zero of odd order, this would violate Corollary 3.4. We may therefore express ϕ˜ = (ω) ˜ 2 for some abelian differential ω˜ on H2 . A priori, we can only conclude that ω˜ descends to an abelian differential ωˆ on an b of 6 determined by an index-2 subgroup b unramified double cover 6 0 ⊂ 0. Let L be a complete noncritical leaf of the horizontal foliation of ϕ. ˜ Choose a point x0 ∈ L and let x¯0 = u(x0 ). We assume that we have chosen x0 so that x¯0 is an edge point.

198


R

E

e

R0 q

e0

Figure 2

Then there is a unique ray R¯ with end point x¯0 leading out to the fixed end E. Let R denote the half-leaf of L starting at x0 and such that a small neighborhood of x0 in R ¯ maps isometrically onto a small subsegment of R. ¯ For suppose to the contrary that We claim that R itself maps isometrically onto R. there is a point y ∈ R such that the portion [x0 , y] of R from x to y maps isometrically ¯ but that this is not true for any y 0 ∈ R \ [x0 , y]. Clearly, the onto a subsegment of R, image of y by u must be a vertex of T . Recall the factorization f : Tϕ˜ → T from Proposition 2.4. Since f is a surjective morphism of trees, the vertices of T are either ˜ or they are images by f of vertices of Tϕ˜ and, hence, images by u of zeros of ϕ, vertices created by a folding of f . Thus, there are two cases to consider: (1) There is a point q such that y and q lie on the same vertical leaf and q is a zero of ϕ. ˜ 0 Moreover, there is a critical horizontal leaf R with one end point equal to q, a small subsegment of which maps isometrically onto a subsegment of R¯ with end point q¯ = u(q) (see Figure 2). (2) There is a point q such that y and q lie on the same vertical leaf, q is connected by a horizontal leaf to a zero p of ϕ, ˜ and the map f folds at π(p), identifying the segment [p, q] with a portion [p, q 0 ] of another horizontal leaf R 0 . Moreover, [p, q 0 ] maps isometrically onto a subsegment of the unique ray from p¯ = u(p) to the end E (see Figure 3). Consider case (1): As indicated in Figure 2, we can find a small neighborhood 1 of y and portions of horizontal leaves e and e0 meeting at q that map isometrically onto segments of T intersecting the image R¯ 0 = u(R 0 ) only in q. ¯ Now, as above, by pushing the image of u in the direction of E and possibly choosing 1 smaller, we can find a harmonic map uε that maps 1 onto a segment with end point q¯ and maps y to the opposite end point—a contradiction. The argument for case (2) is similar: We may find a disk 1 centered at y that maps to the union of segments [p, ¯ q] ¯ and [¯r , q], ¯ with y being mapped to q. ¯ Then, pushing the map in the direction of E as above again leads to a contradiction (see Figure 3). Next, we claim that for any g ∈ b 0 , `(g) is given by the period of Re(ω) ˆ around a curve representing the class [g]. First, by definition of a fixed end, the intersection ¯ contains a subray of R, ¯ and for all x¯ in this subray, `(g) = dT (x, ¯ g(x)) ¯ (cf. R¯ ∩ g(R) ¯ ⊂ R. ¯ Choose a lift of x¯ to [CM, Thm. 2.2]). For simplicity then, we assume g(R)

199


E

E R

r

q0 p

q

R0 u

--->

p

q = q0 r

Figure 3

¯ Suppose g(x) is connected by a (possibly empty) x ∈ R. Then u(g(x)) = g(x) ¯ ∈ R. 0 vertical leaf to a point x on R. Then the curve γ˜ consisting of the portion [x, x 0 ] of R from x to x 0 followed by the vertical leaf to g(x) projects to a curve γ on 6 ¯ `(g) is the length of representing g. Moreover, since R maps isometrically onto R, 0 by ϕ. ˜ Since R contains no [x, x ] with respect to the transverse measure determined R zeros of ϕ, ˜ the latter is simply the absolute value of [x,x 0 ] Re(ω). ˜ Futhermore, since the vertical direction lies in the kernel of Re(ω), ˆ we also have Z Z ˜ = Re(ω) ˆ `(g) = Re(ω) γ˜

γ

as desired. Now consider the possibility that g(x) ∈ g(R) is not connected to R by a vertical ¯ it follows from Proposition 2.4 and the fact that R maps onto R¯ leaf. Since g(x) ¯ ∈ R, that there is an intervening folding of a subray of g(R) onto R. Let y ∈ R project to the vertex in Tϕ˜ at which this occurs. The simplest case is where y is connected by a vertical leaf to a point w ∈ g(R), and the folding identifies the subray of R starting at y isometrically with the subray of g(R) starting at w. The same analysis as above then produces the closed curve γ . A more complicated situation arises when there are intervening vertices (see Figure 4(a)): For example, there may be zeros p, q of ϕ, ˜ a point w0 ∈ g(R), and segments 0 00 e, e , and e of the vertical, horizontal, and vertical foliations, respectively, with endpoints {y, p}, {p, q}, and {q, w 0 }, respectively. Moreover, the map u folds e0 onto a subsegment f of R with endpoints y and y 0 , and then it identifies the subray of R starting at y 0 isometrically with the subray of g(R) starting at w0 . In this way, we see that a subsegment f 0 of g(R) with endpoints w0 and w gets identified with f and e0 ; in particular, the transverse measures of these three segments are all equal. (Strictly speaking, y 0 need not lie on R as we choose it, but this does not affect the argument.) Now consider the prongs at the zero p, for example. These project to distinct segments in Tϕ˜ , which are then either projected to segments in T intersecting R¯ only in y; ¯ or alternatively there may be a folding identifying them with subsegments of

200


g(x)

E y

e

p

q

y0 R

w

e0

f

+

0 or − p

f0 w0

e00

− g(R)

x

+

(a)

+

0 or −

(b) Figure 4

¯ Let us label the prongs with a + sign if there is a folding onto a subsegment of R. [y, ¯ E), with a − sign if there is a folding onto a subsegment of [x, ¯ y], ¯ and with a 0 if no folding occurs or if the edge is folded along some other segment (see Figure 4(b)). Since p is connected by the vertical leaf e to R, we label the adjacent horizontal segments with + and − accordingly. Working our way around p in the clockwise direction, and repeatedly using the “pushing” argument from Section 5 of [DDW], we find that every second prong must be labeled + while the intervening prongs may get either − or 0 (recall Lemma 3.2). Therefore, there must be an odd number of prongs between e0 and the one adjacent to e, which is identified in the leaf space with a portion of f . A similar argument applies to q, e00 , and f 0 . Let γ˜ 0 be the path from y 0 to w obtained by following f, e, e0 , e00 , and then f 0 . Because of the to the folding of the prongs at p and q, one may easily R odd sign ˜ is the just the transverse measure of the segment f . Indeed, verify that γ˜ 0 Re(ω) suppose ϕ˜ has a zero of order 2n at some point p, and choose a local conformal coordinate z such that ϕ(z) ˜ = z2n dz2 . Then the foliation is determined by the leaves of ξ = zn+1 /n + 1. If ζ is a primitive 2n + 2 root of unity, then z 7→ ζ k z takes one radial prong to another, with k − 1pprongs in between (in the counterclockwise direction). The outward integrals of Re ϕ˜ along these prongs to a fixed radius differ by (−1)k . Our analysis implies that k −1 is odd, so k is even, and we have the correct 0 cancellation. If we R extend γ˜ along the horizontal leaves R and g(R) to a path γ˜ from ˜ = dT (x, ¯ g(x)) ¯ as required. In general, there are additional x to g(x), then γ˜ Re(ω) intervening zeros, and the procedure above applies to each of these with no further complication. Thus, ` restricted to b 0 is given by the periods of Re(ω). ˆ Since the real parts of the periods of an abelian differential determine the differential uniquely, ωˆ must agree b of the form in Proposition 3.6; in particular, it descends to 6. with the pullback to 6 This completes the proof of Theorem 3.7. We immediately have the following.


201

Corollary 3.8. Fix an abelian length function `. Then for any tree T with length function ` and any equivariant harmonic map v : H2 → T , we have Hopf(v) = Hopf(u) where u is the equivariant harmonic function from Proposition 3.6 corresponding to `. We are now prepared to define the map (1.1). Take a representative ` of [`] ∈

PL(0). There are two cases: If ` is nonabelian, use Proposition 3.5 to define

H ([`]) = ϕ. If ` is abelian, use Proposition 3.6. The main result of this section is the following. Theorem 3.9. The map H : PL(0) → SQD defined above is continuous.

Proof. Suppose [ì ] → [`], and assume, to the contrary, that there is a subsequence, which we take to be the sequence itself, such that H ([ì ]) → ϕ 6= H ([`]). Choose representatives ì → `. If there is a subsequence {i 0 } consisting entirely of abelian length functions, then ` itself must be abelian, and from the construction of Proposition 3.6, H (ì 0 ) → H (`), a contradiction. Thus, we may assume all the ì ’s are nonabelian. There exist R-trees Ti , unique up to isometry, and equivariant harmonic maps ui : H2 → Ti . We claim that the ui have uniform modulus of continuity (cf. [KS2, Prop. 3.7]). Indeed, by [GS, Thm. 2.4], it suffices to show that E(ui ) is uniformly bounded. If E(ui ) → ∞, then the same argument as in [DDW, proof of Thm. 3.1] would give a contradiction. It follows by [KS2, Prop. 3.7] that there is a subsequence {i 0 } (which we assume is the sequence itself) such that ui converges in the pullback sense to an equivariant harmonic map u : H2 → T , where T is a minimal R-tree with length function equal to `. In addition, by [KS2, Theorem 3.9], Hopf(ui ) → Hopf(u). If ` is nonabelian, we have a contradiction by Proposition 3.1; if ` is abelian, we have a contradiction by Corollary 3.8. 4. Proof of the main theorem. We show how the results of the previous section, combined with those in [KS2] and [DDW], give a proof of the main theorem. We first reduce the proof of the continuity of h¯ to the following. Claim. If [ρi ] ∈ X(0) is a sequence of representations converging to [`] ∈ PL(0), then h([ρi ]) → H ([`]). Suppose the claim holds and h¯ is not continuous. Then we may find a sequence ¯ i ) → y 6= h(x). ¯ If x ∈ PL(0) so that xi ∈ PL(0) ∪ X(0) such that xi → x but h(x ¯h(x) = H (x), the claim rules out the possibility that there is a subsequence of {xi } in X(0). In this case then, there must be a subsequence in PL(0). But this contradicts the continuity of H , by Theorem 3.9. Thus, x must be in X(0). But then we may assume that {xi } ⊂ X(0), so that h¯ = h on {xi }. The continuity of the homeomorphism h : X(0) → MHiggs then provides the contradiction. It remains to prove the claim. Again suppose to the contrary that [ρi ] → [`] but h([ρi ]) → ϕ 6 = H ([`]) for ϕ ∈ SQD. First, suppose that there is a subsequence [ρi 0 ] with reducible representative representations ρi 0 : 0 → SL(2, C). Up to conjugation,

202


which amounts to changing the choice of representative, we may assume each ρi 0 fixes a given vector 0 6 = v ∈ C2 , and that the action on the 1-dimensional line spanned by v is determined by a character χi 0 : 0 → C∗ . The associated translation length functions ì 0 are therefore all abelian, and so [`] must be abelian. We may assume there is a representative ` such that ì 0 → `. By Proposition 3.6 there are harmonic functions u, ui 0 : H2 −→ R ' C∗ /U (1) ,→ H3 , equivariant for the induced action of 0 on C∗ by χ and χi 0 , respectively. These converge (after rescaling) to a harmonic function u : H → R, equivariant with respect to an action on R with translation length function `. Since the length functions converge, it follows from the construction in Proposition 3.6 that Hopf(ui 0 ) → Hopf(u), and so by the definition of H , h([ρi 0 ]) → H ([`]), a contradiction. Second, suppose that there is a subsequence [ρi 0 ] of irreducibles. Then by the main result of [DDW] we can find a further subsequence (which we take to be the sequence itself) of ρi 0 -equivariant harmonic maps ui 0 : H2 → H3 converging in the sense of Korevaar-Schoen to a harmonic map u : H2 → T , where T is a minimal R-tree with an action of 0 by isometries and length function ` in the class [`]. As above, Hopf(ui 0 ) → Hopf(u), so by the definition of H , h([ρi 0 ]) → H ([`]), a contradiction. Since we have accounted for both possible cases, this proves the claim. 5. Convergence of length functions. In this final section we briefly sketch an alternative argument for the convergence to the boundary in the main theorem, based on a direct analysis of length functions, more in the spirit of [W1]. The generalization of estimates for equivariant harmonic maps with target H2 to maps with target H3 has largely been carried out by Minsky [M]. We discuss this point of view, however, since it reveals how and why the folding of the dual tree Tϕ˜ occurs. The first step is to analyze the behavior of the induced metric for a harmonic map u : H2 → H3 of high energy (at the points where u is an immersion). As usual we denote by ϕ˜ the Hopf differential for the map u. Because of equivariance, ϕ˜ is the lift of a holomorphic quadratic differential ϕ on 6. Recall the norm kϕk from the introduction, and let Z(ϕ) ⊂ 6 denote the zero set of ϕ. We also set µ to be the 2 . Beltrami differential associated to the pullback metric u∗ dsH 3 Lemma 5.1. Fix δ, T > 0. Then there are constants B, α > 0 such that for all u, µ, and ϕ as above, kϕk ≥ T , and all p ∈ 6 satisfying dist σ (p, Z(ϕ)) ≥ δ, we have 1 (p) < Be−αkϕk . log |µ| Proof. This result is proven in [M, Lemma 3.4]. One needs only a statement concerning the uniformity of the constants appearing there. However, by using the compactness of SQD, one easily shows the following: For δ > 0 there is a constant c(δ) > 0 such that, for all ϕ ∈ SQD and all p ∈ 6 such that dist σ (p, Z(ϕ)) ≥ δ,


203

the disk U of radius c(δ) ˜ (with respect to the singular flat metric |ϕ|) around p is embedded in 6 and contains no zeros of ϕ. Then the result cited above applies. This estimate is all that is needed to prove convergence in the case where there cannot be a folding of the dual tree Tϕ˜ such that the composition of projection to Tϕ˜ with the folding is harmonic. From Lemma 3.2, this is guaranteed, for example, if ϕ has only simple zeros. For simplicity, in this section we assume all representations are irreducible. Theorem 5.2. Given an unbounded sequence ρj of representations with MorganShalen limit [`], let uj : H2 → H3 be the associated ρj -equivariant harmonic maps. Suppose that for ϕ˜j = Hopf(uj ) we have ϕj /kϕj k → ϕ ∈ SQD, where ϕ has only simple zeros. Then [`] = [`T ], where T = Tϕ˜ . Proof. We prove the convergence of length functions in two steps. First, we compare the length of closed curves γ in the free isotopy class [γ ] with respect to the induced metric from uj to the length with respect to the transverse measure. Second, we compare the length of the image by uj of a lift γ˜ to H2 of γ to the translation length in H3 of the conjugacy class that [γ ] represents. The basic idea is that the image of γ˜ very nearly approximates a segment of the hyperbolic axis for ρj ([γ ]). For ϕ and [γ ] as above, let `ϕ ([γ ]) denote the infimum over all representatives γ of [γ ] of the length of γ with respect to the vertical measured foliation defined by ϕ. If u : H2 → H3 is a differentiable equivariant map, we define ù ([γ ]) as follows: For each representative γ of [γ ], where [γ ] corresponds to the conjugacy class of g ∈ 0, lift γ to a curve γ˜ at a point x ∈ H2 , terminating at gx. We then take the infimum over all such γ˜ of the length of u(γ˜ ). This is ù ([γ ]), and by the equivariance of u it is independent of the choice of x. Finally, recall that the translation length `ρ ([γ ]) for a representation ρ : 0 → SL(2, C) is defined in Section 2. Given ε > 0, let QDε ⊂ QD \{0} denote the subset consisting of holomorphic quadratic differentials ϕ having only simple zeros, and such that the zeros are pairwise at least a σ -distance ε apart. Notice that for t 6= 0, t QDε = QDε . The next result is a consequence of Lemma 5.1. Proposition 5.3. For all classes [γ ] and differentials ϕ ∈ QDε , there exist constants k and η depending on kϕk, [γ ], and ε, so that k `ϕ ([γ ]) + η ≥ ù ([γ ]) ≥ `ϕ ([γ ]) where k → 1 and ηkϕk−1/2 → 0 as kϕk → ∞ in QDε . Sketch of proof. We first need to choose an appropriate representative for the class of [γ ]. Such a choice was explained in [W1]. Namely, for δ > 0 and a given ϕ, we can find a representative γ consisting of alternating vertical and horizontal segments and having the transverse measure of the class [γ ]. Moreover, because the zeros of ϕ are

204


simple, for sufficiently small δ we can also guarantee that γ avoid a δ neighborhood of the zeros. Now the proof follows as in [W1, Lemma 4.6]. Note that along a harmonic maps ray (that is, a sequence ui such that Hopf(ui ) is of the form ti ϕ for a fixed ϕ and an increasing unbounded sequence ti ), we no longer necessarily have monotonicity of the norm of the Beltrami differentials |µ(ti )|. The argument for the estimate still applies, however, since the representatives γ are uniformly supported away from the zeros. There, we apply the estimate Lemma 5.1. The details are omitted. Next, we compare ù with the translation length in H3 . Proposition 5.4. Let ρ : 0 → SL(2, C) and u : H2 → H3 be the ρ-equivariant harmonic map with ϕ˜ = Hopf(u). Suppose ϕ ∈ QDε . For all classes [γ ] there exist constants m and ζ depending on kϕk, [γ ], and ε, so that m `ρ ([γ ]) + ζ ≥ ù ([γ ]) ≥ `ρ ([γ ]), where m → 1 and ζ kϕk−1/2 → 0 as kϕk → ∞ in QDε . Combining Propositions 5.3 and 5.4 proves Theorem 5.2. Sketch of proof of Proposition 5.4. One observes that away from the zeros, the images of the horizontal leaves of the foliation of ϕ˜ closely approximate (long) geodesics in H3 , while by Lemma 5.1 the images of vertical leaves collapse. More precisely, the following is proven in [M, Thm. 3.5]. Lemma 5.5. Fix δ > 0, a representation ρ : 0 → SL(2, C), and let u : H2 → be the ρ-equivariant harmonic map with Hopf differential ϕ. ˜ Let β˜ be a seg˜ ment of the horizontal foliation of ϕ˜ from x to y and suppose that, for all p˜ ∈ β, distσ (p, Z(ϕ)) ≥ δ. Then there is an ε, exponentially decaying in kϕk, such that the following hold: ˜ is uniformly within ε of the geodesic in H3 from u(x) to u(y). (1) u(β) ˜ is within ε of dist H3 (u(x), u(y)). (2) The length of u(β)

H3

The following is a key result. Lemma 5.6. Given g ∈ SL(2, C), let `(g) denote the translation length for the action of g on H3 . Suppose that s ⊂ H3 is a curve that is g invariant and satisfies the following property: For any two points x, y ∈ s, the segment of s from x to y is uniformly within a distance 1 of the geodesic in H3 joining x and y. Then there is a universal constant C such that ¡ inf distH3 x, g(x) ≤ `(g) + C.

x∈s

Proof. The intuition is clear; such an s must be an “approximate axis” for g. The proof proceeds as follows: Choose x ∈ s, and let c denote the geodesic in H3 from x to


205

g(x). By [Cp, Lemma 2.4] there exists a universal constant D and a subgeodesic c˜ of c with the property that |length(c) ˜ − `(g)| ≤ D. Let a and b be the endpoints of c˜ closest to x and g(x), respectively. By the construction of c˜ in the reference cited above, it follows that dist H3 (b, g(a)) ≤ D; hence, distH3 (b, g(b)) ≤ `(g) + 2D. Now by the assumption on c, there is a point y ∈ s close to b, so that dist H3 (y, g(y)) ≤ `(g) + C, where C = 2(D + 1). Proceeding with the proof of Proposition 5.4, choose the representative γ as discussed in Proposition 5.3. We may then lift to γ˜ ⊂ H2 so that γ˜ is invariant under the action of g. Now γ˜ is written as a union of horizontal and vertical segments of the foliation of ϕ. ˜ Let s = u(γ˜ ).Then Lemmas 5.1 and 5.5 imply that s satisfies the hypothesis of Lemma 5.6. Moreover, using Lemma 5.5 again, along with some elementary hyperbolic geometry, one can show that inf x∈s distH3 (x, g(x)) is approximated by the length of a segment of u(γ˜ ) from a point u(x) to u(gx). We leave the precise estimates to the reader. From Lemma 3.2, we see that foldings can only arise when the Hopf differentials converge in SQD to differentials with multiplicity at the zeros. From the point of view taken here, this corresponds to the fact that the representatives for closed curves γ chosen above may be forced to run into zeros of the Hopf differential where the estimate Lemma 5.1 fails. These may cause nontrivial angles to form in the image u(γ˜ ) which, in the limit, may fold the dual tree. Consider again the situation along a harmonic maps ray with differential ϕ. Given [γ ] corresponding to the conjugacy class of an element g ∈ 0, representatives γ still may be chosen as in the proof of Proposition 5.3 so that the horizontal segments remain bounded away from the zeros. However, it may happen that a vertical segment passes through a zero of order two or greater. For simplicity, assume this happens once. Divide γ into curves γ1 , γ2 , and γv , where γv is the offending vertical segment, and lift to segments γ˜1 , γ˜2 , and γ˜v in H2 . Note that one end point of each of the γ˜i ’s corresponds to either end point of γ˜v , and the other end points of the γ˜i ’s are related by g. By the Lipschitz estimate for harmonic maps to nonpositively curved spaces, we have a bound on the distance in H3 between the end points of u(γ˜v ) in terms of the length of γv and the energy E(u)1/2 (cf. [S2]). Thus, the rescaled length is small; in fact, since the length of γv is arbitrary, the distance converges to zero. On the other hand, the previous argument applies to the segments u(γ˜1 ) and u(γ˜2 ), which are connected by u(γ˜v ). Adding the geodesic in H3 joining the other end points of u(γ˜1 ) and u(γ˜2 ) forms an approximate geodesic quadrilateral, which, in the rescaled limit, converges either to an edge | (no folding) or a possibly degenerate tripod a (folding). In both cases, there is an edge that, by the same argument as in the proof of Proposition 5.4, approximates the axis of ρj (g) for large j . At the same time, the rescaled length of this segment is approximated by the translation length of the element g acting on a folding of Tϕ˜ at the zero. An interesting question is whether this approach may be used to determine precisely

206


the fibers of the map h¯ in the main theorem. While the essential ideas are outlined here, a complete description is not yet available. We will return to this issue in a future work. References [Cp] [C] [CM] [CS] [DDW] [D] [FW] [FLP] [GS]

[H] [HM] [KS1] [KS2] [M] [MO] [MS1] [MS2] [S1]

[S2]

[Si] [Sk] [Wan] [W1]

D. Cooper, Degenerations of representations into SL(2, C), preprint, 1998. K. Corlette, Flat G-bundles with canonical metrics, J. Differential Geom. 28 (1988), 361–382. M. Culler and J. Morgan, Group actions on R-trees, Proc. London Math. Soc. (3) 55 (1987), 571–604. M. Culler and P. Shalen, Varieties of group representations and splittings of 3-manifolds, Ann. of Math. (2) 117 (1983), 109–146. G. Daskalopoulos, S. Dostoglou, and R. Wentworth, Character varieties and harmonic maps to R-trees, Math. Res. Lett. 5 (1998), 523–534. S. K. Donaldson, Twisted harmonic maps and the self-duality equations, Proc. London Math. Soc. (3) 55 (1987), 127–131. B. Farb and M. Wolf, Harmonic splittings of surfaces, preprint, 1998. A. Fathi, F. Laudenbach, and V. Poénaru, eds., Travaux de Thurston sur les surfaces: Séminaire Orsay, Astérisque 66–67, Soc. Math. France, Paris, 1979. M. Gromov and R. Schoen, Harmonic maps into singular spaces and p-adic superrigidity for lattices in groups of rank one, Inst. Hautes Études Sci. Publ. Math. 76 (1992), 165–246. N. J. Hitchin, The self-duality equations on a Riemann surface, Proc. London Math. Soc. (3) 55 (1987), 59–126. J. Hubbard and H. Masur, Quadratic differentials and foliations, Acta Math. 142 (1979), 221–274. N. Korevaar and R. Schoen, Sobolev spaces and harmonic maps for metric space targets, Comm. Anal. Geom. 1 (1993), 561–659. , Global existence theorems for harmonic maps to non-locally compact spaces, Comm. Anal. Geom. 5 (1997), 333–387. Y. Minsky, Harmonic maps into hyperbolic 3-manifolds, Trans. Amer. Math. Soc. 332 (1992), 607–632. J. Morgan and J.-P. Otal, Relative growth rates of closed geodesics on a surface under varying hyperbolic structures, Comment. Math. Helv. 68 (1993), 171–208. J. Morgan and P. Shalen, Valuations, trees, and degenerations of hyperbolic structures, I, Ann. of Math. (2) 120 (1984), 401–476. , Free actions of surface groups on R-trees, Topology 30 (1991), 143–154. R. Schoen, “Analytic aspects of the harmonic map problem” in Seminar on Nonlinear Partial Differential Equations (Berkeley, Calif., 1983), ed. S. S. Chern, Math. Sci. Res. Inst. Publ. 2, Springer-Verlag, New York, 1984, 321–358. , “The role of harmonic mappings in rigidity and deformation problems” in Complex Geometry (Osaka, 1990), Lecture Notes in Pure and Appl. Math. 143, Dekker, New York, 1993, 179–200. C. T. Simpson, Moduli of representations of the fundamental group of a smooth projective variety, II, Inst. Hautes Études Sci. Publ. Math. 80 (1994), 5–79. R. K. Skora, Splittings of surfaces, J. Amer. Math. Soc. 9 (1996), 605–616. T. Wan, Constant mean curvature surface, harmonic maps, and universal Teichmüller space, J. Differential Geom. 35 (1992), 643–657. M. Wolf, The Teichmüller theory of harmonic maps, J. Differential Geom. 29 (1989), 449–479.

THE MORGAN-SHALEN COMPACTIFICATION [W2] [W3]

207

, Harmonic maps from surfaces to R-trees, Math. Z. 218 (1995), 577–593. , On realizing measured foliations via quadratic differentials of harmonic maps to R-trees, J. Anal. Math. 68 (1996), 107–120.

Daskalopoulos: Department of Mathematics, Brown University, Providence, Rhode Island 02912, USA; [email protected] Dostoglou: Mathematical Sciences Building, University of Missouri, Columbia, Missouri 65211, USA; [email protected] Wentworth: Department of Mathematics, University of California, Irvine, California 92697, USA; [email protected]

Vol. 101, No. 2


© 2000

S-ARITHMETICITY OF DISCRETE SUBGROUPS CONTAINING LATTICES IN HOROSPHERICAL SUBGROUPS HEE OH 0. Introduction. Let Qp be the field of p-adic numbers, and let Q∞ = R. Let Gp be a connected semisimple Qp -algebraic group. The unipotent radical of a proper parabolic Qp -subgroup of Gp is called a horospherical subgroup. Two horospherical subgroups are called opposite if they are the unipotent radicals of two opposite parabolic subgroups. In [5] and [6], we studied discrete subgroups generated by lattices in two opposite horospherical subgroups in a simple real algebraic group with real rank at least 2. This work was inspired by the following conjecture posed by G. Margulis. Conjecture 0.1. Let G be a connected semisimple R-algebraic group such that R-rank (G) ≥ 2, and let U1 , U2 be a pair of opposite horospherical R-subgroups of G. For each i = 1, 2, let Fi be a lattice in Ui (R) such that H ∩ Fi is finite for any proper normal R-subgroup H of G. If the subgroup generated by F1 and F2 is discrete, then it is an arithmetic lattice in G(R). We settled the conjecture in many cases, including the case when G is an absolutely simple real split group with G(R) not locally isomorphic to SL3 (R) (see [5]). In this paper, we study a problem analogous to the conjecture in a product of real and p-adic algebraic groups. The following is a special case of the main theorem, Theorem 4.3. Theorem 0.2. Let S be a finite set of valuations of Q including the archimedean valuation ∞. For each p ∈ S, let Gp be a connected semisimple algebraic Qp group without any Qp -anisotropic factors, Q and let U1p , U2p be Qa pair of opposite horospherical subgroups of Gp . Set G = p∈S Gp (Qp ), U1 = p∈S U1p (Qp ), and Q U2 = p∈S U2p (Qp ). Assume that G∞ is absolutely simple R-split with rank at least 2 and that if G∞ (R) is locally isomorphic to SL3 (R), then U1∞ is not the unipotent radical of a Borel subgroup of G∞ . Let F1 and F2 be lattices in U1 and U2 , respectively. If the subgroup generated by F1 and F2 is discrete, then it is a nonuniform S-arithmetic lattice in G. If p is a nonarchimedean valuation of Q, then no horospherical subgroup of Gp (Qp ) admits a lattice. Moreover, there is no infinite unipotent discrete subgroup in a p-adic Lie group. Therefore it is necessary to assume in Theorem 0.2 that S contains the archimedean valuation ∞. Received 12 January 1999. 1991 Mathematics Subject Classification. Primary 22E40; Secondary 22E46, 22E50. Author’s work partially supported by National Science Foundation grant number DMS-9801136. 209

210

HEE OH

In the group G = SLm (R) × SLn (Qp ), one can ask if it is possible to generate a discrete subgroup by taking lattices from two opposite horospherical subgroups of G. One interesting aspect of Theorem 0.2 says that, in general, the answer is no. Q Corollary 0.3. Keeping the same notation as in Theorem 0.2, set G = p∈S Gp . Suppose that there exist lattices F1 and F2 in U1 and U2 , respectively, that generate a discrete subgroup of G. Then (1) G is typewise homogeneous; that is, for each p ∈ S, there is an isogeny fp : Gp → G∞ ; in particular, Gp is absolutely almost simple; (2) for each p ∈ S, U1p is isomorphic to U1∞ . In particular, we have the following corollary. Corollary 0.4. With the same notation as in Corollary 0.3, suppose that G is not typewise homogeneous. Then any subgroup generated by lattices in a pair of opposite horospherical subgroups of G is not discrete. Corollary 0.3 follows from Theorem 0.2 simply by the definition of an S-arithmetic subgroup of G (see Section 1.6). Examples. In the following groups there are no discrete subgroups containing lattices in opposite horospherical subgroups: (1) G = SLm (R) × SLn (Qp ) for any m 6= n such that m ≥ 4 and n ≥ 2; (2) G = SLm (R) × SLn1 (Qp ) × SLn2 (Qp ) for any n1 , n2 ≥ 2 and m ≥ 4; (3) G = SO(m, m)R × SLn (Qp ) for any n ≥ 2 and m ≥ 2. As a corollary of Theorem 0.2, we obtain that as long as a discrete subgroup of G intersects a pair of opposite horospherical subgroups as lattices, then it is a lattice in the ambient group G as well. This is not always true in rank-1 simple groups, for instance, in SL2 (R) (see the remark after Theorem 0.2 in [5]). Corollary 0.5. Let S, Gp , p ∈ S, and G be as in Theorem 0.2. Let 0 be a discrete subgroup of G. Then 0 is a nonuniform S-arithmetic lattice in G if and only if for each p ∈ S there exists a pair U1p , U2p of opposite Q horospherical subgroups of Gp such that 0 ∩ Ui is a lattice in Ui , where Ui = p∈S Uip (Qp ) for each i = 1, 2. In that case, G is typewise homogeneous. For the proof of Theorem 0.2, denote by 0F1 ,F2 the subgroup generated by F1 and F2 , and denote by 0F∞1 ,F2 the image of the subgroup 0F1 ,F2 ∩ G∞ (R) × Q p∈S, p6 =∞ Gp (Zp ) under the natural projection G → G∞ (R). Using the results from [5], we first obtain a Q-form on G∞ with respect to which the subgroup 0F∞1 ,F2 is an arithmetic lattice in G∞ (R). Then applying a special case of Margulis’s superrigidity (see Theorem 4.2), we show that this Q-form of G∞ endows a Q-form on each Gp , p ∈ S, so that 0F1 ,F2 becomes an S-arithmetic subgroup in G. We also need some results on the classification of lattices in the product of real and p-adic nilpotent Lie groups (see Corollary 2.7). In fact our method shows that in Theorem 0.2 we can

S-ARITHMETICITY OF SOME DISCRETE SUBGROUPS

211

remove the assumption that G∞ is R-split as long as G∞ is absolutely simple with real rank at least 2 and Conjecture 0.1 holds for G∞ . In [5], we proved directly that any discrete subgroup of G∞ (R) containing lattices in opposite horospherical subgroups is an arithmetic subgroup, rather than using Margulis’s arithmeticity theorem or superrigidity theorem. Therefore, the methods used provide an alternative proof of the arithmeticity theorem in the case of nonuniform lattices for the groups considered in [5]. In the present paper, however, we use a special case of Margulis’s superrigidity theorem in order to extend the arithmetic structure of 0F∞1 ,F2 , obtained in [5], to an S-arithmetic structure of 0F1 ,F2 . Before we close the introduction, we describe the following open case of Margulis’s conjecture, which is believed to be a challenging case. Open problem. Consider the following two subgroups of PSL2 (R) × PSL2 (R): 1 0 1 R2 , U2 = . U1 = R2 1 0 1 For i = 1, 2, choose two linearly independent vectors ui and vi in R2 such that {nui + mvi | n, m ∈ Z} does not contain any element of the form (x, 0) or (0, x) for any x 6 = 0. By the natural isomorphism of Ui with R2 , we consider ui and vi as elements of Ui . Then the question dealt with by Conjecture 0.1 can be regarded as the following discreteness criterion problem: Which four elements u1 , u2 , v1 , and v2 generate a discrete subgroup? From the classification of the Q-forms of PSL2 (R) × PSL2 (R) (cf. [11]), it is not hard to see that Conjecture 0.1 implies that u1 , v1 , u2 , and v2 can generate a discrete subgroup only in the case when the elements u1 , v1 , u2 , and v2 are from some Hilbert modular group of PSL2 (R) × PSL2 (R). It then follows from the results of [12] that the discrete subgroup generated by those four elements is in fact a Hilbert modular group. Here we say that 0 is a Hilbert modular group of PSL2 (R)×PSL2 (R) if there is a real quadratic extension field k of Q such that 0 is conjugate to a subgroup of finite index in ª ¡ σ g, g | g ∈ PSL2 (J ) , where J is the ring of integers of k and σ : k → k is the nontrivial Galois automorphism of k. It seems plausible that an analogous conjecture in the setting P of Theorem 0.2 holds under the assumption that the S-rank of G, that is, the p∈S Qp -rank of G, is at least 2 (without any assumption on G∞ ). The first question in this regard would be to ask whether the conjecture is true for G = PSL2 (R) × PSL2 (Qp ). Acknowledgments. Thanks are due to Professor G. Margulis who drew my attention to Theorem 4.2. This work was done during my visits to the University of Bielefeld and the University of Chicago. I would like to thank the members of the mathematics departments at both universities for their hospitality as well as their invitations.

212

HEE OH

1. Notation 1.1. For a set S of valuations of Q, denote by Sf the subset of S consisting of nonarchimedean valuations (i.e., Sf = S − {∞}), where ∞ denotes the archimedean valuation of Q. Denote by Qp the field of p-adic numbers with the normalized absolute value | |p and set Q∞ = R. Denote by Zp the ring of p-adic integers, that is, Zp = {x ∈ Qp | |x|p ≤ 1}. For a valuation p on Q and a connected Qp -algebraic group Gp , we denote by Gp the Qp -rational points of Gp and by Gp+ the subgroup of Gp generated by all of its unipotent 1-parameter subgroups. 1.2. By Lie Gp we denote the Lie algebra of the group Gp considered as a Lie group over Qp , which is naturally identified with the Qp -points of the Lie algebra Lie Gp . Q 1.3. For each p ∈ S, we denote by prp the natural projection map from p∈S Gp Q to Gp . The notation pr ∞ denotes the projection of G∞ × p∈Sf G(Zp ) to G∞ . For Q a subgroup H of p∈S Gp , the notation H ∞ denotes the image of H ∩ (G∞ × Q ∞ p∈Sf Gp (Zp )) under the projection pr . 1.4. The notation ZS denotes the subring of Q generated by Z and {(1/p) | p ∈ Sf }. Q 1.5. For G = p∈S Gp , we say that G has a Q-form if there exist a connected algebraic Q-group H and a Qp -isomorphism φp : H → Gp for each p ∈ S. A subgroup M ofQG is said to be defined over Q if there is a Q-subgroup M0 of H such that M = p∈S φp (M0 ). For a subring J of Q and a subgroup M of G defined Q over Q, the notation M(J ) denotes the set { p∈S φp (x) ∈ G | x ∈ M0 (J )}, where Q M = p∈S φp (M0 ). 1.6. Q Let Gp be a connected algebraic Qp -group for each p ∈ S. A subgroup 0 of G = p∈S Gp (Qp ) is called an S-arithmetic (or simply arithmetic if S = {∞}) -isogeny fp : subgroup of G if there exist a connected algebraic Q Qp -group G0p , a QpQ 0 0 0 Gp → Gp for each p ∈ S, and a Q-form on G = p∈S Gp such that ( p∈S fp )(0) is commensurable to G0 (ZS ). An epimorphism with finite kernel is called an isogeny. 1.7. A discrete subgroup 0 of a locally compact group G is called a lattice if G/ 0 has a finite G-invariant Borel measure. A lattice 0 in G is called uniform if G/ 0 is compact, and it is nonuniform otherwise. 2. Discrete unipotent subgroups 2.1. Let S be a finite set of valuations of Q Qincluding ∞. For each p ∈ S, let Gp be a connected algebraic Qp -group. Let G = p∈S Gp . Proposition 2.1. If 0 is a discrete subgroup (resp., lattice) in G, then 0 ∞ is a discrete subgroup (resp., lattice) in G∞ .


213

compact, the subgroup 0 ∞ is discrete if 0 is. Proof. Since the kernel of pr ∞ isQ Let 0 be a lattice in G. Since G∞ × p∈Sf Gp (Zp ) is an open subgroup of G, the Q Q intersection 0 ∩ (G∞ × p∈Sf Gp (Zp )) is also a lattice in G∞ × p∈Sf Gp (Zp ). Hence, 0 ∞ is a lattice in G∞ by the compactness of the kernel of pr ∞ . It also follows from the above proof that if 0 is a uniform lattice in G, then 0 ∞ is a uniform lattice in G∞ as well. Lemma 2.2. For p ∈ Sf , let Gp be unipotent. If L is a closed subgroup of Gp such that Gp /L carries a finite Gp -invariant Borel measure, then L = Gp . Proof. Suppose this is not so. The general case is easily reduced to the case when Gp is abelian. Then there is a 1-parameter unipotent subgroup U = Qp x of Gp that is not contained in L. If {p−n x | n > n0 } ∩ L = ∅ for some positive integer n0 , then it contradicts the assumption that Gp /L carries a finite Gp -invariant Borel measure (since Qp /p−n0 Zp is an infinite countable set). Hence there exists a sequence xi ∈ L ∩ U for all i ≥ 1 such that xi → ∞ as i → ∞. Since L is closed and Z(xi ) ⊂ L, we have Zp (xi ) ⊂ L for all i ≥ 1. Note that U = ∪i≥1 Zp (xi ) since xi → ∞ as i → ∞. Therefore U ⊂ L, contradicting the assumption. Lemma 2.3. For each p ∈ S, let Gp be unipotent. If 0 is a lattice in G, then (1) prp (0) is dense in Gp for each p ∈ Sf ; (2) pr ∞ (0) is Zariski-dense in G∞ . Proof. Let p ∈ Sf . Denote by L the closure of prp (0) in Gp . Since G/ 0 has a finite G-invariant measure, then so does Gp /L (see, e.g., [4, Chap. II, Lemma 6.1]). Therefore by Lemma 2.2, L = Gp . This implies (1). For (2), the subgroup pr ∞ (0) is a lattice in G∞ by Proposition 2.1. Note that G∞ is a connected and simply connected nilpotent Lie group as is any real unipotent algebraic group. It is well known that any lattice in a connected and simply connected nilpotent Lie group is Zariski-dense (cf. [8]). This completes the proof. Let V be a connected unipotent algebraic Q-group, Vp = V(Qp ) and V = V . For a subring J of Q, we identify V (J ) with its image under the diagonal p p∈S embedding of V(Q) into V . It is well known that V (ZS ) is a uniform lattice in V (cf. [7, Thm. 5.7]).

Q 2.2.

Lemma 2.4. Let F be a discrete subgroup of V . Then the restriction pr∞ |F is injective. Proof. Without loss of generality, we may assume that V ⊂ GLN . Suppose that there is a nontrivial element x ∈ F such that pr∞ (x) = e. Since x is unipotent, we have that for each (x − e)n = 0 for some n ∈ N. Then, by the binomial formula, Q m m p ∈ Sf , prp (x s ) tends to e as m → ∞, where s = p∈Sf p. Hence, x s → e as m → ∞. This contradicts the assumption that F is discrete and thus proves our claim.

214

HEE OH

Lemma 2.5. For any nonzero integers m and d, there exists a nonzero integer k such that V (kZS ) ⊂ mV (dZS ), where mV (ZS ) = {x m | x ∈ V (ZS )}. Proof. Since V is unipotent, there exists an integer n such that (x − e)n = e for any x ∈ V . Without loss of generality, we may assume that V ⊂ GLN and ª V (dZS ) = x ∈ V | (x − e) is a matrix whose entries are in dZS . P j +1 /j )uj ). Therefore For any x = e + u ∈ V , we have log x 1/m = (1/m)( n−1 j =1 ((−1) we can find k such that if x ∈ V (kZS ), then log x 1/m ∈ log V (dZS ). Hence V (kZS ) ⊂ mV (dZS ). 2.3. It is well known that any lattice in a real algebraic unipotent group is an arithmetic subgroup (cf. [3]). Analogously, we now prove that any lattice in V = Q p∈S Vp is an S-arithmetic subgroup. Q We denote by V the product p∈S Lie Vp . There exists an integer b such that for any subgroup U in V , bhlog U i ⊂ log U , where hlog U i denotes the subring of V generated by log U (cf. [3, Lemma 5.2]). For a discrete subgroup F in V , we set 1F = bhlog F i. It is then clear that 1F is a discrete subgroup in V such that b log F ⊂ 1F ⊂ log F . Proposition 2.6. Let F be a discrete subgroup of V such that pr ∞ (F ) is Zariskidense in V∞ and prp (F ) is dense in Vp for each p ∈ Sf . Then F is an S-arithmetic subgroup of V . Proof. Since log : Vp → Vp is both a rational map and a homeomorphism for each p ∈ S, we have that pr ∞ (1F ) is Zariski-dense in V∞ and prp (1F ) is dense in Vp for each p ∈ Sf . We first show that there exists a Q-form on V such that 1F ⊂ V(Q). Since 1F ∞ is a Zariski-dense discrete subgroup in V∞ , which is a connected and simply connected nilpotent Lie group, then 1F ∞ is a lattice in V∞ (see, e.g., [8]). Therefore there exists a Q-form on V∞ such that 1F ∞ = V∞ (Z). (Note that this Q-form on V∞ does not necessarily coincide with the Q-form on V∞ given by the original Q-form on V with which we started.) We denote by V∞ (Qp ) the completion of V(Q) with respect to the p-adic norm. Note that 1F ∞ is a basis of the vector space V∞ (Qp ) over Qp . Therefore, in order to define a Qp -linear map φp : V∞ (Qp ) → Vp , it is enough to define it on 1F ∞ . For each x ∈ 1F ∞ , there exists an element yx ∈ 1F such that pr∞ (yx ) = x. By Lemma 2.4, such an element yx is unique. We set φp (x) = prp (yx ). We show that the map φp is a Qp -isomorphism. Since dim V∞ (Qp ) = dim Vp , it suffices to show that φp is onto. To show this, it is again enough to show that prp (1F ) ⊂ Im φp , since prp (1F ) is dense in Vp by assumption. For x = log y ∈ 1F , there is an Q n n ∈ N such that pr∞ (y s ) ∈ F ∞ where s = p∈Sf p. Then pr∞ (s n x) ∈ 1F ∞ , and hence prp (x) = φp (s −n pr∞ (s n x)). Therefore prp (1F ) ⊂ Im φp , proving that φp is


215

an isomorphism over Qp for each p ∈ Sf . Set φ∞ to be the identity map of V∞ . Hence, (V∞ , (φp , p ∈ S)) provides a Q-form on V such that 1F ⊂ V(Q). Using the exponential map, we obtain a Q-form on V such that F ⊂ V (Q). We now show that k1V (ZS ) ⊂ 1F for some nonzero integer k. It is easy to see that 1V (mZ) ⊂ 1V (ZS ) ∩ 1F for some nonzero integer m. Now let B be a basis of 1V (mZ) over Z. To show that 1F contains the ZS -module generated by B, it is enough to show that for any x ∈ B, we have p −n x ∈ 1F for all n ≥ 1 and for all p ∈ Sf , since the Z-span of {p−n | p ∈ Sf , n ≥ 1} is equal to ZS . For p ∈ Sf , since prp (1F ) is dense in Vp , there exists ni ∈ N, going to infinity as i → ∞, such that p −ni x ∈ 1F . Let n ≥ 1, and take any integer i such that ni ≥ n. Since pn = p ni −n p −ni and p−ni x ∈ 1F , we have p−n x ∈ 1F . Therefore 1F contains the ZS -module generated by 1V (mZ) as well as by B. Since V (mZ) has finite index in V (Z), we can find a nonzero integer k such that n k1VQ (Z) ⊂ 1V (mZ) . For any x ∈ 1V (ZS ) , there exists n such that s x ∈ 1V (Z) for n s = p∈Sf p, and hence ks x ∈ 1V (mZ) . Therefore kx ∈ 1F since 1F contains the ZS -module generated by 1V (mZ) . This proves that k1V (ZS ) ⊂ 1F . Since kb log V (ZS ) ⊂ k1V (ZS ) , 1F ⊂ log F , we have kbV (ZS ) ⊂ F . By Lemma 2.5, there exists a nonzero integer j such that V (j ZS ) ⊂ kbV (ZS ). Therefore V (j ZS ) ⊂ F and F is commensurable with V (ZS ). This shows that F is an S-arithmetic subgroup of V . 2.4. By Lemma 2.3 and the remark in Section 2.2 that any S-arithmetic subgroup of V is a uniform lattice in V , we obtain the following two corollaries of Proposition 2.6. Corollary 2.7. Any lattice in V is an S-arithmetic subgroup of V . Corollary 2.8. Let F be a discrete subgroup of V . Then the following are equivalent: (1) F is a lattice in V ; (2) pr ∞ (F ) is Zariski-dense in V∞ and prp (F ) is dense in Vp for each p ∈ Sf ; (3) V /F is compact. Proposition 2.9. Let F be a lattice in V . If F ⊂ V (Q), then F is commensurable with V (ZS ). Proof. By Proposition 2.6, there is a Q-form on V with respect to which F is an S-arithmetic subgroup. Since F ⊂ V (Q), this Q-form must coincide with the original Q-form of V. Therefore F is commensurable with V (ZS ). 3. Discrete subgroups in semisimple groups 3.1. Throughout this section, let S be a finite set of valuations of Q including ∞. For each p ∈ S, let Gp be a connected adjoint semisimple Qp -algebraic group without

216

HEE OH

any Qp -anisotropic factors,Q and let U1p , U2pQbe a pair of opposite horospherical Q subgroups of Gp . Set U1 = p∈S U1p , U2 = p∈S U2p , U1 = p∈S U1p (Qp ), and Q U2 = p∈S U2p (Qp ). For lattices F1 and F2 in U1 and U2 , respectively, we denote by 0F1 ,F2 the subgroup generated by F1 and F2 . Lemma 3.1. (1) The subgroups U1p (Qp ) and U2p (Qp ) generate the subgroup Gp+ (see [2]). (2) Any subgroup of Gp normalized by Gp+ is either trivial or contains Hp+ for some nontrivial normal simple Qp -subgroup Hp of Gp (see [10]). If Gp is Qp -simple, it is well known [10] that any subgroup of Gp normalized by Gp+ is either central (and hence trivial in our case since Gp is adjoint) or contains Gp+ . It is not difficult to see that this implies (2) of the above lemma, since a connected adjoint semisimple Qp -algebraic group is a direct product of adjoint Qp -simple groups. Lemma 3.2. Let F1 and F2 be lattices in U1 and U2 , respectively. Then for each p ∈ Sf , prp (0F1 ,F2 ) is dense in Gp+ . Proof. By Lemma 2.3, the closure of prp (Fi ) contains Uip (Qp ). Therefore the closure of prp (0F1 ,F2 ) contains the subgroup generated by U1p (Qp ) and U2p (Qp ), which is Gp+ by Lemma 3.1. Proposition 3.3. If 0 is a discrete subgroup of G containing F1 and F2 , then the restriction pr∞ |0 of pr∞ is injective. Proof. We show that the subgroup 00 = {γ ∈ 0 | pr∞Q(γ ) = e} is trivial. Without loss of generality, we may assume that 00 ⊂ GSf = p∈Sf Gp . Note that 00 is normalized by prSf (0) as well as by 0. We claim that 00 is normalized by G+ Sf = Q + . For each g ∈ G+ , there is a sequence {g | i = 1, 2, . . . } in pr (0) G i Sf p∈Sf p Sf converging to g as i → ∞, since prSf (0) is dense in G+ Sf by Lemma 3.2. Note that gi xgi−1 ∈ 00 for any x ∈ 00 and any i ≥ 1. But 00 is discrete, and in particular, it is closed. Therefore gxg −1 ∈ 00 , proving that 00 is normalized by G+ Sf . Let p ∈ + Sf . Since prp (00 ) is normalized by Gp and prp (00 ) is countable, it follows from Lemma 3.1 that prp (00 ) is trivial. Therefore 00 is trivial, yielding that pr∞ |0 is injective. Theorem 3.4 (See [1] and [4, Chap. I, Thm. Q 3.2.4]). Let G be a connected semisimple Q-algebraic group, and let G = p∈S G(Qp ). Then the S-arithmetic subgroup G(ZS ) is a lattice in G. 3.2. Let G be a connected P Q-simple algebraic group with Q-rank at least 1, S-rank at least 2 (S-rank of G = p∈S Qp -rank of G), and U1 , U2 a pair of opposite horospherical Q-subgroups of G. It was proved by Raghunathan [9] for Q-rank at least 2 and by Venkataramana [12] for Q-rank 1 that for any ideal A of ZS , the


217

subgroup generated by U1 (A) and U2 (A) is of finite index in G(A). It is not hard to see that the following theorem is a consequence of the above result. Theorem 3.5. Let F1 and F2 be lattices in U1 and U2 commensurable to U1 (ZS ) and U2 (ZS ), respectively. If the subgroup 0F1 ,F2 is discrete, then it is commensurable with the S-arithmetic subgroup G(ZS ). 4. Main theorem 4.1. As before, let S be a finite set of valuations of Q including ∞, and for each p ∈ S, let Gp be a connected semisimple Qp -algebraic group without any Qp subgroups anisotropic factors and Q let U1p , U2p be Q a pair of opposite Q horospherical Q of Gp . We set G = p∈S Gp , G = p∈S Gp , U1 = p∈S U1p , U2 = p∈S U2p , Q Q U1 = p∈S U1p (Qp ), and U2 = p∈S U2p (Qp ). Theorem 4.1 (See [5] and [6]). Let S = {∞} and let G be an absolutely simple real algebraic group with R-rank at least 2. Denote by Z(Ui ) the center of Ui for each i = 1, 2. Let the pair (G, U1 ) be as follows: (1) for commutative U1 , assume that G 6= E62 ; (2) for Heisenberg U1 , assume that G 6= A22 , Bn2 , Dn2 ; (3) for U1 such that Z(U1 ) is not the root group of a highest real root, assume that G0 6 = E62 , where G0 is the algebraic subgroup generated by Z(U1 ) and Z(U2 ); (4) for U1 such that Z(U1 ) is the root group of a highest real root, assume that [U1 , U1 ] 6 = Z(U1 ) and G00 6= E62 , where G00 is the algebraic subgroup generated by Z(U10 ) and Z(U20 ) and where Ui0 is the centralizer of the subgroup {g ∈ Ui | gug −1 u−1 ∈ Z(Ui ) for all u ∈ Ui } in Ui . For any lattices F1 and F2 in U1 and U2 , respectively, the subgroup 0F1 ,F2 is discrete if and only if there exists a Q-form on G such that 0F1 ,F2 is a subgroup of finite index in G(Z) and hence a nonuniform arithmetic lattice in G = G(R). Remark. As for the hypothesis on the pair (G, U1 ), if G is split over R and G is not locally isomorphic to SL3 (R), then U1 can be any horospherical subgroup. If G is locally isomorphic to SL3 (R) (i.e., is of type A22 ), then the above hypothesis excludes only the case when U1 is Heisenberg. If R-rank (G) ≥ 3, then U1 can be any commutative or Heisenberg horospherical subgroup. 4.2. The following is a special case of Margulis’s superrigidity theorem (see [4, Chap. VIII, Thm. 3.6]). Theorem 4.2. Let G be a connected almost Q-simple algebraic group without any R-anisotropic factors. Assume that R-rank G ≥ 2 and that 0 ⊂ G(Q) is an arithmetic subgroup of G. Let l be any field of char 0, H a connected adjoint semisimple l-group, and j : 0 → H(l) a homomorphism with the image being Zariski-dense in H.

218

HEE OH

Then there exists a rational l-epimorphism φ : G → H such that φ(x) = j (x) for all x ∈ 0. 4.3. We now prove the main theorem of this paper. The notation continues from Section 4.1. Theorem 4.3. Let Gp be a connected semisimple adjoint Qp -algebraic group without any Qp -anisotropic factors for each p ∈ S. Let F1 and F2 be lattices in U1 and U2 , respectively, such that 0F1 ,F2 is discrete. Assume that (G∞ , U1∞ ) satisfies the conditions in Theorem 4.1. Then there exists a Q-form on G (in the sense of Section 1.5) such that 0F1 ,F2 is a subgroup of finite index in the S-arithmetic subgroup G(ZS ). Hence 0F1 ,F2 is a nonuniform S-arithmetic lattice in G. Proof. Since 0F∞1 ,F2 is a discrete subgroup of G∞ (by Proposition 2.1) containing the lattices F1∞ and F2∞ in U1∞ (R) and U2∞ (R), respectively, Theorem 4.1 implies that there exists a Q-form on G∞ such that 0F∞1 ,F2 is a subgroup of finite index in G∞ (Z). By Proposition 3.3, the map pr∞ |0F1 ,F2 is injective. Therefore we can define a map jp : 0F∞1 ,F2 → Gp as follows: For x ∈ 0F∞1 ,F2 , set jp (x) = prp ◦ (pr ∞ )−1 (x). It is clear from the definition of 0F∞1 ,F2 that jp (0F∞1 ,F2 ) ⊂ Gp (Zp ). We claim that jp (0F∞1 ,F2 ) is Zariski-dense in Gp . Since the subgroup generated by U1p and U2p is Zariski-dense in Gp , it suffices to show that the subgroup jp (Fi∞ ) is Zariski-dense in Uip for each i = 1, 2. It is clear for p = ∞ since j∞ (Fi∞ ) = Fi∞ is a lattice in Ui ∞ (R). For p ∈ Sf , note that jp (Fi∞ ) = prp (Fi ) ∩ Uip (Zp ). Since prp (Fi ) is dense in Uip by Lemma 2.3 and since Uip (Zp ) is open in Uip , jp (Fi∞ ) is dense in Uip (Zp ). Therefore the Zariski closure of jp (Fi∞ ) contains Uip (Zp ) and hence Uip , since it is well known that Uip (Zp ) is Zariski-dense in Uip . By Theorem 4.2, for each p ∈ S, there exists a Qp -epimorphism φp : G∞ → Gp such that φp (x) = jp (x) for all x ∈ 0F∞1 ,F2 . Since G∞ is absolutely simple in our case and hence has no nontrivial normal subgroup, φp is in fact an isomorphism. Therefore (G∞ , (φp , p ∈ S)) endows a Q-form on G with respect to which U1 and U2 are defined over Q. Since Fi ⊂ Ui (Q), Fi is commensurable with Ui (ZS ) by Proposition 2.9. Since 0F1 ,F2 is discrete, it follows from Theorem 3.5 that the subgroup 0F1 ,F2 is commensurable with the S-arithmetic subgroup G(ZS ). Since each Gp is adjoint, we the adjoint representation of Gp . Morecan assume that Gp ⊂ SLN by considering Q over we may assume G(Q) ⊂ { p∈S g | g ∈ SLN (Q)} = SLN (Q) by considering the isomorphisms φp . Since 0F1 ,F2 is an S-arithmetic subgroup contained in G(Q), there exists a ZS -module L in QN of rank N that is invariant by 0F1 ,F2 (cf. [7, Prop. 4.2]); hence 0F1 ,F2 ⊂ GL = {g ∈ G(Q) | g(L) ⊂ L}. Now, by applying the automorphism of SLN (C) that changes the standard basis to a basis of L, we may assume G(ZS ) = GL so that 0F1 ,F2 ⊂ G(ZS ). By Theorem 3.4, 0F1 ,F2 is a lattice in G. Since the lattice 0F∞1 ,F2 in G∞ contains a nontrivial unipotent element, 0F∞1 ,F2 is a nonuniform lattice by Godement’s criterion


219

(cf. [8]). Therefore, by the remark following Proposition 2.1, the lattice 0F1 ,F2 is nonuniform. Proof of Theorem 0.2. The hypothesis on (G∞ , U1∞ ) in Theorem 4.2 is satisfied for the groups considered in Theorem 0.2 by the remark following Theorem 4.1. To go from an adjoint group to its finite covers, we now give a standard argument. For each p ∈ S, there exists a connected semisimple adjoint Qp -group Gp0 and a Qp Q isogeny fp : Gp → Gp0 (cf. [4, Chap. I, Prop. 1.4.11]). Set f = p∈S fp , the direct product of the fp ’s. Set Fi0 = f (Fi ) for each i = 1, 2, and let 0F0 1 ,F2 be the subgroup generated by F10 and F20 . Since the kernel of f is finite, it follows that Fi0 is a lattice in f (Ui ) and 0F0 1 ,F2 is discrete since 0F0 1 ,F2 ⊂ f (0F1 ,F2 ). Hence by Theorem 4.3, there Q exists a Q-form on G0 = p∈S Gp0 such that 0F0 1 ,F2 is a subgroup of finite index in G0 (ZS ). Since f (0F1 ,F2 ) is a discrete subgroup containing the S-arithmetic subgroup 0F0 1 ,F2 , the subgroup f (0F1 ,F2 ) is commensurable with G0 (ZS ). Hence 0F1 ,F2 is an S-arithmetic subgroup of G by the definition in Section 1.6. Hence Theorem 0.2 is proved. References [1] [2] [3]

[4] [5] [6] [7] [8] [9] [10] [11]

[12]

A. Borel, Some finiteness properties of adèle groups over number fields, Inst. Hautes Études Sci. Publ. Math. 16 (1963), 5–30. A. Borel and J. Tits, Homomorphismes “abstraits” de groupes algébriques simples, Ann. of Math. (2) 97 (1973), 499–571. G. A. Margulis, “Non-uniform lattices in semisimple algebraic groups” in Lie Groups and Their Representations (Budapest, 1971), ed. I. M. Gelfand, Wiley, New York, 1975, 371–553. , Discrete Subgroups of Semisimple Lie Groups, Ergeb. Math. Grenzgeb. (3) 17, Springer-Verlag, Berlin, 1991. H. Oh, Discrete subgroups generated by lattices in opposite horospherical subgroups, J. Algebra 203 (1998), 621–676. , On discrete subgroups containing a lattice in a horospherical subgroup, Israel J. Math. 110 (1999), 333–340. V. Platonov and A. Rapinchuk, Algebraic Groups and Number Theory, Pure Appl. Math. 139, Academic Press, Boston, 1994. M. S. Raghunathan, Discrete Subgroups of Lie Groups, Ergeb. Math. Grenzgeb. (3) 68, Springer-Verlag, New York, 1972. , A note on generators for arithmetic subgroups of algebraic groups, Pacific J. Math. 152 (1992), 365–373. J. Tits, Algebraic and abstract simple groups, Ann. of Math. (2) 80 (1964), 313–329. , “Classification of algebraic semisimple groups” in Algebraic Groups and Discontinuous Subgroups (Boulder, Colo., 1965), Proc. Sympos. Pure Math. 9, Amer. Math. Soc., Providence, 1966, 33–62. T. N. Venkataramana, On systems of generators of arithmetic subgroups of higher rank groups, Pacific J. Math. 166 (1994), 193–212.

Department of Mathematics, Princeton University, Princeton, New Jersey 08544, USA; [email protected]

Vol. 101, No. 2


© 2000

A MODULAR INVARIANCE ON THE THETA FUNCTIONS DEFINED ON VERTEX OPERATOR ALGEBRAS MASAHIKO MIYAMOTO

To Professor Toshiro Tsuzuku on his seventieth birthday 1. Introduction. Throughout this paper, V denotes a vertex algebra, or Poperator −n−1 denotes V , Y, 1, ω) with central charge c and Y (v, z) = v(n)z VOA, (⊕∞ n=0 n a vertex operator of v. (Abusing the notation, we also use it for vertex operators of v for V -modules.) o(v) denotes the grade-keeping operator of v, which is given by v(m − 1) for v ∈ Vm and defined by extending it for all elements of V linearly. In particular, o(ω) equals L(0) = ω(1) for the Virasoro element ω of V and o(v) = v(0) for v ∈ V1 . In order to simplify the situation, we assume that dim V0 = 1 so that there is a constant hv, ui ∈ C such that v1 u = −hv, ui1 for v, u ∈ V1 . We call V a rational vertex operator algebra in the case when each V -module is a direct sum of simple modules. Define C2 (V ) to be the subspace of V spanned by elements u(−2)v for u, v ∈ V . We say that V satisfies condition C2 if C2 (V ) has finite codimension in V . For a V -module M with grading M = ⊕Mm , we define the formal character as X (1) dim Mm q m = tr M q −c/24+L(0) . chq M = q −c/24 In this paper, we consider these functions less formally by taking q to be the usual local parameter q = qτ = e2π ιτ at infinity in the upper half-plane ª H = τ ∈ C | Iτ > 0 . Although it is often said that a VOA is a conformal field theory with mathematically rigorous axioms, the axioms of VOA do not assume the modular invariance. However, Zhu [Z] showed the modular (SL2 (Z)) invariance of the space E D |a | q1 1 · · · qn|an | tr W Y (a1 , q1 ) · · · Y (an , qn )q L(0)−c/24 : W irreducible V -modules (2) for a rational VOA V with central charge c and ai ∈ V|ai | under condition C2 , which are satisfied by many known examples, where qj = qzj = e2πιzj and |ai | denotes the Received 13 October 1998. 1991 Mathematics Subject Classification. Primary 17B69; Secondary 11F11. Author’s work supported by the Grants-in-Aid for Scientific Research, number 09440004 and number 10974001, the Ministry of Education, Science and Culture, Japan. 221

222

MASAHIKO MIYAMOTO

weight of aj . For example, the space chq W : W irreducible V -modules is SL2 (Z)-invariant. Recently, Dong, Li, and Mason [DLiM] extended Zhu’s idea and proved a modular invariance of the space (3) tr U φ i q L(0)−c/24 : i ∈ Z, U φ-twisted modules by introducing the concept of φ-twisted modules for a finite automorphism φ. An easy example of automorphism of a VOA is given by a vector v ∈ V1 as φ = e2πιv(0) . Especially if the eigenvalues of o(v) (= v(0)) on modules are in (1/n)Z, then the order of e2π ιo(v) is finite. So for a V -module W and u, v ∈ V1 , we define ZW (v; u; τ ) = tr W e2πι(o(v)−(hv,ui/2)) q L(0)+o(u)−(c+12hu,ui)/24

(4)

and we call Z )c a theta function of W , where u1 u = −hu, ui1 and QW∞(v; 0; τ )η(τ 1/24 n η(τ ) = q n=1 (1 − q ) is the Dedekind eta function. It is worth noting that c + 12hu, ui is the central charge of conformal element ω − L(−1)u and o(ω − L(−1)u) is equal to L(0) + o(u); see [DLnM]. For example, let V be a lattice VOA V2Zx constructed from a 1-dimensional lattice L = 2Zx with hx, xi = 1. It has exactly four irreducible modules (see [D]): W0 = V2Zx ,

W1 = V(2Z+(1/2))x , W2 = V(2Z+1)x , W3 = V(2Z−(1/2))x . P Let θh,k (z, τ ) (:= n∈Z exp(π ι(n + h)2 τ + 2π(n + h)(z + k))) be theta functions for h, k = 0, 1/2. By the construction of a lattice VOA (see [FLMe]), it is easy to check ¡ ¡ ¡ θh,k (z, τ ) = η(τ ) (ι)4hk ZW2h zx(−1)1; 0; τ + (−1)k (ι)4hk ZW2+2h zx(−1)1; 0; τ (5) for h, k = 0, 1/2; their modular transformations πιz2 z −1 , = (i)4hk (−ιτ )1/2 exp θk,h (z, τ ) θh,k τ τ τ are well known (see [Mu]). In particular, there are constants Ahk ∈ C such that X −1 1 2 h = Ak ZWk 0; zx(−1) + z , τ . ZWh zx(−1)1; 0; τ 2

(6)

(7)

Namely, the modular transformations of ZWh (zx(−1)1; 0 : τ ) are expressed by linear combinations of ZWk (u; v; τ ) of (ordinary) modules Wk , but not twisted modules. By this result, for an automorphism φ = ev(0) , we can expect to obtain a modular transformation by using only the ordinary modules, which offers some information about twisted modules. This is the motivation of this paper, and we actually show that the above result is generally true, that is, we prove the following modular transformation by using Zhu’s result (2).

223

THETA FUNCTIONS OF VERTEX OPERATOR ALGEBRAS

Main Theorem. Let V be a rational vertex operator algebra with the irreducible modules {Wi : i = 1, . . . , m} and u, v ∈ V1 . Assume v(0)v = v(0)u = u(0)v = u(0)u = 0 and v(1)v, v(1)u, u(1)u ∈ C1. If V satisfies Zhu’s finite condition C2 , then ª (8) ZWh (v; u; τ ) : h = 1, . . . , m ¡ satisfies a modular invariance. That is, for α = fa db ∈ SL2 (Z), there are constants Ahα,k (see Theorem 4.1) such that ZWh

aτ + b v; u; f τ +d

=

m X k=1

Ahα,k ZWk (av + bu; f v + du; τ ).

(9)

2. Vertex operator algebras Definition. A vertex operator algebra is a Z-graded vector space V = ⊕n∈Z Vn

(10)

satisfying dim Vn < ∞ for all n and Vn = 0 for n 0, equipped with a linear map V −→ (End V ) z, z−1 , X v(n)z−n−1 v −→ Y (v, z) = n∈Z

and with two distinguished vectors, vacuum element 1 ∈ V0 and conformal vector ω ∈ V2 , satisfying the following conditions for u, v ∈ V : u(n)v = 0

for n sufficiently large; Y (1, z) = 1;

Y (v, z)1 ∈ V [[z]]

and

lim Y (v, z)1 = v;

z→0

(z − x)N Y (v, z)Y (u, x) = (z − x)N Y (u, x)Y (v, z)

for N sufficiently large,

where (z1 −z2 )n (n ∈ Z) are to be expanded in nonnegative integral powers of second variable z2 ;

1 L(m), L(n) = (m − n)L(m + n) + (m3 − m)δm+n,0 c 12

for m, n ∈ Z, where L(m) = ω(m + 1) and c is called central charge; L(0)v = nv for v ∈ Vn ; ¡ d Y (v, z) = Y L(−1)v, z . dz

(11)

224

MASAHIKO MIYAMOTO

We also have the notion of modules: Let (V , Y, 1, ω) be a VOA. A weak module W of (V , Y, 1, ω) is a C-graded vector space W = ⊕n∈C Wn

(12)

equipped with a linear map ¡ V −→ End(W ) z, z−1 , X v W (n)z−n−1 v −→ Y W (v, z) =

¡

vn ∈ End(W )

n∈Z

satisfying the following conditions: For u, v ∈ V , w ∈ M, v W (m)w = 0 for m 0; Y W (1, z) = 1; LW (0)w = nw for w ∈ Wn , LW (0) = ωW (1); ¡ d W Y (v, z) = Y L(−1)v, z ; dz and the following Jacobi identity holds: z1 − z2 z2 − z1 Y W (u, z1 )Y W (v, z2 ) − z0−1 δ Y W (v, z2 )Y W (u, z1 ) z0−1 δ z0 −z0 ¡ z1 − z0 −1 = z2 δ Y W Y (u, z0 )v, z2 . z2 A weak module W is called a module if every finitely generated weak submodule M = ⊕r∈C Mr of W satisfies (1) dim Mr < ∞, (2) Mr+n = 0 for n ∈ Z sufficiently large, for any r ∈ C. 3. Formal power series. We use the notation q and qz to denote e2πιτ and e2πιz , respectively. In this paper, the formal power series ∞ X nqz−n q n nqzn 2 + P2 (qz , q) = (2πι) 1 − qn 1 − qn n=1

P ni plays an essential role, where 1/(1 − q n ) is understood as ∞ i=0 q . The limit of P2 (qz , q) (which we still denote as P2 (qz , q)) relates to p(z, τ ) by P2 (qz , q) = p(z, τ ) + G2 (τ ), where G2 (τ ) =

X π2 + 3

X

m∈Z−{0} n∈Z

1 (mτ + n)2

THETA FUNCTIONS OF VERTEX OPERATOR ALGEBRAS

225

is the Eisenstein series and p(z, τ ) is the Weierstrass p-function X 1 1 1 p(z, τ ) = 2 + − . z (z − mτ − n)2 (mτ + n)2 (m,n)6=(0,0)

It is known that G2 and

aτ + b f τ +d

= (f τ + d)2 G2 (τ ) − 2π ιf (f τ + d)

aτ + b z , p f τ +d f τ +d

= (f τ + d)2 p(z, τ ).

In particular, P2

aτ + b z , f τ +d f τ +d

= (f τ + d)2 P2 (z, τ ) − 2π ιf (f τ + d).

(13)

In this paper, we use variables {z1 , . . . , zn } and calculate the products of formal power series P2 (qzi −zj , τ ). In order to simplify notation, we use a transposition (i, j ) of symmetric groups 6n on = {1, . . . , n}. For {(i11 , i12 ), . . . , (it1 , it2 )} with is1 < is2 and iab 6 = icd for (a, b) 6= (c, d), we view σQ= (i11 , i12 ) · · · (it1 , it2 ) as an involution (element of order 2) of 6n and denote tj =1 P2 (qzj 2 −zj 1 , τ ) by Q 2 i

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

Recommend Documents