No title

Vol. 102, No. 1 DUKE MATHEMATICAL JOURNAL © 2000 SINGULAR POINTS AND LIMIT CYCLES OF PLANAR POLYNOMIAL VECTOR FIELDS ...

Author: Jonathan Wahl (Academic Editor)

25 downloads 594 Views 4MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Vol. 102, No. 1

DUKE MATHEMATICAL JOURNAL

© 2000

SINGULAR POINTS AND LIMIT CYCLES OF PLANAR POLYNOMIAL VECTOR FIELDS ILIA ITENBERG and EUGENI˘I SHUSTIN

Contents 0. 1. 2. 3. 4. 5. 6. 7.

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Gluing theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Planar vector fields with many limit cycles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Singular points of planar vector fields: Formulation of results . . . . . . . . . . . . . . Topological classification of collections of singular points . . . . . . . . . . . . . . . . . Elementary vector fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Thin topological classification: Basic construction . . . . . . . . . . . . . . . . . . . . . . . . Thin topological classification: Other cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 4 7 10 11 18 21 30

0. Introduction. The subject of the article is real planar vector fields x˙ = P (x, y), y˙ = Q(x, y), where P , Q are polynomials. We consider two questions related to the second part of the Hilbert 16th problem [Hi]. (1) What can be the number and arrangement of limit cycles of a vector field of degree d in R2 ? (2) Given a classification of singular points of planar vector fields, how many singular points of each type can a vector field of degree d in R2 have? Our approach to these problems comes from the Viro method [V1] to [V4] (see also [IV] and [R]) invented in the framework of the first part of the 16th problem, topology of real algebraic varieties. This method, actually, consists in reducing a problem on polynomials with an arbitrary Newton polyhedron to that on polynomials with smaller Newton polyhedra. Various applications and developments related to the topology of real algebraic varieties and their singularities can be found in [GKZ], [I1], [I2], [IV], [S1] to [S3], and [St]. Note also that Newton polyhedra and diagrams have been used since the last century for the local study of singular points of differential systems. (For the modern account, see, e.g., [Br].) In this paper we prove Viro-type “gluing” theorems for planar polynomial vector fields (see Theorems 1.3.1, 1.4.1, and Corollary 1.4.2). Using gluing theorems we construct vector fields with many limit cycles and vector fields with given numbers of singular points of prescribed types. The exact statements (see Theorem 2.1 on limit Received 11 June 1997. Revision received 19 January 1999. 1991 Mathematics Subject Classification. Primary 34C05. Itenberg partially supported by European Community contract CHRX-CT94-0506. Shustin partially supported by grant number 6836-1-9 of the Ministry of Science and Arts, Israel. 1

2

ITENBERG AND SHUSTIN

cycles and Theorems 3.3 and 3.4 on singular points) are presented below. It is known (see [E] and [Il2]) that a polynomial vector field has only finitely many limit cycles, but no general upper bound (depending only on the degree) is found. On the other hand, one can look for examples of fields with a large number of limit cycles. Among the known examples of vector fields with many limit cycles, one can mention quadratic fields with four limit cycles (see [An], [CW], and [Sh]), cubic fields with 11 limit cycles (see [Z]), vector fields of degree d close to Hamiltonian ones and having (d 2 + 5d − 14)/2 limit cycles (see [O]) or d(d + 1)/2 − 1 limit cycles (see [Il1] and [P]), and the vector fields of degrees d = 2k − 1, k ≥ 2, with (1/2)d 2 log2 d + O(d 2 ) limit cycles (see [CL]). The construction of C. J. Christopher and N. G. Lloyd [CL] provides an asymptotic lower bound (1/8)d 2 log2 d for the maximal number of limit cycles of a planar vector field of degree d. We improve this asymptotic lower bound in the following way (see Theorem 2.1): For any integer d ≥ 3 there exists a planar vector field of degree d with at least (1/2)d 2 log2 d − Cd 2 log2 log2 d limit cycles, where C is a positive number that does not depend on d. Let P0 , P1 , . . . , Pn be polynomials in n variables. Under certain “general position” assumptions on Pi ’s, A. Khovanski˘i [Kh1] obtained sharp estimates (in terms of degrees of Pi ’s) for the total index of the vector field (P1 , . . . , Pn ) in Rn and in the regions P0 > 0 and P0 < 0. In particular, for n = 2, he proved that the total index I of singular points of a vector field of degree d, having no singularities at infinity, satisfies |I | ≤ d, and, under the assumption that there are d 2 real singular points (so all the points have index ±1), all the values of I having the same parity as d in this range can be realized by a suitable vector field (see also [Kh2]). We refine Khovanski˘i’s result for planar vector fields in two steps. First, we distinguish between repellers and attractors (having the same index +1). We call the corresponding classification topological and prove the following statement (see Theorem 3.3): For any nonnegative integers d, s0 , σ1 , σ2 , and σ3 satisfying 2s0 + σ1 + σ2 + σ3 = d 2 ,

|σ1 + σ2 − σ3 | ≤ d,

there exists a planar vector field of degree d with 2s0 imaginary singular points, σ1 attractors, σ2 repellers, and σ3 saddles. Second, we distinguish between a source and a source-focus, and between a sink and a sink-focus. The corresponding classification is called thin topological. The following theorem gives an asymptotically complete thin topological classification of collections of singular points of vector fields satisfying some generality conditions (see Subsection 1.4, Section 3, and Theorem 3.4): For any nonnegative integers d, s0 , s1 , s2 , s3 , s4 , and s5 satisfying 2s0 + s1 + s2 + s3 + s4 + s5 = d 2 ,

|s1 + s2 + s3 + s4 − s5 | ≤ d,

SINGULAR POINTS AND LIMIT CYCLES OF VECTOR FIELDS

and

3

  1,      1,

if d is odd, s1 + s2 + s3 + s4 − s5 ≥ 3, if d is even, (d − 1)(d − 2) < 2s0 < d 2 , s1 + s2 ≥ s1 + s2 + s3 + s4 − s5 ≥ 0,    2, if d is even, 2s0 ≤ (d − 1)(d − 2),     s1 + s2 + s3 + s4 − s5 ≥ 0, there exists a planar vector field of degree d with 2s0 imaginary singular points, s1 sinks, s2 sources, s3 sink-foci, s4 source-foci, and s5 saddles. The last statement is proved by means of the gluing Theorem 1.4.1. To prove the statement on topological classification, we use another gluing theorem (see Theorem 4.1.1, Section 4), which is based on a torus action on (R∗ )2 preserving only topological types of singular points. The reason is that the proof of the result on topological classification becomes much simpler and shorter than possible arguments with general gluing theorems. We point out in this connection that gluing theorems may have various forms in accordance with the problem studied. Remark. An answer to the question on singular points was announced in [IS] with a sketch of the proof. However, that article contained minor flaws in its assertions, and the proofs were incomplete. In the present article we give the corrected statements and the complete proofs. Moreover, we add important details in the approach used. The paper consists of several sections. In Section 1, we formulate and prove the gluing theorems. In the subsequent sections, the gluing theorems are transformed into algorithmic procedures whose outputs are vector fields with many limit cycles (see Section 2) or with a given collection of types of singular points (see Sections 4–7). Such a procedure contains two main steps: (1) analysis of properties of vector fields with small Newton polygons; (2) subdivision of given (large) Newton polygons into appropriate small subpolygons; search for suitable vector fields with these small Newton polygons in order to provide the glued vector field with the required properties. For the problem on limit cycles, both the steps are carried out in Section 2. For the problem on topological classification of collections of singular points, both the steps are performed in Section 4 by means of the methods of [IS]. In Section 5, we carry out the first step for the problem on thin topological classification of collections of singular points. In Section 6, we describe the basic construction and give the thin topological classification for the case when all the singular points are real and the total index is equal to −d. In Section 7, we explain how to complete the proof, but we skip the details, because of the large number (more than ten) of cases, each of them requiring a slight modification of the main algorithm. Acknowledgements. A part of this work was done during the authors’ stay in Mathematisches Forschungsinstitut Oberwolfach under the program “Research in

4


Paris” supported by Volkswagen-Stiftung, and in Max-Planck-Institut für Mathematik. The authors are grateful to these institutions for the hospitality and excellent work conditions. We are also grateful to L. Gavrilov and the referee for valuable remarks. 1. Gluing theorems 1.1. General scheme of gluing vector fields. By polygon, we mean a convex polygon with integral vertices contained in the positive quadrant R2+ = {(λ, µ) ∈ R2 : λ, µ ≥ 0}. First, we define a class of vector fields adapted for the gluing procedure. For a polygon , define polygons x = conv (i, j − 1) : (i, j ) ∈ ∩ Z2 , j ≥ 1 , y = conv (i − 1, j ) : (i, j ) ∈ ∩ Z2 , i ≥ 1 . A polynomial vector field x˙ = P (x, y), y˙ = Q(x, y) is called admissible if the Newton polygons of the polynomials P , Q coincide with x , y , respectively, for some polygon . In this case, the polygon is called the Newton polygon of the given vector field. It is easy to see that Hamiltonian vector fields are admissible. Assume that we are given the following objects: (1) a nondegenerate (i.e., of dimension 2) polygon , (2) a subdivision = 1 ∪ · · · ∪ N , where 1 , . . . , N are convex polygons, (3) a convex piecewise-linear function ν : → R, which is linear on each polygon 1 , . . . , N and nonlinear on each union k ∪ l , k = l. Suppose also that we have a collection of nonzero real numbers Aij indexed by the integral points of the polygon x and that we have a collection of nonzero real numbers Bij indexed by the integral points of the polygon y . We define the functions ν x , ν y on the polygons x , y by ν x (i, j ) = ν(i, j + 1),

(i, j ) ∈ x ,

ν y (i, j ) = ν(i + 1, j ),

(i, j ) ∈ y .

Consider the vector fields Vk with Newton polygons k (1 ≤ k ≤ N ), defined by Aij x i y j , (1.1.1) y˙ = Bij x i y j . Vk : x˙ = (i,j )∈xk

y

(i,j )∈k

Let us introduce a 1-parametric family V (t), t > 0, of vector fields: x y (1.1.2) Aij x i y j t ν (i,j ) , y˙ = Bij x i y j t ν (i,j ) . V (t) : x˙ = (i,j )∈x

(i,j )∈y

Note that all these vector fields are admissible. The idea of the gluing method is that the properties of the block vector fields (1.1.1), stable with respect to small deformations and invariant with respect to multiplication of vector fields by positive


5

constants and linear transformations (x, y) → (αx, βy), α, β > 0, determine the corresponding properties of a glued vector field V (t) for a sufficiently small positive value of t. In the present paper we consider two examples of such properties. 1.2. Limit cycles of vector fields. Let " be a nonsingular limit cycle of a planar vector field V , that is, an isolated nonsingular trajectory of V (for detailed definitions see, e.g., [AIl] and [Ha]). The monodromy map of " is defined as follows. Take a point z ∈ " and the straight line $ passing through z and orthogonal to ". Let w ∈ $ be close to z. Then the trajectory of V , (1.2.1)

x(τ ), y(τ ) ,

τ ≥ 0,

x(0), y(0) = w,

is close to " and returns to a neighborhood U of z after one rotation. We define ϕz (w) ∈ $ as the intersection point of trajectory (1.2.1) with $ ∩ U corresponding to the minimal positive τ . Given a local coordinate s on $ with s(z) = 0, the monodromy map ϕz (s) is analytic and satisfies ϕz (0) = 0. The zero s = 0 of the function ϕz (s)−s is isolated. We call a nonsingular limit cycle " simple if s = 0 is a simple zero of the function ϕz (s) − s. It is clear that the multiplicity of " does not depend on the choice of z ∈ ". The monodromy map and its derivative depend smoothly on small variations of the vector field, and hence a simple limit cycle is stable with respect to such deformations. The limit cycle scheme ᏿(V ) of V (by analogy with the real schemes of plane algebraic curves) is the set of simple limit cycles of V lying in (R∗ )2 partially ordered by the relation "1 ≺ "2 if the limit cycle "1 lies inside the limit cycle "2 . 1.3. Gluing vector fields with limit cycles Theorem 1.3.1. In the notation of Subsections 1.1 and 1.2,

for t > 0 small enough, the disjoint union of partially ordered sets ᏿(V1 ) · · · ᏿(VN ) can be embedded into ᏿(V (t)) (as an induced partially ordered subset). Proof. First, we represent the vector fields (1.1.1), (1.1.2) as follows: ˙ = Vk,x = Vk : xy

Ai,j −1 x i y j ,

yx ˙ = Vk,y =

(i,j )∈k

Bi−1,j x i y j ,

(i,j )∈k

k = 1, . . . , N, V (t) : xy ˙ =

(i,j )∈

Ai,j −1 x i y j t ν(i,j ) ,

yx ˙ =

Bi−1,j x i y j t ν(i,j ) .

(i,j )∈

Let lk (i, j ) = αk i + βk j + γk be the linear function equal to ν(i, j ) on k , and let νk = ν − lk , k = 1, . . . , N . The linear transformation Mk (x, y) = (xt αk , yt βk ) takes

6


the field V (t) into a field Vk (t): xy ˙ = t αk +βk +γk

(i,j )∈

yx ˙ = t αk +βk +γk

Ai,j −1 x i y j t νk (i,j ) = t αk +βk +γk Vk,x + O(t) , Bi−1,j x i y j t νk (i,j ) = t αk +βk +γk Vk,y + O(t) .

(i,j )∈

Let + ⊂ (R∗ )2 be a compact set whose interior contains all the simple limit cycles of V1 , . . . , VN lying in (R∗ )2 . Clearly, for small positive t, the vector fields Vk (t) in + have simple limit cycles close to the limit cycles of Vk , k = 1, . . . , N . The transformations Mk only move limit cycles in (R∗ )2 . Note that |αk − αm | + |βk − βm | > 0,

k = m.

Thus, we can choose t > 0 such that M1−1 (+), . . . , MN−1 (+) are disjoint, and thereby we obtain an embedding of the disjoint union of ᏿(V1 ), . . . , ᏿(VN ) into the partially ordered set of simple limit cycles of V (t) in M1−1 (+) ∪ · · · ∪ MN−1 (+). 1.4. Gluing vector fields with singular points. A point z∗ = (x∗ , y∗ ) ∈ C2 is called singular for a vector field V : x˙ = P , y˙ = Q, if P (x∗ , y∗ ) = Q(x∗ , y∗ ) = 0. A singular point z∗ is called nondegenerate if

det A = 0,

∂P /∂x A= ∂Q/∂x

∂P /∂y

, ∂Q/∂y z ∗

and, for z∗ ∈ R2 , the linearization matrix A has no eigenvalues with zero real part. We call a vector field of degree d nondegenerate if it has d 2 nondegenerate singular points in (C∗ )2 . Let Ei , i ∈ I, be the types of nondegenerate singular points under some equivalence relation. A type Ei is called stable if any singular point z∗ belonging to Ei keeps its type under small variations of the vector field in the class of polynomial vector fields, under multiplication of the vector field by positive constants, and under linear transformations (x, y) → (αx, βy), α, β > 0. For a vector field V , we denote by c(V ) the distribution of all the singular points of V in (C∗ )2 into the stable types Ei , i ∈ I . Namely, c(V ) = (ci , i ∈ I ), where ci is the number of singular points of V in (C∗ )2 of type Ei . Theorem 1.4.1. In the above notation, for a sufficiently small t > 0, the vector field V (t) given by (1.1.2) satisfies ci (V (t)) ≥ N k=1 ci (Vk ) for all i ∈ I . The proof coincides with the proof of Theorem 1.3.1 in the following sense: we repeat the arguments word for word, replacing limit cycles by singular points of a given type.

7


Corollary 1.4.2. In the above notation, suppose that N k=1 i∈I ci (Vk ) is equal to 2 vol(x , y ), where vol(x , y ) denotes the mixed area of x and y . Then N c V (t) = c(Vk ). k=1

Proof. The statement follows from Theorem 1.4.1, since 2 vol(x , y ) is the maximal possible number of singular points in (C∗ )2 of a vector field with Newton polygon (see [Be]). 2. Planar vector fields with many limit cycles. In 1995 C. J. Christopher and N. G. Lloyd [CL] constructed, for any k ≥ 2, a vector field of degree d = 2k − 1 with at least 1 22k−1 (k − 3) + 3 2k − 1 = (d + 1)2 log2 (d + 1) − 3 + 3d 2 limit cycles. This construction holds a record for the number of limit cycles, and, provided the best known asymptotic lower bound for the maximal possible number of limit cycles of planar polynomial vector fields: there exists a family Vd , d = 1, 2, 3, . . . , where Vd is a vector field of degree d with at least (1/8)d 2 log2 d +O(d 2 ) limit cycles. The gluing procedure combined with Christopher and Lloyd’s examples allows us to improve the last statement. Theorem 2.1. There is a positive constant C such that for any integer d ≥ 3 there exists a vector field of degree d with at least 1 2 d log2 d − Cd 2 log2 log2 d 2 limit cycles. To construct the proclaimed vector fields we use the following auxiliary ones. (1)

(2)

Proposition 2.2. For any k ≥ 2 there exist admissible vector fields Vk and Vk with Newton triangles conv{(1, 1), (1 + 2k , 1), (1, 1 + 2k )} and conv{(1 + 2k , 1 + 2k ), (i) (1 + 2k , 1), (1, 1 + 2k )}, respectively, such that each Vk , i = 1, 2, has at least (k − 3)22k−1 + 3(2k − 1) ≥ (k − 3)22k−1 simple limit cycles in (R∗ )2 . Proof. Let k ≥ 2 and x˙ = Pk (x, y), y˙ = Qk (x, y) be a vector field of degree − 1 with at least (k − 3)22k−1 + 3(2k − 1) ≥ (k − 3)22k−1 simple limit cycles in (R∗ )2 . Such a vector field can be obtained from Christopher and Lloyd’s examples by a suitable shift in R2 . The multiplication of a vector field by xy does not affect its limit cycles in (R∗ )2 . Put 2k

k (x, y) = xyPk (x, y) + xpk (x), P

k (x, y) = xyQk (x, y) + yqk (y), Q

where pk , qk are polynomials in one variable of degree 2k . If the absolute values (1) k , Q k ) of the coefficients of pk , qk are sufficiently small, the vector field Vk = (P

8


d

2k1

..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... . ..... ..... ..... ..... . . . . ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... . ..... ..... ..... ..... . . . ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... . ..... . . ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ... .. . .

2k1

. ......

2k1 + 2k2 . . . d

Figure 1

with Newton triangle conv{(1, 1), (1 + 2k , 1), (1, 1 + 2k )} has at least (k − 3)22k−1 (1) limit cycles in (R∗ )2 . Applying to Vk the transformation (x, y) → (1/x, 1/y) and k (2) multiplying the result by (xy)2 , one obtains the required vector field Vk with Newton triangle conv{(1 + 2k , 1 + 2k ), (1 + 2k , 1), (1, 1 + 2k )}. Proof of Theorem 2.1. Without loss of generality we assume that d ≥ 36. We construct a required vector field of degree d using the following subdivision of the triangle T = conv{(0, 0), (0, d + 1), (d + 1, 0)}. Put d = [d − 6 log2 d], and consider the binary decomposition d = 2k1 + 2k2 + · · · . This decomposition defines a natural tiling of the triangle conv{(0, 0), (d , 0), (0, d )} shown in Figure 1. Remove all the triangles of the tiling with the edges of length 2ki < d / log2 d. Then, enlarge by 3 the horizontal and vertical edges of each remaining triangle, and shift the resulting triangles so that their interiors become disjoint (as it is shown in Figure 2). Observe that the union U of the enlarged triangles is contained in the triangle bounded by the vertical axis, the horizontal line j = −3 log2 d, and the line i + j = d + 3 log2 d. Hence the figure U1 , obtained by shifting U up by [3 log2 d], is contained in the triangle T = conv{(0, 0), (d +1, 0), (d +1, 0)}. Denote by T1 , . . . , Ts the triangles of U1 . For any triangle conv (l, m), l + 3 + 2ki , m , l, m + 3 + 2ki or Tr = k k i i , conv (l, m), l − 3 − 2 , m , l, m − 3 − 2 we take the triangle

9


j 2k1 + 2k2 + 6

2k1 + 3

. ....

..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... .......... ..... .. ..... ..... ............ ..... ....... ..... ..... ............. ..... .. ..... ..... ............ ..... ....... ..... ..... ............. ..... .. ..... ..... ............ ..... ....... ..... ..... ............. ..... .. ..... ..... ............ ..... ....... ..... ..... ..... ..... ..... ..... .......... ..... ..... ............ ..... ..... .. ....... ..... ..... . . . . ..... . ............ ..... ..... ....... .. ..... ..... . . ..... ................ ..... ..... .. ....... ..... ..... ..... ................. ..... ..... ....... .. ..... ..... ..... .................. ..... ..... .. ....... ..... ..... ..... ................. ..... ..... ....... .. ..... ..... ..... .................. ..... ..... .. ....... ..... ..... ..... ................. ..... ..... ....... ..... ..... ..... ............. ..... ..... .. ..... ..... ..... . . ..... .......... ..... ..... ....... ..... ..... . ..... ............ ..... ..... .. ..... ..... ..... ..... ............ ..... ..... ....... ..... ..... ..... ............. ..... ..... .. ..... ..... ..... ..... ............ ..... ..... ....... ..... ..... ..... ............. ..... ..... .. ..... ..... ..... ..... ............ ..... ..... ....... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ... .

...... .

i

Figure 2

conv (l + 1, m + 1), l + 1 + 2ki , m + 1 , l + 1, m + 1 + 2ki , τr = conv (l − 1, m − 1), l − 1 − 2ki , m − 1 , l − 1, m − 1 − 2ki

resp.,

(shown by dashes in Figure 2). Then we triangulate the complement T \∪sr=1 τr in an appropriate way in order to obtain a convex triangulation of T . Define block vector fields as follows. For each triangle τr = conv{(l + 1, m + 1), (1) (l +1+2ki , m+1), (l +1, m+1+2ki )} take the vector field Vki from Proposition 2.2 with Newton triangle conv{(1, 1), (1 + 2ki , 1), (1, 1 + 2ki )}, multiplied by x l y m , and for each triangle τr = conv{(l −1, m−1), (l −1−2ki , m−1), (l −1, m−1−2ki )} take (2) the vector field Vki from Proposition 2.2 with Newton triangle conv{(1+2ki , 1+2ki ), k

k

(1 + 2ki , 1), (1, 1 + 2ki )}, multiplied by x l−2−2 i y m−2−2 i . For the remaining integral points in T , take arbitrary generic real numbers to be the coefficients of vector fields corresponding to the elements of the triangulation of T \ ∪sr=1 τr . By Theorem 1.3.1 we can glue all the described block vector fields into a vector field of degree d. To estimate the number of limit cycles of the latter vector field, we note that (1) by Proposition 2.2, the block vector field with Newton triangle τr has at least (ki − 3)22ki −1 simple limit cycles in (R∗ )2 , (2) by construction, all the numbers ki satisfy ki ≥ log2 d − log log2 d − 3, and the sum sr=1 22ki −1 , which is equal to the area of ∪sr=1 τr , satisfies s r=1

22ki −1 ≥

2 2d 1 2d 2 1 d − d− − 6 log2 d . ≥ 2 log2 d 2 log2 d

10


This gives the following lower bound for the total number of limit cycles:

2 1 2d 1 log2 d − log log2 d − 6 · d− − 6 log2 d ≥ d 2 log2 d − Cd 2 log2 log2 d, 2 log2 d 2

for some C > 0 which does not depend on d. 3. Singular points of planar vector fields: Formulation of results. The space of linearization matrices of real nondegenerate singular points has three connected components. They correspond to three topological types of the local phase portraits (see, e.g., [AIl] and [Ha]). Denote these types as follows: 41 attractors, 42 repellers, 43 saddles. A real nondegenerate singular point is called strongly nondegenerate if the eigenvalues λ1 and λ2 of its linearization matrix are distinct. The space of linearization matrices of strongly nondegenerate singular points has five connected components defining thin topological types: S1 sinks, λ1 < λ2 < 0; S2 sources, λ1 > λ2 > 0; S3 sink-foci, λ1 , λ2 ∈ R, Re λ1 = Re λ2 < 0; S4 source-foci, λ1 , λ2 ∈ R, Re λ1 = Re λ2 > 0; S5 saddle points, λ1 > 0 > λ2 . Clearly, 41 ⊃ S1 ∪ S3 , 42 ⊃ S2 ∪ S4 , 43 = S5 . For a nondegenerate vector field, denote by σi and si the numbers of singular points in (R∗ )2 of types 4i (1 ≤ i ≤ 3) and Si (1 ≤ i ≤ 5), respectively, and denote by 2s0 the number of imaginary singular points in (C∗ )2 . Then, clearly, 2s0 + σ1 + σ2 + σ3 = d 2

(3.1) and (see [Kh1]) (3.2)

|σ1 + σ2 − σ3 | ≤ d.

Theorem 3.3 (Topological classification). For any nonnegative integers d, s0 , σ1 , σ2 , and σ3 satisfying (3.1) and (3.2), there exists a vector field of degree d with 2s0 imaginary singular points and σi real singular points of type 4i , i = 1, 2, 3. This is the complete topological classification of collections of singular points of planar polynomial nondegenerate vector fields. Theorem 3.4 (Thin topological classification). For any nonnegative integers d, s0 , s1 , s2 , s3 , s4 , and s5 satisfying 2s0 + s1 + s2 + s3 + s4 + s5 = d 2 ,

|s1 + s2 + s3 + s4 − s5 | ≤ d,

11


and

(3.5)

  1,      1,

if d is odd, s1 + s2 + s3 + s4 − s5 ≥ 3, if d is even, (d − 1)(d − 2) < 2s0 < d 2 , s1 + s2 ≥ s1 + s2 + s3 + s4 − s5 ≥ 0,    2, if d is even, 2s0 ≤ (d − 1)(d − 2),     s1 + s2 + s3 + s4 − s5 ≥ 0,

there exists a vector field of degree d with 2s0 imaginary singular points and si real singular points of type Si , i = 1, 2, 3, 4, 5. This result is an asymptotically complete thin topological classification of collections of singular points of planar polynomial nondegenerate vector fields whose real singular points are strongly nondegenerate. We conjecture that the restriction (3.5) is not necessary. Proofs of Theorems 3.3 and 3.4 are presented in Sections 4 and 5–7, respectively. 4. Topological classification of collections of singular points 4.1. Another gluing theorem. Theorem 3.3 is, in fact, a corollary of Theorem 3.4. However, here we present an alternative proof based on a gluing theorem different from Theorem 1.4.1. This specific version of gluing is adapted for the topological classification. Suppose that we have the following objects: (1) a nondegenerate polygon , (2) a subdivision = 1 ∪ · · · ∪ N , where 1 , . . . , N are convex polygons, (3) a convex piecewise-linear function ν : → R, which is linear on each polygon 1 , . . . , N and nonlinear on each union k ∪ l , k = l, (4) a collection of real numbers Aij , Bij indexed by (i, j ) ∈ ∩ Z2 , such that Aij , Bij = 0 for all vertices (i, j ) of all the polygons 1 , . . . , N . Introduce the vector fields Vk : x˙ = Aij x i y j , y˙ = Bij x i y j , k = 1, . . . , N (i,j )∈k

and the family of vector fields Aij x i y j t ν(i,j ) , V (t) : x˙ = (i,j )∈

(i,j )∈k

y˙ =

Bij x i y j t ν(i,j ) ,

t > 0.

(i,j )∈

Unlike Section 1, here the components of the vector fields have the same Newton polygon. Theorem 4.1.1. Assume that each vector field Vk , k = 1, . . . , N , has in (C∗ )2 exactly 2 · vol(k ) nondegenerate singular points, assume the coefficient ξ22 of the

12


linearization matrix (ξrs )r,s=1,2 of Vk at any real nonsaddle singular point differs from zero, and assume the function ν(λ, µ) satisfies (4.1.2)

∂ν ∂ν > ∂λ ∂µ

at any point in the interior of any linearity domain. Then, for t > 0 small enough, there exists a one-to-one correspondence Ꮿ between the set of singular points of the vector field V (t) in (C∗ )2 and the disjoint union of the sets of singular points of the vector fields V1 , . . . , VN in (C∗ )2 having the following properties: (1) an imaginary singular point of Vk , 1 ≤ k ≤ N, corresponds to an imaginary nondegenerate singular point of V (t), (2) a saddle singular point of Vk , 1 ≤ k ≤ N, corresponds to a saddle singular point of V (t), (3) a real nonsaddle singular point of Vk , 1 ≤ k ≤ N, with the linearization matrix (ξrs )r,s=1,2 corresponds to a real nondegenerate singular point of V (t), which is a repeller if ξ22 > 0 and an attractor if ξ22 < 0. Proof. Let lk (i, j ) = αk i +βk j +γk be a linear function equal to ν(i, j ) on k . We apply the linear transformation Mk (x, y) = (xt αk , yt βk ) to the field V (t) and obtain the vector field y˙ = t βk +γk Vk,y + O(t) . Vk (t) : x˙ = t αk +γk Vk,x + O(t) , As in the proof of Theorem 1.3.1, for a sufficiently small t > 0, one gets an inclusion of the disjoint union of the sets of singular points of the vector fields Vk in (C∗ )2 into the set of singular points of V (t) in (C∗ )2 . This inclusion is, in fact, a bijection due to Bernstein’s theorem [Be]. Clearly, imaginary singular points correspond to imaginary ones. A real singular point (x0 , y0 ) of Vk with the linearization matrix ξ11 ξ12 Ꮽ= ξ21 ξ22 corresponds to a singular point (x0 + O(t), y0 + O(t)) of the field Vk (t) with the linearization matrix α α k ξ k ξ + O(t) t + O(t) t 11 12 Ꮽ = t γk (4.1.3) . t βk ξ21 + O(t) t βk ξ22 + O(t) Since det(Ꮽ) · det(Ꮽ ) > 0 for a positive sufficiently small t and since αk > βk by (4.1.2), one easily derives the relations given in the statement. 4.2. Proof of Theorem 3.3 in the case s0 = 0. Consider the triangle T = {(0, 0), (d, 0), (0, d)} in R2 divided into the following triangles δj k = conv (j, k), (j + 1, k), (j, k + 1) , γj k = conv (j + 1, k), (j, k + 1), (j + 1, k + 1) ,


13

where j , k are nonnegative integers. These triangles are the linearity domains of the convex piecewise-linear function ν = ν1 + ν2 + ν3 : T → R, where ν1 is a convex piecewise-linear function with the linearity domains {(λ, µ) ∈ T : n ≤ λ ≤ n + 1},

n ∈ Z,

ν2 is a convex piecewise-linear function with the linearity domains {(λ, µ) ∈ T : n ≤ µ ≤ n + 1},

n ∈ Z,

and ν3 is a convex piecewise-linear function with the linearity domains {(λ, µ) ∈ T : n ≤ λ + µ ≤ n + 1},

n ∈ Z.

Adding a linear function Kλ with a large K > 0 to ν(λ, µ), we get the property (4.1.2). Given two collections of nonzero real numbers aj k and bj k indexed by the integral points of T , we associate with each triangle Ti = δj k or γj k in the triangulation of T a vector field Vi = (Pi (x, y), Qi (x, y)) such that the triangle Ti is the Newton polygon of Pi and Qi and such that the coefficients of Pi (resp., Qi ) coincide with the numbers aj k (resp., bj k ) at the vertices of Ti . Such a vector field Vi has one singular point in (R∗ )2 (see Subsection 1.4). Now let us glue the fields Vi into the field aj k x j y k t ν(j,k) , y˙ = bj k x j y k t ν(j,k) , t > 0. V (t) : x˙ = j +k≤d

j +k≤d

Let Vδ and Vγ be the vector fields x˙ = aj k x j y k + aj +1,k x j +1 y k + aj,k+1 x j y k+1 , Vδ : y˙ = bj k x j y k + bj +1,k x j +1 y k + bj,k+1 x j y k+1 , Vγ :

x˙ = aj +1,k+1 x j +1 y k+1 + aj +1,k x j +1 y k + aj,k+1 x j y k+1 , y˙ = bj +1,k+1 x j +1 y k+1 + bj +1,k x j +1 y k + bj,k+1 x j y k+1 .

(The Newton triangles δj k and γj k of the components of Vδ and Vγ , resp., are shown in Figure 3.) Proposition 4.2.1. In the above notation, for t > 0 small enough, exactly one of the singular points of the field V (t), assigned by the correspondence Ꮿ (see Theorem 4.1.1) to the singular points of the vector fields Vδ and Vγ , is a saddle. Proof. The linearization matrix at the singular point (x0 , y0 ) of Vδ is a aj,k+1 j Ꮽ(Vδ ) = x0 y0k j +1,k (4.2.2) , bj +1,k bj,k+1

14

ITENBERG AND SHUSTIN .....

k +1

k

..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... .

γj k

δj k

j

j +1

...... .

Figure 3

and the linearization matrix at the singular point (x1 , y1 ) of Vγ is 2 2 j −1 k−1 aj,k+1 y1 aj +1,k x1 . Ꮽ(Vγ ) = −x1 y1 bj,k+1 y12 bj +1,k x12 Hence 2 2j 2j det Ꮽ(Vδ ) · det Ꮽ(Vγ ) = −x0 y02k x1 y12k aj +1,k bj,k+1 − aj,k+1 bj +1,k < 0, which completes the proof. The following statement shows how to control the topological types of singular points of the field V (t) by means of a suitable choice of numbers aj k , bj k . Proposition 4.2.3. In the above notation, given nonnegative integers j, k, nonzero real numbers aj k , aj +1,k , bj k , bj +1,k , and m = 1, 2, 3, there exist nonzero numbers aj,k+1 , bj,k+1 such that, for t > 0 small enough, the singular point of the vector field V (t), assigned by the correspondence Ꮿ to the singular point of the vector field Vδ , belongs to the class 4m . j

Proof. By (4.2.2) one has det(Ꮽ(Vδ )) = (x0 y0k )2 (aj +1,k bj,k+1 − aj,k+1 bj +1,k ). So, by Theorem 4.1.1, to have a saddle at the corresponding singular point of the field V (t), one should choose aj,k+1 , bj,k+1 so that aj +1,k bj,k+1 − aj,k+1 bj +1,k < 0. To have a repeller at the corresponding singular point of V (t), by Theorem 4.1.1 and (4.2.2), one has to satisfy the conditions (in the notation of Theorem 4.1.1) 2j j j det Ꮽ(Vδ ) = x0 y02k aj +1,k bj,k+1 − aj,k+1 bj +1,k > 0, ξ22 = x0 y0 bj,k+1 > 0.


15

Since x0 =

aj,k+1 bj k − aj k bj,k+1 , aj +1,k bj,k+1 − aj,k+1 bj +1,k

y0 =

aj k bj +1,k − aj +1,k bj k , aj +1,k bj,k+1 − aj,k+1 bj +1,k

the previous inequalities are equivalent to the system aj +1,k bj,k+1 − aj,k+1 bj +1,k > 0, j k aj,k+1 bj k − aj k bj,k+1 aj k bj +1,k − aj +1,k bj k bj,k+1 > 0, which always has a solution aj,k+1 = 0, bj,k+1 = 0. Similarly, one proves the statement in the case of an attractor. Now, we note that any nonnegative integers σ1 , σ2 , and σ3 satisfying σ 1 + σ2 + σ 3 = d 2 ,

−d ≤ σ1 + σ2 − σ3 ≤ d,

can be represented as follows: σm = σm + σm , σ1 + σ2 = σ3 =

σm , σm ≥ 0,

d(d − 1) , 2

m = 1, 2, 3,

σ1 + σ2 + σ3 = d.

Consider again the triangle T = conv((0, 0), (d, 0), (0, d)) and its triangulation introduced above. Introduce the following order at the integer points of T : (4.2.4)

(j1 , k1 ) ≺ (j2 , k2 ) ⇐⇒ k1 < k2

or

k1 = k 2 ,

j1 > j 2 .

Associate with any integer point (j, k) of T a pair of numbers aj k , bj k , considering the integer points successively in the order (4.2.4). According to Propositions 4.2.1 and 4.2.3, the collections {aj k } and {bj k } can be chosen in such a way that (1) d vector fields corresponding to the triangles δj k , where j + k = d − 1, have altogether σm points of type 4m , m = 1, 2, 3, (2) d(d −1)/2 vector fields corresponding to the triangles δj k , where j +k < d −1, have altogether σm points of type 4m , m = 1, 2, (3) d(d −1)/2 vector fields corresponding to the triangles γj k , where j +k < d −1, (which are, actually, determined by the previous vector fields according to Proposition 4.2.1) have altogether σ3 = d(d − 1)/2 saddles. Finally, using Theorem 4.1.1, we obtain the desired vector field. 4.3. Proof of Theorem 3.3 in the case 0 < s0 ≤ d(d −1)/2. We define the following subdivision of the triangle T with the vertices (0, 0), (d, 0), (0, d). The triangle T contains d(d − 1)/2 squares sqj k = conv (j, k), (j + 1, k), (j, k + 1), (j + 1, k + 1) .

16


Ordering these squares by (4.2.4), we choose the first s0 squares from this set and then cover the rest of T by the triangles δj k , γj k , introduced in Subsection 4.2. Clearly, there exists a convex piecewise-linear function ν : T → R, whose linearity domains are these s0 squares and triangles. Proposition 4.3.1. For any generic nonzero real numbers aj k , aj +1,k , aj +1,k+1 , bj k , bj +1,k , bj +1,k+1 , there exist nonzero real numbers aj,k+1 , bj,k+1 such that the vector field

x˙ = P (x, y) = x j y k aj k + aj +1,k x + aj,k+1 y + aj +1,k+1 xy , y˙ = Q(x, y) = x j y k bj k + bj +1,k x + bj,k+1 y + bj +1,k+1 xy

has two imaginary nondegenerate singular points in (C∗ )2 . The term generic nonzero real numbers means that the six numbers mentioned in the assertion are chosen in a Zariski-open subset of R6 . Proof. The system P (x, y) = Q(x, y) = 0 in (C∗ )2 is reduced to the equation aj +1,k bj +1,k+1 − aj +1,k+1 bj +1,k x 2 + aj +1,k bj,k+1 + aj k bj +1,k+1 − aj,k+1 bj +1,k − aj +1,k+1 bj k x + aj k bj,k+1 − aj,k+1 bj k = 0, with discriminant 2 D = aj +1,k bj,k+1 + aj k bj +1,k+1 − aj,k+1 bj +1,k − aj +1,k+1 bj k − 4 aj +1,k bj +1,k+1 − aj +1,k+1 bj +1,k aj k bj,k+1 − aj,k+1 bj k . Given generic aj k , aj +1,k , aj +1,k+1 , bj k , bj +1,k , bj +1,k+1 , the equation D = 0 defines a parabola in the plane aj,k+1 , bj,k+1 . Hence, there exist nonzero aj,k+1 , bj,k+1 such that D < 0, which completes the proof. Now, we use Propositions 4.2.1 and 4.2.3 to choose pairs of numbers aj k , bj k at the integer points (j, k), j + k ≤ d of T (considering the points successively in the order (4.2.4)), in such a way that the vector field corresponding to each of s0 chosen squares sqj k has two imaginary nondegenerate points, and the vector fields corresponding to the triangles δj k , γj k in the given subdivision of T provide the prescribed quantities of saddles, repellers, and attractors of the field V (t) : x˙ =

aj k x j y k t ν(j,k) ,

j +k≤d

as was done in the previous subsection.

y˙ =

j +k≤d

bj k x j y k t ν(j,k) ,

17


4.4. Proof of Theorem 3.3 in the case s0 > d(d −1)/2. In this case σ1 +σ2 +σ3 = d 2 − 2s0 = d − 2u, 0 < u ≤ d/2. We introduce the following new element in our construction. Proposition 4.4.1. Let u ≤ d/2 be a positive integer. For any fixed generic polynomials f (x), g(x) of degree d − 2u, there exists a vector field   x˙ = P (x, y) = y 2u f (x) + aj k x j y k ,    k 0, tr(V ) > 0, discr(V ) > 0}, (3) S3 = {det(V ) > 0, tr(V ) < 0, discr(V ) < 0}, (4) S4 = {det(V ) > 0, tr(V ) > 0, discr(V ) < 0}, (5) S5 = {det(V ) < 0}. Straightforward calculations give a description of singular points of NDE vector fields. The exact statements are presented in Propositions 5.5, 5.7, 5.8, 5.12, and 5.13. In these propositions, we suppose that the vertices of Newton triangles are numbered counterclockwise. Proposition 5.5. Let (5.2) be a complete NDE vector field V with the Newton triangle δ = conv((i1 , j1 ), (i2 , j2 ), (i3 , j3 )). Then sign det(V ) = sign (A3 − A1 )(A2 − A3 )(A1 − A2 ) , (5.6) sign discr(V ) = sign @(A1 , A2 , A3 )2 − 4(A3 − A1 )(A1 − A2 )(A2 − A3 ) , where @(A1 , A2 , A3 ) = (i2 −i1 )A2 +j2 −j1 (A3 −A1 )+ (i3 −i1 )A3 +j3 −j1 (A1 −A2 ). To describe the situation with sign(tr(V )), we introduce two types of triangles of area 1/2: a triangle is of type 1 if it has a vertex with both the coordinates odd (clearly, such a vertex is unique), and it is of type 2 otherwise. Proposition 5.7. In the notation of Proposition 5.5, if δ is of type 1 and i1 , j1 are odd, then @(A1 , A2 , A3 ) ; sign tr(V ) = sign(ε1 ) · sign A2 − A 3

20


if δ is of type 2, then sign tr(V ) = sign(ε1 ε2 ε3 ) · sign A(A1 , A2 , A3 ) , where A is a rational function. Proposition 5.8. Let V be a boundary NDE vector field with the Newton triangle δ = conv((0, j1 ), (i2 , j2 ), (i3 , j3 )), i2 i3 j1 j2 j3 = 0, given by x˙ = A1 ε1 y j1 −1 + A2 ε2 x i2 y j2 −1 + A3 ε3 x i3 y j3 −1 , y˙ = ε2 x i2 −1 y j2 + ε3 x i3 −1 y j3 , A1 > 0, A2 A3 = 0, ε1 , ε2 , ε3 = ±1. Then (5.9)

sign det(V ) = sign(A3 − A2 ), sign discr(V ) = sign (i2 A2 − i3 A3 − j3 + j2 )2 − 4(A3 − A2 ) .

If δ is of type 1, then (5.10)

sign ε2 (i2 A2 − i3 A3 − j3 + j2 ) , sign tr(V ) = sign ε3 (i3 A3 − i2 A2 − j2 + j3 ) ,

if i2 , j2 are odd, if i3 , j3 are odd.

If δ is of type 2, then (5.11) sign tr(V ) = sign(ε1 ε2 ε3 ) · sign A1 (A2 − A3 )(i2 A2 − i3 A3 − j3 + j2 ) . Similar formulae hold when a vertex of the Newton triangle lies on the other axis. Proposition 5.12. Let V be a boundary NDE vector field with the Newton triangle δ = conv((i1 , 0), (0, j2 ), (i3 , j3 )), i1 i3 j2 j3 = 0, given by x˙ = A2 ε2 y j2 −1 + A3 ε3 x i3 y j3 −1 ,

y˙ = ε1 x i1 −1 + ε3 x i3 −1 y j3 ,

A2 > 0, A3 = 0, ε1 , ε2 , ε3 = ±1. Then sign det(V ) = sign(A3 ) · sign(i1 j3 + i3 j2 − i1 j2 ), discr(V ) > 0, sign(ε3 ) · sign(i3 A3 + j3 ), if i3 , j3 are odd, sign tr(V ) = sign(ε1 ε2 ε3 ) · sign A2 A3 (i3 A3 + j3 ) , otherwise. Proposition 5.13. Any elementary vector field V = (P , Q) can be turned into a vector field with coefficients 0, ±1 in the y-component ˙ by means of a suitable linear transformation (x, y) → (xα, yβ), α, β > 0, and multiplication of the vector field by


21

a positive number. The parameters A1 , A2 , A3 , ε1 , ε2 , ε3 , defined in Propositions 5.1, 5.5, 5.8, and 5.12 for the corresponding elementary field, are determined uniquely. Moreover, ε1 , ε2 , ε3 are the signs of the coefficients of Q(x, y), and A1 , A2 , A3 are homogeneous functions of order 1 in the coefficients of P (x, y) and of order −1 in the coefficients of Q(x, y). 6. Thin topological classification: Basic construction. Now we can start the proof of Theorem 3.4. The cases d = 0, 1 are obvious. In the sequel, we suppose that d ≥ 2. In this section, we consider the special case s0 = 0,

s1 + s2 + s3 + s4 − s5 = −d.

The construction described here is basically the same for all other cases. In the next section we show what kind of modifications should be done to complete the proof of Theorem 3.4. A vector field of degree d has in general the Newton triangle T = conv (0, 0), (0, d + 1), (d + 1, 0) . We describe a triangulation of T and block vector fields corresponding to this triangulation. Throughout Subsections 6.1–6.5 we assume that d is even. 6.1. Triangulation. We define a piecewise-linear convex function ν1 + ν2 + ν3 : T → R, where ν1 is a convex piecewise-linear function with the linearity domains (λ, µ) ∈ R2 : 2i ≤ µ ≤ 2i + 2 , i ∈ Z, ν2 is a convex piecewise-linear function with the linearity domains (λ, µ) ∈ R2 : 2i ≤ λ ≤ 2i + 2 , i ∈ Z, and ν3 is a convex piecewise-linear function with the linearity domains (λ, µ) ∈ R2 : 2i − 1 ≤ λ + µ ≤ 2i + 1 , i ∈ Z. The intersections of these domains define a subdivision of T into the hexagons Hij , i, j ≥ 0, i + j ≤ (d − 2)/2, with vertices v1 = (2i, 2j + 1), v4 = (2i + 2, 2j + 1),

v2 = (2i + 1, 2j ),

v3 = (2i + 2, 2j ),

v5 = (2i + 1, 2j + 2),

v6 = (2i, 2j + 2),

and triangles covering T \ ∪i,j Hij (see Figure 4). A triangle τ of area 1/2 is called odd if conv((0, 0), τ ) is a nondegenerate quadrangle. It is called even otherwise.

22


d +1

..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ... .. . .

v6 v5 v1 v4 v2 v3

H00

d +1

. ......

Figure 4

For any hexagon Hij , we take a triangulation into six triangles of area 1/2: three odd triangles, one even triangle of type 1, and two even triangles of type 2. Namely, let c = (2i + 1, 2j + 1) be the center of Hij . If j ≥ i, we triangulate Hij by the segments [c, v1 ], [c, v3 ], [c, v5 ], [v1 , v5 ], [v1 , v3 ], [v3 , v5 ]. If i > j , we triangulate Hij by the segments [c, v2 ], [c, v4 ], [c, v6 ], [v2 , v6 ], [v2 , v4 ], [v4 , v6 ] (see Figure 5). Denote by D the described triangulation of T . 6.2. Block vector fields. Suppose that we have certain numbers εkl = ±1, Akl = 0, where (k, l) ∈ T ∩ Z2 . Consider elementary vector fields Vδ with Newton triangles δ ∈ D: x˙ = Akl εkl x k y l−1 , y˙ = εkl x k−1 y l . (k,l)∈δ, l≥1

(k,l)∈δ, k≥1

Proposition 6.2.1. Let Vδ , δ ∈ D, be a complete or boundary elementary vector field close to a Hamiltonian one, that is, any Akl with (k, l) ∈ δ, k > 0 is close to −l/k. Then Vδ has a saddle (resp., a focus) in (R∗ )2 if δ is an odd (resp., even) triangle. Proof. If Vδ is close to a Hamiltonian vector field, the type of the singular point of Vδ in (R∗ )2 is determined by the type of the critical point of the corresponding trinomial with Newton triangle δ. It follows from [S2, Propositions 2.2.1, 2.2.4, and 2.2.5]


23

.....

d +1

..... ..... ..... ..... ..... ..... ..... ..... ..... ..... .... ..... ......... ..... ............. . . ... ... ...... ..... . ... ...... . . . . ... ...... ..... . . ... ..... . . ... ..... ... ..... ..... ... ............. ..... ..... .. ............. ..... ..... ..... ..... ........ ..... ..... ..... ..... ........ ..... ... ..... ..... ........ . ....... ........ ..... ..... ..... ....... ..... .. ..... ..... ....... .... .. ..... ..... ....... ........ ..... ..... ............. ..... ..... ....... .......... .... . . . . . .. ..... ........ ..... ............. ..... ............. . . . . . ... .... ... ..... . .... ..... ..... ... ...... ... ..... . . . . . . . . . ... ...... ... ...... .. ..... . ..... . . ... ...... . . . . . . . . . . ... ..... . .. ... ... ..... ........ ... ..... ... ......... ........ ... ..... ............. ..... ..... . ............ ... ..... ............. . ..... ..... ..... ............. ..... ..... ........ ..... ... ..... .. ..... ........ ..... ..... ........ ..... ... ..... ... ..... ........ ..... ..... ........ . ..... ... ....... ..... ....... ........ ..... ..... ..... ....... ..... ....... ..... .. ..... ..... ....... ......... ..... ..... ....... .... .. ..... . ..... . . ....... ........ ..... ....... ........ ..... ..... ............. ..... ............. ..... ..... . . . .......... ......... ..... ..... .............. ..... .............. ............ . . ..... . . . ..... .................. .................. ..... ..... ............ . . . . . . . . . . . . . . . . . . . ..... . . . . . . . . ..... ... ..... ... ..... ........ . ... ...... ......... ... ..... ....... ..... ..... . ... ...... . . . . . . . . . . . . . . . ....... . ... ..... ............. ......... ... ...... ... ....... ....... ......... ....... ..... ..... ..... . . . . . . . . . . . . . . . . . . . ....... .... ... .... ....... ..... . ... ...... ... ...... ........... ..... ........... ..... ... ..... ..... . . . . . ........... . . . . . . . ......... ..... ... ... ..... .... ... ..... ............. ..... ...... ..... ... ... . . . . . ............. ..... . ..... ..... . . . ..... ..... ........ ..... .... ..... ......... ..... ......... .... ..... ..... . . . ..... ........ ..... ... . ..... ... ..... . ..... ..... ... ..... ........ . ..... ... ..... ..... ..... ..... .. . . . . . . ....... ........ ..... ..... . . ..... .. ..... ..... ... ....... ..... .. ..... ..... ..... ..... ... ..... ..... .. . . . . ....... .... ... . . . . ..... ........ ..... ..... ........ ...... ....... ....... ..... ....... ..... ..... ........ ..... ............. ..... .......... ..... ........ ....... .

. ......

d +1 Figure 5

that if δ is an odd triangle, the trinomial has a saddle (thus, Vδ has a saddle); if δ is even, the trinomial has an extremum (thus, Vδ has a focus). Remark. The statement of Proposition 6.2.1 is true for any triangle δ of area 1/2 such that no straight line through the origin contains an edge of δ. Suppose that each number Akl with k > 0 is sufficiently close to −l/k. Then the vector fields Vδ , δ ∈ D, have a total of d(d + 1)/2 saddles and d(d − 1)/2 foci. 6.3. Refinement procedure. In this subsection we describe a procedure (we call it the refinement procedure) that allows us to obtain a vector field with si singular points of type Si (i = 1, . . . , 5) starting from the block vector fields described in the previous subsection. The procedure is quite general. It can be applied (as it is done in Section 7) to different collections of block vector fields. Step 1. Sources and sinks. Consider the families Vδ (α), δ ∈ D, 1 ≤ α < ∞, of vector fields x˙ = α ·

(k, l)∈δ, l≥1

Akl εkl x k y l−1 ,

y˙ =

εkl x k−1 y l .

(k, l)∈δ, k≥1

Clearly, the sign of the determinant of a singular point does not change in such a family; hence saddles remain saddles, and foci turn into sinks or sources and back. We say that an elementary vector field δ is absolutely generic if one of the following conditions holds:

24


(1) Vδ is a complete NDE vector field, and the sign of the discriminant of Vδ (α) coincides with the sign of a polynomial of degree 4 in α (compare with formula (5.6)); (2) Vδ is a boundary NDE vector field with exactly one vertex on the coordinate axes, and the sign of the discriminant of Vδ (α) coincides with the sign of a quadratic polynomial in α (compare with formula (5.9)); (3) Vδ is an NDE vector field with at least two vertices on the coordinate axes. Proposition 6.3.1. By a small variation of the numbers Akl , one can get the following: (i) all the vector fields Vδ are absolutely generic; (ii) for any α ≥ 1, at most one of the vector fields Vδ (α), δ ∈ D, has a positive determinant and the zero discriminant. Proof. The fact that all the vector fields Vδ can be made absolutely generic by a small variation of the numbers Akl is straightforward: If Vδ is a complete NDE vector field, the coefficient at α 4 in @(A1 , A2 , A3 )2 − 4(A3 − A1 )(A1 − A2 )(A2 − A3 ) (see formula (5.6)) is equal to (i2 − i1 )A2 (A3 − A1 ) + (i3 − i1 )A3 (A1 − A2 ))2 ; if Vδ is a boundary NDE vector field with exactly one vertex on the coordinate y-axis, the coefficient at α 2 in (i2 A2 −i3 A3 −j3 +j2 )2 −4(A3 −A2 ) (see formula (5.9)) is equal to (i2 A2 − i3 A3 )2 . The set of values of α such that at least one of the vector fields Vδ (α) has discriminant zero is finite. Let δ1 , . . . , δr be the triangles in D such that the corresponding vector fields Vδ1 (α0 ), . . . , Vδr (α0 ) have positive determinants and the zero discriminant. We call a vertex of δi interior if it does not belong to a coordinate axis. Consider two triangles δ1 and δ2 . Note that either δ2 has an interior vertex that does not belong to δ1 , or δ1 has an interior vertex that does not belong to δ2 . Varying the coefficient that corresponds to this interior vertex of δ1 or δ2 , we can decrease by 1 the number of vector fields (among Vδ (α0 ), δ ∈ D) having positive determinants and the zero discriminant. Proceeding in this way, we get a collection of absolutely generic vector fields Vδ , δ ∈ D, such that, for any α ≥ 1, there exists at most one vector field Vδ (α) with positive determinant and discriminant zero. Using Proposition 6.3.1, we suppose that all the vector fields Vδ are absolutely generic and that, for any α ≥ 1, at most one of the vector fields Vδ (α), δ ∈ D, has a positive determinant and the discriminant equal to zero. Consider the set of values of α such that one of the vector fields Vδ (α) with positive determinant has discriminant zero. Note that this set does not depend on signs εkl . Moreover, if α0 belongs to this set and the discriminant of Vδ (α0 ) is equal to zero, the type of modification of a singular point of Vδ (α) at α0 is also independent of the choice of the signs εkl . Formulae (5.6) and (5.9) show that, for any δ ∈ D, the discriminant of Vδ (α) is positive if α is sufficiently large. Hence, a focus of Vδ (α) necessarily turns into a source or a sink when α → ∞.


25

.....

v

. .. ... ... .... ... ... ... .... ... ..................... . ... ...... .. .. ... ....... ..... ... ... ..... ... ... ..... .. ... ..... . . ..... ... ..... .... ... ..... .. ... ....... . ... ............ . ... . . ... ... .......... ... ..... .. ..... ... ... ..... ... ... ..... ... ..... .. . ..... .... . ... . . ........ . ... . . ... .... ....... .......... . ... .. . ...... . . . . . ... ... . . . ............. . . . . . . . ... ... ......... ......... ............................... .............................. . ....... ....... ....... . . . . ..

v

δ(v)

l

v

...... ...... .......

....... ....... ....... . . . ...

...... ......

k

...... .

Figure 6

Now we choose α0 ≥ 1 such that, for exactly s1 + s2 vector fields Vδ with even δ, the corresponding vector fields Vδ (α0 ) have sources or sinks and, for the remaining s3 + s4 vector fields Vδ with even δ, the vector fields Vδ (α0 ) have foci. Step 2. Sources or sinks. To distinguish between sources and sinks, foci-sources and foci-sinks, we choose suitable numbers εkl , k + l ≤ d + 1, to prescribe the signs of the traces of the vector fields Vδ (α0 ) with even δ. For any point (2i + 1, 2j + 1) ∈ T , there exists only one even triangle of type 1 with a vertex at this point. Hence, by Proposition 5.7, one can prescribe the sign of the trace of the singular point of the corresponding elementary vector field choosing ε2i+1, 2j +1 independently of all other numbers εkl . To establish the other numbers εkl , we apply the inductive procedure from [S2, Lemma 5.3]. Consider even triangles of type 2. Introduce the set K of all the vertices of these triangles. For any even triangle δ of type 2, we denote by v(δ) ∈ K its vertex contained in the interior of the convex hull of δ and the origin. Clearly v(δ ) = v(δ ) if δ = δ , and we write δ = δ(v) for v = v(δ). In any even triangle δ of type 2, let us mark each edge [v , v ] such that v = v(δ), and let us orient this edge from v to v . Consider the oriented graph E, whose vertex set is K and whose arcs are all the marked edges. Let us show that E does not have oriented cycles. Indeed, take in E the shortest cycle oriented clockwise, and let v = (k, l) be a vertex of this cycle having the minimal slope l/k. Then v belongs to two arcs [v, v ], [v , v], and we see that [v , v] must intersect the triangle δ(v) and that [v, v ] ⊂ δ(v) (see Figure 6). Similarly, we prohibit cycles oriented counterclockwise. Thus, the orientation of E defines a partial order in K. Define the order N(v) of v ∈ K to be the maximal path

26


length from one of the minimal points to v, and put Kr = {v ∈ K | N(v) = r}. Now, we establish the sign distribution by the decreasing induction on the point order. Let us fix the signs of all integral points of T which do not belong to K. Assume that the signs of points of K of order greater than or equal to r are already fixed. According to Proposition 5.7, we provide the prescribed signs of traces of singular points for elementary vector fields with Newton triangles δ(v), v ∈ Kr−1 , by setting suitable εkl at v ∈ Kr−1 . Finally, note that since no v ∈ Kr−1 belongs to triangles δ(w), N (w) ≥ r, the previous operation has no influence on these triangles. Step 3. Gluing. The final stage is the gluing of the elementary vector fields obtained into a vector field of degree d. The only thing we need to verify is that there exists a piecewise-linear convex function ν : T → R whose domains of linearity coincide with the triangles of the triangulation D. This is proved in the following subsection. As a result of the gluing, we obtain a vector field of degree d with si singular points of type Si , i = 1, . . . , 5. 6.4. Convexity of the triangulation D. A triangulation of T is convex if there exists a piecewise-linear convex function ν : T → R whose domains of linearity coincide with the triangles of the triangulation. Proposition 6.4.1. The triangulation D of T is convex. Proof. We define the required function ν inductively. Lemma 6.4.2. Denote by U the square conv((0, 0), (2, 0), (2, 2), (0, 2)). Let a00 , a10 , a20 , a21 , and a22 be real numbers such that a00 + a20 > 2a10 ,

a20 + a22 > 2a21 .

Then, (a) there exists a piecewise-linear convex function ν : U → R such that ν ((0, 0)) = a00 ,

ν ((1, 0)) = a10 ,

ν ((2, 1)) = a21 ,

ν ((2, 0)) = a20 ,

ν ((2, 2)) = a22 ,

whose domains of linearity are the triangles as shown in Figure 7(a); (b) there exists a piecewise-linear convex function ν : U → R such that ν ((0, 0)) = a00 ,

ν ((1, 0)) = a10 ,

ν ((2, 1)) = a21 ,

ν ((2, 0)) = a20 ,

ν ((2, 2)) = a22 ,

whose domains of linearity are the triangles as shown in Figure 7(b).

SINGULAR POINTS AND LIMIT CYCLES OF VECTOR FIELDS .....

.....

.......... ..... ..... ................ ..... ......... ......... ... ... ..... ......... ... ...... ......... ......... ....... ..... ... ...... ....... .... ... ..... ....... .... ... ...... ............ ..... ........... ... .. ..... ... .... ..... ..... ..... ..... ..... ..... ... ..... . ..... .. . . ..... ... ... . . . . ..... ... . ..... .. ..... ........ ........ .......... .

... ............ ..... ............. ..... ... ...... .... . ... ..... . . . .. ... ...... ..... ... ...... .... . . .. . ... . .... ... ......... . . . . . .......... ..... ... ..... ............. ..... ..... ..... ........ ..... ... ..... ........ . ..... ........ ....... ......... ..... ..... . ....... ..... .. ..... ....... .... ... ..... ....... ....... ..... ............ ..... ..... .

. ......

7(a)

27

. ......

7(b)

.....

.....

........ ... ........ ... ......... ....... ... ....... ... ....... ... ....... ....... ... ....... ... ....... ... ....... ... .. ... .... ... ..... ..... ... . . . . ... ..... . . ... . ... ..... ... ......... ........ .

.. ............. ..... ............ ... ..... ..... ..... ... ...... . . . .. ... ..... ..... ... ...... ..... . ... ...... . . ... ..... . ... . . . .. ... ........... ............. ... ..... ........ ... ..... ........ ... ..... ........ ... ....... ..... . ....... ..... ....... ..... ..... ....... .. ..... ....... .. ..... ....... .

...... .

7(c)

...... .

7(d) Figure 7

Proof. (a) Let us choose a value ν (0, 2) such that the point (0, 2, ν ((0, 2))) in is close from above to the plane containing the points (1, 0, ν ((1, 0))), (2, 0, ν ((2, 0))), and (2, 1, ν ((2, 1))). More explicitly, we take R3

a02 = ν ((0, 2)) = 2a10 + 2a21 − 3a20 + F02 , where F02 is a positive sufficiently small number. The projection of the lower part of the polyhedron conv (0, 0, a00 ), (1, 0, a10 ), (2, 0, a20 ), (2, 1, a21 ), (2, 2, a22 ), (0, 2, a02 ) to the xy-coordinate plane gives the triangulation of U as shown in Figure 7(c). We should now subdivide the triangles of area greater than 1/2. To subdivide the triangle of area 3/2, we choose ν ((1, 1)) in such a way that the point (1, 1, ν ((1, 1))) is very close from below to the plane containing (1, 0, ν ((1, 0))), (2, 1, ν ((2, 1))), and (0, 2, ν ((0, 2))). Namely, we take a11 = ν ((1, 1)) = a10 /3 + a21 /3 + a02 /3 − F11 , where F11 ! F02 is a positive sufficiently small number. Now, to subdivide the triangles of area 1, we can take a01 = ν ((0, 1)) = a00 /2 + a02 /2 − F01 ,

28


where F01 is a positive sufficiently small number, and a12 = ν ((1, 2)) = a22 /2 + a02 /2 − F12 , where F12 is a positive sufficiently small number. It remains to extend ν to be linear on each triangle of the triangulation of U as shown in Figure 7(a). (b) We can take a01 = ν ((0, 1)) = 2a10 + a21 − 2a20 − F01 , where F01 is a positive sufficiently small number, a12 = ν ((1, 2)) = a10 + 2a21 − 2a20 − F12 , where F12 = F01 , and a sufficiently large a02 = ν ((0, 2)). The projection of the lower part of the polyhedron conv (0, 0, a00 ), (1, 0, a10 ), (2, 0, a20 ), (2, 1, a21 ), (2, 2, a22 ), (1, 2, a12 ), (0, 2, a02 ), (0, 1, a01 ) , to the xy-coordinate plane, gives the triangulation of U as shown in Figure 7(d). We can now subdivide the triangle of area 3/2 taking a11 = ν ((1, 1)) = a20 /3 + a01 /3 + a12 /3 − F11 , where F11 ! F01 is a positive sufficiently small number. Then, it remains to extend ν to be linear on each triangle of the triangulation of U as shown in Figure 7(b). Now, we can define a function ν˜ : T → R with the following properties: (1) piecewise-linear; (2) convex on each square Uij = conv (2i, 2j ), (2i + 2, 2j ), (2i + 2, 2j + 2), (2i, 2j + 2) (where i, j ≥ 0, i + j ≤ d/2 − 2) and on each quadrangle conv (2i, d − 2i − 2), (2i + 2, d − 2i − 2), (2i + 2, d − 2i − 1), (2i, d − 2i + 1) (where 0 ≤ i ≤ d/2 − 1); (3) the domains of linearity of ν˜ coincide with the triangles of the triangulation D. The function ν˜ can be defined using Lemma 6.4.2 for the squares Uij = conv (2i, 2j ), (2i + 2, 2j ), (2i + 2, 2j + 2), (2i, 2j + 2) , considering them in the following order: Uij ≺ Ui j , if i > i or if i = i and j < j . The precise procedure we use is as follows. First, we define ν˜ in the triangle conv (d + 1, 0), (d, 1), (d, 0)


29

as an arbitrary linear function, and we extend the restriction of this linear function on the interval [(d, 0), (d + 1, 0)] to a convex piecewise-linear function on [(0, 0), (d + 1, 0)] with the intervals of linearity coinciding with the intervals [(i, 0), (i + 1, 0)] (i = 0, . . . , d). Then, we define ν˜ step-by-step in the strips conv (2i, 0), (2i + 2, 0), (2i + 2, d − 2i − 1), (2i, d − 2i + 1) , i = d/2 − 1, . . . , 0. At the step, where ν˜ is already defined in conv (2i + 2, 0), (d + 1, 0), (2i + 2, d − 2i − 1) , we use Lemma 6.4.2 to define the function ν˜ in the squares Uij (where j = 0, . . . , d/2− i − 2) and, then, in the pentagon conv (2i, d − 2i − 2), (2i + 2, d − 2i − 2), (2i + 2, d − 2i − 1), (2i + 1, d − 2i), (2i, d − 2i) . To define the function ν˜ in the pentagons, we use a version of Lemma 6.4.2 in which we ignore the triangle conv((1, 2), (2, 1), (2, 2)). After fixing the values of ν˜ in the pentagon conv (2i, d − 2i − 2), (2i + 2, d − 2i − 2), (2i + 2, d − 2i − 1), (2i + 1, d − 2i), (2i, d − 2i) , we take ν˜ ((2i, d − 2i + 1)) equal to a sufficiently large number and extend ν˜ linearly to the triangle conv (2i, d − 2i), (2i + 1, d − 2i), (2i, d − 2i + 1) . Now we take ν = ν1 + ν2 + F ν˜ , where ν1 and ν2 are defined in Subsection 6.1 and where F is a positive sufficiently small number. 6.5. Gluing. Let us sum up. We glue the compatible vector fields constructed in Subsection 6.3 (using the refinement procedure applied to the block vector fields as described in Subsection 6.2) into the vector field of degree d: x˙ = α0 Akl εkl x k y l−1 t ν(k, l) , k+l≤d+1, l≥1

y˙ =

εkl x k−1 y l t ν(k, l) ,

k+l≤d+1, k≥1

where ν is the function constructed in Subsection 6.4. By Corollary 1.4.2, for a sufficiently small t > 0, the latter vector field has si singular points of type Si , i = 1, . . . , 5. This completes the proof of Theorem 3.4 in the case s0 = 0, with even d.

s1 + s2 + s3 + s4 − s5 = −d

30


d +1 d

...... ............ ............ ................. ............... ............... ......... ...... ... ....... ...... ... ....... ...... ..... ... ...... ..... ..... ..... .... .......... .............. ..... ... ...... ........ ..... ... .. ... ... ..... ..... .. ... ... ... ..... ..... ... ... ... ... ..... ..... ... ... ... ... ..... ....... ... ... .. ..... ...... ... ... ... ..... ..... .. .. ... ..... ..... ... ... ... ....... ..... ... ... .. ........ ..... ... ... ... .. ..... ..... ... ... ... ... ..... ..... .. ...... ... ..... ........ ...... .. ..... ........ ...... ... ..... ..... ........ ..... ... ..... ..... ...... .. ..... ..... ... ..... . ..... ...... .. ... ...... ..... ..... ... ..... ...... ... .... ........ ..... ............. .. ..... .............. .. ..... ............ .. ..... .............. .. ..... .............. ..... .......... ... .

d

...... .

d +1

Figure 8

6.6. Case of odd d. Assume now that d is odd. We modify the previous reasoning as follows. In the triangle T = conv (0, 0), (d, 0), (0, d) we define all the objects as before. The strip conv (d, 0), (d + 1, 0), (0, d + 1), (0, d) is divided into the triangles (see Figure 8) conv (0, d + 1), (i, d − i), (i + 1, d − i − 1) , conv (d, 0), (i, d + 1 − i), (i + 1, d − i) ,

i = 0, . . . , d − 1, i = 0, . . . , d.

The chosen triangulation of T is convex. This easily follows from the fact that the triangulation D of T is convex. All the new triangles are either odd or even of type 2. Thus, we can naturally extend the construction used in the case of even d. 7. Thin topological classification: Other cases. To cover other cases in Theorem 3.4, we modify the above procedure to get (i) the total index s1 + s2 + s3 + s4 − s5 between −d and 1, (ii) the total index s1 + s2 + s3 + s4 − s5 between zero and d, (iii) the prescribed number of imaginary singular points. We show which elements in the main algorithm should be changed but do not describe in detail all possible particular cases. All the cases can be covered by a combination of the techniques presented in this section.


31

The difference between cases (i) and (ii) comes from the fact that a generic polynomial gives a Hamiltonian vector field with a total index less than or equal to 1. So, in the main case treated in Section 6 and in case (i), we basically use vector fields close to Hamiltonian ones. In case (ii), we have to involve non-Hamiltonian vector fields. Remark. The restriction (3.5) appears in our construction, because we cannot always coordinate Hamiltonian and non-Hamiltonian vector fields in the gluing procedure. 7.1. How to get the total index −d < s1 + s2 + s3 + s4 − s5 ≤ 1 for odd d and −d < s1 + s2 + s3 + s4 − s5 < 0 for even d. We have to replace up to (d + 1)/2 saddles by foci in the main construction. (How to obtain sinks or sources from foci is explained in Section 6.) We modify the elementary vector fields corresponding to the triangles adjacent to the side [(d + 1, 0), (0, d + 1)] of the Newton triangle T . If d is odd, we replace the elementary vector fields with Newton triangles σi = conv (d, 0), (d + 1 − 2i, 2i), (d + 2 − 2i, 2i − 1) , σi = conv (d, 0), (d + 2 − 2i, 2i − 1), (d + 3 − 2i, 2i − 2) (giving two saddles) by a vector field close to the Hamiltonian one coming from the polynomial Fi (x, y) = x d + x d+1−2i y 2i + bi x d+2−2i y 2i+1 + x d+3−2i y 2i+2 with the Newton triangle σi ∪ σi . This new vector field gives (for an appropriate choice of bi ) a saddle and a focus due to the following statement. Proposition 7.1.1. If (7.1.2)

4 > bi2 >

16i(i − 1) , (2i − 1)2

then Fi (x, y) has one extremum in (R∗ )2 . The value of Fi,xy (x, y) at that extremum changes its sign when substituting −bi for bi . Proof. It is shown in [S2, Lemma 3.5.2] that, under condition (7.1.2), the polynomial Fi (x, y) has one extremum (R∗ )2 . The change in sign of bi is equivalent to the coordinate transformation (x, y) → (x, −y), which, clearly, moves the extremum to another quadrant and changes the sign of Fi,xy at this point. If d is even, we replace the elementary vector fields with Newton triangles σi = conv (2i − 1, d + 2 − 2i), (2i, d + 1 − 2i), (2i, d − 2i) , σi = conv (2i, d + 1 − 2i), (2i + 1, d − 2i), (2i, d − 2i) (giving a saddle and a focus) by the following vector field.

32


Proposition 7.1.3. For any integer i ∈ [1, d/4] and any k = 3, 4, there exist a, b ∈ R such that the vector field  x˙ = −(d + 2 − 2i)x 2i−1 y d+1−2i − (d − 2i)x 2i+1 y d−1−2i     −(d − 2i)x 2i y d−1−2i − ax 2i y d−2i , Vi (a, b) :  y˙ = (2i − 1)x 2i−2 y d+2−2i + (2i + 1)x 2i y d−2i    +2ix 2i−1 y d−2i + bx 2i−1 y d+1−2i , with the Newton triangle σi ∪ σi , has two foci of type Sk in (R∗ )2 . Proof. The field V (0, 0) is Hamiltonian with the integral H (x, y) = x 2i−1 y d+2−2i + x 2i+1 y d−2i + x 2i y d−2i , which has two extrema (x ∗ , y ∗ ) and (x ∗ , −y ∗ ) in (R∗ )2 according to [S2, Section 3.3]. Hence for a, b close to zero, V (a, b) has, in a neighborhood of (x ∗ , y ∗ ), (x ∗ , −y ∗ ), two foci with the trace (d + 1 − 2i)b − 2ia (x ∗ )2i−1 (y ∗ )d−2i + O(a 2 + b2 ) at each point. 7.2. How to get the total index 3 ≤ s1 + s2 + s3 + s4 − s5 ≤ d for odd d and 0 ≤ s1 +s2 +s3 +s4 −s5 ≤ d for even d. In the case of odd d, we divide the triangle T into the triangle T = conv (0, 0), (d, 0), (0, d) and the band

T = conv (0, d), (d + 1, 0), (0, d + 1), (d, 0) .

Subdivide the band T into the elementary triangles σi = conv (d + 1, 0), (d − i, i), (d − i − 1, i + 1) , i = 0, . . . , d − 1, σi = conv (0, d), (d + 1 − i, i), (d − i, i + 1) , i = 0, . . . , d. Take block vector fields of the main construction in T , vector fields close to Hamiltonian ones (all having foci) with Newton triangles σi , and non-Hamiltonian vector fields with Newton triangles σi constructed inductively using the following statement. Proposition 7.2.1. For any nonzero real numbers a, b1 , b2 such that b1 /b2 > 1, for any positive integer i < d, and for any j = 1, . . . , 5, there exist nonzero real numbers c1 and c2 such that c1 /c2 > 1 and the vector field V : x˙ = b1 x i+1 y d−i−1 + c1 x i y d−i + ay d−1 , y˙ = b2 x i y d−i + c2 x i−1 y d+1−i , with Newton triangle conv((0, d), (i +1, d −i), (i, d +1−i)), has a singular point of type Sj in (R∗ )2 .


33

Proof. The proof has straightforward calculations. If i = d in Proposition 7.2.1, a corresponding vector field with Newton triangle conv((0, d), (d + 1, 0), (d, 1)) cannot have a focus. It produces the restriction (3.5) in our construction. Now let d be even. We again divide the triangle T into the triangle T = conv (0, 0), (d, 0), (0, d) and the band T = conv (0, d), (d + 1, 0), (0, d + 1), (d, 0) . The band T is subdivided into the elementary triangles σi = conv (d + 1, 0), (d − i, i), (d − i − 1, i + 1) , i = 0, . . . , d − 1, σi = conv (0, d), (d + 1 − i, i), (d − i, i + 1) , i = 0, . . . , d. Take in T block vector fields of the construction described above in the case of odd d with the total index equal to d. Then, take vector fields compatible with those already taken with Newton triangles σi (these vector fields are non-Hamiltonian and have saddles), and take non-Hamiltonian vector fields with Newton triangles σi constructed inductively using Proposition 7.2.1. In this construction, we have s1 + s2 ≥ 2. 7.3. How to obtain imaginary singular points. The idea is to use the vector fields described in the following four statements. Proposition 7.3.1. For any integers k ≥ 2, 0 ≤ s ≤ k/2, and any real a, b = 0, there exists a nondegenerate vector field  j 0

and the conclusion follows from the isomorphism Eb(ρ) E.

46

PROSMANS AND SCHNEIDERS

Lemma 1.3. Let E and F be two objects of ᐀c. Then lim lim Hom᐀c EB , FB B(E, F ), ← − − → B∈ᏮE B ∈ᏮF

where

B(E, F ) = f : E −→ F : f linear, f (B)-bounded in F if B-bounded in E .

Remark 1.4. If E and F are objects of ᐀c, we have Hom᐀c (E, F ) ⊂ B(E, F ). In general, this inclusion is strict, but, as is well known, it turns into an equality if E is bornological (i.e., if any absolutely convex subset of E that absorbs any bounded subset is a neighborhood of zero). Proposition 1.5. Let E and F be two objects of ᐀c. If E is bornological and F complete, then HomᏵnd(Ꮾan) IB(E), IB(F ) Hom᐀c (E, F ). Proof. Since the inclusion ᏮF ⊂ ᏮF is cofinal, we have HomᏵnd(Ꮾan) IB(E), IB(F ) HomᏵnd(Ꮾan) “lim” EB , “lim” FB − → − → B∈ᏮE

lim ← −

B ∈ᏮF

B . B , F lim HomᏮan E − →

B∈ᏮE B ∈ᏮF

Since F is complete, FB is a Banach space, and B Hom᐀c EB , FB . B , F HomᏮan E It follows that

HomᏵnd(Ꮾan) IB(E), IB(F ) lim ← −

lim Hom᐀c EB , FB − →

B∈ᏮE B ∈ᏮF

B(E, F ) Hom᐀c (E, F ), where the second isomorphism follows from Lemma 1.3 and the last isomorphism from Remark 1.4. Proposition 1.6. Let IL : Ᏽnd(Ꮾan) −→ ᐀c denote the functor defined by

IL “lim” Ei = lim Ei . − → − → i∈Ᏽ

i∈Ᏽ

A TOPOLOGICAL RECONSTRUCTION THEOREM FOR Ᏸ∞ -MODULES

47

Let E be an object of Ᏽnd(Ꮾan), and let F be a complete object of ᐀c. Then HomᏵnd(Ꮾan) E, IB(F ) Hom᐀c IL(E), F . Proof. Assuming E “lim” Ei , we have − → i∈Ᏽ HomᏵnd(Ꮾan) E, IB(F ) lim HomᏵnd(Ꮾan) “Ei ”, IB(F ) ← − i∈Ᏽ lim HomᏵnd(Ꮾan) IB(Ei ), IB(F ) ← − i∈Ᏽ lim Hom᐀c (Ei , F ) Hom᐀c IL(E), F , ← − i∈Ᏽ

where the second isomorphism follows from Remark 1.2 and the third follows from Proposition 1.5. Corollary 1.7. Let Ᏽ be a small category. For any functor X : Ᏽop −→ ᐀c such that X(i) is complete for any i ∈ Ᏽ, we have IB lim X(i) lim IB X(i) . ← − ← − i∈Ᏽ

i∈Ᏽ

Proof. For any object E of Ᏽnd(Ꮾan), we have HomᏵnd(Ꮾan) E, IB lim X(i) Hom᐀c IL(E), lim X(i) ← − ← − i∈Ᏽ i∈Ᏽ lim Hom᐀c IL(E), X(i) ← − i∈Ᏽ lim HomᏵnd(Ꮾan) E, IB X(i) , ← − i∈Ᏽ

where the first and last isomorphisms follow from Proposition 1.6. The conclusion follows from the theory of representable functors. Proposition 1.8. Assume that (Fn , fm,n )n∈N is an inductive system of Fréchet spaces with injective transition morphisms and that lim Fn − →

n∈N

is complete. Then the canonical morphism

lim IB Fn −→ IB lim Fn − → − →

n∈N

is an isomorphism.

n∈N

48


Proof. Applying IB to the canonical morphisms rn : Fn −→ lim Fn − → n∈N

and using the characterization of inductive limits, we get the canonical morphism lim IB Fn −→ IB lim Fn . (∗) − → − → n∈N

n∈N

Let B be a closed absolutely convex bounded subset of lim n∈N Fn . It follows from, − → for example, [8, Chapter IV, §19] that for some n ∈ N, B is the image of a closed absolutely convex bounded subset Bn of Fn by the canonical morphism rn . Since rn is injective, it induces the isomorphism of seminormed spaces ∼ Fn B −−→ lim Fn . n − → B

n∈N

Hence we get the isomorphism of Banach spaces ∼

n . lim Fn −−→ F Bn − → B

n∈N

Composing with the morphism

n ” −→ IB Fn −→ lim IB Fn , “ F Bn − → n∈N

we get a canonical morphism “ lim Fn ” −→ lim IB Fn . − → − → B

n∈N

n∈N

Finally, using the characterization of inductive limits, we obtain a canonical morphism IB lim Fn = lim “ lim Fn ” −→ lim IB Fn . − → − → − → − → n∈N

B∈Ꮾlim Fn − →

n∈N

B

n∈N

A direct computation shows that this morphism is a left and right inverse of (∗). Remark 1.9. Note that, due to [10, Proposition 7.2 and Corollary 7.2], a countable filtering inductive system of Fréchet spaces which is lim-acyclic in ᐀c is essentially − → equivalent to an inductive system that satisfies the assumptions of the preceding proposition. Hence, IB also commutes with the inductive limit functor in such a situation.


49

Definition 1.10. Let E and F be two objects of ᐀c. As usual, we denote by L b (E, F ) the vector space Hom᐀c (E, F ) endowed with the system of seminorms pB : p-continuous seminorm of F, B-bounded subset of E , where

pB (f ) = sup p f (e) . e∈B

Lemma 1.11. Let E and F be two objects of ᐀c. Assume E is bornological. Then L b (E, F ) lim L b EB , F ← − B∈ᏮE

in ᐀c. Moreover, assume that F is complete. Then B , F L b (E, F ) lim L b E ← − B∈ᏮE

in ᐀c. Proof. Keeping in mind the properties of bornological spaces, it is clear from the definition of L b (E, F ) that L b (E, F ) lim L b EB , F . ← − B∈ᏮE

B is included in the closure of a semiball of EB , any bounded Since any ball of E subset of EB is included in the closure of a bounded subset of EB . This property and the completeness of F show that B , F . L b EB , F L b E Hence we have the conclusion. Lemma 1.12. If E is a Banach space and F is a complete object of ᐀c, then IB L b (E, F ) L “E”, IB(F ) . Proof. For any B ∈ ᏮF , set Bb = f ∈ Hom᐀c (E, F ) : e ≤ 1 ⇒ f (e) ∈ B . Clearly Bb belongs to ᏮL b (E,F ) . Moreover, if B is closed in F , then Bb is closed in L b (E, F ), and one checks easily that L b (E, F ) B L E, FB b

50


as Banach spaces. Hence, one has successively IB L b (E, F ) = “lim” L b (E, F ) B “lim” L b (E, F ) B − → − → b B∈ᏮL b (E,F )

B ∈ᏮF

“lim” L E, FB L “E”, IB(F ) , − → B ∈ᏮF

where the second isomorphism follows from the fact that the inclusion Bb : B ∈ ᏮF ⊂ ᏮL b (E,F ) is cofinal. Proposition 1.13. Let E and F be two objects of ᐀c. Assume that E is bornological and F is complete. Then IB L b (E, F ) L IB(E), IB(F ) . Proof. We have successively (1) (2) (3)

lim L b EB , F ← − B∈ᏮE B , F lim IB L b E ← − B∈ᏮE B ”, IB(F ) lim L “E ← − B∈ᏮE L IB(E), IB(F ) ,

IB L b (E, F ) IB

where the isomorphism (1) follows from Lemma 1.11, (2) follows from Corollary 1.7, and (3) follows from Lemma 1.12. Remark 1.14. (1) Let E, F , G be three objects of ᐀c. Recall that a bilinear application b : E × F −→ G is continuous if and only if for any continuous seminorm r of G, there are continuous seminorms p and q of E and F , respectively, such that r b(x, y) ≤ p(x)q(y). (2) Let E and F be two objects of ᐀c with P and Q as systems of seminorms. As usual, if p ∈ P and q ∈ Q, we let p ⊗q denote the seminorm on E ⊗F defined by (p ⊗q)(u) = inf p(xi )q(yi ). u=

xi ⊗yi

i


51

Recall that E ⊗π F is the object of ᐀c obtained by endowing E ⊗F with the system of seminorms induced by p ⊗q : p ∈ P , q ∈ Q . From this definition, it follows immediately that any continuous bilinear map b : E × F −→ G factors uniquely through a continuous linear map E ⊗π F −→ G. ˆ π q is the ˆ π F denotes the completion of E ⊗π F and that p ⊗ Finally, recall that E ⊗ ˆ seminorm of E ⊗π F induced by p ⊗q. Proposition 1.15. There is a canonical morphism ˆ IB(E) ⊗IB(F ) −→ IB E ⊗π F . Proof. For B ∈ ᏮE and B ∈ ᏮF , let B ⊗B denote the absolutely convex hull of b ⊗b : b ∈ B, b ∈ B . This is clearly a bounded absolutely convex subset of E ⊗F . As a matter of fact, (p ⊗q) b ⊗b ≤ p(b)q(b ) ≤ sup p(b) sup q(b ). b∈B

b ∈B

Moreover, we have a canonical linear map EB ⊗FB −→ E ⊗π F B⊗B . This map is clearly continuous because e ⊗ f ∈ B ⊗ B when e ∈ B and e ∈ B . Applying the completion functor, we get a morphism B ⊗ B −→ E ˆF E ⊗π F B⊗B and, hence, a morphism B ” ⊗“ B ” IB E B ⊗ B −→ IB E ⊗ F . ˆ F ˆF “E π Using the definition of inductive limits, we get a morphism ˆ IB(E) ⊗IB(F ) lim − →

B ” ⊗“ B ” −→ IB E ⊗ F . ˆ F lim “E π − →

B∈ᏮE B ∈ᏮF

52


ˆ in Ᏽnd(Ꮾan). Hereafter, we let c0 2. Some acyclicity results for L and ⊗ 1 (resp., l ) denote as usual the Banach spaces formed by the sequences x = (xn )n∈N of complex numbers that converge to 0 (resp., that are summable); the norm is defined by ∞

xn . xc0 = sup xn resp., xl 1 = n∈N

n=0

For any Banach space X, we also set for short D(X) = L (X, C). Lemma 2.1. Let X, Y be two Banach spaces, and let f : X → Y be a nuclear map. Then there is a continuous linear map p : X → c0 and a nuclear map c : c0 → Y making the diagram 0 ? c ?? ~ ??c p ~~ ?? ~~ ? ~ ~ /Y X f commutative. Proof. Since f : X → Y is nuclear, there is a bounded sequence xn∗ of D(X), a bounded sequence yn of Y , and a summable sequence λn of complex numbers such that +∞ f (x) = λn xn∗ , x yn ∀x ∈ X. n=0

Since λn is summable, one can find a sequence rn of nonzero complex numbers converging to zero such that λn /rn is still summable. It is easy to check that the maps p : X → c0 and c : c0 → Y , defined by

p(x)n = rn xn∗ , x

c(s) =

and

+∞ λn n=0

rn

yn sn ,

have the requested properties. Definition 2.2. A projective system E : I op → Ꮾan, where I is a filtering ordered set, is nuclear if for any i ∈ I , there is j ∈ I , j ≥ i such that the transition morphism ei,j : Ej −→ Ei is nuclear. Lemma 2.3. Let I be an infinite filtering ordered set, and let E : I op → Ꮾan be a nuclear projective system. Then, in ᏼro(Ꮾan), we have “lim” Ei “lim” Xk , ← − ← − i∈I

k∈K


53

where X : K op → Ꮾan is a projective system with nuclear transition morphisms such that X k = c0 for any k ∈ K and #K = #I . Proof. Consider the set K = (i, j ) ∈ I × I : j ≥ i, ei,j : Ej −→ Ei nuclear . The relation “≥,” defined by setting (i , j ) ≥ (i, j ) if (i , j ) = (i, j ) or i ≥ j , turns K into a filtering ordered set. By Lemma 2.1, for any k = (i, j ) ∈ K, we may choose a continuous linear map pk : Ej → c0 and a nuclear map ck : c0 → Ei , which make the diagram 0 ? c ?? ~ ?? ck pk ~~ ?? ~~ ?? ~ ~ / Ei Ej e i,j

commutative. For any k ∈ K, we set Xk = c0 and xk,k = idXk . If k = (i , j ) > k = (i, j ), we set xk,k = pk # ej,i # ck : Xk −→ Xk . Because the map ck is nuclear, xk,k is also nuclear. An easy computation shows that if k < k < k , then xk,k # xk ,k = xk,k . Consider the functors 6 : K −→ I

7 : K −→ I

and

defined by 6((i, j )) = i and 7((i, j )) = j . They are clearly cofinal, and if k ≥ k in K, the diagrams Xk

ck

xk,k

Xk

/ E6(k )

ck

E7(k )

e6(k),6(k )

e7(k),7(k )

/ E6(k)

pk

xk,k

E7(k)

/ Xk

pk

/ Xk

are commutative. Hence we get the two morphisms “lim” Xk −→ “lim” E6(k) “lim” Ei , ← − ← − ← − k∈K

k∈K

i∈I

“lim” Ej “lim” E7(k) −→ “lim” Xk . ← − ← − ← − j ∈I

k∈K

k∈K

Since these morphisms are easily checked to be inverse of each other, the proof is complete. Remark 2.4. Hereafter, as usual, we let en denote the element of c0 defined by (en )m = δn,m , and we let en∗ denote the element of D(c0 ) defined by ∗ en , x = x n .

54


Lemma 2.5. For any Banach space Y and any nuclear map u : c0 −→ Y, the sequence u(en )Y is summable, and for any x ∈ c0 , we have u(x) =

+∞ n=0

en∗ , x u(en ).

Proof. Since u is nuclear, we can find a bounded sequence xn∗ of D(c0 ), a bounded sequence yn of Y , and a summable sequence λn of complex numbers such that u(x) =

+∞ n=0

λn xn∗ , x yn

for any x ∈ c0 . Using the isomorphism D(c0 ) l 1 , we see that +∞

∗

x , em = x ∗

(∗)

m=0

n D(c0 ) .

n

Therefore, M

+∞ M

∗

λn

x , em yn Y

u(em ) ≤

m=0

≤ ≤

m=0 n=0 +∞

n

∗

λn x

n=0 +∞ n=0

n D(c0 ) yn Y

λn sup x ∗ 0 sup yn Y , n D(c ) n∈N

n∈N

and the sequence u(en )Y is summable. Moreover, u(x) = = =

+∞ +∞

λn xn∗ , em xm yn

n=0 m=0 +∞ +∞

xm

m=0 +∞ m=0

n=0

λn xn∗ , em yn

∗ , x u(em ), em

where the permutation of sums is justified using (∗).


55

Lemma 2.6. Let I be an infinite filtering ordered set, and let X : I op → Ꮾan be a nuclear projective system. Assume that Y is a Fréchet space. Then the morphisms ˆ π Y −→ L b D(Xi ), Y , ϕi : X i ⊗ defined by setting ˆ π y (x ∗ ) = x ∗ , x y ϕi x ⊗

∀x ∈ Xi , y ∈ Y, x ∗ ∈ D(Xi ),

induce an isomorphism ˆ π Y “lim” L b D(Xi ), Y . “lim” Xi ⊗ ← − ← − i∈I

i∈I

In particular, for Y = C, we have

“lim” Xi “lim” D D(Xi ) . ← − ← − i∈I

i∈I

Proof. By Lemma 2.3, we may assume that Xi = c0 for any i ∈ I and that the transition morphisms xi,j : Xj −→ Xi (j > i) are nuclear. It is easy to check that ϕi is a well-defined continuous map. By Lemma 2.5, we know that the sequence (xi,j (en )Xi )n∈N is summable and that xi,j (c) =

+∞ n=0

en∗ , c xi,j (en ) ∀c ∈ Xj .

Therefore, we may define a continuous linear map ˆπY ψi,j : L b D(Xj ), Y −→ Xi ⊗ by setting ψi,j (h) =

+∞ n=0

ˆ π h en∗ . xi,j (en ) ⊗

It is easy to see that the morphisms ϕi and ψi,j induce morphisms of pro-objects ˆ π Y −→ “lim” L b D(Xi ), Y “lim” Xi ⊗ ← − ← − i∈I

and

i∈I

ˆ π Y. “lim” L b D(Xi ), Y −→ “lim” Xi ⊗ ← − ← − i∈I

i∈I

A direct computation shows that these morphisms are inverse of each other.

56


Definition 2.7. We say that a filtering projective system E : I op → ᐀c satisfies condition ML if for any i ∈ I , any seminorm p of Ei , and any ; > 0, there is i ≥ i such that ei,i (Ei ) ⊂ bp (;) + ei,i (Ei ) ∀i ≥ i , where bp (;) denotes as usual the semiball of radius ; and center 0 associated to the seminorm p. Remark 2.8. By [15, Proposition 1.2.9] (which is a direct consequence of [14, Theorem 5.6]), a countable filtering projective system of Fréchet spaces is lim-acyclic ← − in ᐀c if and only if it satisfies condition ML. Lemma 2.9. Let E : I op → ᐀c and F : J op → ᐀c be two filtering projective systems. If E and F satisfy condition ML, then the projective system ˆ π F : (I × J )op −→ ᐀c E⊗ defined by

ˆ π F (i, j ) = Ei ⊗ ˆ π Fj E⊗

satisfies condition ML. ˆ π Fj . It follows ˆ π q be a seminorm of Ei ⊗ Proof. Let (i, j ) ∈ I × J and let p ⊗ from our assumptions that there is i ≥ i and j ≥ j such that (∗)

ei,i (Ei ) ⊂ bp (1) + ei,i (Ei ) ∀i ≥ i

and (∗∗)

fj,j Fj ⊂ bq (1) + fj,j Fj

∀j ≥ j .

Fix (i , j ) ≥ (i , j ). Since the maps ei,i , fj,j and ei,i are continuous, we can find a seminorm p of Ei , a seminorm q of Fj , and a seminorm p of Ei such that p # ei,i ≤ p ,

q # fj,j ≤ q ,

p # ei,i ≤ p .

Consider ; > 0, and let z be an element of Ei ⊗π Fj of the type x ⊗π y , where x ∈ Ei , y ∈ Fj . Using (∗) and (∗∗) above, we obtain x ∈ Ei and y ∈ Fj such that ; p ei,i x − ei,i x ≤ 2 1 + q y and

; . q fj,j y − fj,j y ≤ 2 1 + p x


57

For z = x ⊗π y ∈ Ei ⊗π Fj , we get

p ⊗π q

ei,i ⊗π fj,j z − ei,i ⊗π fj,j z = p ⊗π q ei,i x − ei,i x ⊗π fj,j y + ei,i x ⊗π fj,j y − fj,j y ≤ p ei,i x − ei,i x q fj,j y + p ei,i x q fj,j y − fj,j y ≤ ;.

Since any element of Ei ⊗π Fj is a finite sum of elements of the type considered above, we see that for any ; > 0, ei,i ⊗π fj,j Ei ⊗π Fj ⊂ bp⊗ q (;) + ei,i ⊗π fj,j Ei ⊗π Fj . π

ˆ π Fj . The conclusion follows directly since Ei ⊗π Fj is dense in Ei ⊗ Remark 2.10. Let E be an object of ᐀c. Recall that E is of type FN if it is a nuclear Fréchet space and that E is of type DFN if it is isomorphic to the strong dual of a nuclear Fréchet space. Lemma 2.11. Assume X is an FN space. Then there is a projective system Xn , xn,m n∈N of Banach spaces such that (a) there is an isomorphism X lim Xn ; ← − n∈N

(b) for m > n, the transition map xn,m : Xm −→ Xn is nuclear and has a dense range; (c) there is an isomorphism Db (X) lim D(Xn ), − → n∈N

where Db (X) denotes the strong dual of X; (d) for m > n, the transition map D(xn,m ) : D(Xn ) −→ D(Xm ) is nuclear and injective.

58


Proof. Since X is an FN space, there is a cofinal increasing sequence (pn )n∈N of continuous seminorms of X such that the canonical map Xpn+1 −→ Xpn is nuclear. For such a sequence, the canonical map pn+1 −→ X pn X is also nuclear and has a dense range. Moreover, it is well known (see, e.g., [8, Chapter IV, §19]) that . X lim X ← − pn n∈N

Clearly

pn , Di (X) lim D Xpn lim D X − → − → n∈N

n∈N

where Di (X) is the inductive dual of X. Recall that an absolutely convex subset V is a neighborhood of 0 in Di (X) if it absorbs any equicontinuous subset of X . Hence, it is clear that a neighborhood of 0 in Db (X) is a neighborhood of 0 in Di (X). We know that X is reflexive (see, e.g., [11, §5.3.2]). Hence, Db (X) is bornological (see, e.g., [8, Chapter VI, §29]). The space X being itself bornological, the bounded subsets of Db (X) are equicontinuous. So any neighborhood of 0 in Di (X) is a neighborhood of 0 in Db (X), and Di (X) Db (X). Since (d) follows directly from (b), the proof is complete. Proposition 2.12. Assume that E is a DFN space and F is a Fréchet space. Then the canonical morphism Hom IB(E), IB(F ) −→ RHom IB(E), IB(F ) is an isomorphism. Proof. Since E is an DFN space, there is an FN space X such that E Db (X). Let (Xn , xn,m ) be a projective system of the kind considered in Lemma 2.11. We have E Db (X) lim D(Xn ). − → n∈N

Since the transition morphisms D(xn,m ) : D(Xn ) −→ D(Xm )

(m > n)

are injective and E is complete, Proposition 1.8 and Remark 1.2 show that IB(E) lim “D(Xn )”. − → n∈N


59

Using Lemma 2.3, we find a nuclear projective system (Yn , yn,m ) with Yn = c0 such that “lim” Xn “lim” Yn . ← − ← − n∈N

n∈N

It follows that IB(E) lim “D(Yn )”. − → n∈N

Hence we have successively (1) (2) (3) (4) (5)

RHom IB(E), IB(F ) RHom L lim “D(Yn )”, IB(F ) − → n∈N R lim RHom “D(Yn )”, IB(F ) ← − n∈N R lim Hom “D(Yn )”, IB(F ) ← − n∈N R lim Hom IB D(Yn ) , IB(F ) ← − n∈N R lim Hom᐀c D(Yn ), F , ← −

n∈N

where (1) follows from the fact that filtering inductive limits are exact in Ᏽnd(Ꮾan), (2) follows from [13, Proposition 3.6.3], (3) follows from the fact that “D(Yn )” “D(c0 )” “l 1 ” is projective in Ᏽnd(Ꮾan), (4) follows from Remark 1.2, and (5) follows from Proposition 1.5. By Lemma 2.6, we have the isomorphism ˆ π F “lim” L b D(Yn ), F . “lim” Yn ⊗ ← − ← − n∈N

n∈N

Forgetting the topologies and applying the derived projective limit functor for proobjects (see [13]), we obtain the isomorphism ˆ π F R lim Hom᐀c D(Yn ), F . R lim Yn ⊗ ← − ← − n∈N

n∈N

Since (Xn , xn,m )n∈N satisfies condition ML, it is lim-acyclic in ᐀c (see Remark 2.8). ← − It follows that (Yn , yn,m )n∈N is also lim-acyclic in ᐀c and hence satisfies condition ← − ˆ π F ) is concentrated in degree 0. ML. Using Lemma 2.9, we see that R lim n∈N (Yn ⊗ ← − It follows that the projective system

Hom᐀c D(Yn ), F n∈N

is lim-acyclic, and the conclusion follows. ← −

60


Theorem 2.13. Assume that E is a DFN space and F is a Fréchet space. Then the canonical morphism L IB(E), IB(F ) −→ RL IB(E), IB(F ) is an isomorphism. Proof. It is sufficient to show that LH k RL IB(E), IB(F ) 0 for k > 0. This is the case if

Hom “l 1 (I )”, RL IB(E), IB(F )

is concentrated in degree 0 for any set I . Let I be an arbitrary set. Since “l 1 (I )” is a projective object of Ᏽnd(Ꮾan) and since F is complete, we have RL “l 1 (I )”, IB(F ) L IB l 1 (I ) , IB(F ) IB L b l 1 (I ), F IB l ∞ (I, F ) , where l ∞ (I, F ) is the Fréchet space formed by the bounded families (xi )i∈I of F (a fundamental system of seminorms given by pI : p-continuous seminorm of F , where pI ((xi )i∈I ) = supi∈I p(xi )). Therefore, we have the chain of isomorphisms Hom “l 1 (I )”, RL IB(E), IB(F ) RHom “l 1 (I )”, RL IB(E), IB(F ) ˆ L IB(E), IB(F ) RHom “l 1 (I )” ⊗ ˆ L “l 1 (I )”, IB(F ) RHom IB(E) ⊗ RHom IB(E), RL “l 1 (I )”, IB(F ) RHom IB(E), IB l ∞ (I, F ) , and the conclusion follows from Proposition 2.12. Lemma 2.14. Let X, Y be two Banach spaces and let f : X → Y be a nuclear map. Then there is a nuclear map p : X → l 1 and a continuous linear map c : l 1 → Y making the diagram 1 ? l ?? ??c p ?? ? /Y X f commutative.


61

Proof. The result follows from an argument entirely similar to the one used in the proof of Lemma 2.1. Definition 2.15. An inductive system E : I → Ꮾan, where I is a filtering ordered set, is nuclear if for any i ∈ I , there is j ∈ I , j ≥ i such that the transition morphism ej,i : Ei −→ Ej is nuclear. An object of Ᏽnd(Ꮾan) is nuclear if it corresponds to a nuclear inductive system. Remark 2.16. Working as in the proof of Proposition 2.12, we see easily that IB(E) is nuclear if E is a DFN space. Lemma 2.17. Let I be an infinite filtering ordered set, and let E : I → Ꮾan be a nuclear inductive system. Then “lim” Ei “lim” Xk , − → − → i∈I

k∈K

where X : K → Ꮾan is an inductive system with nuclear transition morphisms such that X k = l 1 for any k ∈ K and #K = #I . Proof. The result is obtained by working as in the proof of Lemma 2.3, provided that Lemma 2.14 is used in place of Lemma 2.1. Lemma 2.18. Let I be a filtering ordered set. For any F ∈ D − (Ᏽnd(Ꮾan)) and any E ∈ D − (Ᏽnd(Ꮾan)I ), we have ˆ L F lim Ei ⊗ ˆ LF . lim Ei ⊗ − → − → i∈I

i∈I

Proof. If P· is a projective resolution of F , we have successively L ˆ F lim Ei ⊗P ˆ · lim Ei ⊗P ˆ · lim Ei ⊗ ˆ LF . lim Ei ⊗ − → − → − → − → i∈I

i∈I

i∈I

i∈I

Proposition 2.19. Let E and F be objects of Ᏽnd(Ꮾan). Assume that E is nuclear. Then ˆ L F E ⊗F. ˆ E⊗ Proof. Using Lemma 2.17, we may assume that E = “lim” Xi , − → i∈I

where X : I → Ꮾan is a filtering inductive system with Xi = l 1 , the transition morphisms xj,i : Xi −→ Xj

62


being nuclear. We may also assume that F = “lim” Yj , − → j ∈J

where Y : J → Ꮾan is a filtering inductive system. Then we have ˆ L F “lim” Xi ⊗ ˆ L “lim” Yj E⊗ − → − → i∈I

j ∈J

L

ˆ “Yj ” lim lim “Xi ” ⊗ → − →−

(1)

i∈I j ∈J

ˆ j” lim lim “Xi ” ⊗“Y − →− → i∈I j ∈J ˆ “lim” Xi ⊗ “lim” Yj − → − →

(2)

i∈I

j ∈J

ˆ E ⊗F, where (1) follows from Lemma 2.18 and (2) follows from the fact that “Xi ” “l 1 ” is projective in Ᏽnd(Ꮾan). 3. A topological version of Cartan’s Theorem B Proposition 3.1. Let X be a topological space with a countable basis. If F is a presheaf of Fréchet spaces on X that is a sheaf of vector spaces, then U −→ IB F (U ) (U open of X) is a sheaf with values in Ᏽnd(Ꮾan). Proof. Let U be an open subset of X and let ᐁ be an open covering of U . Consider the sequence (∗)

α

0 −→ F (U ) −−→

V ∈ᐁ

β

F (V ) −−→

F (V ∩ W ),

V ,W ∈ᐁ

where α and β are the continuous applications defined by pV # α = rV ,U

and

pV ,W # β = rV ∩W,V # pV − rV ∩W,W # pW ,

where pV and pV ,W are the canonical projections and rV ,U is the restriction map. Since F is a sheaf of vector spaces, this sequence is algebraically exact. Let us show that it is strictly exact. (1) If ᐁ is countable, F (U ), V ∈ᐁ F (V ), and V ,W ∈ᐁ F (V ∩ W ) are Fréchet spaces. Then, by the homomorphism theorem, the sequence (∗) is strictly exact.


63

(2) Assume that ᐁ is not countable. Since X has a countable basis, there is a countable set A of open subsets of X such that for any open V of X, V =

Uk ,

Uk ∈ A.

k∈N

Then consider the countable set

ᐂ = V ∈ A : ∃ V ∈ ᐁ such that V ⊂ V .

For any U ∈ ᐁ, we may assume that U = k∈N Uk , with Uk ∈ ᐂ. It follows that ᐂ covers any U in ᐁ and therefore is a covering of U . Hence, by (1), the sequence α

0 −→ F (U ) −−→

β F V −−→

V ∈ᐂ

F V ∩W

V ,W ∈ᐂ

is strictly exact. Now consider a map f : ᐂ → ᐁ such that V ⊂ f (V ) for any V ∈ ᐂ. Then consider the commutative diagram 0

/ F (U )

0

/ F (U )

α

/

V ∈ᐁ F (V )

β

/

γ

id

α

/ V ∈ᐂ F V

V ,W ∈ᐁ F (V

∩W)

δ

β

/

F V ∩W , V ,W ∈ᐂ

where γ and δ are, respectively, defined by pV # γ = rV ,f (V ) # pf (V ) and pV ,W # δ = rV ∩W ,f (V )∩f (W ) # pf (V ),f (W ) . To prove that the sequence (∗) is strictly exact, it is sufficient to establish that α is a kernel of β. Let h : X → V ∈ᐁ F (V ) be a morphism of ᐀c such that β #h = 0. Since β # γ # h = δ # β # h = 0 and since α is a kernel of β , there is a unique morphism h : X → F (U ) such that α # h = γ # h. Set h = h − α # h . We clearly have γ # h = 0 and β # h = 0. Fix V ∈ ᐁ. For any V ∈ ᐂ such that V ⊂ V , we have 0 = pV ,f (V ) # β # h = rV ∩f (V ),V # pV # h − rV ∩f (V ),f (V ) # pf (V ) # h .

64


It follows that rV ,V # pV # h = rV , V ∩f (V ) # rV ∩f (V ), V # pV # h = rV , V ∩f (V ) # rV ∩f (V ), f (V ) # pf (V ) # h = rV , f (V ) # pf (V ) # h = pV # γ # h = 0. Since {V ∈ ᐂ : V ⊂ V } is a covering of V and since F is a sheaf of vector spaces, we get pV # h = 0 ∀V ∈ ᐁ. It follows that h = 0 and that h = α # h . Since α is injective, h is the unique morphism of ᐀c such that h = α # h . Therefore, α is a kernel of β, and the sequence (∗) is strictly exact. Finally, since the functor IB preserves projective limits of complete objects of ᐀c (see Corollary 1.7), the sequence IB(β) IB(α) IB F (V ) −−−−→ IB F (V ∩ W ) 0 −→ IB F (U ) −−−−→ V ∈ᐁ

V ,W ∈ᐁ

is strictly exact in Ᏽnd(Ꮾan) . Hence, we reach the conclusion. Definition 3.2. We let IB(F ) denote the sheaf with values in Ᏽnd(Ꮾan) associated to a presheaf F of the kind considered in Proposition 3.1. Hereafter, let X denote a complex analytic manifold of complex dimension dX . We let ᏻX denote the sheaf of holomorphic functions on X. Recall that for any open subset U of X, ᏻX (U ) has a canonical structure of FN space. Moreover, recall that if V is a relatively compact open subset of U , the restriction morphism ᏻX (U ) −→ ᏻX (V )

is nuclear. In particular, if K is a compact subset of X, then ᏻX (K) lim ᏻX (U ),

− →

U ⊃K U open

topologized as an inductive limit, is a DFN space. Proposition 3.3. For any compact subset K of X, we have K, IB(ᏻX ) IB ᏻX (K) .


65

Proof. We know that K has a fundamental system (Un )n∈N of relatively compact open neighborhoods such that U n+1 ⊂ Un for any n ∈ N. If necessary, replacing Un by the union of those of its connected components that meet K, we may even assume that any connected component of Un meets K. In this case, it follows from the principle of unique continuation that the restriction ᏻX (Un ) −→ ᏻX Un+1 is injective. Moreover, by cofinality,

ᏻX (K) lim ᏻX Un .

− →

n∈N

Hence, by Proposition 1.8, it follows that IB lim ᏻX Un lim IB ᏻX Un . − → − → n∈N

n∈N

Since K is a taut subspace of X, a cofinality argument shows that K, IB ᏻX lim Un , IB ᏻX , − → n∈N

and the conclusion follows. Hereafter, we let Ꮿ∞,X denote the sheaf of rings formed by functions of class C∞ . (p,q) More generally, we let Ꮿ∞,X denote the sheaf of differential forms of class C∞ and (p,q)

of bitype (p, q). Recall that for any open subset U of X, Ꮿ∞,X (U ) has a canonical (p,q)

structure of FN space. Since the conditions of Proposition 3.1 are satisfied, IB(Ꮿ∞,X ) is a sheaf with values in Ᏽnd(Ꮾan). (p,q)

Proposition 3.4. The sheaf IB(Ꮿ∞,X ) is (U, ·)-acyclic for any open subset U of X. Proof. For any object E of Ᏽnd(Ꮾan), let hE : Ᏽnd(Ꮾan) −→ Ꮽb denote the functor defined by setting hE (F ) = Hom(E, F ). Using the techniques developed in [18], it is easy to show that (p,q) (p,q) Hom P , R U, IB Ꮿ∞,X R U, hP IB Ꮿ∞,X for any projective object P of Ᏽnd(Ꮾan). Therefore, the result is true if the sheaf of (p,q) abelian groups hP (IB(Ꮿ∞,X )) is soft. This follows from the fact that it clearly has a canonical structure of a Ꮿ∞,X -module.

66


Theorem 3.5. If U is an open subset of X such that H k U, ᏻX 0 (k > 0) algebraically, then

R U, IB ᏻX IB ᏻX (U ) . (p,q)

Proof. As is well known, since Ꮿ∞,U is a soft sheaf, the Dolbeault complex (0,0)

∂

(0,1)

∂

(0,n)

0 −→ Ꮿ∞,X −→ Ꮿ∞,X · · · −→ Ꮿ∞,X −→ 0 is a (U, ·)-acyclic resolution of ᏻX . Therefore, R(U, ᏻX ) is given by the complex ∂ ∂ (0,0) (0,1) (0,n) 0 −→ U, Ꮿ∞,X −→ U, Ꮿ∞,X · · · −→ U, Ꮿ∞,X −→ 0. Moreover, since H k (U, ᏻX ) 0 for k > 0, the sequence ∂ ∂ (0,0) (0,1) (0,n) 0 −→ U, ᏻX −→ U, Ꮿ∞,X −→ U, Ꮿ∞,X · · · −→ U, Ꮿ∞,X −→ 0 (p,q)

is algebraically exact. Since ᏻX (U ) and Ꮿ∞,X (U ) are FN spaces, the last sequence is strictly exact in ᐀c. Using [18, Proposition 3.2.26], one sees easily that the sequence (∗)

(0,0) (0,n) 0 −→ U, IB ᏻX −→ U, IB Ꮿ∞,X · · · −→ U, IB Ꮿ∞,X −→ 0

is strictly exact in Ᏽnd(Ꮾan). For any open ball b of X, Cartan’s Theorem B shows that H k b, ᏻX 0 (k > 0). Hence the sequence (0,0) (0,n) 0 −→ b, IB ᏻX −→ b, IB Ꮿ∞,X · · · −→ b, IB Ꮿ∞,X −→ 0 is strictly exact in Ᏽnd(Ꮾan). Filtering inductive limits being exact in Ᏽnd(Ꮾan), we see that (0,0) (0,1) (0,n) 0 −→ IB ᏻX −→ IB Ꮿ∞,X −→ IB Ꮿ∞,X · · · −→ IB Ꮿ∞,X −→ 0 is a strictly exact sequence of sheaves with values in Ᏽnd(Ꮾan). Moreover, since by (p,q) Proposition 3.4, IB(Ꮿ∞,U ) is (U, ·)-acyclic, R(U, IB(ᏻX )) is given by (0,0) (0,1) (0,n) 0 −→ U, IB Ꮿ∞,X −→ U, IB Ꮿ∞,X · · · −→ U, IB Ꮿ∞,X −→ 0. The sequence (∗) being strictly exact, we get R U, IB ᏻX U, IB ᏻX .


67

Proposition 3.6. If X is a Stein manifold and K is a holomorphically convex compact subset of X, we have R K, IB ᏻX IB ᏻX (K) . Proof. It is well known that K has a fundamental system ᐂ of Stein open neighborhoods. By tautness, it follows that for k > 0, we have LH k K, IB ᏻX lim LH k V , IB ᏻX 0, − → V ∈ᐂ

where the second isomorphism follows from Theorem 3.5. Hence, using Proposition 3.3, we get R K, IB ᏻX K, IB ᏻX IB ᏻX (K) . Remark 3.7. Note that all the results in this section clearly hold if we replace ᏻX by the sheaf of holomorphic sections of a holomorphic vector bundle. In particular, p they hold for the sheaf !X of holomorphic p-forms. 4. A factorization formula for IB(ᏻX×Y ) Definition 4.1. For any ρ = (ρ1 , . . . , ρp ) ∈ ]0, +∞[p , we set

)ρ = z ∈ Cp : z1 < ρ1 , . . . , zp < ρp , and we denote by Aρ the object of ᐀c defined by endowing

aα ρ α < +∞ Aρ = (aα )α∈Np : α

with the norm

α (aα )α∈Np =

aα ρ . α

p

Lemma 4.2. For any ρ ∈ ]0, +∞[ , we have the isomorphism A ρ l 1 Np . In particular, Aρ is a Banach space. Proof. This follows directly from the fact that the application u : Aρ −→ l 1 Np , defined by u((aα )α∈Np ) = (aα ρ α )α∈Np , is continuous and bijective. Lemma 4.3. For any p ∈ N, lim − →

ρ∈]0,+∞[p

IB ᏻCp )ρ

lim − →

ρ∈]0,+∞[p

IB Aρ .

68


Proof. This follows directly from the fact that the canonical restriction morphism ᏻCp )ρ −→ ᏻCp )ρ may be factored through Aρ for ρ > ρ. Proposition 4.4. Assume that X, Y are complex analytic manifolds. Then there is a canonical isomorphism L ˆ IB ᏻY IB ᏻX×Y . IB ᏻX Proof. Let U , V be open subsets of X and Y . The map uU,V : ᏻX (U ) × ᏻY (V ) −→ ᏻX×Y (U × V ), defined by setting uU,V (f, g)(u, v) = f (u) g(v), is clearly bilinear and continuous. Hence, it induces a morphism ᏻX (U ) ⊗π ᏻY (V ) −→ ᏻX×Y (U × V ),

and by Proposition 1.15, we get a morphism ˆ µU,V : IB ᏻX (U ) ⊗IB ᏻY (V ) −→ IB ᏻX×Y (U × V ) , which is clearly well behaved with respect to the restriction of U or V . Therefore, we get a canonical morphism ˆ IB ᏻY −→ IB ᏻX×Y . µ : IB ᏻX To show that it is an isomorphism, it is sufficient to work at the level of germs and to prove that ˆ µ(x,y) : IB ᏻX x ⊗IB ᏻY y −→ IB ᏻX×Y (x,y)

is an isomorphism. The problem being local, we may assume that X = Cp , Y = Cp , x = 0, y = 0. In this case, Lemma 4.3 shows that IB ᏻX x lim IB Aρ , lim IB Aρ , IB ᏻY y − → p − → ρ∈]0,+∞[

and

IB ᏻX×Y (x,y)

ρ ∈]0,+∞[p

lim − →

(ρ,ρ )∈]0,+∞[p+p

IB A(ρ,ρ ) .

A direct computation shows that through these isomorphisms, µx,y corresponds to the inductive limit of the maps ˆ τρ,ρ : IB Aρ ⊗IB Aρ −→ IB A(ρ,ρ )


69

associated to the continuous bilinear maps tρ,ρ : Aρ × Aρ −→ A(ρ,ρ ) defined by

tρ,ρ aα α∈Np , aα α ∈Np = aα aα (α,α )∈Np+p .

Since the diagram

ˆ IB Aρ ⊗IB Aρ τρ,ρ

/o

IB A(ρ,ρ ) /o

/ “Aρ ⊗A ˆ ρ ”

“tρ,ρ ”

/ “A(ρ,ρ ) ”

is clearly commutative, to prove that µ(x,y) is an isomorphism, it is sufficient to prove that tρ,ρ is an isomorphism. Thanks to Lemma 4.2, this fact is an easy consequence of the well-known isomorphism 1 p ˆ N l 1 Np+p . l 1 Np ⊗l By Proposition 3.3, IB ᏻX x {x}, IB ᏻX IB ᏻX ({x}) . Since ᏻX ({x}) is a DFN space, Proposition 2.19 shows that L ˆ IB ᏻY IB ᏻX ⊗IB ˆ IB ᏻX x ⊗ ᏻY y . y x Therefore, L ˆ IB ᏻY IB ᏻX ˆ IB ᏻY IB ᏻX×Y , IB ᏻX as requested. Corollary 4.5. If A, B are subsets of X and Y , then L ˆ R c B; IB ᏻY . R c A × B, IB ᏻX×Y R c A, IB ᏻX ⊗ In particular, if X, Y are Stein manifolds and if K, L are holomorphically convex compact subsets of X and Y , then ˆ IB ᏻX×Y (K × L) IB ᏻX (K) ⊗IB ᏻY (L) . Proof. The first part is a direct consequence of Theorem 4.4 and the Künneth theorem for sheaves with values in Ᏽnd(Ꮾan). The second part follows from the first part, using Proposition 3.6, Proposition 2.19, and the fact that ᏻX (K) is a DFN space.

70


5. Poincaré duality for IB(ᏻX ) Proposition 5.1. Assume that X, Y are complex analytic manifolds of dimension dX and dY . Then there is a canonical integration morphism : RqY ! IB !X×Y dX×Y −→ IB !Y dY . X

Proof. Recall that integration along the fibers of qY (i.e., on X) defines morphisms p+dX ,q+dX p,q −→ Ꮿ∞,Y (p, q ∈ Z), (∗) : qY ! Ꮿ∞,X×Y X

which are compatible with ∂ and ∂. Fix p, q ∈ Z. Let K be a compact subset of X, and let U be an open subset of Y . It is easy to check that the morphism p+dX ,q+dX p,q : K×U X × U ; Ꮿ∞,X×Y −→ U ; Ꮿ∞,Y X

is continuous for the canonical topologies. Applying IB, we get a morphism p+dX ,q+dX p,q K×U X × U ; IB Ꮿ∞,X×Y −→ U ; IB Ꮿ∞,Y . Taking the inductive limit on K, we get a morphism p+dX ,q+dX p,q U ; qY ! IB Ꮿ∞,X×Y −→ U ; IB Ꮿ∞,Y and, hence, a morphism p+dX ,q+dX p,q qY ! IB Ꮿ∞,X×Y −→ IB Ꮿ∞,Y of sheaves with values in Ᏽnd(Ꮾan). Thanks to the compatibility of (∗) with ∂ and ∂, we also get a morphism of complexes dX×Y ,· X ,· qY ! IB Ꮿ∞,X×Y dX×Y −→ IB Ꮿd∞,Y dY . Using the properties of Dolbeault resolutions, we get the requested integration morphism : RqY ! IB !X×Y dX×Y −→ IB !Y dY . X

Remark 5.2. Assume that X, Y , Z are complex analytic manifolds. Then one checks easily that the Fubini theorem gives rise to the commutative diagram:

RqZ ! RqY ×Z ! IB !X×Y ×Z dX×Y ×Z

X

/ RqZ IB !Y ×Z dY ×Z !

O

RqZ ! IB !X×Y ×Z dX×Y ×Z

X×Y

Y

/ IB !Z dZ .


71

Moreover, using the linearity of the integral, one gets the commutative diagram:

RqY ! IB !X×Y dX×Y

ˆ ⊗IB ᏻY

ˆ

X ⊗id

/ IB !Y dY ⊗IB ˆ ᏻY

projection

ˆ Y−1 IB ᏻY RqY ! IB !X×Y dX×Y ⊗q

cup product

cup product

RqY ! IB !X×Y dX×Y

X

/ IB !Y dY .

Theorem 5.3. Assume that X is a complex analytic manifold of dimension dX , and let aX : X → {pt} denote the canonical map. Then the morphism p d −p dX −→ D IB !X , IB !XX induced by adjunction from d −p p ˆ # I: aX! IB !XX dX ⊗IB !X −→ IB(C), X

is an isomorphism. Proof. Because the problem is local, it is sufficient to treat the case p = 0 and to show that the morphism R U ; IB !U dU −→ RL R c U ; IB ᏻU , IB(C) , obtained by adjunction from L ˆ R c U ; IB ᏻU −→ IB(C), # I: R U ; IB !U dU ⊗ X

is an isomorphism for any open interval U of CdU . This follows directly from Proposition 5.5 with V reduced to a point. Remark 5.4. As we will show elsewhere, the preceding theorem may be used to simplify the topological duality theory for coherent analytic sheaves. Proposition 5.5. Assume that U is an open interval of CdU and that V is an open interval of CdV . Then the canonical morphism ϕU,V : R U × V , IB !U ×V dU ×V −→ RL R c U ; IB ᏻU , R V ; IB !V dV ,

72


obtained by adjunction from

L ˆ R c U ; IB ᏻU # I: R U × V , IB !U ×V dU ×V ⊗ X −→ R V ; IB !V dV ,

is an isomorphism. Proof. Let W be an open interval of CdW , and assume that ϕU,V ×W and ϕV ,W are isomorphisms. Then we have successively

(1) (2) (3) (4)

R U × V × W ; IB !U ×V ×W dU ×V ×W RL R c U ; IB ᏻU , R V × W, IB !V ×W dV ×W RL R c U ; IB ᏻU , RL R c V ; IB ᏻV , R W ; IB !W dW L ˆ R c V ; IB ᏻV , R W ; IB !W dW RL R c U ; IB ᏻU ⊗ RL R c U × V ; IB ᏻU ×V , R W ; IB !W dW ,

where (1) and (2) follow from our assumptions, (3) is obtained by adjunction, and (4) comes from Corollary 4.5. Using Remark 5.2, we easily check that the composition of the preceding isomorphisms is equal to ϕU ×V ,W . Hence, an induction on dU reduces the problem to the case where dU = 1. This is dealt with in Proposition 5.6. Proposition 5.6. Assume that U is an open interval of C and that V is an open interval of Cn . Then the canonical morphism R U × V , IB !U ×V dU ×V −→ RL R c U ; IB ᏻU , R V ; IB !V dV is an isomorphism. Proof. For P = U , sheaf theory gives us the two distinguished triangles R ∂P ×V C × V , IB !C×V −→ R P ×V C × V , IB !C×V +1 −→ R U × V , IB !U ×V −−→ and +1 R c U, IB ᏻU −→ R P , IB ᏻC −→ R ∂P , IB ᏻC −−→, where ∂P denotes the boundary of P . If we apply the functor RL(·, IB(!V (V ))) to the last triangle, we obtain the morphism of distinguished triangles


R ∂P ×V C × V , IB ᏻC×V [1]

α

/ RL R ∂P , IB ᏻC , IB !V (V )

R P ×V C × V , IB ᏻC×V [1]

β

/ RL R P , IB ᏻC , IB !V (V )

R U × V , IB ᏻU ×V [1]

+1

γ

73

/ RL R c U, IB ᏻC , IB !V (V )

+1

where α and β are isomorphisms of the type considered in Proposition 5.7 (∂P is a finite union of closed intervals of C). It follows that γ is an isomorphism. Proposition 5.7. Assume that K is a finite union of closed intervals of C and that V is an open interval of Cn . Then the canonical morphism R K×V C × V ; IB !C×V dC×V −→ RL R K; IB ᏻC , R V ; IB !V dV , obtained by adjunction from

L ˆ R K; IB ᏻC # I: R K×V C × V ; IB !C×V dC×V ⊗ C −→ R V ; IB !V dV ,

is an isomorphism. Proof. Assume first that K is a closed interval of C. Since P × V is closed in C × V , we have the distinguished triangle R P ×V C × V , IB ᏻC×V −→ R C × V , IB ᏻC×V +1 −→ R C \ P × V , IB ᏻC×V −−→ . By Cartan’s Theorem B and Theorem 3.5, we have the isomorphisms R C × V , IB ᏻC×V IB ᏻC×V C × V and

R (C \ P ) × V , IB ᏻC×V IB ᏻC×V (C \ P ) × V .

Hence, the long exact sequence associated to the preceding distinguished triangle ensures that LH kP ×V C × V , IB ᏻC×V = 0 ∀k ≥ 2

74


and that the sequence 0 GF @A

/ LH 0 P ×V C × V , IB ᏻC×V

/ IB ᏻC×V (C × V )

/ IB ᏻC×V (C \ P ) × V

/ LH 1 P ×V C × V , IB ᏻC×V

ED BC /0

is strictly exact. Applying the functor IB to the sequence of Proposition 5.9, we get the split exact sequence 0 −→ IB ᏻC×V (C × V ) −→ IB ᏻC×V (C \ P ) × V (∗) −→ IB L b ᏻC (P ), ᏻV (V ) −→ 0 in Ᏽnd(Ꮾan). Therefore, LH 0P ×V C × V , IB ᏻC×V = 0 and

LH 1P ×V C × V , IB ᏻC×V IB L b ᏻC (P ), ᏻV (V ) .

Combining these results with Proposition 1.13, Theorem 2.13, Theorem 3.5, and Proposition 3.6, we obtain successively R P ×V C × V , IB ᏻC×V L IB ᏻC (P ) , IB ᏻV (V ) [−1] RL R P ; IB ᏻC , R V ; IB ᏻV [−1]. From Proposition 5.8, it follows easily that the canonical morphism R P ×V C × V ; IB !C×V dC×V −→ RL R P ; IB ᏻC , R V ; IB !V dV is an isomorphism. Now assume that the result has been established when K is a union of k < N closed intervals of C, and let us prove it when K=

N

Pi ,

i=1

where Pi (i = 1, . . . , N ) is a closed interval of C. Set L = N−1 i=1 Pi and Q = PN . By the Mayer-Vietoris theorem associated to the decomposition K = L∪Q, we have the


75

distinguished triangle R K, IB ᏻC R L, IB ᏻC ⊕ R Q, IB ᏻC R L ∩ Q, IB ᏻC . +1

Applying the functor RL(·, IB(!V (V ))), we obtain the distinguished triangle A = RL R L ∩ Q, IB ᏻC , IB !V (V ) B = RL R L, IB ᏻC ⊕ R Q, IB ᏻC , IB !V (V )

C = RL R K, IB ᏻC , IB !V (V ) . +1

Now consider the Mayer-Vietoris distinguished triangle A = R (L∩Q)×V C × V , IB !C×V B = R L×V C × V , IB !C×V ⊕ R Q×V C × V , IB !C×V

C = R K×V C × V , IB !C×V .

Since L∩Q = morphisms

N−1 i=1

(Pi ∩PN ) is a union of N −1 closed intervals of C, the canonical A [1] −→ A

and

B [1] −→ B

are isomorphisms. The canonical diagram A [1]

/ B [1]

/ C [1]

O A

O /B

/C

+1

+1

/

/

76


being commutative, the canonical morphism C [1] −→ C is also an isomorphism, and the conclusion follows. Proposition 5.8. Let P be a compact interval of C and let V be an open interval of Cn . Then : HP1 ×V C × V , !C×V −→ H 0 V , !V C

sends the class of ω = h(z, v) dz ∧ dv ∈ H 0 C \ P , !C×V

to

∂P

h(z, v) dz dv,

where P is a compact interval of C such that P ◦ ⊃ P . Proof. Let Ᏽ· be an injective resolution of !C×V . Let (v+1,·)

u· : Ꮿ∞,C×V −→ Ᏽ· denote a morphism extending id : !C×V → !C×V . The class c of ω in HP1 ×V C × V , !C×V H 1 P (C × V , Ᏽ· ) is represented by dσ , where σ ∈ (C × V , Ᏽ0 ) extends u0 (ω) ∈ ((C \ P ) × V , Ᏽ0 ). Let ϕ be a function of class C∞ on C equal to 1 on C \ P and equal to 0 on P , where P and P are compact intervals such that P ◦ ⊃ P and P ◦ ⊃ P . Then it is clear that u0 (ϕω) ∈ (C × V , Ᏽ0 ) and that σ − u0 (ϕω) ∈ c×V C × V , Ᏽ0 . Therefore, dσ and du0 (ϕ) give the same class in H 1 ( c×V (C × V , Ᏽ· )). It follows (v+1,·) that c corresponds to the class of ∂(ϕω) in H 1 ( c×V (C × V , Ꮿ∞,C×V )). Since c represents the image of c by the canonical map 1 HP1 ×V C × V , !C×V −→ Hc×V C × V , !C×V , we see that

C

c=

C

∂(ϕω) =

Hence, we have the conclusion.

P \P

∂ϕω =

∂P

ϕω −

∂P

ϕω =

∂P

ω.


77

Proposition 5.9. Let P be a closed interval of C, and let V be an open interval of Cn . Then in ᐀c, we have a split exact sequence of the form T r 0 −→ ᏻC×V (C × V ) −→ ᏻC×V (C \ P ) × V −−→ L b ᏻC (P ), ᏻV (V ) −→ 0, where r is the canonical restriction map and T is defined by setting h(z, v) g(z) dz T (h)(ϕ)(v) = ∂P

where g is a holomorphic extension of ϕ ∈ ᏻC (P ) on an open neighborhood U of P and P is a compact interval of C such that P ◦ ⊃ P and P ⊂ U . Proof. Note that the definition of T is meaningful since the right-hand side clearly does not depend on the choices of U , g, and P . It is also clear that the function T (h)(ϕ) is holomorphic on V and that the operator T is linear. Let us show that T is continuous. Let p be a continuous seminorm of Lb (ᏻC (P ), ᏻV (V )). We may assume that there is a bounded subset B of ᏻC (P ) and a compact subset K of V such that

p(τ ) = sup sup τ (ϕ)(v) , τ ∈ L b ᏻC (P ), ᏻV (V ) . ϕ∈B v∈K

For n > 0, set Un = {u ∈ C : d(u, P ) < 1/n}. By cofinality, we have ᏻC (P ) lim ᏻC (Un ).

− →

n>0

Moreover, for any n > 0, ᏻC (Un ) is a Fréchet space, and the restriction ᏻC (Un ) −→ ᏻC Un+1 is injective. Hence, by [8, Chapter IV, §19], there is n ∈ N and a bounded subset Bn of ᏻC (Un ) such that B ⊂ rUn (Bn ). Choosing a compact interval Pn of C such that Pn ◦ ⊃ P and Pn ⊂ Un , we see that

p T (h) ≤ sup sup h(z, v) g(z) dz ,

g∈Bn v∈K ∂Pn and we can find C > 0 such that

p T (h) ≤ C sup sup g(z) g∈Bn z∈∂Pn

sup

(z,v)∈∂Pn ×K

h(z, v) .

Let us consider the linear map S : L b ᏻC (P ), ᏻV (V ) −→ ᏻC×V (C \ P ) × V ,

78


defined by setting S(τ )(z, v) =

1 τ 2iπ

1 (v). z−u

Let us check that S is continuous. Consider a compact subset K of C \ P and a compact subset L of V . Since the set 1 BK = :z∈K z−u is bounded in ᏻC (C \ K), rP ,C\K (BK ) is a bounded subset of ᏻC (P ), and we have

S(τ )(z, v) ≤ 1 2π (z,v)∈K×L sup

sup

sup τ (f )(v) .

f ∈rP ,C\K (BK ) v∈L

For any τ ∈ Lb (ᏻC (P ), ᏻV (V )) and ϕ ∈ ᏻC (P ), there is an open U of C, containing P and g ∈ ᏻC (U ) such that ϕ = rU (g). Let K be a closed interval included in U and such that K ◦ ⊃ P , and let Ꮿ be the oriented boundary of K. For any v ∈ V , using the continuity of τ and Cauchy representation formula, we have 1 1 T S(τ ) (ϕ)(v) = (v)g(z) dz τ 2iπ Ꮿ z−u g(z) 1 dz (v) =τ 2iπ Ꮿ z − u = τ (g)(v) = τ (ϕ)(v). It follows that T # S = id or, in other words, that S is a section of T . Let us consider the continuous linear map R : ᏻC×V (C \ P ) × V −→ ᏻC×V (C × V ), defined as follows. Let h ∈ ᏻC×V ((C \ P ) × V ) and z ∈ C. Consider R > 0 such that z ∈ PR◦ = z : d(z, P ) < R . Then, for any v ∈ V , we set R(h)(z, v) =

1 2iπ

ᏯR

h(u, v) du, u−z

where ᏯR is the oriented boundary of PR . Since, for any f ∈ ᏻC×V (C × V ) and any (z, v) ∈ C × V , we have 1 1 r(f )(u, v) f (u, v) R r(f ) (z, v) = du = du = f (z, v), 2iπ ᏯR u − z 2iπ ᏯR u − z


79

we see that R # r = id. The map R is thus a retraction of r. Using a well-known result of homological algebra, the proof is complete if we show that r # R + S # T = id . To this end, consider h ∈ ᏻC×V ((C\P )×V ) and (z, v) ∈ (C\P )×V . Fix R > 0 such that z ∈ PR◦ and let ᏯR denote the oriented boundary of PR . Let Ꮿ be the oriented boundary of a closed interval K ⊂ PR such that z ∈ K and K ◦ ⊃ P . Letting R denote the oriented boundary of PR \ K ◦ and using Cauchy integral formula, we get 1 2iπ ᏯR 1 = 2iπ ᏯR 1 = 2iπ R = h(z, v).

(r # R + S # T )(h)(z, v) =

1 h(ξ, v) 1 dξ + T (h) (v) ξ −z 2iπ z−u 1 h(ξ, v) h(ξ, v) dξ + dξ ξ −z 2iπ Ꮿ z − ξ h(ξ, v) dξ ξ −z

Remark 5.10. The preceding result is a slightly more precise form of a special case of the Köthe-Grothendieck duality theorem (see [7], [3], [4]). 6. A holomorphic Schwartz kernel theorem (r,s)

Definition 6.1. Let X and Y be complex analytic manifolds. We define !X×Y to be the subsheaf of !r+s X×Y whose sections are the holomorphic differential forms that are locally a finite sum of forms of the type ωi,j dxi1 ∧ · · · ∧ dxir ∧ dyi1 ∧ · · · ∧ dyis , where x and y are holomorphic local coordinate systems on X and Y . (r,s)

Remark 6.2. Clearly (W ; !X×Y ) has a canonical structure of FN space for any (r,s)

open subset W of X × Y . Therefore, using Proposition 3.1, we see that IB(!X×Y ) is a sheaf with value in Ᏽnd(Ꮾan). Moreover, using Proposition 4.4, one can check easily that L s (r,s) ˆ IB !Y . IB !X×Y IB !rX Theorem 6.3. Assume that X, Y are complex analytic manifolds of dimension dX , dY . Then we have a canonical isomorphism −1 r ! (dX −r,s) IB !X×Y dX Rᏸ qX IB !X , qY IB !sY .

80


Proof. We have successively (1)

−1 r ! −1 r ! dY −s R ᏸ qX IB !X , qY IB !sY dY Rᏸ qX IB !X , qY D IB !Y −1 r −1 dY −s Rᏸ qX IB !X , D qY IB !Y −1 r IB !X , Rᏸ qY−1 IB !dYY −s , ωX×Y R ᏸ qX L dY −s ˆ IB ! , ωX×Y Rᏸ IB !rX Y (r,d −s) (2) , ωX×Y Rᏸ IB !X×YY (r,d −s) D IB !X×YY (dX −r,s) IB !X×Y (3) dX×Y , where ωX×Y denotes the dualizing complex on X × Y for sheaves with values in

Ᏽnd(Ꮾan). Note that (1) and (3) follow from Theorem 5.3 and that (2) comes from

Remark 6.2. As a consequence, we may now give Propositions 5.6 and 5.7 their full generality. Corollary 6.4. Let X, Y be complex analytic manifolds of dimension dX and dY . Assume that K is a compact subset of X. Then (dX −r,s) R K×Y X × Y ; IB !X×Y dX RL R K; IB !rX ; R Y ; IB !sY . Moreover, if X and Y are Stein manifolds and K is holomorphically convex in X, then these complexes are concentrated in degree 0 and isomorphic to IB L b !rX (K), !sY (Y ) . Proof. Transposing a classical result of the theory of abelian sheaves to sheaves with values in Ᏽnd(Ꮾan), we see that −1 Ᏺ, qY! Ᏻ RL R(K; Ᏺ); R(Y ; Ᏻ) R K×Y X × Y ; Rᏸ qX if Ᏺ and Ᏻ are objects of ᏿hv(X; Ᏽnd(Ꮾan)) and ᏿hv(Y ; Ᏽnd(Ꮾan)). This formula, combined with Theorem 6.3, gives the first part of the result. The second part follows from Proposition 3.6, Theorem 3.5 (using Remark 3.7), Theorem 2.13, and Proposition 1.13. Corollary 6.5. Let X, Y be complex analytic manifolds of dimension dX and dY . Then (dX −r,s) R X × Y ; IB !X×Y dX RL R c X; IB !rX , R Y ; IB !sY .


81

Proof. This follows directly from the general isomorphism −1 R X × Y ; Rᏸ qX Ᏺ, qY! Ᏻ RL R c (X; Ᏺ), R(Y ; Ᏻ) , which holds for any objects Ᏺ and Ᏻ of ᏿hv(X; Ᏽnd(Ꮾan)) and ᏿hv(Y ; Ᏽnd(Ꮾan)). Lemma 6.6. Let X be a complex analytic manifold of dimension dX , and let Y be a complex analytic submanifold of X of dimension dY . Then 0 LH k R Y IB ᏻX for k = dX − dY . Proof. Since the problem is local, it is sufficient to show that 0 LH k R {0}×V U × V ; IB ᏻU ×V for k = dX − dY if U and V are Stein open neighborhoods of 0 in CdX −dY and CdY . In this situation, {0} is a holomorphically convex compact subset of U , and from Corollary 6.4 we get that R {0}×V U × V ; IB ᏻU ×V dX − dY IB L b ᏻU ({0}), ᏻV (V ) . The conclusion follows directly. Theorem 6.7. For any morphism of complex analytic manifolds f : X → Y , we have a canonical isomorphism (0,d ) Rᏸ f −1 IB ᏻY , IB ᏻX δf−1 R )f IB !X×YY dY , where )f is the graph of f in X × Y and where δf : X → X × Y is the associated graph embedding. In particular, LH k Rᏸ f −1 IB ᏻY , IB ᏻX =0 for k = 0, and

R Ᏼom f −1 IB ᏻY , IB ᏻX Ᏸ∞ X→Y .

Proof. Using Theorem 6.3, we see that ! (0,d ) IB !X×YY dY Rᏸ qY−1 IB ᏻY , qX IB ᏻX . Applying δf! , we get successively ! (0,d ) IB ᏻX δf! IB !X×YY dY δf! Rᏸ qY−1 IB ᏻY , qX ! IB ᏻX Rᏸ δf−1 qY−1 IB ᏻY , δf! qX −1 ! IB ᏻY , qX # δf IB ᏻX R ᏸ qY # δ f Rᏸ f −1 IB ᏻY , IB ᏻX .

82


This gives the first part of the result. To get the second part, it is sufficient to use Lemma 6.6 if we remember that, following [16], we have (0,dY ) −1 Ᏸ∞ X→Y δf R )f !X×Y dY . Corollary 6.8. For any complex analytic manifold X of dimension dX , we have a canonical isomorphism (0,d ) Rᏸ IB ᏻX , IB ᏻX δ −1 R ) IB !X×XX dX , where ) is the diagonal of X × X and where δ : X → X × X is the diagonal embedding. In particular, =0 LH k Rᏸ IB ᏻX , IB ᏻX for k = 0, and

R Ᏼom IB ᏻX , IB ᏻX Ᏸ∞ X.

Remark 6.9. Note that the fact that continuous endomorphisms of ᏻX may be identified with partial differential operators of infinite order was conjectured by Sato and proved in [5]. The vanishing of the topological Ᏹxt k (k > 0) is, to our knowledge, entirely new. 7. Reconstruction theorem. Let ᏾ be a ring on X with values in Ᏽnd(Ꮾan) (i.e., a ring of the closed category ᏿hv(X; Ᏽnd(Ꮾan)) (see [18])). Let ᏹod(᏾) denote the quasi-abelian category formed by ᏾-modules. If ᏹ, ᏺ are two ᏾-modules, it is easy to see that ᏸ (ᏹ, ᏺ) is endowed with both a structure of a right ᏾-module and a compatible structure of a left ᏾-module. These structures give two maps / ᏸ (ᏹ , ᏺ ) / ᏸ ᏾ , ᏸ (ᏹ , ᏺ ) . As usual, we denote their equalizer by ᏸ ᏾ (ᏹ, ᏺ). In this way, we get a functor ᏸ ᏾ (·, ·) : ᏹod(᏾)op × ᏹod(᏾) −→ ᏿hv X; Ᏽnd(Ꮾan) , which is clearly continuous on each variable and in particular left exact. Using the techniques of [18], it is easy to see that ᏹod(᏾) has enough injective objects, and working as in [18, Proposition 2.3.10], one sees that the functor ᏸ ᏾ (·, ·) has a right derived functor op Rᏸ ᏾ (·, ·) : D − ᏹod(᏾) × D + ᏹod(᏾) −→ D + ᏿hv X; Ᏽnd(Ꮾan) . Now let Ᏹ be a sheaf on X with values in Ᏽnd(Ꮾan) and let ᏺ be an ᏾-module. Since ᏸ (Ᏹ, ᏺ) is canonically endowed with a structure of ᏾-module, we get a functor op ᏸ (·, ·) : ᏿hv X; Ᏽnd(Ꮾan) × ᏹod(᏾) −→ ᏹod(᏾).


83

We check directly that this functor may be derived on the right by resolving the first argument by a complex of K − (᏿hv(X; Ᏽnd(Ꮾan))) with components of the type Pi U i∈I

i

(where Pi is a projective object of Ᏽnd(Ꮾan) and Ui is an open subset of X) and the second argument by a complex of K + (ᏹod(᏾)) with flabby components. This gives us a derived functor op Rᏸ (·, ·) : D − ᏿hv X; Ᏽnd(Ꮾan) × D + ᏹod(᏾) −→ D + ᏹod(᏾) , which reduces to the usual Rᏸ functor if we forget the ᏾-module structures. Finally, recall that an object ᏹ of D b (ᏹod(᏾)) is perfect if there are integers p ≤ q such that for any x ∈ X, there is a neighborhood U of x with the property that ᏹ|U is isomorphic to a complex of the type 0 −→ ᏼp −→ · · · −→ ᏼq −→ 0, where each ᏼk is a direct summand of a free ᏾U -module of finite type. We denote b (ᏹod(᏾)) the triangulated subcategory of D b (ᏹod(᏾)) formed by perfect by Dpf objects. Proposition 7.1. Let ᏺ be a sheaf on X with values in Ᏽnd(Ꮾan) such that LH k Rᏸ (ᏺ, ᏺ) = 0 (k = 0), and let ᏾ be the ring ᏸ (ᏺ, ᏺ) of internal endomorphisms of ᏺ. Then ᏺ is an ᏾module, and the functor b ᏹod(᏾) −→ D b ᏿hv X; Ᏽnd(Ꮾan) Rᏸ ᏾ (·, ᏺ) : Dpf is well defined. Moreover, we have a canonical isomorphism Rᏸ Rᏸ ᏾ (ᏹ, ᏺ), ᏺ ᏹ b (ᏹod(᏾)). In particular, Rᏸ (·, ᏺ) identifies in D(ᏹod(᏾)) for any ᏹ ∈ Dpf ᏾ b b Dpf (ᏹod(᏾)) with a full triangulated subcategory of D (᏿hv(X; Ᏽnd(Ꮾan))). b (ᏹod(᏾)), it is clear that Proof. For any ᏹ ∈ Dpf

Rᏸ ᏾ (ᏹ, ᏺ) ∈ D b ᏿hv X; Ᏽnd(Ꮾan) since Rᏸ ᏾ (᏾, ᏺ) ᏺ. The canonical morphism L

ˆ Rᏸ ᏾ (ᏹ, ᏺ) −→ ᏺ ᏹ⊗

84


induces by adjunction a morphism

ᏹ −→ Rᏸ Rᏸ ᏾ (ᏹ, ᏺ), ᏺ .

If ᏹ ᏾, Rᏸ ᏾ (ᏹ, ᏺ) ᏺ and Rᏸ Rᏸ ᏾ (ᏹ, ᏺ), ᏺ Rᏸ (ᏺ, ᏺ) ᏸ (ᏺ, ᏺ) ᏾, and the preceding morphism is an isomorphism. It follows that it is also an isomorphism for ᏹ ᏾k and, hence, if ᏹ is a direct summand of a free ᏾-module of finite type. Thanks to the local structure of perfect complexes, the conclusion follows easily. Let us consider the two functors Iᐂ : ᐂ −→ Ᏽnd(Ꮾan) E −→

“lim” F − →

F ⊂E dim F 2, the corresponding problem was raised by several authors. Problem 1. Is every closed generic (2n − 2)-ball contained in a strictly pseudoconvex boundary in Cn , n > 2, removable? Note that the “simplest” nonremovable singularity (in the sense of Definition 2) is the boundary of a complex variety of codimension 1 contained and relatively closed in with “not too bad behaviour near ∂” (for more detailed information, see [Ch] and the aforementioned references on removable singularities). In well-behaved cases, the boundary of this variety is a closed manifold of codimension 3 in Cn , and this way we get the only nonremovable compact sets contained in codimension 3 submanifolds of Cn (see [ChSt], [Jö2]). Hence the problem is closely related to the following geometric problem. Problem 2. Can the boundary of a codimension 1 analytic subvariety of a strictly pseudoconvex domain be a smooth submanifold of a generic ball of real dimension 2n − 2 contained in ∂? Analytic varieties play an important role in complex analysis and geometry, so the question itself is interesting. The problem becomes much easier if we weaken the assumptions. It is not so hard to see that a generic (2n − 2)-manifold M in Cn with more complicated topology than balls (e.g., a manifold with infinite fundamental group) may contain the boundary of a codimension 1 analytic variety in such a way that the mentioned boundary bounds a domain on M. One such example for the case n = 3 was given in [Do]. Note that, in this example, the manifold M is not contained in the boundary of a strictly pseudoconvex domain. The easy part of our proof (see Step 2 of the proof of Lemma 2) gives an example of such a manifold M contained in the unit sphere of C3 . Note that, in the case n = 2, Theorem 0 implies that Problem 2 has a negative answer. In general, generic 2-balls in C2 may contain boundaries of 1-dimensional

A NONREMOVABLE 4-BALL

89

analytic varieties as was known long before from Wermer’s example [We]. Note also that for n ≥ 3, most of the suggestions were towards a negative answer to Problem 2 (and, moreover, towards a positive answer to Problem 1). For example, it is not difficult to see that (under the conditions imposed in Problem 2) the boundary of the mentioned analytic variety cannot be of class C 2 and diffeomorphic to a (2n−3)sphere. (Otherwise, there would exist a nonsingular vector field on the (2n − 2)-ball, which is transversal to the aforementioned (2n − 3)-sphere by [Fo].) A proof of a similar statement in a slightly more general situation is written, for example, in [Do]. Moreover, for any smooth generic (2n − 2)-ball contained in a strictly pseudoconvex boundary, there exist arbitrarily small (in certain C k -norm) perturbations that are removable. This assertion is true in much more general situations. (A generic CRmanifold that is imbedded into Cn and in general position consists of one CR-orbit.) The authors will give a proof in a forthcoming paper. Nevertheless, for dimensions n ≥ 3, Problem 2 has a positive answer. More precisely, we prove the following theorem. We let Bj denote the unit ball in Cj and we let ∂Bj denote its boundary. Theorem 1. There exist a smooth generic 4-ball Ꮾ4 imbedded into the unit sphere ∂B3 and a smooth (simple) zero-set X of an entire function in C3 which intersects ∂B3 transversally such that X ∩ ∂B3 ⊂ Ꮾ4 . As a corollary, we get the following theorem. Theorem 2. There exists a smooth generic nonremovable 4-ball Ꮾ4 contained in ∂B3 . Moreover, Ꮾ4 is not (L1 , ∂¯b )-removable either. Proof of Theorem 2. Indeed, let f be an entire function in C3 such that f −1 (0) =X and grad f = 0 on X. The function 1/f restricted to ∂B3 \ X is continuous and satisfies the tangential Cauchy-Riemann equations there. Moreover, by assumption, we have |1/f (z)| ≤ const · dist(z, X). Since X intersects ∂B3 transversally, it follows that 1/f is in L1 on ∂B3 with respect to 5-dimensional surface measure. But 1/f does not extend analytically to B3 , which would be the case if Ꮾ4 (and hence also X ∩ ∂B3 ) were removable or (L1 , ∂¯b )-removable, respectively. Acknowledgments. The authors are grateful to several people for useful discussions, especially concerning explanation of the material. Among them are A. Juhl, N. Kruzhilin, and O. Viro. O. Viro pointed out the reference on the isotopy extension theorem (our original proof differed slightly from the present one; it was longer and more technical, but a bit more explicit). N. Kruzhilin read the manuscript and gave useful comments. The present proof of Lemma 4 actually follows his advice. The rest of the paper is devoted to the proof of Theorem 1. 2. Proof of Theorem 1: A reduction. We start by guessing the manifold X. Dimca told us the following example of a family of smooth algebraic hypersurfaces

90

JÖRICKE AND SHCHERBINA

X in C3 whose intersections with the unit sphere are compact maximally complex 3-manifolds with group. For notational convenience, we work √ infinite fundamental √ with the ball 3B3 of radius 3 centered at zero instead of the unit ball and with its boundary def √ S5 = ∂ 3B3 . (1) Put X = (z1 , z2 , z3 ) ∈ C3 : z1 z2 z3 = ,

> 0,

(2)

and A = X ∩ S5 .

(3)

Lemma 1. A is empty for > 1 and is diffeomorphic to (S 1 )3 for 0 < < 1. For the latter choice of , it is a compact maximally complex CR-manifold. For = 1, it degenerates and is diffeomorphic to (S 1 )2 . Here S 1 is a circle (say, the unit circle in the complex plane). Proof. Use coordinates zj = rj ζj

with rj > 0, ζj ∈ S 1 .

(4)

In these coordinates, A is the direct product of the 2-torus ᐀2 = ζ1 , ζ2 , ζ1−1 ζ2−1 : |ζ1 | = |ζ2 | = 1 and the set ᏾ =

r 1 , r2 , r1 r2

: r1 > 0, r2 > 0,

2 def m(r1 , r2 ) = r12 + r22 + 2 2 r1 r2

(5) =3 .

The smooth function m has a minimum at ( 1/3 , 1/3 ) and m( 1/3 , 1/3 ) = 3 2/3 . All the other points are regular for m. Therefore, ᏾ is empty if > 1, it is a point if = 1, and it is diffeomorphic to a circle for < 1. √ Note that for = 1, the algebraic hypersurface X1 is tangent to S5 (= ∂( 3B3 )) at the points of A1 = ᐀2 ; hence for p ∈ ᐀2 , we have Tp X1 = Tpc S5 . Our goal is to find a smooth generic real 4-ball Ꮾ4 imbedded into S5 which contains A for certain < 1 close to 1. The following lemma reduces the problem to a simpler one. Lemma 2. Suppose that there is a C ∞ -smooth 4-ball B 4 imbedded into S5 which contains the 2-torus A1 = ᐀2 and has the following property. For each p ∈ ᐀2 ⊂ B 4 , the tangent space Tp B 4 coincides with the complex tangent space Tpc S5 of the sphere:


91

Tp B 4 = Tpc S5

(6)

for p ∈ ᐀2 .

Then there is a smooth generic 4-ball Ꮾ4 that contains A for some positive close to 1, < 1. Of course, B 4 is not generic (see (6)). Proof. Step 1. We want to change B 4 slightly near ᐀2 . The new 4-ball coincides near ᐀2 with a perturbation of the following auxiliary 4-manifold ᏹ4 = ᏹδ4 ,

ᏹδ4 = r1 ζ1 , r2 ζ2 , 3 − r12 − r22 ·ζ1−1 ζ2−1 : |ζ1 | = |ζ2 | = 1, |1−r1 | < δ, |1−r2 | < δ . Here δ > 0 is sufficiently small. ᏹδ4 contains A for close to 1; in particular, it contains ᐀2 and Tp ᏹδ4 = Tpc S5

for p ∈ ᐀2 .

In other words, ᏹδ4 and B 4 have first-order contact along ᐀2 . We use this later. It is convenient to write ᏹδ4 as a graph: ᏹδ4 = (z1 , z2 , z3 ) ∈ C3 : (z1 , z2 ) ∈ δ , z3 = Ᏻ(z1 , z2 ) ,

(7)

(8)

where δ =

r1 eiϕ1 , r2 eiϕ2 ∈ C2 : |1 − r1 | < δ, |1 − r2 | < δ

and

Ᏻ r1 eiϕ1 , r2 eiϕ2 =

3 − r12 − r22 · e−i(ϕ1 +ϕ2 ) .

(9)

(10)

The complex tangencies of ᏹδ4 are those points (z1 , z2 , Ᏻ(z1 , z2 )) for which both derivatives ∂z¯ 1 Ᏻ(z1 , z2 ) and ∂z¯ 2 Ᏻ(z1 , z2 ) vanish. (Recall that the complex tangencies of a manifold imbedded into Cn are the points for which the tangent space is complex. In our case these are exactly the nongeneric points of ᏹδ4 .) In polar coordinates this means that ((r1 ∂/∂r1 ) + (i∂/∂ϕ1 ))Ᏻ and ((r2 ∂/∂r2 ) + (i∂/∂ϕ2 ))Ᏻ vanish at those points. It is now easy to compute directly that the set of complex tangencies of ᏹδ4 is exactly ᐀2 . (There is another way to see that ᏹδ4 \᐀2 does not contain complex tangencies. Indeed, if p ∈ A ⊂ ᏹδ4 is a complex tangency, then, since Tp ᏹδ4 contains Tp A , it also contains the complexification of this space, namely, Tp X . This is impossible if < 1, since by a theorem of Forstneriˇc [Fo], Tp A ⊂ Tpc S5 for < 1 and for every p ∈ A .) In the next step, we make a small perturbation of ᏹδ4 , keeping ᏹδ4 \ ᏹδ41 fixed for an arbitrarily small δ1 > 0, δ1 < δ, in order to get a generic manifold. The perturbed ˜ 4 ) still contains a lot of A ’s. Moreover, it is arbitrarily close (say, ˜ 4 (= ᏹ manifold ᏹ δ δ,δ1 in C 1 ) to ᏹδ4 and is contained in S5 . (Note that the existence of such a perturbation

92


can also be obtained using the general h-principle of Gromov (see [Gr]). Instead we give a short explicit construction.) Step 2. Replace on δ1 the function Ᏻ by the following function Ᏻ1 : Ᏻ1 r1 eiϕ1 , r2 eiϕ2 = Ᏻ r1 eiϕ1 , r2 eiϕ2 · eig(r1 ,r2 ) (11) = 3 − r12 − r22 · e−i(ϕ1 +ϕ2 )+ig(r1 ,r2 ) , with a smooth real function g with small C 1 norm, compactly supported in {|1 − ¯ r1 | < δ1 , |1 − r2 | < δ1 }. The ∂-derivatives ∂z¯ 1 Ᏻ1 and ∂z¯ 2 Ᏻ1 of Ᏻ1 , multiplied by 2r1 e−iϕ1 and 2r2 e−iϕ2 , respectively, are equal to the following expressions: ∂ ∂ Ᏻ1 r1 eiϕ1 , r2 eiϕ2 +i r1 ∂r ∂ϕ1 1   −r 2 + 3 − r 2 − r 2  ∂g 1 1 2 = + ir1 3 − r12 − r22 · (r1 , r2 ) e−i(ϕ1 +ϕ2 )+ig(r1 ,r2 ) , (12)   ∂r1 3 − r2 − r2 1

2

∂ ∂ r2 +i Ᏻ1 r1 eiϕ1 , r2 eiϕ2 ∂r2 ∂ϕ2    −r 2 + 3 − r 2 − r 2  ∂g 2 1 2 = + ir2 3 − r12 − r22 · (r1 , r2 ) e−i(ϕ1 +ϕ2 )+ig(r1 ,r2 ) . (13)   ∂r2 3 − r12 − r22

Since 3 − r12 − r22 = r32 , we see that the first term in (12) vanishes if and only if r1 = r3 and the first term in (13) vanishes if and only if r2 = r3 . Moreover, the first terms in brackets are real and the second ones are imaginary. Now choose g with compact support in {|1 − rj | < δ1 , j = 1, 2} in such a way that at least one of the derivatives (∂/∂r1 )g or (∂/∂r2 )g does not vanish for r1 = r2 = 1. Then at all points ¯ of δ1 , at least one of the ∂-derivatives of Ᏻ1 does not vanish. Let Ᏻ1 = Ᏻ on δ \δ1 ˜ 4 be the graph of Ᏻ1 over δ . Then, as desired, ᏹ ˜ 4 ⊂ S5 , ᏹ ˜ 4 is generic, is and let ᏹ δ δ δ close (in C 1 ) to ᏹδ4 , and contains some A . Step 3. It follows from assumption (6) that for δ small enough, a part of the ball B 4 can be represented as the graph of a function z3 = ᏳB (z1 , z2 ) defined on the set δ . Take δ2 ∈ (δ1 , δ) close to δ, so that the graph of Ᏻ1 over δ2 \ δ1 still contains some A , and take δ3 ∈ (δ2 , δ). For a smooth nonnegative cut-off function χ that is 1 on ¯ δ2 and 0 on \δ3 , put Ᏻ2 = χ Ᏻ1 +(1−χ)ᏳB . Let Bˇ 4 denote the ball B 4 with the graph of ᏳB over δ replaced by the graph of Ᏻ2 over δ . By Thom’s transversality theorem (see [GoGu, Theorem 4.9] or [Hi]), there is a 4-ball B˜ 4 ⊂ S5 arbitrarily close (say, in C 1 ) to Bˇ 4 with only isolated complex tangencies which coincides with Bˇ 4 ¯ δ2 . over the set Shrinking B˜ 4 a little, we may assume that the number of complex tangencies is finite. Moreover, by construction, they are outside the relatively compact domain


93

bounded by some A on B˜ 4 . Remove from B˜ 4 a suitable small neighbourhood of a smooth curve which joins one complex tangency with a boundary point of B˜ 4 and does not meet A and the other complex tangencies. This decreases the number of complex tangencies by 1. After finitely many steps, we get the desired 4-ball Ꮾ4 . The following almost obvious lemma gives a further reduction. Lemma 3. To obtain the B 4 of Lemma 2, it is enough to find a C ∞ -smooth 3-ball B 3 ⊃ ᐀2 imbedded into S5 together with a smooth nonsingular normal vector field n on B 3 which is orthogonal to Tpc S5 at the points p ∈ ᐀2 . Orthogonality is always understood with respect to the usual Euclidean scalar product of Cn . Normality of n is meant with respect to the imbedding of B 3 into S5 , that is, n(p) ∈ Tp S5 , and n(p) is orthogonal to Tp B 3 . Proof. For each p ∈ B 3 ⊂ S5 , the set of vectors in Tp S5 which are normal to B 3 is a 2-plane. Choose an orientation on B 3 and S5 , and orient the 2-planes so that the orientations are compatible. For each p, we rotate the vector n(p) in the corresponding normal 2-plane in “positive” direction around the origin by the angle π/2 and denote the thus obtained vector by t(p). In other words, the pair (n(p), t(p)) gives the above chosen orientation on the normal plane, and t is a smooth, nonsingular normal vector field on B 3 orthogonal to n. Using exponential mappings, we may slightly extend B 3 in the direction of t and get a 4-ball B 4 , B 3 ⊂ B 4 ⊂ S5 . The obtained 4-ball has the following property: At points p ∈ B 3 ⊂ B 4 , the tangent space Tp B 4 is spanned by Tp B 3 and t; hence it is orthogonal to n. In other words, B 4 satisfies the conditions of Lemma 2. Lemmas 1, 2, and 3 reduce Theorem 1 to the following slightly easier assertion, which we state explicitly and prove in the following section. Proposition 1. There exists a C ∞ -smooth 3-ball B 3 imbedded into S5 which contains ᐀2 and a C ∞ -smooth nonsingular normal vector field n on B 3 which is orthogonal to Tpc S5 at the points p ∈ ᐀2 . 3. Proof of Proposition 1. First note that for p = eiϕ1 , eiϕ2 , e−i(ϕ1 +ϕ2 ) ∈ ᐀2 ,

(14)

the outer normal to S5 is the vector given by (eiϕ1 , eiϕ2 , e−i(ϕ1 +ϕ2 ) ). The vector def 1 N(p) = √ ieiϕ1 , ieiϕ2 , ie−i(ϕ1 +ϕ2 ) (15) 3 is of length 1 and orthogonal to the previous vector; hence it is tangential to the sphere S5 and orthogonal to the complex tangent space Tpc S5 . In other words, N may serve as the restriction of the desired vector field n to ᐀2 . The first step of the proof is to give an explicit description of a 3-sphere ᏿3 ⊂ S5 that contains ᐀2 and has the property that N is normal to ᏿3 at points of ᐀2 . The part

94


of the desired ᏿3 which is contained in S5 \ {z1 · z2 · z3 = 0} is in coordinates (4) the direct product of ᐀2 with a smooth curve (16) t −→ R(t) = R1 (t), R2 (t), R3 (t) , t ∈ (−1, 1), running in the r-variables. (We choose the functions Rj below.) In other words, let ᐀2 (t), t ∈ (−1, 1), denote the following 2-torus, which is isotopic in S5 to ᐀2 : def

᐀2 (t) =

R1 (t)eiϕ1 , R2 (t)eiϕ2 , R3 (t)e−i(ϕ1 +ϕ2 ) : eiϕ1 ∈ S 1 , eiϕ2 ∈ S 1 .

(17)

Then the part of the desired ᏿3 which is outside {z1 · z2 · z3 = 0} can be described by ᏿3 \ {z1 · z2 · z3 = 0} = ∪t∈(−1,1) ᐀2 (t).

(18)

Now choose a smooth curve R(t), t ∈ (−1, 1), with R12 (t) + R22 (t) + R32 (t) = 3,

(19)

which has the following crucial properties: (a) near −1, R3 (t) = R1 (t) and R1 (t) → 0 for t → −1; (b) near +1, R3 (t) = R2 (t) and R2 (t) → 0 for t → 1; (c) R1 (0) = R2 (0) = R3 (0) = 1, and hence ᐀2 (0) = ᐀2 . For example, R can be obtained in the following way. Define a curve R˜ that satisfies (19) and such that R˜ 3 (t) = R˜ 1 (t) if t ∈ (−1, 0] and R˜ 3 (t) = R˜ 2 (t) if t ∈ [0, 1). Then ˜ The curve R is obtained by making R˜ (a)–(c) are satisfied for R replaced by R. smooth near zero in a suitable way. For simplicity, we also assume that R satisfies the following condition (d). (d) The curve t → R(t), t ∈ (−1, 1), has nondegenerate projection (R1 (t), R2 (t)) onto the (r1 , r2 )-plane; hence the image R((−1, 1)) is the graph of the function R3 over a smooth curve in the (r1 , r2 )-plane. Moreover, near t = 0, the curve t → R(t) has nondegenerate projection onto the r1 -axis as well as onto the r2 -axis. Figure 1 shows an example of a curve R that satisfies all four conditions. Condition (a) means that for t = −1, the 2-torus ᐀2 (t) degenerates: We may put ᐀2 (−1) to be equal to the circle def

def

᐀2 (−1) = *1 =

√ iϕ iϕ 0, 3e 2 , 0 : e 2 ∈ S 1 .

Analogously, for t = +1, we have the degeneration def

def

᐀2 (1) = *2 =

√ iϕ 3e 1 , 0, 0 : eiϕ1 ∈ S 1 .

Joining the circles *1 and *2 to the set (18) described above, we get the desired def

᏿3 = ∪t∈(−1,1) ᐀2 (t) ∪ *1 ∪ *2 = ∪t∈[−1,1] ᐀2 (t).

(20)

95


r3

R(0) = (1, 1, 1)

R(t), t ∈ (−1, 1)

r2 R(+1)

R(−1)

r1 Figure 1

Over each point R(t) with t ∈ (−1, 1), we have a 2-torus in ζ -coordinates; over R(1) and R(−1), we have a circle. The degeneration of the 2-torus to a circle for t = +1 and t = −1 is crucial to obtain a 3-sphere. To prove that (20) actually defines a 3sphere, we consider its covering by two √ solid tori Z1 and Z2 which intersect along a “collar” Ꮿ. Here Z1 contains *1 = {(0, 3eiϕ2 , 0) : eiϕ2 ∈ S 1 }, and near *1 , the set Z1 is given by

def Zδ1 = z1 , 3 − 2|z1 |2 · eiϕ2 , z¯ 1 e−iϕ2 : z1 ∈ (1 − δ)D, eiϕ2 ∈ S 1 . (21) Here δ > 0 is small. D is the open unit disc in the complex plane, S 1 is its boundary, and (1 − δ)D is the disc of radius (1 − δ) centered at zero. Globally, Z1 is given by the formula

Z1 = a1 (|z1 |) · z1 , a2 (|z1 |) · 3 − 2|z1 |2 · eiϕ2 ,

a3 (|z1 |) · z¯ 1 · e−iϕ2 : z1 ∈ (1 + δ)D, eiϕ2 ∈ S 1 (22) with smooth positive functions aj on (1+δ)D, depending only on the radius |z1 | and equal to 1 on (1 − δ)D (see (21)). These functions are uniquely determined by the condition that Z1 lies on the sphere ᏿3 . Similarly,

Z2 = b1 (|z2 |) · 3 − 2|z2 |2 · eiϕ1 , b2 (|z2 |) · z2 ,

b3 (|z2 |) · z¯ 2 · e−iϕ1 : z2 ∈ (1 + δ)D, eiϕ1 ∈ S 1 , (23)

96


with smooth positive functions bj on (1+δ)D, depending only on the radius |z2 | and equal to 1 on (1 − δ)D. Then the collar Ꮿ = Z1 ∩ Z2 can be written as Ꮿ = R1 (t)eiϕ1 , R2 (t)eiϕ2 , R3 (t)e−i(ϕ1 +ϕ2 ) : t ∈ (−σ, σ ), eiϕ1 ∈ S 1 , eiϕ2 ∈ S 1 (24) for a suitable σ ∈ (0, 1). Here ᐀2 ⊂ Ꮿ = Z1 ∩ Z2 ; that is, aj (1) = bj (1) = 1 for j = 1, 2, 3. It is now clear that ᏿3 is actually a 3-sphere. Moreover, since Tp ᐀2 ⊂ Tpc S5 for p ∈ ᐀2 , it follows from (19) and (c) that N is orthogonal to ᏿3 at points of ᐀2 . Unfortunately, N does not extend to a nonsingular normal field on ᏿3 (otherwise, removing from ᏿3 a small closed 3-ball, we would be done by Lemma 3). This is easy to understand and is based on the following fact. Consider, for example, Z1 . Let ϕ0

ϕ0

Z1 2 be the set of points in Z1 with fixed argument ϕ2 = ϕ20 . Then Z1 2 is topologically a disc with nondegenerate projection onto the disc (1 + δ)D in the first coordinate; ϕ0

ϕ0

᐀2 ∩ Z1 2 projects onto the unit circle; the set *1 ∩ Z1 2 projects onto the origin. The set of nonvanishing normal vectors for Z1 in S5 retracts to a circle with nondegenerate projection onto the third coordinate. Indeed, a neighbourhood of Z1 on S5 can be

parametrized by

iϕ2 iϕ2 2 2 z1 , e , z3 −→ z1 , 3 − |z1 | − |z3 | · e , z3 ,

(25)

and Z1 can be given by an equation of the form z3 = h1 z1 , eiϕ2 = a3 |z1 | · z¯ 1 · e−iϕ2 .

(26) ϕ0

It is now easy to see that N has “nontrivial winding” along the circle ᐀2 ∩ Z1 2 . This is the obstruction for extending N. On the other hand, we see now that there is a smooth nonsingular normal vector field on ᏿3 . Lemma 4. On every 3-sphere S 3 that is smoothly imbedded into S5 , there exists a smooth nonsingular vector field m that is normal to S 3 (normal with respect to the imbedding of S 3 into S5 ). Proof. Let E be the bundle over S 3 of the unit vectors that are tangent to S5 and normal to S 3 . Our goal is to prove that there exists a smooth section of this bundle. Let B13 and B23 be a covering of S 3 by two smooth closed 3-balls such that B13 ∩ B23 is a smooth 2-sphere. We denote this 2-sphere by S 2 . Since the restriction of E to B13 is trivial, there exists a smooth section s1 of E over B13 . Since the fibers of E are diffeomorphic to the circle S 1 and since the restriction of E to B23 is also trivial, the restriction of s1 to S 2 defines (in the last trivialization) an element of the homotopy group π2 (S 1 ). Then, from the triviality of π2 (S 1 ), it follows that there

97


n(ζ )

τ

Figure 2

exists an extension s2 of s1 | S 2 to a smooth section of E over B23 . Define a section s of E over the whole of S 3 to be equal to s1 over B13 and equal to s2 over B23 . After smoothing s, if necessary, we obtain the required section of E, which is the desired smooth nonsingular normal vector field m on S 3 . The plan of the last step of the proof of Proposition 1 is the following. We replace the pair (Ꮿ, m) (the collar (see (24)) together with the restriction of the transversal vector ˜ ,m ˜ ) that is identical to the previous one outside field m to this collar) by a new pair (Ꮿ ˜ and m ˜ | ᐀2 = N (see a smaller tubular neighbourhood of ᐀2 , and moreover, ᐀2 ⊂ Ꮿ (15) for the definition of N). def ˜ ,m ˜ ) is the fact that N and n = m | ᐀2 are The key to the existence of the pair (Ꮿ homotopic as normal vector fields on ᐀2 imbedded into S5 . Indeed, we give below an explicit description of the homotopy N(t, ζ ), ζ ∈ ᐀2 , t ∈ [0, 1], of N and n, and we associate with this homotopy a homotopy of orthogonal transformations of the fibers of the normal bundle of ᐀2 . Using exponential mappings, this gives an isotopy of imbeddings of a small tubular neighbourhood of ᐀2 on S5 into S5 , which fixes ᐀2 . This isotopy describes how to bend a small part of the collar Ꮿ around ᐀2 to get a ˜ . The isotopy extension theorem (see below for more details) allows small part of Ꮿ ˜. us to get the whole collar Ꮿ For psychological reasons we include Figure 2, which is an example of how to ˜ around ᐀2 in dependence on the relation between n(ζ ) and N(ζ ). The figure bend Ꮿ ˜ with a normal to ᐀2 3-manifold through a given point shows the intersection of Ꮿ ζ ∈ ᐀2 . It corresponds to the case n(ζ ) = −N(ζ ). The aforementioned intersection is contained in the plane spanned by n(ζ ) and τ (ζ ) (compare with the notation used below for the description of the homotopy N(t, ζ )). The intersection of the old collar Ꮿ with the 3-manifold is the τ (ζ ) axis.

98


The following lemma realizes the plan of the proof of Proposition 1. Lemma 5. There is a diffeomorphism F of S5 which is identical on ᐀2 and outside a small tubular neighbourhood of ᐀2 such that: (a) F∗ (m) | ᐀2 = N; (b) the vector field N is orthogonal to F (᏿3 ) on ᐀2 . Here, as usual, F∗ (m) denotes the pushforward on vector fields induced by F . Lemma 5 implies Proposition 1. We get the desired 3-ball B 3 by removing from the smooth 3-sphere F (᏿3 ) a small closed 3-ball that does not meet ᐀2 . From the transversal vector field F∗ (m) on F (᏿3 ), we get the required normal vector field n on B 3 . Indeed, for each point p in F (᏿3 ), we take the orthogonal projection of the vector F∗ (m)(p) onto the normal space to F (᏿3 ). Since, by Lemma 5, F∗ (m) | ᐀2 = N is already orthogonal to F (᏿3 ), the normal field n coincides with F∗ (m) = N on ᐀2 . The proof of Proposition 1 (and hence of Theorem 1) is completed with the proof of Lemma 5. Proof of Lemma 5. Recall that the key for the existence of the diffeomorphism def F is the fact that N and n = m | ᐀2 are homotopic as normal vector fields on ᐀2 imbedded into S5 . We start with the explicit description of the homotopy N(t, ζ ), ζ ∈ ᐀2 , t ∈ [0, 1], of n and N. More precisely, we construct a smooth mapping N(t, ζ ), ζ ∈ ᐀2 , t ∈ [0, 1], such that for each ζ and t, the vector N(t, ζ ) is tangent to S5 and normal to ᐀2 and, moreover, N(0, ζ ) = n(ζ ) and N(1, ζ ) = N(ζ ). Let N be the bundle over ᐀2 of the unit vectors that are tangent to S5 and normal to ᐀2 . The fiber of this bundle over a point ζ ∈ ᐀2 is the unit 2-sphere S 2 (ζ ) in the 3-dimensional linear space which is tangent to S5 and normal to ᐀2 (the latter space considered as a real linear subspace of the tangent space Tζ C3 of C3 ). In the following we identify Tζ C3 and C3 . To describe the homotopy of N and n, we use the round metric on S 2 (ζ ) (i.e., the metric induced from the Euclidean metric on C3 ) and geodesics in this metric. Recall that a geodesic on S 2 (ζ ) in the round metric, which joins two points p1 and p2 , is an arc of the “big” circle, which is the intersection of S 2 (ζ ) with the 2-plane through the points 0, p1 , and p2 . We always take the shorter arc (in our case, it will always be a quarter, not three quarters of the big circle) without mentioning this explicitly further. Consider a unit vector field τ on ᐀2 which is tangent to ᏿3 and normal to ᐀2 (there are two such vector fields opposite to each other). Divide [0, 1] into two intervals of equal length, J1 = [0, 1/2] and J2 = [1/2, 1]. For t ∈ J1 = [0, 1/2], ζ ∈ ᐀2 , we let the point N(t, ζ ) move with some velocity s(t) (in contrast to the parametrization by length) along the geodesic from n(ζ ) to τ (ζ ), starting with n(ζ ) for t = 0 and reaching τ (ζ ) for t = 1/2. In other words, N(t, ζ ) is the point on the (shorter) geodesic between n(ζ ) and τ (ζ ) with geodesic t 1/2 distance 0 s(t ) dt from n(ζ ). Moreover, 0 s(t ) dt = π/2, which is the length of the quarter circle, the geodesic joining n(ζ ) and τ (ζ ).


99

The velocity s depends only on t ∈ [0, 1/2], not on ζ ∈ ᐀2 ; it is of class C ∞ , is positive on the interior of the interval, and vanishes of infinite order at the endpoints. For t ∈ J2 = [1/2, 1], we let the point N(t, ζ ) move on S 2 (ζ ) with the velocity s(t − (1/2)), t ∈ J2 , (s as defined above), along the geodesic from N(1/2, ζ ) = τ (ζ ) to N(ζ ). (For each ζ ∈ ᐀2 , this geodesic is a quarter of a big circle.) Here ζ is an arbitrary point of ᐀2 . Since the velocity s vanishes of infinite order at 0 and 1/2, we get a C ∞ map N : [0, 1] × ᐀2 → N such that N(t, ζ ) = n(ζ ) for t = 0, and N(t, ζ ) = N(ζ ) for t = 1. Define the homotopy R(t, ζ ), t ∈ [0, 1], of orthogonal transformations of the fibers of N associated with N(t, ζ ) in the following way. R(t, ζ ) = Id (the identity on S 2 (ζ )) for t = 0, ζ ∈ ᐀2 . For t ∈ [0, 1/2], define t R(t, ζ ) to be rotation of S 2 (ζ ) by the angle 0 s(t ) dt around the axis orthogonal to the 2-plane containing the geodesic in the direction from n(ζ ) towards τ (ζ ). R(t, ζ ) is chosen in such a way that N(t, ζ ) = R(t, ζ )n(ζ ). Note that R(1/2, ζ ) is rotation by the angle π/2 around the mentioned axis. t For t ∈ [1/2, 1], define R(t, ζ ) to be R(1/2, ζ ) followed by rotation by the angle 1/2 s(t − 1/2) dt around the new axis, which is now orthogonal to the 2-plane that contains the geodesic joining τ (ζ ) and N(ζ ). Note that R(1, ζ ) n(ζ ) = N(ζ ).

(27)

The exponential mapping applied in the normal directions to ᐀2 defines a diffeomorphism of a small neighbourhood of ᐀2 in the normal bundle of ᐀2 onto a small open subset of S5 which contains ᐀2 (a “tubular neighbourhood of ᐀2 in S5 ”). Composing the homotopy R(t, ζ ) with the aforementioned exponential mappings, we get a smooth isotopy of imbeddings of the mentioned tubular neighbourhood of ᐀2 into S5 . This isotopy fixes ᐀2 and is infinitesimally orthogonal on ᐀2 . The isotopy extension theorem (see [Hi, Theorem 1.4]) allows us to extend this isotopy to a diffeotopy of S5 . Indeed, put in the notation of [Hi] M = S5 , U equal to the mentioned tubular neighbourhood of ᐀2 and A equal to the closure of a smaller tubular neighbourhood of ᐀2 , A ⊂ U . The openness condition required in Theorem 1.4 for the isotopy of U is then satisfied. Let F be the diffeomorphism of the diffeotopy of Theorem 1.4 which is obtained for t = 1. It coincides with the identity on ᐀2 and outside some tubular neighbourhood of ᐀2 . By construction (recall that R(t, ζ ) are orthogonal transformadef tions of the fibers of N and (27) holds with n(ζ ) = m(ζ ), ζ ∈ ᐀2 ), it follows that F∗ (m) = N on ᐀2 and F (᏿3 ) is orthogonal to N on ᐀2 . This proves Lemma 5. References [Ch] [ChSt]

E. M. Chirka, Complex Analytic Sets (in Russian), “Nauka”, Moscow, 1985; English transl. in Math. Appl. (Soviet Ser.) 46, Kluwer Acad. Publ., Dordrecht, 1989. E. M. Chirka and E. L. Stout, “Removable singularities in the boundary” in Contributions to Complex Analysis and Analytic Geometry, Aspects Math. E 26, Vieweg,

100 [Di] [Do]

[Du]

[Fo] [FoSt] [GoGu] [Gr]

[Hi] [Jö1] [Jö2] [Jö3] [Lu] [St]

[We]

JÖRICKE AND SHCHERBINA Braunschweig, 1994, 43–104. A. Dimca, private communication. A. V. Domrin, On the spanning of maximally complex cycles by CR-submanifolds (in Russian), Mat. Zametki 60 (1996), 776–777; English transl. in Math. Notes 60 (1996), 582–583. J. Duval, “Surfaces convexes dans un bord pseudoconvexe” in Colloque d’Analyse Complexe et Géométrie (Marseille, 1992), Astérisque 217, Soc. Math. France, Montrouge, 1993, 6, 103–118. F. Forstneriˇc, Regularity of varieties in strictly pseudoconvex domains, Publ. Mat. 32 (1988), 145–150. F. Forstneriˇc and E. L. Stout, A new class of polynomially convex sets, Ark. Mat. 29 (1991), 51–62. M. Golubitsky and V. Guillemin, Stable Mappings and Their Singularities, Grad. Texts in Math. 14, Springer-Verlag, New York, 1973. M. L. Gromov, Convex integration of differential relations, I (in Russian), Izv. Akad. Nauk SSSR Ser. Mat. 37 (1973), 329–343; English transl. in Math. USSR-Izv. 7 (1973), 329–343. M. W. Hirsch, Differential Topology, Grad. Texts in Math. 33, Springer-Verlag, New York, 1976. B. Jöricke, Removable singularities of CR-functions, Ark. Mat. 26 (1988), 117–143. , Boundaries of singularity sets, removable singularities, and CR-invariant subsets of CR-manifolds, J. Geom. Anal. 9 (1999), 257–300. , Removable singularities of Lp CR-functions on hypersurfaces, J. Geom. Anal. 9 (1999), 429–456. G. Lupacciolu, Characterization of removable sets in strongly pseudoconvex boundaries, Ark. Mat. 32 (1994), 455–473. E. L. Stout, “Removable singularities for the boundary values of holomorphic functions” in Several Complex Variables: Proceedings of the Special Year Held at the MittagLeffler Institute (Stockholm, 1987/1988), Math. Notes 38, Princeton Univ. Press, Princeton, 1993, 600–629. J. Wermer, On a domain equivalent to the bidisk, Math. Ann. 248 (1980), 193–194.

Jöricke: Department of Mathematics, University of Uppsala, S-75106, Uppsala, Sweden; [email protected] Shcherbina: Department of Mathematics, University of Göteborg, S-412 96 Göteborg, Sweden; [email protected]

Vol. 102, No. 1


© 2000

GROUP ACTIONS ON S 6 AND COMPLEX STRUCTURES ON P3 ALAN T. HUCKLEBERRY, STEFAN KEBEKUS, and THOMAS PETERNELL

Contents 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 Setup and general results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 The case where G is semisimple . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 Main methods for the elimination of solvable groups . . . . . . . . . . . . . . . . . . . . . . 106 Elimination of the solvable groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 Proof of the C∗ × C∗ -action argument . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 Considerations concerning the C∗ -action argument . . . . . . . . . . . . . . . . . . . . . . . 111 The case of a discrete fixed-point set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 Proof of the C∗ -action argument if F is a curve . . . . . . . . . . . . . . . . . . . . . . . . . . 120 Some further remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

1. Introduction. This note is motivated by the following classical problem: Is there a complex structure on the 6-sphere S 6 , in other words, is S 6 a complex manifold? It has been known since the paper [BS] that all other spheres S 2n , n > 1, do not even admit an almost complex structure. On the other hand S 6 admits many almost complex structures; see the presentation in [Ste, Part III, Sect. 41.17]. It is generally believed that none of them is integrable. Suppose S 6 has a complex structure X. Then by [CDP] every meromorphic function on X is constant. Moreover X is not Kähler, since b2 (X) = 0. Therefore the problem is quite inaccessible by standard methods of complex geometry. In this paper we prove the following theorem. Theorem 1.1. X is not almost homogeneous. In other words, the automorphism group Aut ᏻ (X) does not have an open orbit. This is related as follows to the question of existence of complex structures on the underlying differentiable manifold of P3 (C): As above, assume X = S 6 has the structure of a complex manifold. For p ∈ X, let πp : Xp → X denote the blow-up of X at p with πp−1 (p) =: Ep . Of course Ep = P2 (C). Since sufficiently small neighborhoods of a hyperplane in Pn (C) are differentiably identifiable with neighborhoods Received 5 May 1999. 1991 Mathematics Subject Classification. Primary 53C15; Secondary 32M05, 32M12. 101

102

HUCKLEBERRY, KEBEKUS, AND PETERNELL

of a blown-up point, it follows that Xp is diffeomorphic to P3 (C). Suppose that for given points p, q ∈ X there exists a biholomorphic mapping ψ : Xp → Xq . Note that, since ψ(Ep ) generates the cohomology H 2 (Xq , Z), it follows that ψ(Ep ) ∩ Eq = ∅. If C := ψ(Ep ) ∩ Eq = Eq , then πq | ψ(Ep ) would be a modification that maps the curve C ⊂ ψ(Ep ) ∼ = P2 (C) to a point. Since this is impossible, ψ(Ep ) = Eq , and ψ induces an automorphism gψ : X → X with gψ (p) = q. Let Ᏺ denote the orbit space X/ Autᏻ (X). For ξ, η ∈ Ᏺ with ξ = η and p ∈ ξ, q ∈ η representatives, it follows that Xp and Xq are not biholomorphically equivalent. Of course Ᏺ may very well be non-Hausdorff and, from certain points of view, an unreasonable parameter space, but by abuse of language we nevertheless refer to it as a family of complex structures on P3 (C). Since Aut ᏻ (X) does not have an open orbit, its generic orbit in X is at least 1-codimensional. In particular, for general p, the semiuniversal deformation space of Xp is at least 1-dimensional. In this sense we have the following consequence of Theorem 1.1. Corollary 1.2. If S 6 admits a complex structure, then there is a 1-dimensional family of complex structures on P3 . Corollary 1.3. Let X be a complex structure on S 6 . Then X carries at most two linearly independent holomorphic vector fields: h0 (T X) ≤ 2. Proof. If the generic Aut ᏻ (X)-orbit is 2-dimensional, then there are globally defined holomorphic vector fields V1 and V2 such that U := {p ∈ X : V1 ∧ V2 (p) = 0} is a dense, Zariski-open subset. If V3 is any holomorphic field on X, then, since V1 ∧ V2 ∧ V3 ≡ 0, there exist m1 , m2 ∈ ᏻ(U ) with V3 = m1 V1 + m2 V2 on U . By explicit computation in local coordinates, one verifies that these coefficients are in fact meromorphic on X and the relation between the fields extends to X. On the other hand, it has been proven in [CDP] that X possesses only the constant meromorphic functions. Thus, it follows in this case that dimC (X, T X) = 2. The other cases, that is, those where the generic orbit dimension is smaller, are handled analogously. By semicontinuity we obtain from Corollary 1.3 the following corollary. Corollary 1.4. The inequality h0 (T X ⊗ L) ≤ 2 holds for generic L ∈ Pic0 (X). Our theorem should be seen in a more general context, or rather program, for investigating complex structures on S 6 . Namely, we would like to prove that the tangent bundle T X is stable with respect to a Gauduchon metric. It would then follow that X carries a Hermite-Einstein connection (see [LY]). At this point one could employ powerful analytic tools to investigate the problem further. In order to prove the stability, it would seem necessary to verify the statements: (A) H 0 (X, T X ⊗ L) = 0 for all L ∈ Pic0 (X); (B) H 0 (X, 1X ⊗ L) = 0 for all L ∈ Pic0 (X). Hence our theorem is the first approach to (A). The next step should be to investigate

GROUP ACTIONS ON S 6 AND COMPLEX STRUCTURES ON P3

103

group actions in general. We feel that the study of group actions on highly nonalgebraic manifolds is of independent interest, and we hope the methods that we develop in this paper are useful in a broader context. 2. Setup and general results. In this section we gather results that are used throughout the paper and give the general setup. 2.1. Setup and outline of proof. The entire paper is devoted to proving that a complex structure on S 6 cannot be almost homogeneous. We argue by contradiction. More precisely, we make the following assumption. Assumptions 2.1. Throughout the paper, X denotes a complex manifold whose underlying differentiable manifold is S 6 , and G denotes a connected, simply connected complex Lie group that acts holomorphically and almost effectively on X and has an open orbit G · x0 =: . The G-isotropy at x0 is always denoted by . Remark 2.2. Recall that the automorphism group Autᏻ (X) is a complex Lie group that acts holomorphically on X. Of course it may not be simply connected. However, its universal cover G is also a complex Lie group that is acting holomorphically. Thus, since we are willing to accept a discrete ineffectivity, I = {g ∈ G : g(x) = x, ∀x ∈ X}, the assumptions above are equivalent to assuming that Aut ᏻ (X) has an open orbit, that is, that X is almost homogeneous. The structure of the proof can be outlined as follows: Using the fact that X has only finitely many analytic hypersurfaces in connection with topology and Lie theory of the situation at hand, it is shown in Sections 3 that if G exists, then it must be solvable. In Section 4, we give a number of methods that are applied in Section 5 in order to rule out this case, too. Two of the methods involve a lengthy proof that we have preferred to give separately in Section 6 and Sections 7–9. The technical heart of this work lies in Sections 7–9 where we rule out the following situation: we suppose that there exists a subgroup C∗ < Aut ᏻ (X) and that E := X \ is an irreducible, nonnormal, rational surface where C∗ acts as an algebraic transformation group. The nonnormal locus N ⊂ E and its preimage N˜ in the normalization E˜ play a key role in the remainder of the proof. It follows from the Betti number information on X that N and N˜ are connected and have the same first Betti numbers. An analysis of the ˜ + 1 (see Sections 8–9). C∗ -action on E shows, however, that in fact b1 (N) = b1 (N) 2.2. Meromorphic functions, discrete isotropy, and the dimension of E. Since there are no nonconstant meromorphic functions on X by [CDP], we have that dim G = h0 (T X) = 3. For this, observe that if h0 (T X) ≥ 4, then KX has two linearly independent sections. Proposition 2.3. The G-isotropy at a point x0 ∈ is discrete. Furthermore, E := X \ is nonempty and purely 1-codimensional in X.

104


Proof. Since dim G = 3, it is clear that is discrete. In particular, since χTop (X) > 0, every vector field has zeroes and thus E = ∅. If the vector fields X1 , X2 , X3 form a basis of (X, T X), then E = {X1 ∧ X2 ∧ X3 = 0}. In particular, E is of pure codimension 1 and −KX = ᏻX irreducible components of E and λi > 0.

λi Ei , where Ei are the

Let G = R · S be the Levi-Malcev decomposition, so that R is the radical of G, that is, the maximal connected solvable normal subgroup of G and S is semisimple; moreover R ∩ S is discrete. Since a semisimple complex Lie group has dimension at least 3, we have only two cases; namely, that G is semisimple or solvable. 2.3. Topology of and E. Since the topology of X is well known, the Betti numbers of and E are closely related. Notation 2.4. If Y is a topological space, set bi (Y ) := hi (Y ; Q). Proposition 2.5. For q ∈ {1, . . . , 4}, the following equation of Betti numbers holds: bq (E) = b5−q (). Proof. Recall from algebraic topology that there is an exact cohomology sequence associated to the pair (X, E): · · · −→ H q (X; Q) −→ H q (E; Q) −→ H q+1 (X, E; Q) −→ H q+1 (X, Q) −→ · · · . Since X is homeomorphic to the 6-sphere, bq (X) = bq+1 (X) = 0 for all numbers 1 ≤ q ≤ 4, so that H q (E; Q) ∼ = H q+1 (X, E; Q). An application of the Alexander duality theorem yields H q+1 (X, E; Q) ∼ = H5−q (; Q) and, hence, the claim. 2.4. Fixed points of reductive groups. Many of our arguments involve linearization of group actions at fixed points. We recall the following theorem on faithful linearization. Theorem 2.6. Assume that a reductive complex Lie group H acts holomorphically on a complex manifold M, and assume that x ∈ M is an H -fixed point. For h ∈ H , let T (h) : Tx → Tx be the tangential map. Then there exist neighborhoods U of x and V of 0 ∈ Tx M and an isomorphism φ : U → V such that φ ◦ h = T (h) ◦ φ for all h in a given maximal compact subgroup of H . Furthermore, if W is a neighborhood of the maximal compact subgroup and U ⊂ U open so that W U ⊂ U , then (T (w) ◦ φ)(x) = (φ ◦ w)(x) for all x ∈ U . In this setting we call U a linearizing neighborhood of x. See [Huc] or [HO, p. 11f] for more information about linearization. The fixed-point set of a reductive group also possesses certain topological properties. In our special case this implies the following proposition.


105

Proposition 2.7. Suppose that Aut(X) contains a subgroup I ∼ = C∗ . Let F be ∼ the fixed-point set of I . Then either F = P1 , or F consists of two disjoint points. Proof. As a first point, note that it follows from the linearization theorem that F is smooth; in particular, its irreducible components are disjoint. Furthermore, χTop (F ) = χTop (X) (see [KP] for a proof in the algebraic setting which carries over immediately to, for example, compact complex spaces.) In our situation, for p ∈ F , the group I stabilizes the complement X \ {p} ∼ = R6 and thus F \{p} is acyclic (see [Bre]). Consequently, F consists of two points or it is irreducible. Thus it remains to show that F is at most 1-dimensional. Suppose F is 2-dimensional. Since H 4 (X, Z) = 0, it follows that c2 (T X|F ) = 0 and, since H 2 (X, Z) = 0, the normal bundle NF,X is topologically trivial. Consequently 0 = c2 (T X|F ) = c2 (F ) + NF,X · KF = χTop (F ), which is contrary to χTop (F ) = 2. 3. The case where G is semisimple. In this section we treat the case that G is semisimple. Since dim G = 3, it follows that G ∼ = SL2 . The most basic property of almost transitive SL2 -actions on 3-folds is found in the following lemma. Lemma 3.1. The G-action on X does not have a fixed point. In particular, if E˜ i is the normalization of Ei , then E˜ i is smooth. Proof. Assume that x ∈ X is a G-fixed point. Linearize the G-action at x and recall from the representation theory of SL2 that no 3-dimensional SL2 -representation space is SL2 -almost homogeneous (see [MU, Lemma 1.12] for a more detailed proof). This is a contradiction. Lemma 3.2. If Ei ⊂ E is an irreducible component, then G acts almost transitively on Ei . In particular, the normalization E˜ i is either rational or a Hopf surface. Proof. If G did not have an open orbit in Ei , then by Lemma 3.1 all G-orbits would be 1-dimensional. Thus the normalization E˜ i would be the product P1 (C)×C, where G operates transitively on the first factor and C is a smooth curve. In particular, it follows that a maximal torus T ∼ = C∗ would have two disjoint copies of C as a fixedpoint set in Ei . Note that, since the normalization map E˜ i → Ei is equivariant with respect to G ∼ = SL2 , the T -fixed-point set in Ei is the disjoint union of two curves, contrary to Proposition 2.7. The “In particular . . . ” clause of the statement results from the classification of the almost homogeneous surfaces; see [HO, p. 92] or [Pot]. Now we exclude both possibilities in the following proposition. Proposition 3.3. The group G is not semisimple.

106


Proof. It follows from Lemma 3.2 that the normalization E˜ i of a component of E is an almost homogeneous Hirzebruch surface, P2 (C) equipped with either the defining representation of SO3 (C) or the representation of SL2 with a fixed point or a homogeneous Hopf surface (see [HO] or [Pot]). Consequently, a maximal torus T ∼ = C ∗ < G has only isolated fixed points in X and, if Ei were rational, it would already have three or four fixed points in Ei alone. Thus we may assume that every such component is a homogeneous Hopf surface. But this is also not possible, because such a surface is a homogeneous space G/H , where H 0 is unipotent; that is, T has no fixed points. 4. Main methods for the elimination of solvable groups. We begin by presenting several methods that involve the normalizer of subgroups of isotropy groups. Recall that G is a connected, simply connected complex Lie group acting almost transitively on X with an open orbit = G · x0 . By Proposition 3.3 we may assume that G is solvable and, by the remarks in Section 2, that the isotropy := Gx0 is discrete. 4.1. The normalizer arguments. Throughout the paper if H is a subgroup of G, then N(H ) denotes its normalizer in G and N(H )0 its identity component. If H is discrete, it follows that N(H )0 = Z(H )0 , where Z(H ) denotes its centralizer. 4.1.1. The 2-dimensional normalizer argument. Note that as a subgroup of the discrete group , the ineffectivity I is itself discrete and, being normal, is therefore central. Thus, if H < is not normal in G, then it acts nontrivially on . Proposition 4.1 (2-dimensional normalizer argument). If H < is an arbitrary subgroup, then dim N(H ) = 2. Proof. Suppose not and note that the 2-dimensional orbit F := Z(H )0 · x0 is a component of the set of H -fixed points in . Since the full set X H of H -fixed points is closed, F is Zariski open in its closure F , which is 1-codimensional in X. Observe that {gF | g ∈ G} is an infinite set of hypersurfaces and consequently there exists a nonconstant meromorphic function on X (see [Kra, Thm. 1]1 ). However, such a function does not exist (see [CDP]). 4.1.2. The 1-dimensional normalizer argument Proposition 4.2 (1-dimensional normalizer argument). If H < is any subgroup and dim N(H ) = 1, then N(H )0 ∩ is a lattice of rank 2. Proof. The 1-dimensional group Z := Z(H )0 acts transitively on every component of the H -fixed-point set H ⊂ . These components are of course Zariski open in their closures. Set C := Z · x0 and note that if C = C, then C · E > 0, contrary to h2 (X; Z) = 0. Thus the orbit C is a compact 1-dimensional complex torus. 1 See

[FF] for a more general result.


107

4.2. Arguments involving reductive subgroups of Aut(X). In many cases we are able to rule out the existence of certain subgroups of Aut(X) under additional assumptions on the topology of . For convenience, use the following notation. Notation 4.3. If Y and Z are topological spaces, we say that Y has “Betti-type Z” if all the Betti numbers of Y and Z agree. 4.2.1. The C∗ × C∗ -action argument Proposition 4.4. If E is connected and has at least two irreducible components, then there does not exist an (effective) action of (C∗ )2 on X. The proof is given in Section 6. 4.2.2. The torus-action argument Proposition 4.5 (Torus-action argument). The automorphism group of X does not contain a compact complex torus. In particular, if < is an arbitrary subgroup that is normal in G and contained in a 1-dimensional Abelian subgroup A, then rank( ) = 1. Proof. Assume that T < Aut(X) is a compact complex torus. Since χTop (X) = 2, the Lefschetz fixed-point formula shows that every vector field must have zeros. In particular, T must have a fixed point in X. On the other hand, all representations of T are trivial, a contradiction to the theorem on faithful linearization! 4.2.3. The C∗ -action argument Proposition 4.6 (C∗ -action argument). If the open orbit ⊂ X has the same Betti numbers as the circle S 1 , then the automorphism group of X does not contain a subgroup that is isomorphic to C∗ . The proof of the preceding proposition turns out to be a rather involved matter. We give it in Sections 7–9. 4.3. Topological observations. Recall that G is solvable and simply connected, for example, it is a cell, and = G/ , where is discrete. Thus the topology of is completely determined by . It follows directly from Proposition 2.5 that = {e}. Restriction on the Betti type of Proposition 4.7 (Betti-type argument). The open orbit ⊂ X does not have the Betti type of a S 1 × S 1 . Proof. Assume to the contrary. Then, by Proposition 2.5, we have b4 (E) = 2 and b3 (E) = 1. Similar to the proof of Proposition 2.5, the sequence of the pair (E, ) and Alexander duality give 0 −→ H 0 (X, E; Q) −→ H 0 (X; Q) −→ H 0 (E; Q) −→ H 1 (X, E; Q) −→ · · · =H6 (;Q)=0

=Q

=H5 (;Q)=0

108


so that b0 (E) = 1. In particular, E is connected and has exactly two irreducible components E1 and E2 . Now E1 |E2 is a Cartier divisor that is effective, nonzero, but homologeously equivalent to zero, so that no desingularization of E2 is Kähler. By the Mayer-Vietoris sequence · · · −→ H 3 (E) −→ H 3 (E1 ) ⊕ H 3 (E2 ) −→ H 3 (E1 ∩ E2 ) −→ · · · , =0

we obtain b3 (E1 ) + b3 (E2 ) ≤ b3 (E) = 1. Hence we may assume that b3 (E1 ) = 0. Let ν : E˜ 1 → E1 be the normalization. Let N ⊂ E be the (set-theoretical) nonnormal locus and N˜ := ν −1 (N). The sequence · · · −→ H q (E1 ) −→ H q E˜ 1 ⊕ H q (N) −→ H q N˜ −→ · · · , q ≥ 1, yields b3 (E1 ) ≥ b3 (E˜ 1 ) so that b3 (E˜ 1 ) = 0. (See [BK, Prop. 3.A.7, p. 98] for an explanation of the sequence.) Now consider the minimal desingularization π : Eˆ 1 → E˜ 1 . Since R q π∗ (Z) is a skyp,q scraper sheaf with support only finitely many points for q > 0, E2 := H p (E˜ 1 , R q p,q ∼ p,q π∗ (Z)) = 0 for all p, q > 0. Thus, E2 = E∞ in the Leray spectral sequence, and since R 3 π∗ (Z) = 0 and H 3 (E˜ 1 , Z) = 0, we obtain b3 (Eˆ 1 ) = 0. Thus b1 (Eˆ 1 ) = 0, and the classification of surfaces yields that Eˆ 1 is Kähler2 . This is a contradiction!

5. Elimination of the solvable groups. Our goal here is to eliminate the possibility that X is almost homogeneous with respect to the action of a solvable group. The proof requires a bit of the special knowledge given by the classification of the simply connected 3-dimensional solvable Lie groups. 5.1. Classification of the relevant Lie groups. We now recall the classification mentioned above (see, e.g., [Jac]). In this case, G is biholomorphic to C3 as a complex manifold and the group structure is isomorphic to one of the following: G0 : This is the well-known Abelian group C3 . G1 : We could also denote this group by G2 (0). The multiplication is given as a1 , b1 , c1 a2 , b2 , c2 = a1 + a2 , ea1 b2 + b1 , c1 + c2 . G2 (τ ): Here τ is any complex number other than zero. The multiplication is a1 , b1 , c1 a2 , b2 , c2 = a1 + a2 , ea1 b2 + b1 , eτ a1 c2 + c1 . G3 : The multiplication is a1 , b1 , c1 a2 , b2 , c2 = a1 + a2 , ea1 b2 + b1 + a1 ea1 c2 , ea1 c2 + c1 . 2 Actually

are Kähler.

it is easy to see that κ(Eˆ 1 ) = −∞ so that we do not have to use the fact that K3-surfaces


109

H3 : This is the Heisenberg group, where the multiplication is

a1 , b 1 , c 1 a 2 , b 2 , c 2 = a 1 + a 2 , b 1 + b 2 + a 1 c 2 , c 1 + c 2 .

For a detailed study of the discrete subgroups of such groups, see [ES, Sect. 1]. 5.2. The elimination. Here we eliminate the possibility of an action of a solvable group by utilizing a general strategy along with some knowledge of the Lie groups that occur. Proposition 5.1. If is normal in G and A < G is any closed connected Abelian subgroup, then ⊂ A. In particular, the group G is not isomorphic to G0 ∼ = C3 . Proof. Assume to the contrary, that is, that ⊂ A. Since G/A is acyclic, G/ has the same homotopy type as A/ , that is, the same homotopy type as a real torus. Our argument depends on rank(). rank() = 1: Then has the Betti type of the circle S 1 . Take a 1-dimensional subgroup A < G, A ∼ = C with ⊂ A and observe that A / ∼ = C∗ acts nontrivially on X, contrary to the C∗ -action argument in Proposition 4.6. rank() = 2, 3, 4: Betti number considerations show that the assumptions of Proposition 4.4 are satisfied. However, the assumption on rank() together with the torusaction argument allow us to construct a (C∗ )2 -action coming from the group A. rank() = 5: Here G = A is Abelian and has two ends so that Proposition 4.4 may not be applied. Let E0 ⊂ E be an irreducible component and x0 ∈ E0 a generic point so that the dimension of the G-orbit through x0 is maximal. We consider the isotropy group Gx0 , which is positive-dimensional. Let H < Gx0 be a closed 1dimensional connected subgroup and note that, since rank() = 5, H ∩ = {e}: otherwise the image of would be a lattice of rank 5 in G/H ∼ = C2 . Thus, we obtain ∗ ∼ an action of H /( ∩ H ) = C on X which fixes the closure of the G-orbit G · x0 pointwise. If G · x0 is a point, then G, and thus H , fix E0 pointwise, contrary to Proposition 2.7. If G · x0 is 1-dimensional and closed, then it is an elliptic curve. In this case Fix(H ) also does not have the required properties of Proposition 2.7. Finally, if G.x0 is 1-dimensional and is not closed, then G0x0 ∩ has rank 4. In this case, after moding out by ineffectivity, the 2-dimensional compact complex torus T := G0x0 /(G0x0 ∩ ) acts holomorphically on X with a nonempty fixed-point set. Thus, contrary to almost ineffectivity of the G-action, it acts trivially; see the torusaction argument in Proposition 4.5. This finishes the proof. A weaker statement holds if is not necessarily normal. Lemma 5.2. For all groups A < G with A ∼ = C, we have ⊂ A. Proof. Assume to the contrary. Again the Betti types of G/ and of A/ agree. We may assume that is not normal.

110


Since A is contained in the normalizer of , the 2-dimensional normalizer argument implies that A = NG ()0 . In this setting the 1-dimensional normalizer argument yields that ∼ = Z2 , contrary to the Betti-type argument. We need the following technical lemma on centralizers of elements in G. ∼ G1 , G2 (τ ), G3 , or H3 and g ∈ G, then dim ZG (g) ≥ 1 and Lemma 5.3. If G = one of the following holds: (1) dim ZG (g) ≥ 2, or (2) ZG (g)0 ⊂ (0, C, C), or (3) ZG (g)0 = ZG (g), that is, ZG (g) is connected. Here ZG (g) denotes the set of elements commuting with g. Proof. The statement that dim ZG (g) ≥ 1 can be checked directly. To show that one of (1), (2), or (3) holds, for any number a ∈ C define the set Za := (b, c) ∈ C2 | (a, b, c) ∈ G commutes with g . Note that it is sufficient to show that, for all a, either dim Za > 0 or #Za ∈ {0, 1} holds. The last statement follows by an elementary calculation of commutators. Now we start with the the following proposition. Proposition 5.4. The group G is not solvable. Proof. By Proposition 5.1, we may assume that G ∼ = G1 , G2 (τ ), G3 , or H3 . If G∼ = G1 or H3 , then for all g ∈ G we have that dim ZG (g) ≥ 2. Thus, if ZG ⊂ G denotes the centralizer of G, then, by the 2-dimensional normalizer argument, < ZG . A direct calculation shows that ZG is contained in a connected Abelian subgroup, contrary to Proposition 5.1. Now assume that G ∼ = G2 (τ ) or G3 . Note that G ∼ = A G where G = (0, C, C) is the commutator group of G, and A as well as G are connected and Abelian. Furthermore, dim ZG (g ) = 2 for all g ∈ G . Thus, by the 2-dimensional normalizer argument, ∩ G = {e}, and the projection π1 : G → A is an injective group morphism, if restricted to . This shows that is Abelian, which in turn implies that < ZG (γ ) for all γ ∈ . If dim ZG (γ ) ≥ 2 for all γ ∈ , then is again central and contained in a connected Abelian subgroup. Thus the same argument as above applies. Thus we may assume that there exists a γ ∈ with dim ZG (γ ) = 1. If ZG (γ ) is connected, then ⊂ ZG (γ ) and we obtain a contradiction to Lemma 5.2. If ZG (γ ) is not connected, then Lemma 5.3 asserts that ZG (γ )0 ⊂ G . But this is also not possible: use the 1-dimensional normalizer argument to see that there exists an element γ ∈ ∩ G . However, we have already ruled out this possibility in the previous paragraph. This finishes the proof of the main Theorem 1.1 up to the proof of Propositions 4.4 and 4.6.


111

6. Proof of the C∗ × C∗ -action argument. In this section we carry out the proof of the C∗ × C∗ -action argument in Proposition 4.4. Proof of Proposition 4.4. For ease of notation, let T ∼ = = C∗ ×C∗ act on X and C∗ ∼ H < T be a subgroup. By Proposition 2.7 there exists an H -fixed point y0 ∈ X. Since its G-isotropy is positive-dimensional, it follows directly from the characterization of Proposition 2.3 that y0 ∈ , that is, y0 ∈ E. Let E0 ⊂ E be any irreducible component containing y0 , let η : E˜ 0 → E0 be the normalization, and let ρ : Eˆ 0 → E˜ 0 be a T -equivariant desingularization. Let E1 be a component of E with E0 ∩ E1 = ∅. It follows that the surface Eˆ 0 contains an effective divisor that is numerically equivalent to zero, namely, (η ◦ ρ)∗ (E1 ). This implies that the algebraic dimension a(Eˆ 0 ) is at most 1. Observe that if T has a fixed point x0 ∈ Eˆ 0 , then linearization at x0 shows either that the T -action on Eˆ 0 is almost transitive or that the T -action on Eˆ 0 contains a positive-dimensional ineffectivity C∗ ∼ = I < T . Notice that the latter possibility is ruled out by Proposition 2.7. Assume that Eˆ 0 is T -almost homogeneous but contains no T -fixed points. In this case, for x ∈ (η ◦ ρ)−1 {y0 } the orbit T · x is closed and 1-dimensional, that is, an elliptic curve that is blown down by ρ to an isolated boundary point of the open T -orbit in E˜ 0 . It follows that E˜ 0 is a homogeneous cone over P1 (C) (see [HO]). This is again contrary to a(Eˆ 0 ) ≤ 1. Thus we may assume that all T -orbits are 1-dimensional. It follows that, for every x ∈ Eˆ 0 , the analytic set Fix(Tx ) = x ∈ Eˆ 0 | tx = x for all t ∈ the isotropy group Tx is 1-dimensional and a finite union of T -orbits. In particular every T -orbit is closed, that is, an elliptic curve. Since there are infinitely many such orbits, a(Eˆ 0 ) = 1. Let h : Eˆ 0 → C denote the algebraic reduction. Since its fibers are connected, h is T -equivariant. Furthermore, all curves are contained in h-fibers. Thus the T -orbits, which are now known to be 1-dimensional and closed, are components of h-fibers; that is, the h-fibers are exactly the T -orbits. In particular, no T -orbit can be blown down by ρ and Eˆ 0 = E˜ 0 . Now H ∼ = C∗ has ˜ a fixed point z ∈ E0 . But this implies that FixE˜ 0 (H ) contains the elliptic curve T · z, contrary to Proposition 2.7. Thus the final case has been eliminated and the proof is complete. 7. Considerations concerning the C∗ -action argument. The remainder of the paper is devoted to proving Proposition 4.6. As was indicated at the beginning of Section 2, this completes the proof of our main theorem, that is, that X is not almost homogeneous. Accepting the situation presented to us by Proposition 4.6, we operate here under

112


the assumptions that has the Betti type of S 1 and that there exists an effective C∗ action on X. We begin by deriving some topological consequences of the assumption on the Betti type of . 7.1. Topological constraints. At this place, it seems advisable to fix some notation that is used in the sequel. Notation 7.1. Let ν : E˜ → E be the normalization and π : Eˆ → E˜ be the minimal desingularization. Write δ := π ◦ν. Let N ⊂ E be the nonnormal locus equipped with the complex conductor ideal structure and N˜ := ν −1 (N) the analytic preimage. ˜ and E are Cohen-Macaulay space; see [Mor, p. 166, 3.34(2)] for Notice that N , N, a proof. It follows from Serre’s criterion that the nonnormal locus N ⊂ E must be of ˜ pure dimension 1. The same holds for N˜ ⊂ E. Proposition 7.2. The divisor E is irreducible and not normal. Its normalization E˜ has only rational singularities and the minimal desingularization Eˆ is rational. In particular, E˜ is Q-factorial. Proof. Since has Betti-type S 1 , it follows from Proposition 2.5 that b4 (E) = 1; that is, E is irreducible. Suppose that E is normal. Since b3 (E) = 0, we have ˆ = 0 (see the proof of Proposition 4.7); hence b1 (E) ˆ = 0, and Eˆ is Kähler. b3 (E) ∗ ∗ ˆ ≤ 0, Since KEˆ ⊂ π (KE ) and π (KE ) is linearly equivalent to zero, we have κ(E) ˆ ˆ and by b3 (E) = 0, E is either rational or birational to a K3-surface or an Enriques surface. But because Eˆ possesses a C∗ -action, the latter cases are excluded. So Eˆ (and hence E) are rational surfaces. As a consequence, note that R 1 π∗ (Q) = 0 by the Leray spectral sequence and ˆ Q = H 2 (E, Q) = 0. H 1 E, ˆ Q) = H 0 (E, R 2 π∗ (Q)) and we conclude that H 2 (E, ˆ Q) is generated Thus H 2 (E, by the π -exceptional curves. ˆ Then we find m ∈ N, λi ∈ Z, and π-exceptional Now take an ample divisor A on E. ˆ curves Ci ⊂ E such that

c1 ᏻEˆ (mA) = c1 ᏻEˆ λi Ci . Since Eˆ is rational, we have the linear equivalence mA = λi Ci , which is absurd. Consequence: E is not normal. Using the formula ωE˜ = ν ∗ (ωE ) − N˜ and the fact that N˜ is an effective Weil divisor supported on the preimage of the nonnormal locus (observe that E is a Gorenstein space!), the “old” arguments still apply and give the rationality of E. In order to show that E˜ has only rational singularities, we check R 1 π∗ ᏻEˆ = 0.


113

Since H 1 (ᏻEˆ ) = 0, the Leray spectral sequence yields an embedding H 0 R 1 π∗ ᏻEˆ −→ H 2 ᏻE˜ .

Now E˜ is a Cohen-Macaulay space and therefore H 2 (ᏻE˜ ) ∼ = H 0 (ωE˜ ). Since ωE˜ ⊂ ∗ ∗ 2 ν (ωE ) and ωE˜ = ν (ωE ) = ᏻE˜ , we have H (ᏻE˜ ) = 0. Therefore R 1 π∗ (ᏻEˆ ) = 0. ˜ and Proposition 7.3. The following Betti numbers are equal: b0 (N) = b0 (N) ˜ b1 (N ) = b1 (N). Proof. We use the following Mayer-Vietoris sequence for reduced cohomology: ˜ Q) ⊕ H˜ q (N; Q) · · · −→ H˜ q (E; Q) −→ H˜ q (E; ˜ Q) −→ H˜ q+1 (E; Q) −→ · · · −→ H˜ q (N; (see [BK, Prop. 3.A.7, p. 98] for information about this sequence). ˜ Q) = h0 (N; Q), since H 1 (E; Q) = 0 by Proposition 2.5. Furthermore So h0 (N; ˜ ˜ − b1 (N ), since H 2 (E; Q) = 0. b1 (E) = b1 (N) Lemma 7.4. The space N is connected. Proof. Using the fact that E is a Cohen-Macaulay space and actually a Gorenstein space, by [Mor, p. 166, 3.34(2)] there exists an exact sequence −1 0 −→ ᏻE −→ ν∗ ᏻE˜ −→ ωE ⊗ ωN −→ 0. Since c1 (ν ∗ (ωE )) = 0 so that ν ∗ (ωE ) = ᏻE˜ , 0 −→ ωE −→ ωE ⊗ ν∗ ᏻE˜ −→ ωN −→ 0 =ν∗ ν ∗ (ωE )=ν∗ (ᏻE˜ ) is also exact. Since N is a Cohen-Macaulay space, Serre duality holds. Thus, the associated long cohomology sequence gives ˜ ᏻ ˜ −→ H 1 (N, ωN ) −→ H 2 (E, ωE ) −→ · · · · · · −→ H 1 E, E =0

∼ =H 0 (N,ᏻN )

∼ =H 0 (E,ᏻE )∼ =C

so that h0 (N, ᏻN ) ≤ 1, and the reduced subspace Nred is connected. 7.2. Orbits of the C∗ -action. For the sake of completeness we outline some known facts of C∗ -actions on rational surfaces: The orbits are always constructible. As a consequence we see that E necessarily contains attractive and repulsive fixed points, a fact that is crucial in the sequel. More information can be found in the works of Białynicki-Birula and of Sommese (see, e.g., [BBS]).

114


Lemma 7.5. If H is a linear algebraic group and ι : C∗ → H is a holomorphic map that is a (set-theoretical) group morphism, then ι is algebraic and, in particular, the image ι(C∗ ) is a closed algebraic subgroup of H . Proof. Let J < H be the Zariski closure of ι(C∗ ) in H . It is immediate that J is connected and Abelian and, since it is affine, J ∼ = Cn × (C∗ )m . Now use the fact that there is no nonconstant holomorphic group morphism from C∗ to C and that any holomorphic group morphism from C∗ to C∗ is given by z → zk . Lemma 7.6. Let S be a (possibly singular) irreducible compact surface with a holomorphic action of C∗ . Suppose additionally that S is rational. Then: (1) All C∗ -orbits are constructible, and their closures are rational. In particular, for all x ∈ S, the limits limλ∈C∗ , λ→0 λ · x and limλ∈C∗ , λ→∞ λ · x exist. (2) If F ⊂ S is the set of the C∗ -fixed points, then there are two irreducible subspaces F0 , F∞ ⊂ F (not necessarily distinct) and a Zariski-open set U ⊂ S such that for all x ∈ U : lim

λ∈C∗ , λ→0

λ · x ∈ F0 ,

lim

λ∈C∗ , λ→∞

λ · x ∈ F∞ .

Proof. Suppose for the moment that S is smooth. Then Aut(S) is linear algebraic and acts algebraically; this is a consequence of the fact that, since b1 (S) = 0, S can be equivariantly embedded into some Pn —see [Bla] for a proof. By Lemma 7.5, any closed subgroup C∗ < Aut(S) is linear algebraic and acts algebraically. In particular, C∗ -orbits are constructible, and all C∗ -stable curves in S contain fixed points. This already proves (1). In order to prove assertion (2), embed C∗ → P1 in the usual way. Then there exists a rational morphism φ : P1 × S S. Since the set of fundamental points of φ −1 is of codimension ≥ 2, there exists an open set U ⊂ S such that φ|P1 ×U is regular. Consequence: for all x ∈ U the limits limλ∈C∗ , λ→0 λ · x and limλ∈C∗ , λ→∞ λ · x exist. Set F0 = φ(0 × U ) and F∞ := φ(∞ × U ). If S is singular, then let δ : Sˆ → S be an equivariant resolution and find Fˆ0 , ˆ F∞ , and Uˆ ⊂ Sˆ as above. It is sufficient to set F0 := δ(Fˆ0 ), F∞ := δ(Fˆ∞ ), and U := δ(U ) ∩ Sreg , where Sreg is the (open) set of regular points in S. 8. The case of a discrete fixed-point set. Our goal here is to prove the C∗ -action argument under the assumption that the set F of C∗ -fixed points is discrete. We have seen in Proposition 2.7 that F necessarily consists of two distinct points, which we denote by F0 and F∞ . Since its G-isotropy contains C∗ , it follows again from Proposition 2.3 that F0 , F∞ ∈ E. It is shown that the discreteness assumption, which ˜ (see Proposition 8.21). is made throughout this section, is contrary to b1 (N) = b1 (N)


115

Notation 8.1. In the sequel, U0 and U∞ are disjoint linearizing neighborhoods of the points F0 and F∞ in X. Since E ∩ U0 and E ∩ U∞ are closed in U0 and U∞ , respectively, after shrinking U0 and U∞ we can assume that E ∩ U0 and E ∩ U∞ are connected and F0 (resp., F∞ ) is contained in every component. 8.1. Symmetry lemmas. Here we prove several lemmas which show that the situations at F0 and at F∞ are very similar. First, we investigate the following. 8.1.1. Weights of the C∗ -actions on TF0 X and TF∞ X. The main result of this section is the following proposition. Proposition 8.2. The locally closed spaces E ∩ U0 and E ∩ U∞ are reducible. One of the components of each space is smooth, and the C∗ -action has a totally attractive (resp., repulsive) fixed point there. Since the proof is somewhat lengthy, and we use a number of partial results later, we subdivide the proof into a sequence of lemmas and corollaries. Lemma 8.3. After swapping F0 and F∞ and, if necessary, replacing the C∗ -action by (C∗ )−1 , the weights of the C∗ -action on the tangent space TF0 X have the following signs: (+ + −). Proof. By symmetry, we only have to exclude the following distribution of signs of the weights of the C∗ -action on TF0 X and TF∞ X: (+ + +), (+ + +): This contradicts Lemma 7.6. (+ + +), (− − −): Let x ∈ E be a generic point. Choose a sequence (xn )n∈N ⊂ such that limn→∞ xn = x. Then there exist numbers λ, λ ∈ C∗ such that λ · x ∈ U0 and λ · x ∈ U∞ and there exists an n ∈ N such that λ · xn ∈ U0 , and λ xn ∈ U∞ . Linearization shows that limλ→0 λxn = F0 and limλ→∞ λxn = F∞ so that C := C∗ · xn is a closed curve in X and C ∩E = F0 ∪F∞ . But then E ·C ≥ 2, a contradiction to H 2 (X, Z) = 0. Assume from now on that the C∗ -action on TF0 X has weights of type (+ + −). Let x, y, and z be coordinates for the associated weight spaces. By the linearization Theorem 2.6, we can view x, y, and z as giving local coordinates on U0 . After performing a C∗ -equivariant change of coordinates on TF0 X, we may assume that the unit ball in TF0 X is contained in the image of U0 . Corollary 8.4. After swapping F0 and F∞ and, if necessary, replacing the C∗ action by (C∗ )−1 , the weights of the C∗ -action on the tangent space TF0 X have signs (+ + −) and the locally closed subspace E ∩ U0 is reducible. One of the components of E ∩ U0 is smooth, and the C∗ -action has a totally attracting fixed point there. Proof. It follows directly from Lemma 7.6 that E ∩ U0 contains infinitely many C∗ -stable curves containing F0 . Since it is closed in U0 , we have that {z = 0} ∩ U0 ⊂ E. Since {z = 0} is smooth, in order to see that E ∩U0 is reducible, it suffices to show E ∩ U0 is not normal. Since E is a hypersurface in X, it is a Cohen-Macaulay space,

116


and it follows from Serre’s criterion that the nonnormal locus N ⊂ E must be of codimension 1. By Lemma 7.6, N contains C∗ -fixed points. If N contains F0 , then we can stop here. Otherwise, if N contains F∞ but not F0 , then the signs of the weights at F∞ are necessarily (− − +); we show this by ruling out all other possibilities: (+ + +): does not occur, or else we obtain a contradiction to Lemma 7.6; (+ + −): is similar; (− − −): then limλ→0 λx does not exist for generic x ∈ N . Since every 1-dimensional component of N contains a C∗ -fixed point, it is rational. Thus, the limit exists on the normalization of N, which in turn implies that the limit exists on N. In this case swap F0 and F∞ and start anew. For the remainder of this section, fix F0 and F∞ so that we are in the situation of the above corollary. Notation 8.5. Let E0,i denote the irreducible components of E ∩ U0 , and let E0,0 be the smooth component in the preceding corollary. Lemma 8.6. Choose numbers a, b, and c ∈ N+ and let the group C∗ act on C3 by λ : (x, y, z) → (λa x, λb y, λ−c z). Let S ⊂ C3 be an irreducible C∗ -stable divisor with S = {z = 0}, but S ∩ {z = 0} = ∅. Then {x = y = 0} ⊂ S. Proof. Choose a point s ∈ {z = 0} ∩ S. Let sn = (xn , yn , zn ) be a sequence in S with zn = 0 and lim sn = s. Choose λn ∈ (zn )1/c . Note that lim λn = 0. Then λn · sn ∈ E and lim λn sn = lim(λan xn , λbn yn , 1) = (0, 0, 1). Corollary 8.7. Every component E0,i (i = 0) contains {x = y = 0}. Proof. In view of the preceding Lemma 8.6, we must show that E0,i ∩{z = 0} = ∅. This, however, is clear since all components E0,i contain F0 . Lemma 8.8. If the signs of the weights of the C∗ -action on TF∞ X are all negative, then there exists a curve C − ⊂ E satisfying (1) C − ∩ E0,0 is a curve (i.e., dim C − ∩ E0,0 = 1), and (2) F∞ ⊂ C − . Proof. By Corollary 8.4, E ∩U0 is reducible, and by Corollary 8.7, {x = y = 0} ⊂ E ∩U0 . The z-axis is the weight space to the negative weight, so that limλ→∞ (0, 0, 1) = F0 . But the limit limλ→0 (0, 0, 1) exists; this is because Eˆ is rational, and the limit exists there. Due to the negative weights, it is impossible that limλ→0 (0, 0, 1) = F∞ . Thus limλ→0 (0, 0, 1) = F0 and, for all λ sufficiently small, λ · (0, 0, 1) ∈ E0,0 . Lemma 8.9. The weights of the C∗ -action on TF∞ X have signs (− − +). Proof. It is clear that at least two of the signs must be negative—this is because for generic x ∈ E, limλ→∞ λ·x = F0 . Now suppose the weights were (a, b, c) which


117

were all negative. The weights of the C∗ -action on TF0 E0,0 are denoted by d and e. We consider the weighted projective spaces P(−a,−b,−c) (TF∞ X) and P(d,e) (TF0 E0,0 ). These are parameter spaces for C∗ -stable curves in X and E0,0 passing through F∞ or F0 , respectively. The analytic subspace E ∩ U∞ gives a closed subspace in the weighted projective space E ⊂ P(−a,−b,−c) (TF∞ X) parameterizing curves in E ∩ U∞ . We now construct a map from E to P(d,e) (TF0 E0,0 ). First we fix some notation. Let A0 : E0,0 → TF0 E0,0 and A∞ : U∞ → TF∞ X be the linearizing maps, and let π0 : TF0 E0,0 \ {0} → P(d,e) (TF0 E0,0 ) be the canonical projection. Given an arbitrary point x ∈ E, there exists a neighborhood U (x) ⊂ E and a section σ : U (x) → A∞ (U∞ ) ⊂ TF∞ X. After shrinking U (x), if necessary, there is a λ ∈ C∗ such that (λ ◦ σ )(U (x)) ⊂ E0,0 \ {F0 }: simply choose a λ such that (λ ◦ σ )(v) ∈ E∞ and set U (x) := λ−1 ((λ ◦ σ )(U (x)) ∩ E0,0 ); this is the shrinkage that might be unavoidable. This way we obtain a map ιx := (π0 ◦ λ ◦ σ ) : U (x) → P(d,e) (TF0 E). In order to obtain a global map ι : E → P(d,e) (TF0 E), it suffices to show that ιx does not depend on the choice of σ and λ. Indeed, choosing different σ and λ for all y ∈ U (x), there is a unique number λy ∈ C∗ such that (λ◦σ )(y) = λy (λ ◦σ )(y). This already shows that (π0 ◦ λ ◦ σ )(y) = (π0 ◦ λ ◦ σ )(y), and the existence of the global map ι is shown. It is obvious that ι is injective. This is how we make use of ι : by Lemma 8.8, π(C − ∩E0,0 )\{F0 } is not contained in the image of ι, so that the image must be contained in P(d,e) (TF0 E)\(π(C − ∩E0,0 \ {F0 })) ∼ = C. By the maximum principle, the image must be a point, a contradiction to ι being injective. Corollary 8.10. E ∩ U∞ is reducible. There exists a smooth component E∞,0 with totally repulsive fixed point. Proof. The existence of E∞,0 follows exactly as in Corollary 8.4. Similarly, it follows from the argumentation of corollary 8.4 that E ∩ U∞ is reducible if we show that the nonnormal locus N ⊂ E intersects E∞,0 . Suppose this is not the case. If x ∈ E∞,0 , then limλ→0 λx = F0 and the argumentation used in the proof of Lemma 8.9 yields a contradiction. Notation 8.11. In analogy to the notation introduced above, let E∞,i denote the irreducible components of E ∩ U∞ , and let E∞,0 be the smooth component whose existence is asserted by Corollary 8.10. 8.1.2. Loops. Now turn to the nonnormal locus of N ⊂ E . The following is an important notion. Notation 8.12. If S is a compact complex space with a holomorphic action of C∗ , call a C∗ -stable curve C ⊂ S a “loop” if C is the closure of an orbit and contains exactly one fixed point.

118


Lemma 8.13. There is at most one loop containing F0 , and at most one containing F∞ . The number of loops containing F0 is equal to the number of loops containing F∞ . Proof. There is only one curve in E ∩U0 containing a point x such that limλ→∞ λ· x = F0 (namely, the z-axis). The situation at F∞ is similar. Argue as in the proof of Lemma 8.9, using a map P(a,b) TF0 E0,0 −→ P(c,d) TF∞ E∞,0 to exclude the possibility that there is no loop at F0 and one at F∞ or vice versa. Lemma 8.14. The number of irreducible components of N ∩E0,0 equals the number of irreducible components of N ∩ E∞,0 .

Proof. Decompose N = NL ∪ i Ni , where the NL are loops and the Ni are other components. By Lemma 8.13, the claim is true if one considers loops only. If Ni is one of the other components and x ∈ Ni a generic point, then limλ→0 λ · x = F0 and limλ→∞ λ·x = F∞ so that Ni ∩E0,0 and Ni ∩E∞,0 are both irreducible components of N ∩ E0,0 and N ∩ E∞,0 , respectively. 8.2. Preparations: C∗ -action on normal surfaces. Let S be a smooth connected algebraic surface equipped with an algebraic C∗ -action with an attractive (resp., repulsive) fixed point F∞ (resp., F0 ). Let U ⊂ S be as in Lemma 7.6. Notation 8.15. A curve C = C+ ∪ C1 ∪ · · · ∪ Ck ∪ C− , k ≥ 0, as in the diagram F0 •

> C+

•p1

> C1

•p2

> C2

···

> C−

•F∞ ,

which is invariant under the C∗ -action, is called an “external chain.” A C∗ -fixed point different from F0 and F∞ is an “external fixed point.” Lemma 8.16. The complement of U in S is a union of external chains without common components. Proof. At every external fixed point the weights of the linearization must be of type (+−). Now allow S to have normal singularities at external fixed points. Applying the above argument to the desingularization Sˆ and blowing back down, we have the same result. Lemma 8.17. If S is a normal compact rational surface equipped with a C∗ -action having a smooth point F0 as a source and a smooth point F∞ as a sink, then the complement S \ U is a union of external chains. Corollary 8.18. In the setting of the preceding lemma there are no loops in S.


119

8.3. Computation of b1 of curves Lemma 8.19. Let C be the union of c rational curves having s singular points. Let ν : C˜ → C be the normalization of C, and let s˜ be the number of points in ν −1 (CSing ). For convenience, let δ = s˜ − s. Then b1 (C) = 1 − c + δ. Proof. Apply the Mayer-Vietoris sequence (see the proof of Proposition 7.3), where E := C, N := CSing , and so forth: 0 = h˜ 0 (E) − h˜ 0 E˜ + h˜ 0 (N) + h˜ 0 N˜ − h1 (E) + h1 E˜ + h1 (N) . 0

c−1

s−1

s˜ −1

b1

0

8.4. The proof in the case where F is discrete Notation 8.20. An “outer orbit” is an orbit that flows in the opposite direction from the generic orbit, that is, with source F∞ and sink F0 . An irreducible C∗ -invariant curve C ⊂ E containing F0 as a source and F∞ as a sink is called a “crossing curve” if E is not locally irreducible at the generic point of C. Proposition 8.21. The case where F is discrete does not occur. Proof. Assume to the contrary and use the notation introduced above. Let n be the number of components in N ∩ E0,0 . Then, by Lemma 8.19,   n − 1 if N consists of crossing curves only, b1 (N ) = n if N contains an outer orbit,   n + 1 if N contains loops; on the other hand, since the number of external chains in E˜ plus the number of curves in U˜ ∩ N˜ is also n (note that there are n components of N˜ containing F˜0 ), we have b1 N˜ = n − 1. It remains to show that the case where N consists only of crossing curves does not occur. In that case, since all components E0,i , i = 0, contain {x = y = 0} and N does not contain an outer orbit, there is only one such component E0,1 . Furthermore, E0,0 ∩ E0,1 consists of closures of C∗ -orbits that flow from F0 to F∞ . ˜ If p ∈ N˜ is an Observe that in this situation there cannot be external chains in E: external fixed point, then it is the intersection point of a curve flowing into p with a ˜ flow out of curve flowing out of p. But if ν(p) = F0 , then all curves in N = ν(N) ν(p), and we would have a contradiction. The analogue holds if ν(p) = F∞ .

120


9. Proof of the C∗ -action argument if F is a curve. If F is a curve, then F ∼ = P1 by Proposition 2.7. It is very easy to calculate b1 (N). However, we first have to show that F ⊂ N. Lemma 9.1. Suppose dim F = 1. If x ∈ F is an arbitrary point and U is a linearizing neighborhood of C∗ about x, then the weights of the C∗ -action on Tx X have signs (0+−), and if x, y, and z are coordinates associated to the weight spaces, then {yz = 0} ⊂ E ∩ U . Recall from the theorem on linearization (Theorem 2.6) that x, y, and z can be viewed to give coordinates on U . Proof. We have to exclude the possibility that the signs of the nonzero weights are equal. Suppose they were both positive. Then for a point y ∈ (E \ F ) ∩ U , the limit limλ→∞ λy would not exist in F. This contradicts the fact that F is the full C∗ -fixed-point set. Now it is a direct consequence of Lemma 7.6 that {zy = 0} ⊂ E ∩ U . Note that F 0 = F∞ = F . Lemma 9.2. If F ∼ = P1 , then b1 (N) = (# of irreducible components of N) − 1.

Proof. Decompose N = F ∪ i Ni . We show by induction that b1(F ∪ i=1,...,k Ni ) = k. Start: k = 0. It is clear that b1 (F ) = b1 (P1 ) = 0. Step: Since for all x ∈ Ni , the limits limλ→0 λx and limλ→∞ λx exist and are C∗ -fixed, that is, contained in F , there are only two possibilities: (1) limλ→0 λx = limλ→∞ λx for all x ∈ Nk , that is, Nk is a loop (see the notation 8.1.2). Then b1 (Nk ) = 1 and Nk ∩ F is a single point. (2) limλ→0 λx = limλ→∞ λx. Then the normalization N˜ k → Nk is injective and thus b1 (Ni ) = 0. Furthermore Nk ∩ F are two points. sequence associated to the decomposition F ∪

In any case the Mayer-Vietoris

N = (F ∪ N ) i=1,...,k i i=1,...,k−1 i ∪ Nk shows directly that b1 F ∪

i=1,...,k

Ni = b 1 F ∪

Ni + 1 = k.

i=1,...,k−1

˜ and finally Again we use a description of the rational surface E˜ to calculate b1 (N) to derive a contradiction. Lemma 9.3. There exists a surjective morphism with connected fibers φ : E˜ → P1 such that the induced C∗ -action on P1 is trivial. ˜ Since the Proof. Let C and C be two generic irreducible C∗ -stable curves in E. ∗ ˜ C -fixed-point set in E does not contain a totally attractive fixed point (this is because ˜ there is none in E), C and C are disjoint and do not intersect the singular locus of E.


121

Since E˜ is rational, C and C are linearly equivalent as divisors and yield the desired map. Lemma 9.4. The situation that F ∼ = P1 does not occur. This finishes the proof of the C∗ -action argument in Proposition 4.6. Proof. We show that b1 (N˜ ) = (# of irreducible components of N) − 2, contradicting Lemma 9.2. In accordance with the notation and the results of Lemma 7.6, let F0 and F∞ ⊂ E˜ denote the C∗ -fixed-point curves. These are sections for φ : E˜ → P1 , ˜ and they are contained in the preimage ν −1 (F ). In particular, F0 ∪ F∞ ⊂ N.

−1 ˜ Again decompose N = F ∪ i Ni . Set Ni := ν (Ni ). First, we show that N˜ i ∩ F0 and N˜ i ∩ F∞ are both single points. Realize from the description of Lemma 9.1 that ν|F0 : F0 → N is injective, and note that if y ∈ Ni is a generic point, then

N˜ i ∩ F0 ⊂ ν −1 lim λy ∩ F0 . λ→0

The expression on the right-hand side denotes a single point. A similar argumentation holds for F∞ . Second, we claim that b1 (N˜ i ) = 0. Recall that N˜ i are C∗ -stable and do not contain F0 or F∞ as an irreducible component. Thus, the irreducible components of N˜ i are irreducible components of φ-fibers. This is sufficient information to apply the Mayer-Vietoris sequence. For brevity, let k := (# of irreducible components of N). The beginning of the cohomological Mayer˜ Vietoris sequence associated to the decomposition N˜ = (F0 ∪ F∞ ) ∪ i Ni yields:     h0 (F0 ∪ F∞ ) + h0 N˜ i h0 N˜ −    i =2 =1 ≥h0 (

In other words,

i Ni )=k−1

  0 +h (F0 ∪ F∞ ) ∩ N˜ i −h1 N˜ = 0.   i =2(k−1)

˜ ≤ k − 2. h1 (N)

10. Some further remarks. We end with some general remarks. Again, let X denote a hypothetical complex structure on S 6 , this time not necessarily almost homogeneous. Proposition 10.1. There is no nonzero 3-form on X; that is, H 0 (KX ) = 0. Proof. Let σ ∈ H 0 (3X ). Then dσ = 0. Since H 3 (X; Q) = 0, the de Rham theorem yields a 2-form η with σ = dη. Thus σ ∧σ = dη ∧ σ = d(η ∧ σ ) − η ∧ d σ = 0. X

Hence σ = 0.

X

X

X

122


Corollary 10.2. We have the following: (1) H 3 (X, ᏻX ) = 0; (2) H 0 (KX ⊗ L) = 0 for generic L ∈ Pic(X); (3) h1 (X, ᏻX ) = h2 (X, ᏻX ) + 1; (4) Pic(X) = Pic0 (X) # H 1 (X, ᏻX ) is a positive-dimensional complex vector space. Proof. Items (1) and (2) are clear, (3) follows from the Riemann-Roch theorem, and (4) from H 1 (X, Z) = H 3 (X, Z) = 0. Observe that it is not at all clear whether κ(X) = −∞. Proving this would seem to be as complicated as to show that there are no divisors on X at all. (Notice that KX cannot be torsion by the above results!) See Proposition 10.4. We now prove a result that is the analogue of our main theorem for the cotangent bundle. The following version of this statement was pointed out to us by M. Toma. It improved our original bound by 1. Proposition 10.3. We have h0 (1X ) ≤ 1. Proof. By Proposition 10.1, H 0 1X ⊗ H 0 2X

∧

/ H 0 3 = 0. X

Now assume that ω1 and ω2 are two linearly independent 1-forms. Then, since a(X) = 0, ω1 (x) and ω2 (x) are linearly independent for a generic point x ∈ X. Consequently, if µ ∈ H 0 (2X ), then ω1 (x) ∧ ω2 (x) = c(x)µ(x). Thus any two 2-forms must be linearly dependent at all points of X. Again using that a(X) = 0, h0 (2X ) = 1. 1 (X) = 0 and therefore d : H 0 (1 ) → H 0 (2 ) is injective. On the other hand, HdR X X Thus there do not exist such 1-forms. Note that by semicontinuity of dimension it follows as above that h0 (1X ⊗ L) ≤ 2 for L a generic line bundle. Finally, we observe that statement (B) of the introduction is valid under the assumption that there are no curves in X. Proposition 10.4. (1) Assume that X has no compact curves. Then H 0 (1X ⊗ L) = 0 for all line bundles L, and h0 (T X ⊗ L) ≤ 1. (2) Assume that X has no divisors. Then h0 (1X ⊗ L) ≤ 2 for all line bundles L. Proof. (1) Let ω ∈ H 0 (1X ⊗ L). Suppose ω = 0. Then there is a divisor D (possibly empty) such that the section ω ∈ H 0 (1X ⊗ L ⊗ ᏻX (−D)) has no zeroes in codimension 1; hence it has only finitely many zeroes by our assumption. Thus c3 1X ⊗ L ⊗ ᏻX (−D) = c3 1X ≥ 0. But c3 (X) = −c3 (1X ) = 2, which is a contradiction. The inequality h0 (T X ⊗L) ≤ 1 is an immediate consequence from H 0 (1X ⊗ L) = 0.


123

(2) Assume h0 (1X ⊗L) = 3, and take a basis (ωi ). Then ω1 ∧ω2 ∧ω3 is a nonzero section of KX ⊗(3L). Since there are no divisors on X, the ωi are linearly independent everywhere; hence 1X ⊗ L # ᏻ⊕3 X . This again contradicts c3 (X) = 2. Acknowledgement. We thank the referee for a number of suggestions that led to improvement of the exposition as well as to simplifications of several mathematical points. References [BK]

[BBS] [Bla] [BS] [Bre] [CDP]

[ES] [FF] [Huc]

[HO] [Jac] [KP] [Kra]

[LY]

[Mor] [MU]

[Pot]

G. Barthel and L. Kaup, “Sur la topologie des surfaces complexes compactes” in Topologie des surfaces complexes compactes singulières, Sem. Math. Sup. 80, Presses Univ. Montréal, Montréal, 1982, 61–297. A. Białynicki-Birula and A. Sommese, Quotients by C∗ ×C∗ actions, Trans. Amer. Math. Soc. 289 (1985), 519–543. A. Blanchard, Sur les variétés analytiques complexes, Ann. Sci. École. Norm. Sup. (3) 73 (1956), 157–202. A. Borel and J.-P. Serre, Groupes de Lie et puissances réduites de Steenrod, Amer. J. Math. 75 (1953), 409–448. G. Bredon, Introduction to Compact Transformation Groups, Pure Appl. Math. 46, Academic Press, New York, 1972. F. Campana, J.-P. Demailly, and T. Peternell, The algebraic dimension of compact complex threefolds with vanishing second Betti number, Compositio Math. 112 (1998), 77–91. J. Erdman-Snow, “On the classification of solv-manifolds in dimension 2 and 3” in Journées Complexes 85 (Nancy, 1985), Inst. Élie Cartan 10, Univ. Nancy, Nancy, 1986, 57–103. G. Fischer and O. Forster, Ein Endlichkeitssatz für Hyperflächen auf kompakten komplexen Räumen, J. Reine Angew. Math. 306 (1979), 88–93. A. Huckleberry, “Actions of groups of holomorphic transformations” in Several Complex Variables, VI: Complex Manifolds, Encyclopaedia Math. Sci. 69, Springer-Verlag, Berlin, 1990, 143–196. A. Huckleberry and E. Oeljeklaus, Classification Theorems for Almost Homogeneous Spaces, Inst. Élie Cartan 9, Univ. Nancy, Nancy, 1984. N. Jacobson, Lie Algebras, Interscience Tracts in Pure Appl. Math. 10, Interscience Publishers, New York, 1962. H. Kraft and V. L. Popov, Semisimple group actions on the three-dimensional affine space are linear, Comment. Math. Helv. 60 (1985), 466–479. V. A. Krasnov, Compact complex manifolds without meromorphic functions (in Russian), Mat. Zametki 17 (1975), 119–122; English translation in Math. Notes 17 (1975), 69–71. J. Li and S.-T. Yau, “Hermitian-Yang-Mills connection on non-Kähler manifolds ” in Mathematical Aspects of String Theory (San Diego, Calif., 1986), Adv. Ser. Math. Phys. 1, World Scientific, Singapore, 1987, 560–573. S. Mori, Threefolds whose canonical bundles are not numerically effective, Ann. of Math. (2) 116 (1982), 133–176. S. Mukai and H. Umemura, “Minimal rational threefolds” in Algebraic Geometry (Tokyo and Kyoto, 1982), Lecture Notes in Math. 1016, Springer-Verlag, Berlin, 1983, 490– 518. J. Potters, On almost homogeneous compact complex analytic surfaces, Invent. Math. 8 (1969), 224–266.

124 [Ste]

HUCKLEBERRY, KEBEKUS, AND PETERNELL N. Steenrod, The Topology of Fibre Bundles, Princeton Math. Ser. 14, Princeton Univ. Press, Princeton, 1951.

Huckleberry: Fakultät und Institut für Mathematik, Ruhr-Universität Bochum, D44780 Bochum, Germany; [email protected] Kebekus: Lehrstuhl Mathematik VIII, Universität Bayreuth, D-95440 Bayreuth, Germany; [email protected] Peternell: Lehrstuhl Mathematik I, Universität Bayreuth, D-95440 Bayreuth, Germany, [email protected]

Vol. 102, No. 1


© 2000

SOLUTIONS, SPECTRUM, AND DYNAMICS FOR SCHRÖDINGER OPERATORS ON INFINITE DOMAINS A. KISELEV and Y. LAST

1. Introduction and main results. In this paper we investigate the relations between the rate of decay of solutions of Schrödinger equations, continuity properties of spectral measures of the corresponding operators, and dynamical properties of the corresponding quantum systems. The first main result of this paper shows that, in great generality, certain upper bounds on the rate of growth of L2 norms of generalized eigenfunctions over expanding balls imply certain minimal singularity of the spectral measures. Consider an operator HV defined by the differential expression HV = − + V (x) on some connected infinite domain with a smooth boundary and with Dirichlet boundary conditions on ∂. The case of = Rd is not excluded; no boundary conditions are needed in this case. To every vector φ ∈ L2 () we associate a spectral measure µφ in the usual way (namely, µφ is the unique Borel measure on R obeying f (E) dµφ (E) = (f (HV )φ, φ) for any Borel function f ). For any measure µ, we define the upper α-derivative D α µ(E) in the standard way: D α µ(E) = lim sup δ→0

µ(E − δ, E + δ) . δα

We denote by BR the ball of radius R centered at the origin, and we use the notation f BR for the L2 norm of the function f restricted to BR . We denote by Wml the usual Sobolev spaces of functions f such that D l f exists in the distributional sense l and (|u|m +|D l u|m )dx < ∞. We say that f (x) ∈ Wm,loc () if f (x) ∈ Wml (∩BR ) for every R < ∞. One of the main theorems that we prove here is the following. Theorem 1.1. Assume that the potential V (x) belongs to L∞ loc and is bounded from below, and that is a domain with piecewise smooth boundary. Suppose that there exists a distributional solution u(x, E) of the generalized eigenfunction equation HV − E u(x, E) = 0 (1) Received 25 August 1998. Revision received 21 May 1999. 1991 Mathematics Subject Classification. Primary 35J10, 81Q10; Secondary 35P05. Kiselev partially supported by National Science Foundation grant number DMS-9801530. Last partially supported by National Science Foundation grant number DMS-9801474. 125

126

KISELEV AND LAST

satisfying the boundary conditions and such that for some α, 0 ≤ α ≤ 1, we have lim inf R −α (2) |u(x, E)|2 dx < ∞. R→∞

BR ∩

Fix some compactly supported φ(x) ∈ L2 () such that φ(x)u(x, E) dx = 0.

Then we have D α µφ (E) > 0. Remarks. (1) Notice that under our assumptions on the potential, we have u ∈ 2 by standard results on Sobolev estimates for elliptic operators (see, e.g., [12]), W2,loc and the boundary values for u are well defined. (2) We choose not to formulate Theorem 1.1 for more general classes of potentials, domains, and boundary conditions in order to be able to give a transparent proof. Certainly, we can extend this theorem to wider classes of potentials and boundary conditions. The nature of the limitations is clear from the proof and the Stark operators example in Appendix B. For instance, when = Rd , we only ask that the negative part of the potential, V− , belong to the Kato class K d (see, e.g., [3], [39] for the definition of Kato classes). (3) If we replace < ∞ in equation (2) by = 0, we obtain that D α µφ (E) = ∞. Theorem 1.1 provides information on the pointwise behavior of spectral measures from rather simple and natural assumptions about the behavior of generalized eigenfunctions. From this theorem follow new criteria for the existence of absolutely continuous spectrum or singular continuous spectrum of given dimensional characteristics (see Section 2, and, in particular, Theorem 2.5 for more details). This contrasts the well-known result (see [5], [38], [39]) that existence of a polynomially bounded (but not L2 ) solution of (1) implies that the energy E belongs to the essential spectrum of HV but gives no further information on the structure of the essential spectrum. To the best of our knowledge, Theorem 1.1 is the first rigorous result providing a relation between the behavior of solutions and pointwise properties of the spectral measures for multidimensional Schrödinger operators. A result analogous to Theorem 1.1 also holds for discrete Schrödinger operators defined on some ⊂ Zd by (hv u)(n) = u(m) + v(n)u(n). |m−n|=1, m∈

We discuss this extension in Section 3. In Appendix A, we also indicate that results similar to Theorem 1.1 hold for more general elliptic and higher-order operators.

127

SOLUTIONS, SPECTRUM, AND DYNAMICS

The motivation for seeking relations between the pointwise in energy behavior of solutions and properties of spectral measures comes from the fact that in many problems the solutions are among the objects we can hope to investigate. When we are interested in the fine structure of the spectrum of Schrödinger operators for which the methods of scattering theory are not applicable, there are very limited tools in higher dimensions that may be effectively used for spectral analysis. On the other hand, for one-dimensional Schrödinger operators, the subordinacy theory created by Gilbert and Pearson [15], [14] and further extended by Jitomirskaya and Last [19], [20], [21] provides a powerful method for spectral analysis. The main results of these papers give a necessary and sufficient link between the behavior of solutions and the singularity of the spectral measure. Subordinacy theory played an important role in many recent results in one-dimensional spectral theory (see [7], [9], [18], [19], [20], [21], [24], [26], [30], [35]). In this paper, we derive only a sufficient-type relation between the solutions and the spectrum, but in much greater generality. However, in contrast to subordinacy theory, which requires comparison of different solutions, we need information about only one solution—the one obeying the appropriate boundary conditions. We remark that for one-dimensional Schrödinger operators, the result of Theorem 1.1 can be derived from subordinacy theory [20], [21]. Our second major result in this paper establishes a fundamental relation between spectral properties, generalized eigenfunctions, and quantum dynamics, and in particular, provides new bounds for the transport properties of quantum systems. We study the behavior of the time-averaged moments of the position operator X under the Schrödinger evolution. Pick some initial state ψ and consider

m

|X|

T

1 = T

T 0

m |X| exp − iH t ψ, exp − iH t ψ dt. V

V

Recall that a measure µ is called α-continuous if it gives zero weight to any set of zero α-dimensional Hausdorff measure (we recall the definition of these measures in Section 2). Let us denote by Pαc the spectral projector on the α-continuous spectral subspace, the set of all vectors ξ such that µξ is α-continuous (see [29]). In particular, if µψ has an α-continuous component (i.e., Pαc ψ = 0), then the following lower bound holds [8], [16], [17], [29]: m |X| T ≥ Cm T mα/d (here d is the space dimension, and Cm is a constant depending on µψ and m). Recall that for a wide class of Schrödinger operators, one has a generalized eigenfunction expansion theorem (see, e.g., [5], [30], [39]). In particular, for every ψ, there is a unique unitary map Uψ from the cyclic subspace Ᏼψ , generated by the vector ψ and the operator HV , to L2 (R, dµψ (E)). This map sends ψ to a function equal to 1 everywhere and realizes a unitary equivalence Uψ HV |Ᏼψ Uψ−1 = E, where E stands

128

KISELEV AND LAST

for the operator of multiplication by E. The operator Uψ is an integral operator with kernel u(x, E), where, for each fixed E, the u(x, E)’s solve (1) and are called generalized eigenfunctions. We say that the u(x, E)’s correspond to ψ if they constitute the kernel of the unitary map Uψ described above. Note that they are only defined a.e. with respect to µψ . We prove the following theorem, which holds in both the discrete and continuous settings. Theorem 1.2. Let ψ be a vector for which there exists a Borel set S ⊂ R of positive µψ measure, such that the restriction of µψ to S is α-continuous and, in addition, the generalized eigenfunctions u(x, E) for all E ∈ S satisfy (3)

lim sup R −γ u(x, E)2BR < ∞ R→∞

for some γ such that 0 < γ < d. Then for any m > 0, there exists a constant Cm such that m |X| T ≥ Cm T mα/γ (4) for all T > 0. Remarks. (1) Theorem 1.2 is somewhat related to (although it does not coincide with) some recent heuristic results in [23]. (2) It may be seen from Theorem 1.1 that we cannot have γ < α, since it would follow that the upper γ -derivative of the spectral measure is positive on too large a set (see Corollary 2.6). The physical reason is that when V is bounded from below, the velocity is bounded, and the propagation rate is at most ballistic. However, the range of applicability of Theorem 1.2 is wider than that of Theorem 1.1. In particular, it is applicable to operators with strongly negative potentials, such as Stark operators, which exhibit faster-than-ballistic transport. See Appendix B. The somewhat striking aspect of Theorem 1.2 is that for a fixed nonzero spectral dimension, faster decay of u(x, E) leads to faster transport. Theorem 1.2 shows that the behavior of the generalized eigenfunctions plays an important role in determining dynamical properties of quantum systems. We apply Theorem 1.2 to investigate the dynamics in the random decaying potentials model studied in [26]. When weakly coupled, these systems have (almost surely) some singular continuous spectrum with local dimensions that depend on the energy, but we show that the dynamical spreading of wavepackets, for any energy region where the spectrum is continuous, is almost ballistic with probability 1. More precisely, we show that for almost every realization, we have, for every - > 0, a bound of the form m |X| T ≥ Cm,- T m(1−-) . The paper is organized as follows. In Section 2, we prove Theorem 1.1 and its corollaries, rendering new spectral criteria. In Section 3, we sketch the argument for


129

similar results in the discrete setting. In Section 4, we consider some simple examples, in particular, showing that the result of Theorem 1.1 provides only a sufficient but not necessary criterion for positivity of the derivative of the spectral measure. It is, however, an optimal result in the sense that one cannot, in general, say more by looking only at the rate of growth of the L2 norm (see Section 5). It remains an interesting open question to find additional properties of solutions that determine the spectrum (or other important characteristics of the operator, such as transport properties) completely. In Section 5, we study the relationship between solutions, spectral dimension, and quantum dynamics, in particular, proving Theorem 1.2. In the appendices, we indicate further possible generalizations for elliptic and higher-order operators and consider dynamics for strongly perturbed one-dimensional Stark operators. The example of Stark operators provides another illustration of the relationship between the behavior of solutions and transport properties. Acknowledgements. We thank Y. Avron, I. Guarneri, R. Ketzmerick, B. Simon and S. Tcheremchantsev for stimulating discussions. We are grateful to the referees for useful sugestions and corrections. 2. Solutions and spectrum: Continuous case. We begin the proof of Theorem 1.1 with the following simple observation. Lemma 2.1. Let A be a selfadjoint operator acting on a Hilbert space Ᏼ and fix a vector φ ∈ Ᏼ. Let z ∈ C \ R. Then 2 = Im (A − z)−1 φ, φ . Im z(A − z)−1 φᏴ Proof. Consider the spectral representation associated with a vector φ and perform a straightforward computation:

dµφ dµφ Im = Im z . 2 R t −z R |t − z| The first idea in the proof of Theorem 1.1 is to estimate from below Im((HV − E − i-)−1 φ, φ) as - → 0. Such an estimate is equivalent to an estimate on the upper α-derivative of the spectral measure by the following lemma. β

Lemma 2.2. Let Qµ (E) denote Qβµ (E) = lim sup - β Im -→0

dµ(t) . t − E − i-

Then 1−α D α µ(E) ≤ C1 Qµ (E) ≤ C2 D α µ(E),

where C1 , C2 are positive constants depending only on α.

130

KISELEV AND LAST

Proof. The proof is a direct computation. For details, we refer to [11, Lemmas 3.2 and 3.3]. To derive an estimate on the Borel transform, we use Lemma 2.1, namely, estimates from below on the norm of the function −1 θ (x, E + i-) = HV − E − i- φ(x) over balls of radius of order 1/- as - goes to zero over some properly chosen sequence. The last technical lemmas that we need for the proof concern estimation of the W21 norms of u(x, E) and θ (x, z) in terms of their L2 norms. Lemma 2.3. Let ⊂ Rd be a domain with piecewise smooth boundary. Suppose that the potential V belongs to L∞ loc and is bounded from below, and let HV denote an operator with Dirichlet boundary conditions on ∂. Suppose that the function g(x, z) satisfies Dirichlet boundary conditions and HV − z g(x, z) = φ(x), where φ ∈ L2 () is compactly supported and real valued, and z is in general complex. Then (5) gW 1 (BR ∩) ≤ C(z, V− ) gL2 (BR+1 ∩) + φL2 () . 2

The constant in (5) depends only on the lower bound on V and on z, and may be chosen uniformly for z in any compact set. Proof. The proof is standard, and we provide it for the sake of completeness. See, for example, [3], [39] for detailed exposition of similar results and further references. Throughout the proof, we assume that the function g is sufficiently smooth to justify integration by parts (local W22 is sufficient). Clearly this is the case under our assumptions on V (see, e.g., [12]). To prove the bound (5) with the constant independent of R, let g(x, z) = g1 (x, z) + ig2 (x, z), where g1 , g2 are real valued. For any ψ ∈ C ∞ () such that 1 ≥ ψ(x) ≥ 0, ψ(x) = 1 when x ∈ BR ∩ , ψ(x) = 0 when x ∈ / BR+1 ∩ , we have (∇g1 )2 dx ≤ ψ(∇g1 )2 dx BR ∩ B ∩ R+1 ∂g1 g1 dσ − ψ (∇ψ)(∇g1 )g1 dx = (6) ∂n ∂(∩BR+1 ) BR+1 ∩ ψg1 g1 dx, − BR+1 ∩

131


where dσ is the surface measure on ∂( ∩ BR+1 ) induced from Rd . The first term vanishes because g1 vanishes on ∂ and ψ vanishes on (∂BR ) ∩ . Furthermore, by Green’s formula, ∂ψ 2 (g1 ) dσ − (7) 2 (∇ψ)(∇g1 )g1 dx = ψ(g1 )2 dx. BR+1 ∩ ∂(BR+1 ∩) ∂n BR+1 ∩ The boundary term in this equality is also equal to zero. Substituting (7) into (6), we find 1 2 (∇g1 ) dx ≤ ψ(g1 )2 dx 2 BR ∩ B ∩ R+1 ψg1 (Re z − V )g1 + φ − (Im z)g2 dx. + BR+1 ∩

Therefore,

g1 2W 1 (B ) ≤ Cψ φ2L2 + 2(1 + |z|)+V− L∞ g1 2L2 (B 2

R

R+1

+| Im z|g2 2L2 (B )

R+1 )

.

A similar estimate holds for g2 . Combining these two estimates, we obtain the result of the lemma. Remarks. (1) We have not tried to determine the most general classes of potentials and boundary conditions for which Lemma 2.3 holds. With slightly more technical effort, we can treat some other boundary conditions, such as Neumann, for instance. (2) For the case of the whole space, the lemma is true under the assumption that V− ∈ K d , the Kato class, which allows singularities in the negative part of the potential (see [39] for the definition and properties of potentials from these classes). This result follows from the technique developed in [3], [39], which uses Brownian motion to derive subsolution estimates implying bounds like in Lemma 2.3. Although [3], [39] consider only real z (and homogeneous equation), it is not hard to see that their arguments extend to give results like (5). We now introduce an important object in our consideration. Suppose S is a do2 (S). We denote by main with piecewise smooth boundary and f , g belong to W2,loc W∂S [f, g] the following expression:

∂f ∂g (8) f (t) (t) − (t)g(t) dσ (t), W∂S [f, g] = ∂n ∂n ∂S where σ is the surface measure induced from Rd , and ∂/∂n is the derivative in the 2 outer normal direction. The definition makes sense for W2,loc functions by Sobolev trace theorems (see, e.g., [13]). The notation W stresses the fact that in one dimension, the corresponding expression is related to the Wronskian of two functions (precisely, it is the difference of the Wronskians taken at the endpoints of the interval S). We abuse verbal notation and call the expression (8) the Wronskian of f and g over ∂S for the rest of this paper. The final lemma we need is the following.

132

KISELEV AND LAST

Lemma 2.4. Suppose that two functions f , g are locally W22 and satisfy Dirichlet boundary condition on ∂. Then for every R,

R

0

W∂(B

r ∩)

[f, g] dr ≤ f W 1 (BR ∩) gW 1 (BR ∩) . 2

2

Proof. We have W∂∩BR [f, g] = 0 since f and g satisfy the boundary conditions. Next note that R W(∂B ∩) [f, g] dr ≤ (|f ||∇g| + |∇f ||g|) dx r BR ∩

0

≤ f W 1 (BR ∩) gW 1 (BR ∩) . 2

2

We used the Cauchy-Schwartz inequality in the last step. Now we are ready to prove Theorem 1.1. Proof of Theorem 1.1. An interplay of the scales in space and in the spectral parameter plays an important role in the analysis. Let us assume that φ(x)u(x, E) dx = c = 0.

Take sufficiently large R0 , such that supp φ ⊂ BR0 . By Green’s formula, we have E θ (x, E + i-)u(x, E) dx BR0 ∩ = W∂(BR0 ∩) [θ, u] + HV θ(x, E + i-)u(x, E) dx BR0 ∩

= W∂(BR0 ∩) [θ, u] + (E + i-) +

BR0 ∩

BR0 ∩

θ(x, E + i-)u(x, E) dx

φ(x)u(x, E) dx.

In the above computation, we used the definition of θ(x, z) and the fact that the function u satisfies (HV − E)u = 0. Hence we obtain (9) θ(x, E + i-)u(x, E) dx. W∂(BR0 ∩) [θ, u] = −c − iBR0 ∩

Let us integrate (9) from R0 to some larger value of R: R R W∂(B )∩ [θ, u] dr ≥ |c|(R − R0 ) − dr r R0

R0

Br ∩

θ(x, E + i-)u(x, E) dx .

133


Using Lemmas 2.3 and 2.4, we see that (10) C 2 θ (x, E + i-)L2 (BR+1 ∩) + φL2 uL2 (BR+1 ∩) R ≥ |c|(R − R0 ) − drθ(x, E + i-)L2 (Br ∩) uL2 (Br ∩) . 0

According to assumption (2) of the theorem, there exists a sequence Rn → ∞, such that (11)

α/2

uL2 (BRn ∩) ≤ C1 Rn .

Let us set -n = C2 /Rn , and pick R + 1 = Rn and - = -n in (10). We obtain 2 C + C2 θ (x, E + i-n )L2 (BRn ∩) + φL2 uL2 (BRn ∩) ≥ |c|(Rn − R0 − 1). Substituting (11) into the last inequality, we find that there exists some constant C3 such that for n large enough, we have (12)

1−(α/2)

θ (x, E + i-n )L2 (BRn ∩) ≥ C3 Rn

− φL2 .

Now it remains to invoke Lemma 2.1 and note that

−1 2 Im HV − E − i-n φ, φ ≥ -n θ x, E + i-n L

2 (BRn )

for every n. Using the estimate (12) and the relation between Rn and -n , we find −1 Im HV − E − i-n φ, φ ≥ C4 -nα−1 for sufficiently small -n. The application of Lemma 2.2 now completes the proof. Remarks. (1) Theorem 1.1 also holds for wider classes of potentials and boundary conditions. The restrictions of the classes come from Lemma 2.3, the necessary estimate on the energy norms. With the help of smooth mollifiers to justify integration by parts, Theorem 1.1 can be extended to the classes to which one can extend Lemma 2.3. (2) We also note that the same argument as in the proof implies that D α µφ (E) = ∞ if, instead of (2) in the assumption of Theorem 1.2, we suppose that lim inf R −α u(x, E)2BR = 0. R→∞

We use this fact in the proof of Corollary 2.6. The next question that we would like to discuss is a sufficient condition for the existence of the various components of the spectrum. Let us recall the definition of

134

KISELEV AND LAST

Hausdorff measures and dimension. For α ∈ [0, 1] and any S ⊂ R, the α-dimensional Hausdorff measure of S is defined by hα (S) = lim

inf

δ→0 δ-covers

∞

|Iγ |α ,

γ =1

where Iγ are the intervals constituting the cover. The Hausdorff dimension of a set S is the infimum of all values of α such that hα (S) = 0. First, we prove the following theorem. Theorem 2.5. Let HV be a Schrödinger operator, with V and satisfying the same conditions as in Theorem 1.1. Suppose that for a measurable set S of positive hα measure, for each E ∈ S, there exists a nontrivial solution u(x, E) of the generalized eigenfunction equation (1) satisfying the boundary conditions such that lim inf R −α u(x, E)2BR < ∞. R→∞

Then there exists a vector ϕ ∈ L2 (R n ) such that µϕ (S1 ) > 0 for any S1 ⊂ S of positive hα measure. In particular, if α = 1, we have an absolutely continuous spectrum filling the set S. Remark. In many applications, particularly in one dimension, one applies a reasoning different from that suggested by Theorem 2.5 to derive the existence of various dimensional spectral components from results like Theorem 1.1. One proves the existence of solutions as in (2) for a.e. E, and then uses rank-one perturbation arguments (see, e.g., [20], [26]). Proof of Theorem 2.5. Recall that for every self-adjoint operator, there is an associated spectral measure of maximal type, µ, such that for every ψ and any measurable set S, µψ (S) > 0 implies µ(S) > 0. A vector χ is of the maximal type if for any measurable set S, µχ (S) > 0 given that µ(S) > 0. We show that for any S1 ⊂ S of positive α-dimensional Hausdorff measure, there exists a vector ψ with µψ (S1 ) > 0. By the standard argument for the existence of vectors of maximal type (see, e.g., [6]), this would imply existence of the vector ϕ as in the theorem. Pick some ball BR0 such that u(x, E)L2 (BR ∩ ) = 0 for energies E in a subset S2 of S1 of positive hα 0 measure (it is easy to see that such a ball exists, because of the σ -additivity of hα ). We remark that for a wide class of operators HV , an arbitrary ball will do because of the unique continuation (solutions u(x, E) cannot vanish identically on any ball), but there is no need to invoke these results. Pick a basis {ψn (x)}∞ n=1 in the Hilbert space L2 (BR0 ∩ ). Since {ψn } forms a basis, for every E ∈ S2 , there exists an n such that ψn (x)u(x, E) dx = 0. BR0 ∩

Consider the functions D α µψn on the set S2 . By Theorem 1.1, for every E ∈ S2 , there exists an n such that D α µψn (E) > 0. In particular, by σ -additivity of hα , there


135

exists an n0 such that D α µψn0 (E) > 0 for every E in a set Sn0 ⊂ S2 of positive hα measure. By the results of Rogers-Taylor theory (see [36, Theorem 63]), it follows that the measure µψn0 gives positive weight to the set Sn0 , and hence to the set S1 . The case of the absolutely continuous spectrum corresponds to α = 1; in this case, the application of Rogers-Taylor theory may be replaced by the well-known fact that a measure gives positive weight to a set of positive Lebesgue measure when its derivative is positive a.e. in this set. From Theorem 2.5 (or, essentially, from its proof and the remark after the proof of Theorem 1.1), we have the following corollary. Corollary 2.6. For any α, the set S of energies E, for which there exists a solution u(x, E) satisfying lim inf R −α u(x, E)2BR = 0,

(13) has zero

R→∞

hα

measure.

Remark. The fact that there may be only countably many values of E (counting multiplicities) for which equation (1) has L2 solutions satisfying the boundary conditions, is an obvious consequence of the separability of the Hilbert space L2 (). This corollary may be viewed as a less trivial generalization for slower rates of decay. Proof of Corollary 2.6. Suppose that S has positive hα measure. By the remark α φ after the proof of Theorem 1.1, (13) implies that D µ (E) = ∞ for every E ∈ S and finitely supported φ such that u(x, E)φ(x) = 0. Proceeding as in the proof of Theorem 2.5, we can find a vector ϕ such that D α µϕ (E) = ∞ for any E in some set of positive hα measure. This is not possible by Rogers-Taylor theory (see [36, Theorem 67]) and therefore gives a contradiction. We remark that for α = 1, this argument reduces to the well-known statement that a finite Borel measure µψ cannot have an infinite derivative on a set of positive Lebesgue measure. We would like to end this section by drawing a link with the well-known results of Rellich [34] and Kato [22], who showed, respectively, that for the free Laplacian and the Laplacian with a short-range perturbation (i.e., a potential that satisfies |V (x)| ≤ C(1 + |x|)−1−- ), there are no solutions satisfying (13) with α = 1 for any energy. Corollary 2.6 shows that for a much larger class of potentials, such solutions are still in some sense “exceptional” and can only occur on a set of energies of zero Lebesgue measure. 3. Solutions and spectrum: Discrete case. In this section, we consider discrete Schrödinger operators. All the results of the previous section extend to the discrete setting. In fact, the proofs are easier due to the absence of the Sobolev estimates issue, and there are no restrictions on potential. Let be some connected infinite domain in Zd . We define the Schrödinger operator hv on L2 () with Dirichlet boundary conditions by

136

KISELEV AND LAST

h v f (n) =

f (m) + v(n)f (n).

|n−m|=1, m∈

It is easy to check that the operator defined in this way is self-adjoint. We need an analog of Green’s formula in the discrete setting. For any domain S ⊂ Z d , let us denote by ∂S the set of points outside S that have a point of S within a unit distance. We have, for any two functions f , g,   f (m) h g(l) − g(m) f (l) , v f (n)g(n) − f (n)hv g(n) = n∈S

m∈∂S

l∈NS (m)

l∈NS (m)

where NS (m) denotes the set of neighbors of the point m ∈ ∂S lying in S (so that |m − n| = 1 for any n ∈ NS (m)). Therefore, we say that the analog of the Wronskian over ∂S of two functions is, in the discrete setting,   f (m) g(l) − g(m) f (l) . w∂S [f, g] = m∈∂S

l∈NS (m)

l∈NS (m)

For convenience, in all considerations for the discrete case, we replace the balls BR with cubes CR . The point n = (n1 , . . . , nd ) of the lattice belongs to CR if and only if |ni | ≤ R for all i = 1, . . . , d. We now formulate and prove an analog of Theorem 1.1 in the discrete case. Theorem 3.1. Suppose that there exists a solution u(n, E) of the generalized eigenfunction equation (14) hv − E u(n, E) = 0 satisfying the Dirichlet boundary conditions on ∂. Suppose that for some α, 0 ≤ α ≤ 1, we have (15) |u(n, E)|2 dx < ∞. lim inf R −α R→∞

n∈CR ∩

Fix some vector φ of compact support such that u(n, E)φ(n) = 0. n

Then we have D α µφ (E) > 0. In particular, if u(n0 , E) = 0, then D α µδn0 (E) > 0 (here δn0 is a function equal to 1 at n0 and 0 otherwise).


137

Proof. The argument repeats the proof of Theorem 1.1, except that we do not need Lemma 2.3. The analog of Lemma 2.4 is proven directly by the observation that w∂(∩Cr ) [f, g] = w∂Cr \∂ [f, g] and R w∂C

r \∂

r=1

[f, g] ≤ df L2 (CR+1 ∩) gL2 (CR+1 ∩) .

Remark. As in the continuous case, the result is also true for more general boundary conditions. The following theorem is an analog of Theorem 2.5. Theorem 3.2. Suppose that for each energy E in some measurable set S of positive hα measure, there exists a nontrivial solution u(n, E) of (14) satisfying Dirichlet boundary conditions and having the property (15). Then there exists a vector φ ∈ L2 (Zd ), such that µφ (S1 ) > 0 for any set S1 ⊂ S of positive hα measure. In particular, if α = 1, the set S is an essential support of the absolutely continuous part of the measure µφ restricted to S. The proof of this theorem is the same as the proof of Theorem 2.5. 4. Examples and discussion. The purpose of this section is purely illustrative—to show where the solutions we are studying are known to occur. However, these observations also partly lead us to the issue that is the topic of the next section: the relationships between generalized eigenfunctions, spectrum, and dynamics. In addition, we show that the criteria given by Theorems 1.1 and 3.1 are sufficient but not, in general, necessary for the positivity of the derivatives of spectral measures. We give an explicit example to confirm this statement. Our first remark is that solutions u(x, E) satisfying lim inf R −1 u(x, E)2BR < ∞ R→∞

exist for every energy E = 0 in the spectrum in the case of the free Laplacian operator in Rd or in the cylinder with Dirichlet boundary conditions. In the cylinder case, we may take u(x, E) = exp i E − El x1 Zl (x2 , . . . , xd ), where x1 is the coordinate along the rotation axis, El is any eigenvalue (less than E) of the Laplace operator with Dirichlet boundary conditions on the d−one-dimensional ball, and Zl (x2 , . . . , xd ) is any eigenfunction corresponding to this eigenvalue. In the free case, we can take any function √ u(x, E) = r −(d/2)+1 Jν Er Yl (θ),

138

KISELEV AND LAST

where Yl is any of the spherical harmonics corresponding to the eigenvalue El = l(l + d − 2) of the Laplace-Beltrami operator on the d-dimensional sphere, and Jν is a Bessel function (without singularity at the origin) with ν defined by ν 2 = l(l + d − 2) + ((d/2) − 1)2 . Note that for large r,

√ √ 2πν − π −1/2 Jν Er ∼ Cr 1 + o(1) . cos Er − 4 See, for example, [4], [44] for more information on spherical harmonics and Bessel functions. Using the results of Agmon theory and related estimates on the Fourier transform (see [1] or [33], and [2]), it is straightforward to show that the existence for every E ∈ (0, ∞) of solutions with the rate of growth of the L2 norm as in (2) with α = 1 extends to perturbations of the free Laplacian by short-range potentials, |V (x)| ≤ C(1+|x|)−1−- , if C is sufficiently small. In one dimension, it was recently shown in [7], [35] that such solutions exist for a.e. E ∈ (0, ∞) for any potential V satisfying |V (x)| ≤ C(1+|x|)−(1/2)−- . This implies that the absolutely continuous spectrum of the free operator in one dimension is stable under all perturbations decaying at this rate. This result is optimal: there are potentials that satisfy |V (x)| ≤ C(1 + |x|)−1/2 and lead to purely singular spectrum in (0, ∞). The corresponding question about the borderline decay for the stability of the absolutely continuous spectrum is open in higher dimensions, with any power in [1, 1/2] a possible candidate, in principle. We make the following conjecture. Conjecture 4.1. Suppose that HV is a Schrödinger operator in Rd for which |V (x)| ≤ C(1 + |x|)−(1/2)−- , - > 0. Then the absolutely continuous spectrum of the operator HV fills the whole positive semiaxis. This conjecture would, in particular, be clear from the following. Conjecture 4.2. Under the conditions of the previous conjecture, for a.e. E ∈ (0, ∞), there exists a solution u(x, E) of the generalized eigenfunction equation satisfying (2) with α = 1. Our next example concerns Schrödinger operators with periodic potentials. Let V (x) be a smooth periodic potential of period 1 in all variables x1 , . . . , xd . Given E in the spectrum of HV , consider the boundary value problem

(16)

(HV − E)b(x, E) = 0, ∂ j b = exp(iθl ) j , l = 1, . . . , d, j = 0, 1. ∂xl xl =1 ∂xl xl =0 j

∂j b

The set of all values of θ ∈ [0, 2π)d for which there exist solutions of the boundary value problem (16) is called the real (physical) Fermi surface FE . From well-known results on spectral properties of periodic differential operators (see [28]), it follows

139


that for all but a countable set of energies in the spectrum (exceptional points corresponding to band edges), we can find solutions u(x, E) of the generalized eigenfunction equation (1) of the following type: u(x, E) = b(x, θ, E)γ (θ) dσ, (17) S

where S ⊂ FE is a piece of an analytic (d −1)-dimensional surface, γ (θ) is a C0∞ (S)function, and b(x, θ, E) are Bloch functions satisfying (16): b(x, θ, E) = exp(iθx)f (x, θ, E), where f (x, θ, E) is periodic with period 1 in all directions in x, continuous in x, and analytic (as an L2 ([0, 1)d ) vector) in θ ∈ S. We claim that u(x, E) satisfies lim inf R −1 u(x, E)2BR ≤ ∞. R→∞

This can be shown in a way similar to the proof of this property in the case of Fourier transforms of measures supported on (d − 1)-dimensional smooth surfaces (see [2]). Represent the equation of the surface S as θd = s(θ1 , . . . , θd−1 ) (we can assume that S is small enough and θd is chosen so that this is possible). Then we can rewrite (17) as exp iθ x + is θ xd f x, θ , E γ θ dθ , u(x, E) = S

where the integration is now over the projection S of S on the hyperplane θd = 0, θ denotes first d − 1 coordinates, and γ includes the Jacobian from the change of variables. Fix the value of xd and integrate over the cube CR in the other coordinates x = x1 , . . . , xd−1 : |u(x, E)|2 dx = γ θ γ θ˜ exp i s θ − s θ˜ xd CR

S S

×

CR

exp i θ − θ˜ x f x , θ , E f x , θ˜ , E dx dθ d θ˜ .

Without loss of generality, take R to be an integer. Then we obtain   d−1 sin R + (1/2) θj − θ˜j |u(x, E)|2 dx = dθ d θ˜  (18)  ψ θ , θ˜ , sin(1/2) θj − θ˜ C S S j =1

R

j

where ψ θ , θ˜ = γ θ γ θ˜ exp i s θ − s θ˜ xd f x , θ , E f x , θ˜ , E exp i θ − θ˜ x dx . × C1

140

KISELEV AND LAST

Figure 1. The spiral domain

Due to the properties of f and γ , the function ψ is smooth, and hence the right-hand side in (18) converges as R → ∞ to the constant 2 2 f x , θ , E dx . C= dθ ψ θ , θ = dθ γ θ S

S

C1

Therefore, integrating in xd from −R to R, we obtain |u(x, E)|2 dx ≤ CR, BR

as claimed. Our last example in this section shows that the criteria for the positivity of derivatives of spectral measures, given by Theorems 1.1 and 3.1, provide a sufficient, but, in general, not necessary condition. The example is especially simple and transparent in the discrete setting. Let us consider the discrete plane Z2 and let be an infinite “spiral” in this plane (see Figure 1; we marked by × the points that do not belong to the domain). Consider h 0 defined on the spiral with Dirichlet boundary conditions. 2 By inspection, we see that h 0 acts on l () as a free one-dimensional Jacobi matrix. Hence the spectrum is absolutely continuous in [−2, 2], and for every E in this interval, there exists an explicitly computable unique solution u(n, E) of the generalized eigenfunction equation satisfying the boundary conditions:

E u(n, E) = sin cos−1 n . 2 This is a standard discrete plane wave. If we measure the linear distance N along the spiral, the square of the l 2 -norm of this solution grows as N. However, in Z2 , we have u(x, E)2BR ∩ ∼ R 2 .


141

Hence in this case, we cannot find solutions as in Theorem 1.1. We remark that [40] contains an example of a bounded spiral jelly roll domain on which the Laplace operator with Neumann boundary conditions has absolutely continuous spectrum. In this case, for a.e. E in the spectrum, the norm of solutions becomes infinite for finite R. 5. Solutions and dynamics. In this section, we prove Theorem 1.2 and apply it to study quantum dynamics in the random decaying potentials model studied in [10] and more recently in [26], [27]. The previous section provided us with several examples of operators with absolutely continuous spectrum and solutions satisfying the condition (2) in Theorem 1.1 for α = 1, and one example of an operator with absolutely continuous spectrum, but without such solutions. For the former three, the transport is ballistic for every vector (i.e., |X|m T ∼ T m ); for the latter, it is easy to see that the transport is not ballistic (it is diffusive in Z2 ). Theorem 1.2 indicates that this is not a coincidence. The proof of Theorem 1.2 is an extension of the proof of Theorem 6.1 in [29], and it is essentially the same in both the discrete and continuous settings. We use a discrete notation that formally only covers the discrete case, but the continuous case follows from it in a totally straightforward manner (which essentially amounts to replacing n by x and some summations by integrals). We note that in [29, Theorem 6.2] the continuous case gets an independent treatment, based on semigroup kernel inequalities. This is not needed here, since we assume the existence of eigenfunction expansions with suitable properties. This allows our Theorem 1.2 to cover some cases, such as Stark operators, that are excluded from [29, Theorem 6.2]. Recall that a measure µ is called uniformly α-Hölder continuous (denoted UαH) if there exists a constant C such that for every interval I with |I | ≤ 1, we have (19)

µ(I ) ≤ C|I |α .

α-continuous measures (recall that this means measures giving zero weight to all sets of zero hα measure) can be approximated by UαH measures in the following sense. Theorem (Rogers-Taylor [37]). A finite Borel measure µ on R is α-continuous if and only if, for every - > 0, there exist two mutually singular Borel measures µ-1 and µ-2 , such that µ = µ-1 + µ-2 , where µ-1 is UαH and µ-2 (R) < -. For UαH measures, we can study dynamics with the aid of the following Strichartz estimate. Theorem (Strichartz [43]). Let µ be a finite UαH measure, and for each f ∈ L2 (R, dµ), denote f µ(t) = exp(−ixt)f (x) dµ(x). Then there exists a constant C1 , depending only on µ (more precisely, only on C in (19)), such that for any f ∈ L2 (R, dµ) and T > 0,

142

KISELEV AND LAST

|f µ|2

< C1 f 2 T −α ,

T

where f is the L2 norm of f . We now prove Theorem 1.2. Proof of Theorem 1.2. Without loss of generality, assume ψ = 1. We first establish the existence of a Borel set S˜ ⊂ S, for which the following three properties are true: (i) PS˜ ψ > 0. (ii) The restriction of the spectral measure µψ to S˜ is UαH. (iii) There exists a constant C2 , such that for each E ∈ S˜ and R > 0, the corresponding generalized eigenfunction u(n, E) satisfies |u(n, E)|2 < C2 R γ . |n|0

|n| 0.

For example, if we take idependent, √ √identically distributed random variables aω (n) with uniform distribution in [− 3, 3], then vω (n) = λn−1/2 aω (n) satisfy all the conditions. The half-line random Schrödinger operators hω with, say, Dirichlet boundary conditions at zero and potential vω exhibit very rich spectral structure. Such operators where studied by Delyon, Simon, and Souillard [10], and more recently by Kotani and Ushiroya [27] and Kiselev, Last, and Simon [26]. Our study here is based mainly on the results of the last paper. In particular, the following was proven in [26]. Theorem (KLS [26]). For all ω, the essential spectrum of hω is [−2, 2]. If |λ| < 2, then for a.e. ω, hω has purely singular continuous spectrum in {E | |E| < (4−λ2 )1/2 } and only dense pure point spectrum in {E | (4 − λ2 )1/2 < |E| < 2}. For a.e. ω and E ∈ (−2, 2), (22)

λ2 log TE (n, 0) = , n→∞ log n 8 − 2E 2 lim

and there exists an initial condition θ(ω) at zero such that (23)

λ2 log TE (n, 0)uθ(ω) =− , n→∞ log n 8 − 2E 2 lim

where uθ (ω) is the 2-vector corresponding to the boundary condition θ(ω) at 0, and TE (n, 0) is the transfer matrix from 0 to n at energy E. √ √ This theorem implies that for a.e. ω and E ∈ (− 4 − λ2 , 4 − λ2 ), the spectral measure µ (corresponding to the vector δ1 ) has local Hausdorff dimension (24)

α(E, λ) =

4 − E 2 − λ2 4 − E2

at energy E, in the sense that for any - > 0, there is a δ so that µ(A) = 0 if A is a subset of (E − δ, E + δ) of Hausdorff dimension less than α(E, λ) − -, and there is a subset B of Hausdorff dimension less than α(E, λ) + - such that µ((E − δ, E + δ) \ B) = 0. These properties of the spectral measure follow from (22), (23) by subordinacy theory [19], [20]. See [26] for details. Remark. The KLS theorem also provides an example indicating that the criterion of Theorem 1.1 is optimal in the sense that one cannot, in general, say more by looking at the rate of growth of the L2 -norm. Indeed, by (22), for a.e. ω, all solutions


145

u(n, ˜ E) for every energy E in the continuous spectrum satisfy R −ρ u(n, ˜ E)2BR ≤ C, for any λ2 4 − E2 and all R. In particular, for every ρ > 1, we can take λ sufficiently small to ensure the existence of an interval Iρ around E = 0 such that for a.e. ω, all solutions (and in particular the one obeying the boundary condition) satisfy ρ > 1+

˜ E)2BR < C R −ρ u(n, for E ∈ Iρ . Yet for a.e. E ∈ Iρ , we have Dµ(E) = 0 since the measure is purely singular. This shows that no condition of type (2) with α > 1 leads, in general, to pointwise estimates on the derivatives of spectral measures. This remark sounds trivial in one dimension, but it is straightforward (using the analysis of [26] for the continuous analog Vω (x) of the family of random potentials we study) to give a similar example that works in any dimension (in the continuous case). Set HVω = −+λVω (r) with spherically symmetric potential. Using spherical symmetry, one shows that the spectrum of HVω is purely singular with probability 1. However, for every ρ > 1, there are solutions for a.e. ω and all energies E sufficiently large such that (2) holds with α = ρ. The following theorem shows that as long as our operators hω have some continuous spectrum (which may be of arbitrarily small dimension), their transport properties are arbitrarily close to ballistic. Theorem 5.1. Consider the family hω of random Schrödinger operators defined on Z+ with potential λvω (n), where λ < 2 and the potential satisfies (21). Then for a.e. ω, for every ψ such that Pc (ω)ψ = 0 (where Pc (ω) is the projector on the continuous spectrum of the operator hω ), we have that for every - > 0 and m > 0, there is a positive constant C-,m,ω such that for any T > 0, m |X| ψ(t), ψ(t) T ≥ C-,m,ω T m(1−-) . (25) Proof. By the results of the Gilbert-Pearson theory, the spectral measure µ is supported on the set of the energies E for which the decaying solution (23) satisfies the boundary condition (namely, θ(ω) coincides with the Dirichlet boundary condition). Moreover, these decaying solutions, which we denote by u(n, E), are exactly the generalized eigenfunctions in the sense of Theorem 1.2, if we normalize them by setting u(1, E) = 1. Fix ω such that the results of the KLS theorem hold. (23) implies that the generalized eigenfunctions u(n, E) of the operator hω satisfy (26)

lim sup R −γ u(n, E)2BR ≤ ∞ R→∞

146

KISELEV AND LAST

for√ every γ √ > α(E, λ) given by (22). Pick an open energy interval I = (E1 , E2 ) ⊂ (− 4 − λ2 , 4 − λ2 ), such that 0 ∈ / I , and µψ (I ) > 0. Let α1 = α(E1 , λ), α2 = α(E2 , λ). α(E, λ) is monotone on I . Assume, without loss, that α1 < α2 . The restriction of µψ to I is α1 -continuous, and by (26), lim supR→∞ R −α2 u(n, E)2BR < ∞ for any generalized eigenfunction u(n, E) with E ∈ I . Thus, by Theorem 1.2, for each m > 0, there is a constant Cm,I,ω such that for all T > 0, m |X| ψ(t), ψ(t) T ≥ Cm,I,ω T mα1 /α2 . Since Pc (ω)ψ = 0, we can clearly choose such an interval I with α1 /α2 > 1 − - and µψ (I ) > 0. Thus, Theorem 5.1 follows. Remark. By using an extension of the proof of Theorem 1.2, we can show that there is actually a component of the wave packet of size corresponding to Pc (ω)ψ that is spreading on average at a rate that is arbitrarily close to ballistic. More explicitly, we can show that for a.e. ω, for every - > 0 and ρ > 0, there exists a constant Cω,ρ,such that if RT = Cω,ρ,- T 1−- , then (27) PRT ψ(t)2 T ≤ ψ − Pc (ω)ψ2 + ρ. This easily yields Theorem 5.1 and is thus a stronger statement. Appendices A. Generalizations. The whole proof of Theorem 1.1 readily extends to more general settings. Namely, we can replace the operator HV with a general, uniformly elliptic self-adjoint operator J such that J = ∂l − iAl (x) alk (x) ∂k − iAk (x) + V (x), provided that alk , Al , and V are “nice enough” (for example, bounded and sufficiently smooth). The proof for this case is very similar. Green’s formula leads us to consider the following modified Wronskian: W∂S [f, g] = cos n, xl alk ∂k − iAk u v − u cos n, xk ∂l − iAl alk v dσ. ∂S

It is clear that under our assumptions, the analog of Lemma 2.4 holds. The estimate of Lemma 2.3 also holds with the constant independent of R by the standard Sobolev estimates for bounded sufficiently smooth coefficients (see, e.g., [13], [31]). The rest of the proof does not change. A similar remark applies to some higher-order operators and systems. In particular, in one dimension, a self-adjoint half-line differential operator of order 2n is given by the expression (n) (n−1) (Lf )(x) = (−1)n p0 f (n) + (−1)n−1 p1 f (n−1) + · · · + pn f


147

and a set of self-adjoint boundary conditions at zero. The analog of the Wronskian in this case is determined by integration by parts: (28) WL,x [f, g] =

j n

(m−1) (j −m) (m−1) (−1)m pj f (j ) g − f (j −m) pj g (j ) ,

j =1 m=1

where all values are taken at the point x. The analog of the Sobolev estimates of Lemma 2.3 is now the claim that for a solution of (L − E)u = φ, uW2m (BR ) ≤ CuL2 (BR+1 ) holds for m ≤ 2n−1. Such estimates (in fact, for m ≤ 2n) are well known to hold for operators with bounded sufficiently smooth coefficients (see, e.g., [31]). The analog of Lemma 2.4 follows directly from (28); the rest of the proof of Theorem 1.1 does not change. In particular, we have the following theorem. Theorem A.1. Let L denote the self-adjoint differential operator of order 2n with bounded sufficiently smooth (say, infinitely differentiable) coefficients. Suppose that for every E in a set S of positive Lebesgue measure, there exists a bounded solution u(x, E) of the generalized eigenfunction equation (L − E)u = 0 satisfying the boundary conditions. Suppose that for a compactly supported function φ ∈ L2 , we have u(x, E)φ(x) dx = 0 for a.e E ∈ S. Then the absolutely continuous part of the spectral measure µφ fills S (so that µφ (S1 ) > 0 for any S1 ⊂ S of positive Lebesgue measure). Remark. Of course, we can also allow for φ that are not L2 , but from the Sobolev space H−2 (HV ), such as the δ function and its derivatives up to 2n−1, which are often used in the setting of one-dimensional differential operators. The spectral measure is not finite in this case, but nothing else changes. Theorem A.1 follows from the above discussion and proof of Theorem 2.5. This result may be viewed as a sort of analog of [41], [42] for the higher-order case. It is typical, though, that our condition involves only one solution ([41], [42] require all solutions to be bounded) because the possible multiplicity of the spectrum makes it unreasonable to demand all solutions to be bounded (in higher-order cases) to get absolutely continuous spectrum. On the other hand, our result does not guarantee pure absolute continuity.

148

KISELEV AND LAST

B. One-dimensional perturbed Stark operators. In this appendix, we make a remark concerning dynamical properties of a certain class of perturbed Stark operators. We denote by HV ,S the operator defined on the whole axis by the differential expression −

d2 − x + V (x). dx 2

Our results are based on the following theorem, proved in [25]. Theorem [25]. Suppose that |V (x)| ≤ C(1 + |x|)−(1/3)−- , or that V is bounded and has a derivative V that is bounded and Hölder continuous. Then the whole axis (−∞, ∞) is an essential support of the absolutely continuous part of the spectral measure µ. Moreover, for a.e. E ∈ R, there exist two linearly independent solutions u± (x, E), such that u± (x, E) = x

−1/4

2 3/2 exp ± i x + f± (x, E) 1 + o(1) 3

as x → +∞, where |f± (x, E)| ≤ C(1 + x)−1/2 . Stark operators do not fit into the framework provided by Theorem 1.1 because of the strong negative part of the potential (and resulting failure of Lemma 2.3). Indeed, for a.e. energy E, we have a solution u(x, E) that satisfies R −1/2 u(x, E)BR ≤ C(E), which, if Theorem 1.1 were true, would imply D 1/2 (E) > 0 a.e. E. It should be possible to prove an analog of Theorem 1.1 for some perturbed Stark operators, taking into account that instead of the Sobolev estimates of Lemma 2.3, we have ∇u2BR ≤ CRu2BR . However, the criterion of Theorem 1.2 applies, immediately giving the following theorem. Theorem B.1. Under the conditions of the previous theorem, for every vector ψ with nonzero projection on the absolutely continuous subspace, we have m |X| ψ(t), ψ(t) T ≥ CT 2m . We note that there are examples (see [32]) of potentials V satisfying |V (x)| ≤ C(x)(1 + |x|)−1/3 , where C(x) tends to infinity as x → ∞, but arbitrarily slowly, such that for a corresponding Stark operator, there is a dense set of eigenvalues embedded in the absolutely


149

continuous spectrum. Theorem B.1 shows that such potentials, nevertheless, do not slow down dynamics corresponding to the absolutely continuous component. References [1] [2] [3] [4] [5] [6]

[7]

[8]

[9] [10] [11]

[12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22]

S. Agmon, Spectral properties of Schrödinger operators and scattering theory, Ann. Scuola Norm. Sup. Pisa Cl. Sci. (4) 2 (1979), 151–218. S. Agmon and L. Hörmander, Asymptotic properties of solutions of differential equations with simple characteristics, J. Anal. Math. 30 (1976), 1–38. M. Aizenman and B. Simon, Brownian motion and Harnack inequality for Schrödinger operators, Comm. Pure Appl. Math. 35 (1982), 209–273. Bateman Manuscript Project, Higher Transcendental Functions, 2, McGraw-Hill, New York, 1953–1955. J. M. Berezanskii, Expansions in Eigenfunctions of Selfadjoint Operators, Transl. Math. Monogr. 17, Amer. Math. Soc., Providence, 1968. M. Birman and M. Solomyak, Spectral Theory of Selfadjoint Operators in Hilbert Space, Trans. S. Khrushchëv and V. Peller, Math. Appl. (Soviet Ser.), D. Rédel Publishing Co., Dordrecht, 1987. M. Christ and A. Kiselev, Absolutely continuous spectrum for one-dimensional Schrödinger operators with slowly decaying potentials: Some optimal results, J. Amer. Math. Soc. 11 (1998), 771–797. J. M. Combes, “Connections between quantum dynamics and spectral properties of timeevolution operators” in Differential Equations with Applications to Mathematical Physics, Math. Sci. Engrg. 192, Academic Press, Boston, 1993, 59–68. D. Damanik, α-continuity properties of one-dimensional quasicrystals, Comm. Math. Phys. 192 (1998), 169–182. F. Delyon, B. Simon, and B. Souillard, From power pure point to continuous spectrum in disordered systems, Ann. Inst. H. Poincaré Phys. Théor. 42 (1985), 283–309. R. del Rio, S. Jitomirskaya, Y. Last, and B. Simon, Operators with singular continuous spectrum, IV. Hausdorff dimensions, rank one perturbations, and localization, J. Anal. Math. 69 (1996), 153–200. L. C. Evans, Partial Differential Equations, Grad. Stud. Math. 19, Amer. Math. Soc., Providence, 1998. D. Gilbarg and N. Trudinger, Elliptic Partial Differential Equations of Second Order, Grundlehren Math. Wiss. 224, Springer-Verlag, Berlin, 1977. D. Gilbert, On subordinacy and analysis of the spectrum of Schrödinger operators with two singular endpoints, Proc. Roy. Soc. Edinburgh Sect. A 112 (1989), 213–229. D. Gilbert and D. Pearson, On subordinacy and analysis of the spectrum of one-dimensional Schrödinger operators, J. Math. Anal. Appl. 128 (1987), 30–56. I. Guarneri, Spectral properties of quantum diffusion on discrete lattices, Europhys. Lett. 10 (1989), 95–100. , On an estimate concerning quantum diffusion in the presence of a fractal spectrum, Europhys. Lett. 21 (1993), 729–733. S. Jitomirskaya, in preparation. S. Jitomirskaya and Y. Last, Dimensional Hausdorff properties of singular continuous spectra, Phys. Rev. Lett. 76 (1996), 1765–1769. , Power law subordinacy and singular spectra. I. Half-line operators, preprint. , Power law subordinacy and singular spectra. II. Line operators, in preparation. T. Kato, Growth properties of solutions of the reduced wave equation with a variable coefficient, Comm. Pure Appl. Math. 12 (1959), 403–425.

150 [23] [24]

[25] [26]

[27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44]

KISELEV AND LAST R. Ketzmerick, K. Kruse, S. Kraut, and T. Geisel, What determines the spreading of a wave-packet?, Phys. Rev. Lett. 79 (1997), 1959–1963. A. Kiselev, Absolutely continuous spectrum of one-dimensional Schrödinger operators and Jacobi matrices with slowly decreasing potentials, Comm. Math. Phys. 179 (1996), 377–400. , Absolutely continuous spectrum of perturbed Stark operators, to appear in Trans. Amer. Math. Soc. A. Kiselev, Y. Last, and B. Simon, Modified Prüfer and EFGP transforms and the spectral analysis of one-dimensional Schrödinger operators, Comm. Math. Phys. 194 (1998), 1–45. S. Kotani and N. Ushiroya, One-dimensional Schrödinger operators with random decaying potentials, Comm. Math. Phys. 115 (1988), 247–266. P. Kuchment, Bloch solutions of periodic partial differential equations, Funktsional Anal. i Prilozhen 14 (1980), 65–66. Y. Last, Quantum dynamics and decompositions of singular continuous spectra, J. Funct. Anal. 142 (1996), 406–445. Y. Last and B. Simon, Eigenfunctions, transfer matrices, and absolutely continuous spectrum of one-dimensional Schrödinger operators, Invent. Math. 135 (1999), 329–367. S. Mizohata, The Theory of Partial Differential Equations, Trans. K. Miyahara, Cambridge University Press, New York, 1973. S. N. Naboko and A. B. Pushnitskii, Point spectrum on a continuous spectrum for weakly perturbed Stark type operators, Funct. Anal. Appl. 29 (1995), 248–257. M. Reed and B. Simon, Methods of Modern Mathematical Physics, IV. Analysis of Operators, Academic Press, New York, 1978. F. Rellich, Jber das asymptotische Verhalten der Lösungen von u + λu = 0 in unendlichen Gebieten, Über. Deutsch. Math. Verein 53 (1943), 57–65. C. Remling, The absolutely continuous spectrum of one-dimensional Schrödinger operators with decaying potentials, Comm. Math. Phys. 193 (1998), 151–170. C. A. Rogers, Hausdorff Measures, Cambridge University Press, London, 1970. C. A. Rogers and S. J. Taylor, Additive set functions in Euclidean space. II, Acta Math. 109 (1963), 207–240. I. Sch’nol, On the behavior of the Schrödinger equation, Mat. Sb. 42 (1957), 273–286. B. Simon, Schrödinger semigroups, Bull. Amer. Math. Soc. (N.S.) 7 (1982), 447–526. , The Neumann Laplacian of a jelly roll, Proc. Amer. Math. Soc. 114 (1992), 783–785. , Bounded eigenfunctions and absolutely continuous spectra for one-dimensional Schrödinger operators, Proc. Amer. Math. Soc. 124 (1996), 3361–3369. G. Stolz, Bounded solutions and absolute continuity of Sturm-Liouville operators, J. Math. Anal. Appl. 169 (1992), 210–228. R. S. Strichartz, Fourier asymptotics of fractal measures, J. Funct. Anal. 89 (1990), 154–187. N. Vilenkin, Special functions and the theory of group representations (in Russian), “Nauka,” Moscow, 1991.

Kiselev: Department of Mathematics, University of Chicago, Chicago, Illinois 60637, USA; [email protected] Last: Department of Mathematics, California Institute of Technology, Pasadena, California 91125, USA; [email protected]

Vol. 102, No. 1


© 2000

THE GRIFFITHS GROUP OF A GENERAL CALABI-YAU THREEFOLD IS NOT FINITELY GENERATED CLAIRE VOISIN 1. Introduction. If X is a Kähler variety, the intermediate Jacobian J 2k−1 (X) is defined as the complex torus J 2k−1 (X) = H 2k−1 (X, C)/F k H 2k−1 (X) ⊕ H 2k−1 (X, Z), k 2k−1(X), where F k H 2k−1(X) is the set of classes representable by a closed form in F A that is, which is locally of the form I,J αI,J dzI ∧ dzJ , with |I | + |J | = 2k − 1 and |I | ≥ k. Griffiths [9] has defined the Abel-Jacobi map

kX : ᐆkhom (X) −→ J 2k−1 (X), where ᐆkhom (X) is the group of codimension k algebraic cycles homologous to zero on X. Using the identification n−k+1 2n−2k+1 ∗ F H (X) 2k−1 J , n = dim X (X) = H2n−2k+1 (X, Z) given by Poincaré duality, kX associates to the cycle Z = ∂, where is a real chain of dimension 2n − 2k + 1 well defined up to a 2n − 2k + 1-cycle, the element ∗ ∈ F n−k+1 H 2n−2k+1 (X) /H2n−2k+1 (X, Z),

which is well defined using the isomorphism F n−k+1 A2n−2k+1 (X)c F n−k+1 H 2n−2k+1 (X) ∼ . = dF n−k+1 A2n−2k (X) If (Zt )t∈C is a flat family of codimension k algebraic cycles on X parametrized by a smooth irreducible curve C, the map t → kX (Zt − Z0 ) factors through a homomorphism from the Jacobian J (C) to J 2k−1 (X), and one can show that the image of this morphism is a complex subtorus of J 2k−1 (X) whose tangent space is contained in H k−1,k (X) ⊂ H 2k−1 (X, C)/F k H 2k−1 (X). Defining the subgroup Received 9 March 1999. 1991 Mathematics Subject Classification. Primary 14C25, 14C30, 14M99, 32J25. Author’s work partially supported by the project Algebraic Geometry in Europe. 151

152

CLAIRE VOISIN

ᐆkalg (X) ⊂ ᐆkhom (X) of cycles algebraically equivalent to zero as the subgroup gen-

erated by the cycles Zt − Z0 for any family as above and defining the Griffiths group Griff k (X) as the quotient ᐆkhom (X)/ᐆkalg (X), it follows that the Abel-Jacobi map induces a morphism kX : Griff k (X) −→ J 2k−1 (X)tr ,

where J 2k−1 (X)tr is the quotient of J 2k−1 (X) by its maximal subtorus having its tangent space contained in H k−1,k (X). In this paper, we are mainly interested in the case where n = 3, k = 2. We use then the notation J (X), X . In [10], Griffiths proved the following theorem. Theorem 1. If X is a general quintic threefold and Z is the difference of two distinct lines in X, X (Z) is not a torsion point in J (X). Furthermore, J (X)tr = J (X). From this it follows that Griff(X) contains nontorsion elements. In [3] Clemens, using the countably many isolated rational curves in X, proved the following theorem. Theorem 2. If X is a general quintic threefold, Im X ⊗ Q is not a finitedimensional Q–vector space. In particular, Griff(X) ⊗ Q is not a finite-dimensional Q–vector space. Clemens’s theorem has been extended to complete intersections by Paranjape [15] and to Abelian threefolds by Nori [14]. (In the last case, J (X)tr is different from J (X), and one considers the Abel-Jacobi map with value in J (X)tr .) Notice that it is conjectured (see [13]) that for codimension-two cycles, the Abel-Jacobi map 2X : Griff(X) → J (X)tr is injective, so both statements should be equivalent. More recently, Nori [13] proved that there may exist nontorsion cycles in Griff k (X) for any k ≥ 3 (so X has to be of dimension at least 4), which are annihilated by the Abel-Jacobi map. Combining Nori’s ideas and the study of the Abel-Jacobi map for the general cubic sevenfold in P8 , Albano and Collino [1] even proved that for k ≥ 3 the kernel of the Abel-Jacobi map kX : Griff k (X) → J 2k−1 (X)tr may be nonfinitely generated. In this paper, we consider another kind of generalization of the Clemens theorem: Instead of a quintic threefold, we consider a Calabi-Yau threefold X; that is, X is a Kähler threefold with trivial canonical bundle such that H 2 (ᏻX ) = 0 (so, in particular, X is projective). For such X it is well known that the local moduli space of X is smooth of dimension dim H 1 (TX ) = dim H 1,2 (X). In [17] we proved the following. Theorem 3. Let X be a Calabi-Yau threefold. If h1 (TX ) = 0, the general deformation Xt of X satisfies that the Abel-Jacobi map Xt : ᐆ2 (Xt ) −→ J 2 (Xt ) of Xt is nontrivial, even modulo torsion.

THE GRIFFITHS GROUP OF A GENERAL CALABI-YAU THREEFOLD

153

It is easy to check that J (Xt )tr = J (Xt ) for a general point t, so the theorem implies that Griff(Xt ) contains nontorsion elements. We prove in this paper the following result. Theorem 4. Let X be a Calabi-Yau threefold. If h1 (TX ) = 0, the general deformation Xt of X has the property that the Abel-Jacobi map Xt : ᐆ2 (Xt ) −→ J (Xt ) is such that Im Xt ⊗ Q is an infinite-dimensional Q–vector space. In particular, Griff(Xt ) ⊗ Q is an infinite-dimensional Q–vector space. The one-cycles in Xt we use to prove this result are the same as in [18]. Namely, we consider for |Lt | a sufficiently ample linear system on Xt , the surfaces S ∈ |Lt |, jS S → Xt having a class λ ∈ Ker(jS ∗ : H 2 (S, Z) → H 4 (Xt , Z)), which is in F 1 H 2 (S); that is, λ is algebraic, λ = c1 (Dλ ) for some divisor Dλ on S, by the Lefschetz theorem on (1, 1)-classes. It was proved in [17] that there are countably many isolated such surfaces in Xt , and the countably many corresponding one-cycles Zλ = jS ∗ (Dλ ) homologous to zero in Xt were proved in [18] to generate a nontorsion subgroup of J (Xt ) by the Abel-Jacobi map. We were unable to show, however, that this subgroup is nonfinitely generated. The method we use is in some sense related to a suggestion of Clemens in [4]. He suggested that a proof of the nonfinite generation of the Griffiths group of the general quintic threefold could be obtained by studying the ramification loci of the various generically finite coverings πd : ᏹd → ᏹ, where ᏹ is the moduli space for the quintic threefold and ᏹd parametrizes a quintic threefold X and a degree d isolated rational curve C in it. Along the ramification divisor of πd , the curve C ⊂ X has an infinitesimal deformation η in X, and there is a corresponding element X∗ (η) ∈ H 1,2 (X), which is the differential of X applied to the deformation η of the corresponding cycle in X. However, another important ingredient is the complexified Abel-Jacobi map; we use the complexified infinitesimal Abel-Jacobi map to prove Theorem 4. The “complexjS

ified” objects we study are the following: If S → X, and λ ∈ Ker(jS ∗ : H 2 (S, C) → H 4 (Xt , C)), we define Uλ as the set of deformations (Xt , St ) of the pair (X, S) such that the fixed class λt ∈ H 2 (St , C) ∼ = H 2 (S, C) belongs to F 1 H 2 (St ). It turns out that when X is a Calabi-Yau threefold and ᏹ is its local moduli space, most varieties Uλ are generically finite covers of ᏹ (by the map (Xt , St ) → Xt ). A point (Xt , St ) of ramification of this map then corresponds to a surface St ⊂ Xt that admits an infinitesimal deformation η such that λt ∈ F 1 H 2 (St ) remains (infinitesimally) in η F 1 H 2 (St ). There is then an associated complexified infinitesimal Abel-Jacobi invariant Xt ∗ (η) ∈ H 1,2 (Xt ). Notice that if λ is integral, it is the class of a divisor in St and we get the same invariant as above.

154

CLAIRE VOISIN

In Section 2, we introduce various Hodge theoretic objects and study the varieties Uλ defined above. We also define the complexified Abel-Jacobi map and “compute” its differential. In Section 3, we give a very simple infinitesimal criterion, which implies that the infinitesimal invariants above are nonzero and that if the image of the Abel-Jacobi map of Xt were finitely generated, these infinitesimal invariants would vanish. It follows that if this criterion is satisfied, then Theorem 4 is true. This infinitesimal criterion concerns the infinitesimal variation of Hodge structure of a generic sufficiently ample surface S ⊂ X. In Section 4, we check this criterion, which reduces (see [7]) to the study of Jacobian rings, that is, quotients of rings of functions by Jacobian ideals, generated by the derivatives of the defining equation of the surface along vector fields. 2. Noether-Lefschetz loci and infinitesimal Abel-Jacobi map. Part of the material in this section works in the general situation of a family of smooth surfaces ᏿ → B contained in a family of smooth threefolds ᐄ → B; however, we restrict π → B is the local universal family of the discussion to the following situation: ᐄ − deformations of a Calabi-Yau threefold X. B is a smooth ball, which can be assumed to be as small as we want. We have dim B = dim H 1 (TX ). Now let L be an ample line bundle on X; since H 1 (ᏻX ) = H 2 (ᏻX ) = 0 and H i (L) = 0, i > 0, by Kodaira vanishing and KX trivial, L extends uniquely to a line bundle ᏸ on ᐄ, and p → B is smooth dim H 0 (Xt , Lt ) = dim H 0 (X, L) for any t ∈ B. Then P(R 0 π∗ ᏸ) − over B, and we denote by U ⊂ P(R 0 π∗ ᏸ) the open set parametrizing smooth surπS πX faces. Let then ᏿ −→ U be the universal family, ᐄU −→ U be the pullback to U π → B, and j : ᏿ → ᐄU be the natural inclusion. First we have the of the family ᐄ − following lemma. Lemma 1. For sufficiently ample L, the tangent space TU,t at a point t identifies to H 1 (TSt ) by the Kodaira-Spencer map. It is also isomorphic to H 1 (TXStt ) by the Kodaira-Spencer map for pairs, where TXStt is the kernel of the natural map TXStt −→ NSt /Xt . Proof. We have the exact sequence 0 −→ TXt (−Lt ) −→ TXStt −→ TSt −→ 0, which induces the natural map H 1 TXStt −→ H 1 TSt , from the deformations of the pair to the deformations of the surface. So by Serre vanishing, the map above is an isomorphism for sufficiently ample L.


155

Next we have the exact sequence p∗ 0 −→ H 0 Lt |St −→ TU,t −−→ TB,p(t) −→ 0 and the exact sequence defining TXStt , 0 −→ TXStt −→ TXt → Lt |St −→ 0, which induces the exact sequence 0 −→ H 0 Lt |St −→ H 1 TXStt −→ H 1 (TXt ) −→ 0, since H 0 (TXt ) = 0 and H 1 (Lt |St ) = 0. Finally, the Kodaira-Spencer map TU,t → H 1 (TXStt ) fits into the commutative diagram 0

/ H 0 Lt |S t

/ TU,t

0

/ H 0 Lt |S t

/ H 1 T St Xt

p∗

/ TB,p(t)

/0

/ H 1 (TX ) t

/ 0,

where the first and last vertical maps are the identity. It follows immediately that the middle map is an isomorphism. We have on U the primitive variation of Hodge structure of the family of surfaces

᏿: namely, let

j∗ HZ2 := Ker R 2 πS ∗ Z −−→ R 4 πX∗ Z

be the local system whose fiber at t is jt ∗ H 2 (St , Z)0 := Ker H 2 (St , Z) −−→ H 4 (Xt , Z) . Let Ᏼ2 := HZ2 ⊗ ᏻU , with its Gauss-Manin connection ∇ S : Ᏼ2 → Ᏼ2 ⊗ *U , whose local system of flat sections is HC2 = HZ2 ⊗ C. Let F i Ᏼ2 , 0 ≤ i ≤ 2 be the Hodge bundles, with fiber F i Ᏼt2 = F i H 2 (St ) ∩ Ker jt ∗ ,

F i H 2 (St ) = ⊕p≥i H p,2−p (St )

and associated quotients Ᏼi,2−i = F i Ᏼ2 /F i+1 Ᏼ2 . By transversality, we have ∇ S F i Ᏼ2 ⊂ F i−1 Ᏼ2 ⊗ *U . We denote by

S

∇ : Ᏼi, 2−i −→ Ᏼi−1, 3−i ⊗ *U

156

CLAIRE VOISIN S

the ᏻU -linear map deduced from ∇ S by transversality, so that ∇ fits into the commutative diagram / F i Ᏼ2 ⊗ * ∇ S : F i+1 Ᏼ2 U ∇ S : F i Ᏼ2 S

∇ :

Ᏼi, 2−i

/ F i−1 Ᏼ2 ⊗ * U / Ᏼi−1, 3−i ⊗ * . U

S

For λ ∈ H i,j (Sv )0 we then have ∇ (λ) ∈ Hom(TU,v , H i−1,j +1 (Sv )0 ). For η ∈ TU,v , S we denote by ∇ η the induced map H i,j (Sv )0 → H i−1,j +1 (Sv )0 . Let V be a simply connected open subset of U . Then the local system HC2 is trivial on V , so that if v0 ∈ V and λ ∈ H 2 (Sv0 , C)0 , we can view λ as a section of HC2 on V . We then define the component of the Noether-Lefschetz locus determined by λ as Vλ = t ∈ V , λt ∈ F 1 H 2 (St )0 . Vλ is an analytic subvariety of V , defined by the vanishing of the projection in Ᏼ0,2 of the flat, hence holomorphic, section λ ∈ Ᏼ2 . If t ∈ Vλ , λt ∈ F 1 H 2 (St )0 and hence has a projection λ1,1 ∈ H 1,1 (St )0 = Ᏼt1,1 . Then the next lemma follows from the t S definition of ∇ . S

Lemma 2. The Zariski tangent space to Vλ at t is equal to Ker ∇ (λ1,1 t ), where 1,1 0,2 ∇ (λt ) ∈ Hom(TV ,t , H (St )). S

Note that usually the terminology of the Noether-Lefschetz locus is reserved to the case where λ is rational. In this case, by the Lefschetz theorem on (1, 1)-classes, Vλ is the set of points v ∈ V where the class λv is algebraic; that is, any multiple mλ λv that is an integral class is the class [Dλ,v ] of a divisor on Sv . Then since jv ∗ λv = 0, jv ∗ (Dλ,v ) is a one-cycle homologous to zero in Xv . We have the following convenient interpretation of Vλ : Let v0 be any point of V ; then HC2 ∼ = V × H 2 (Sv0 , C)0 . Viewing F 1 Ᏼ2 , Ᏼ2 as vector bundles, we have a map (2.0) φ : F 1 Ᏼ2 −→ H 2 Sv0 , C 0 ∼ obtained as the composition of the inclusion F 1 Ᏼ2 ⊂ Ᏼ2 , the isomorphism Ᏼ2 = H 2 (Sv0 , C)0 × V given by the trivialization of HC2 , and the first projection. Then we have that Vλ is naturally isomorphic to φ −1 (λv0 ). Indeed, by definition, Vλ × λv0 ⊂ V × H 2 (Sv0 , C)0 ∼ = Ᏼ2 is the scheme-theoretic intersection of Vλ × λv0 and F 1 Ᏼ2 in 2 Ᏼ ; but this is also the definition of the fiber φ −1 (λv0 ). 2 , gives the In other words, the flat section λ restricted to Vλ , which is in F 1 Ᏼ|V λ −1 reverse isomorphism Vλ → φ (λv0 ). We abuse notation in Section 3 and view, by 2 . this isomorphism, Vλ as a subvariety of F 1 Ᏼ|V


157

We denote by 1,1 λ1,1 ∈ Ᏼ|V λ

(2.1)

2 . Now if v ∈ V and λ1,1 ∈ H 1,1 (S ) , let the projection of the section λ ∈ F 1 Ᏼ|V v 0 1 2 1,1 λ1 , λ2 ∈ F H (Sv )0 be two liftings of λ , so that λ1 = λ2 + η, for some η ∈ H 2,0 (Sv ). By Lemma 2 the tangent spaces to Vλi at v coincide, and the two sections / λ1,1 i , which are defined on the first-order neighbourhood Vλ of v in Vλi (where i = 1 or 2) are equal at v. However, their derivatives do not coincide. In fact, we have the next lemma. 1,1 1,1 Lemma 3. The derivative at v of the section λ1,1 1 − λ2 of Ᏼ|Vλ/ (which vanishes 1,1 at v), is equal to −∇(η)|TVλ ,v : TVλ ,v → H (Sv )0 .

Proof. Let h ∈ TVλ/ ,v and let Zh be the scheme of length two supported on v with tangent vector h. Then the section λh1 = λ1|Zh of F 1 Ᏼ2 is the flat section that extends λ1 ∈ F 1 H 2 (Sv )0 and that remains in F 1 Ᏼ2 . Now, η being given above, let µh2 := λh1 + /∇hS (η) ˜ − η, ˜ where η˜ is a section of F 2 Ᏼ2 on Zh extending η. Then clearly µh2 is flat and its value at v is equal to λ2 . Furthermore, µh2 is a section of F 1 Ᏼ2 on Zh by transversality. It follows that λh2 = µh2 . Hence, ˜ + η, ˜ λh1 − λh2 = −/∇h (η) S

so that by projecting to Ᏼ1,1 and using the definition of ∇ , we get

λh1

1,1

1,1 S − λh2 = −/∇ (η)(h),

which proves the lemma. We now turn to the generalized Abel-Jacobi map and its infinitesimal version. For u ∈ U , let Yu = Xu − Su . We have an exact sequence Res

0 −→ H 3 (Xu ) −→ H 3 (Yu ) −−→ H 2 (Su )0 −→ 0 of cohomology groups with integral coefficients, and H 3 (Yu , C) carries a mixed Hodge structure compatible with the Hodge structures on H 3 (Xu ) and H 2 (Su )0 . Namely, we have a decreasing filtration F i H 3 (Yu ), 0 ≤ i ≤ 3, such that F i H 3 (Yu ) ∩ H 3 (Xu ) = F i H 3 (Xu ), Res F i H 3 (Yu ) = F i−1 H 2 (Su )0 , where F i H 3 (Xu ) = ⊕p≥i H p,3−p (Xu ) is the Hodge filtration of Xu .

158

CLAIRE VOISIN

Working in families, we get the local system 3 HY,Z = R 3 πY ∗ Z,

where πY = πX|ᐅ , ᐅ = ᐄU − ᏿. We then define the associated Hodge bundles ᏴY3 by tensorizing the local system with ᏻU . We denote by ∇ Y the Gauss-Manin connection on ᏴY3 . This bundle is equipped with the Hodge filtration by holomorphic subbundles F i ᏴY3 , which satisfy Griffiths transversality ∇ Y F i ᏴY3 ⊂ F i−1 ᏴY3 ⊗ *U . We denote by HZ3 , Ᏼ3 , F i Ᏼ3 , and ∇ X the analogous objects on B that describe the variation of Hodge structure of the family π : ᐄ → B; that is, HZ3 = R 3 π∗ Z, F i Ᏼ3 ⊂ Ᏼ3 with Ᏼ3 = HZ3 ⊗ ᏻB , and ∇ X : Ᏼ3 → Ᏼ3 ⊗ *B with ∇ X F i Ᏼ3 ⊂ F i−1 Ᏼ3 ⊗ *B . We then have an exact sequence of variation of mixed Hodge structures 3 0 −→ p∗ HZ3 −→ HY,Z −→ HZ2 −→ 0.

(2.2)

3 of (2.2). Denoting by On our open set V , let us choose a splitting rZ : HZ2 → HY,Z 1 2 1 2 P : F Ᏼ → B the composite of the bundle map F Ᏼ → U and the map p : U → B, the section rZ allows us to construct a section

s∈

P ∗ Ᏼ3 F 2 Ᏼ3

(2.3)

2 as follows: If (v, λ) ∈ F 1 Ᏼ2 , that is, λ ∈ F 1 H 2 (S ) , let λ be a lifting over F 1 Ᏼ|V v 0 F 2 3 of λ in F H (Yv ). Then we define

s(v, λ) = λF − rZ (λ) mod .F 2 H 3 (Xv ). This is a well-defined element of H 3 (Xv , C)/F 2 H 3 (Xv ), since clearly λF − rZ (λ) belongs to H 3 (Xv , C) and λF (v) is defined up to F 2 H 3 (Xv ). In fact, we are mainly interested with the restriction of s to the subvarieties 3 as the comφ −1 (λ0 ) ∼ = Vλ . We may then consider these sections of p∗ Ᏼ3 /F 2 Ᏼ|V λ plexified Abel-Jacobi map, as we explain now. Suppose that λ ∈ H 2 (Sv , Z)0 ∩ F 1 H 2 (Sv ). Then λ = [Dλ ] for some divisor Dλ on Sv , and jv ∗ (Dλ ) is a one-cycle homologous to zero on Xv . It is then well known that the element Xv jv ∗ (Dλ ) ∈ J (Xv ) = H 3 (Xv , C)/F 2 H 3 (Xv ) ⊕ H 3 (Xv , Z)


159

is equal to λF − rZ (λ) mod .F 2 H 3 (Xv ) ⊕ H 3 (Xv , Z). (The fact that we consider it modulo H 3 (Xv , Z) makes it independent of the retraction rZ .) In other words, for integral λ we find that s|Vλ mod .HZ3 is equal to the section νλ of the pullback to Vλ of the family of intermediate Jacobians J (Xb )b∈B given by νλ (v) = Xv jv ∗ (Dλ ) ∈ J (Xv ), v ∈ Vλ . We now want to study the infinitesimal properties of the map φ defined in (2.0) or equivalently of the varieties Vλ . Recall that for v ∈ U, λ ∈ H 1,1 (Sv )0 we have the map S ∇ (λ) : H 1 TSu = TU,u −→ H 2 ᏻSu , which induces

S ∇ (λ) : H 0 Lu |Su = Ker p∗ ⊂ TU,u −→ H 2 ᏻSu .

Note that by Serre duality and because KXv is trivial, both spaces have the same dimension. We have the following lemma. Lemma 4. The following are equivalent: S (i) ∇ (λ) : H 0 (Lu |Su ) → H 2 (ᏻSu ) is an isomorphism; (ii) for any λ˜ ∈ F 1 H 2 (Sv )0 projecting to λ modulo F 2 H 2 (Sv ), the map (P , φ) : F 1 Ᏼ02 −→ B × H 2 Sv0 , C 0 ˜ is étale at λ. Proof. We may clearly assume that u = v0 since the change of base point simply composes φ with the natural isomorphism H 2 (Sv0 )0 ∼ = H 2 (Su )0 . Consider (P , φ)∗ : 1 2 TF 1 Ᏼ2 ,λ˜ → TB,p(u) ×TH 2 (Su )0 ,φ(λ) ˜ . Since on F H (Su )0 ⊂ TF 1 Ᏼ2 ,λ˜ this map is simply the inclusion F 1 H 2 (Su )0 ⊂ H 2 (Su )0 = TH 2 (Su )0 ,φ(λ) ˜ , this map induces

2 (P , φ)0,2 ∗ : TU,u −→ TB,p(u) × H ᏻSu .

S S It is then immediate, using the definition of ∇ , to show that (P , φ)0,2 ∗ = p∗ , ∇ . But (P , φ)∗ is an isomorphism if and only if (P , φ)0,2 ∗ is an isomorphism. Since p∗ is sur2 jective, this is also equivalent to (P , φ)0,2 ∗ |Ker p∗ being an isomorphism onto H (ᏻSu ), S

that is, to ∇ (λ) : H 0 (Lu |Su ) → H 2 (ᏻSu ) being an isomorphism. So Lemma 4 is proved. In fact, the proof shows the following lemma. Lemma 5. The kernel of (P , φ)∗ at λ˜ identifies naturally via the projection to TU,u to S Ker ∇ (λ) : H 0 Lu |Su −→ H 2 ᏻSu ,

160

CLAIRE VOISIN p(u) λ

that is, to the vertical part TV p(u) of TVλ˜ , where V ˜ P(H 0 (L

p−1 (p(u)).

p(u) ))

λ˜

is the intersection of Vλ˜ with

= The reverse isomorphism is given by the differential of p(u) 1 2 ˜ the natural section λ of F Ᏼ on V ˜ . λ

We now study the infinitesimal variation of mixed Hodge structure of the family πY

ᐅ → U . It is described as above, by transversality, by a series of maps Y

∇ : F i /F i+1 ᏴY3 −→ F i−1 /F i ᏴY3 ⊗ *U , which fit into the commutative diagram i i+1 3 X ∗ ∇ ◦ p ∗ : p F /F Ᏼ

/ p ∗ F i−1 /F i Ᏼ3 ⊗ *U

i /F i+1 Ᏼ3 F ∇ : Y

/ F i−1 /F i Ᏼ3 ⊗ *U Y

i−1 /F i Ᏼ2 F ∇ :

/ F i−2 /F i−1 Ᏼ2 ⊗ * , U

Y

S

(2.4)

where the first vertical maps are injective and the last ones are surjective. Composing Y ∇ with the restriction map *U,u → H 0 (Lu |Su )∗ then gives a map F 2 /F 3 H 3 (Yu ) −→ Hom H 0 Lu |Su , F 1 /F 2 H 3 (Yu ) , which obviously factors through F 1 /F 2 H 2 (Su )0 since the composition of p∗ with the restriction to H 0 (Lu |Su ) = Ker p∗ is zero. (This simply means that there is no variation of Hodge structure for X in the fibers of p.) So we have constructed a map µ0 : H 1 *Su 0 −→ Hom H 0 Lu |Su , F 1 /F 2 H 3 (Yu ) , which induces

µ1 : H 0 Lu |Su −→ Hom H 1 *Su 0 , F 1 /F 2 H 3 (Yu ) .

We then have the following. Lemma 6. There is a natural isomorphism (depending on the choice of a trivialization of KXu ) ∗ F 1 /F 2 H 3 (Yu ) ∼ = (TU,u )∗ = H 1 TSu such that for any η ∈ H 0 (Lu |S ) ∼ = H 0 (KS ), the map u

u

µ1 (η) : TU,u −→ H 1 *Su 0

t

S

identifies to ∇ (η).


161

Notice that in the identification H 0 (Lu |Su ) ∼ = H 0 (KSu ), we use the same trivialization of KXu . Proof. Recall the isomorphisms TU,u = H 1 (TSu ) = H 1 TXSuu of Lemma 1. Now TXSuu is dual to *Xu (log Su ), so that, choosing a trivialization of KXu , we get an isomorphism H 1 (TXSuu ) ∼ = (H 2 (*Xu (log Su )))∗ , and taking into account the natural isomorphism (see [5]) H 2 (*Xu (log Su )) = F 1 /F 2 H 3 (Yu ), we get the first assertion. Next it is known (this is an easy generalization of [9]) that the map Y

∇ : F 2 /F 3 ᏴY3 −→ F 1 /F 2 ᏴY3 ⊗ *U identifies to the map given by the interior product H 1 *2Xu (log Su ) −→ Hom H 1 TXSuu , H 2 *Xu (log Su ) .

(2.5)

The image of the map (2.5) is contained in the set of symmetric homomorphisms from H 1 (TXSuu ) to its dual; indeed the dual of (2.5) is equal to the (symmetric) cup product

2 Su Su Su 1 1 2 TXu , H TXu ⊗ H TXu −→ H taking into account the isomorphism 2 TXSuu = (*2Xu (log Su ))∗ , the triviality of KXu , and Serre duality. It follows that for λ ∈ H 1 (*2Xu (log Su )), η, χ ∈ H 1 (TXSuu ), we have

Y Y ∇ (λ)(η), χ = ∇ (λ)(χ), η .

(2.6)

Now note that the inclusion H 0 (Lu |Su ) → H 1 TXSuu is dual to the residue map Res : H 2 (*Xu (log Su )) → H 2 (ᏻSu ), so that for η ∈ H 0 (Lu |Su ), (2.6) gives

Y Y ∇ (λ)(η), χ = Res ∇ (λ)(χ) , η ,

(2.7)

where the second pairing in (2.7) is the duality between H 0 (Lu |Su ) and H 2 (ᏻSu ). (We always use the same trivialization of KXu to compute the pairings.) But by Y S diagram (2.4), we have Res(∇ (λ)(χ)) = ∇ (Res(λ))(χ)) and by definition of µ1 , Y we have ∇ (λ)(η) = µ1 (η)(Res(λ)). So we have proved for any λ ∈ H 1 (*Su )0 , for any η ∈ H 0 (Lu |Su ) and χ ∈ H 1 TXSuu , the equality

S µ1 (η)(λ), χ = ∇ (λ)(χ), η ,

(2.8)

where the first pairing is the duality above between H 1 TXSuu and H 2 (*Xu (log Su )), while the second one is the duality between H 0 (Lu |Su ) and H 2 (ᏻSu ). Finally, for

162

CLAIRE VOISIN

S η ∈ H 0 (Lu |Su ) ∼ = H 0 (KSu ), λ ∈ H 1 (*Su )0 , χ ∈ H 1 TXuu , we have the equalities

t

S (2.8) S S ∇ (η) (λ), χ = λ, ∇ (η)(χ) = ∇ (λ)(χ), η = µ1 (η)(λ), χ ,

where the second equality is standard and follows from the fact that the intersection pairing on H 2 (Su )0 is flat with respect to the Gauss-Manin connection and that, for this S pairing, H 2,0 (Su ) is perpendicular to F 1 H 2 (Su )0 . This proves that t (∇ (η)) = µ1 (η), as we wanted. We conclude this section with the “complexified infinitesimal Abel-Jacobi map.” Recall that from Lemma 5 we get, for λ ∈ H 1 (*Su )0 with lifting λ˜ ∈ F 1 H 2 (Su )0 , S a natural identification between Ker ∇ (λ)|H 0 (Lu|S ) and Ker P∗ : TV ˜ ×λ˜ ,λ˜ → TB,p(u) , u λ where by definition of V ˜ , V ˜ × λ˜ ⊂ U × H 2 (Su0 , C)0 is in fact contained in F 1 Ᏼ2 . λ

λ

Now consider the section s of the bundle P ∗ Ᏼ3 /F 2 Ᏼ3 constructed in (2.3). Since this bundle is naturally trivial on the fibers of P , it makes sense to differentiate s|V ˜ ×λ˜ λ in the direction contained in Ker P∗ . It follows that we have a map S

ds : Ker P∗ = Ker ∇ (λ)|H 0 (Lu|S ) −→ H 3 (Xu )/F 2 H 3 (Xu ). u

On the other hand, the map µ0 : H 1 *Su 0 −→ Hom H 0 Lu |Su , F 1 /F 2 H 3 (Yu ) S

satisfies Res ◦µ0 (λ) = ∇ (λ)|H 0 (Lu|S

u)

and hence induces a map

S

µ2 : Ker ∇ (λ)|H 0 (Lu|S ) −→ H 3 (Xu )/F 2 H 3 (Xu ). u

Now we have the following. Lemma 7. We have the equality ds = µ2 . Proof. Indeed, recall that s|V ˜ ×λ˜ is equal to λ˜ F − r Z (λ˜ ) mod . F 2 Ᏼ3 , where λ˜ F is λ 2 in F 2 Ᏼ3 ˜ any lifting of λ˜ ∈ F 1 Ᏼ|V Y |V . Since λ is flat, we have ˜ λ˜

λ

∇ X λ˜ F − rZ λ˜ = ∇ Y λ˜ F ; Y

since λ˜ F is a section of F 2 ᏴY3 |V , we have, by definition of ∇ , λ˜

Y ∇ Y λ˜ F mod . F 2 ᏴY3 = ∇ λF ,


163

where λF is the projection of λ˜ F in F 2 ᏴY3 /F 3 ᏴY3 . But then for h ∈ TVλ˜ ,u ∩ Ker p∗ = S

Ker ∇ (λ)|H 0 (Lu|S ) , we have u

Y ds(h) = ∇hX λ˜ F − r Z λ˜ mod . F 2 H 3 (Xu ) = ∇ λF (h), and by definition of µ2 the right-hand side is equal to µ2 (h). The reason we call ds or µ2 the complexified infinitesimal Abel-Jacobi map is, again, that if λ˜ is an integral class, we have shown that s|V ˜ ×λ˜ is a lifting of the λ normal function νλ˜ (u) = Xu jSu ∗ Dλ˜ ,u ∈ J (Xu ) to a section of p ∗ (Ᏼ3 /F 2 Ᏼ3 ) on Vλ˜ . Then if h ∈ Ker p∗ , ds(h) is simply the differential of νλ˜ in the direction h, which makes sense since Xu , and hence J (Xu ) remains constant in the direction h. 3. An infinitesimal criterion for the nonfinite generation of the image of the Abel-Jacobi map. With the notation of Section 2, we now assume that dim B > 0. Recall that we have defined for Su ⊂ Xu and for λ ∈ H 1 (*Su )0 , η ∈ H 0 (KSu ) the maps S ∇ (η) : H 1 TSu −→ H 1 *Su 0 , S ∇ (λ) : H 1 TSu −→ H 2 ᏻSu . We prove in this section the following infinitesimal criterion for the infinite generation of the Griffiths group of the general fiber Xb . Proposition 1. Assume that Lu is sufficiently ample and that for generic u ∈ U and generic η ∈ H 0 (KSu ), we have that S (i) the map ∇ (η) : H 1 (TSu ) → H 1 (*Su )0 is injective; S (ii) for generic λ ∈ H 1 (*Su )0 such that ∇ (λ)(η) = 0, we have that S Ker ∇ (λ) : H 0 Lu |Su −→ H 2 ᏻSu is generated by η. Then for the general point t ∈ B, the Abel-Jacobi map φXt of Xt satisfies that Im Xt ⊗ Q is an infinite-dimensional Q–vector space. In assumption (ii), η is viewed as an element of H 0 (Lu |Su ) ⊂ H 1 (TSu ). To start the proof, we first note the following lemma. Lemma 8. Assumption (ii) implies that for generic u and generic λ ∈ H 1 (*Su )0 , the map S ∇ (λ) : H 0 Lu |Su −→ H 2 ᏻSu is an isomorphism.

164

CLAIRE VOISIN

Proof. Identifying H 0 (Lu |Su ) with H 0 (KSu ) by a trivialization of KXu , the maps ∇ (λ) : H 0 (Lu |Su ) → H 2 (ᏻSu ) are symmetric, with respect to Serre duality. Hence S ∇ (λ) determines a quadric qλ on P(H 0 (Lu |Su )). If λ is as in assumption (ii), the quadric qλ has η for only singular point, and since η is generic, it is not in the base locus of the system of quadrics qλ . Hence the tangent space at λ to the discriminant hypersurface, parametrizing singular quadrics qλ being equal to the set of qλ vanishing at η, is a proper subspace of H 1 (*Su )0 , so the generic qλ is smooth. S

S

Now note that the condition that ∇ (λ) : H 0 (Lu |Su ) → H 2 (ᏻSu ) is an isomorphism is Zariski open on H 1 (*Su )0 , which is the complexification of HR1,1 (Su )0 := H 1,1 (Su ) ∩ H 2 (Su , R)0 . So if it is satisfied at some point, it will be satisfied at some real point λ ∈ HR1,1 (Su )0 , which obviously has a natural (real) lifting λ in F 1 H 2 (Su )0 . From Lemma 4 we know that at such a λ ∈ F 1 Ᏼ2 the map (P , φ) : F 1 Ᏼ2 −→ B × H 2 (Su0 , C)0 is étale, so it is a local isomorphism for the usual topology. Hence there are open connected neighbourhoods B ⊂ B of p(u), V ⊂ H 2 (Su0 , C)0 of φ(λ), and W ⊂ (P ,φ) F 1 Ᏼ2 of λ, with W ∼ = B × V . Finally, note that since φ(λ) is real, the rational 2 points in V ∩H (Su0 , Q)0 are Zariski dense in V . For any such rational point λ ∈ V , the fiber φ −1 (λ) ∩ W is then naturally isomorphic to B by P , and it parametrizes jt then the pairs St → Xt such that λt is algebraic on St . For each such λ, we choose an integer mλ such that mλ λ is integral, and then mλ λ = c1 (Dλ,t ) on St . Hence we get a normal function νλ on B , that is, a section of the sheaf ᏶ = Ᏼ3 /F 2 Ᏼ3 ⊕ HZ3

defined by

νλ (t) = Xt jt ∗ (Dλ,t ) ∈ J (Xt ).

, in order to prove Proposition 1. So we assume We use the countably many νλ , λ ∈ VQ by contradiction the following assumption:

(∗) For any general point t ∈ B , the image of Xt tensorized by Q is finitely generated. Then we have the following. such that for any λ ∈ V , Lemma 9. If (∗) holds, there exists λ1 , . . . , λN ∈ VQ Q there exist integers m = 0, m1 , . . . , mN , satisfying the equality mνλ = mi νλi in ᏶. i . For any sequence Proof. Choose an ordering λi , i ∈ N, of the elements of VQ (αi )i∈N of integers with only finitely many nonzero elements, let


Bα · = t ∈ B ,

165

αi νλi (t) = 0 in J (Xt ) .

i

Then Bα · is an analytic subset of B , so any point in B = B − Bα · Bα · =B

is general. On B , any relation with integral the other hand, by definition, if t ∈ coefficients i αi νλi (t) = 0 in J (Xt ) implies that i αi νλi = 0 in ᏶. Lemma 9 follows, taking any t ∈ B at which (∗) holds. Coming back to the section s of P ∗ (Ᏼ3 /F 2 Ᏼ3 )|W ) defined in (2.3), we get the following corollary. Corollary 1. Under the same assumption (∗), for any λ ∈ V , s|B ×λ belongs to the finite vector space K of holomorphic sections of (Ᏼ3 /F 2 Ᏼ3 )|B generated by the image of HC3 |B in (Ᏼ3 /F 2 Ᏼ3 )|B and by liftings ν˜ λi of νλi in (Ᏼ3 /F 2 Ᏼ3 )|B for i = 1, . . . , N . Here we use the isomorphism W ∼ = B × V given by (P , φ). . Indeed, a relation mν = Proof. By Lemma 9, the conclusion is true for λ ∈ VQ λ m ν in ᏶ is equivalent to a relation i λ i i Ᏼ3 m˜νλ = mi ν˜ λi + α in , F 2 Ᏼ3 |B i

where α ∈ HZ3 and ν˜ λi , ν˜ λ are liftings of our normal functions in (Ᏼ3 /F 2 Ᏼ3 )|B . On the other hand, we have shown in Section 2 that mλ s|B ×λ , mλi s|B ×λi give such liftings. In order to deduce from this that the conclusion is true for any λ ∈ V , we use now in V . To be precise, using a trivialization of the bundle the Zariski density of VQ (Ᏼ3 /F 2 Ᏼ3 )|B , Corollary 1 will follow now from the next lemma. Lemma 10. Let K be a finite-dimensional set of functions on B . Let f be a function on B × V , where V is a connected open set of Ck meeting Rk , such that for any λ ∈ V ∩ Qk , f|B ×λ ∈ K. Then for any λ ∈ V , f|B ×λ ∈ K. Proof. Let k = dim K and let p1 , . . . , pk be points on B such that the restriction map K → ⊕i ᏻpi is an isomorphism. Then we have a basis (ki ) of K such that ki (pj ) = δij . So any element g of K satisfies g = i g(pi )ki . It follows that the function f (b, v) − i f (pi , v)ki (b) on B × V vanishes on B × (V ∩ Qk ), hence everywhere, by the (analytic) Zariski density of B × (V ∩ Qk ) in B × V . Now we use analytic continuation to conclude the following.

166

CLAIRE VOISIN

Corollary 2. Let U be any open subset of U contained in the image of the 2 and that V is open projection W → U . (We recall that W is an open subset of F 1 Ᏼ|V 2 such that (P , φ) is in U .) Then under the same assumption (∗), for any λu ∈ F 1 Ᏼ|U ∗ étale at λu , the section s|Uλ ,0 belongs to P (K), where Uλu ,0 denotes the irreducible u component of Uλ u ∼ = φ −1 (φ(λu )) containing u (which is unic since by hypothesis Uλ u is smooth at u). 2 ) denote the Zariski-dense open subset of F 1 Ᏼ2 where Proof. Let (F 1 Ᏼ|U et |U 2 1 (P , φ) is étale. We can cover (F Ᏼ|U )et by connected open sets Wi isomorphic to Bi × Vi by (P , φ) for some open subsets Bi of B and Vi of H 2 ((Su0 , C)0 ). Then for λu ∈ Wi , Bi × φ(λu ) is open in Uλ u ,0 . So if we show that s|Bi ×φ(λu ) belongs to K, there is a k ∈ K such that

P ∗ k|Bi ×φ(λu ) = s|Bi ×φ(λu ) , and this will be true everywhere on Uλ u ,0 by analytic continuation. So it suffices to prove the following: For any Wi ∼ = Bi × Vi , we have that for any λ ∈ Vi , s|Bi ×λ belongs to K. But using the same argument as in Corollary 1, we see that if this is true for Wi and if Wi ∩ Wj = ∅, this is true as well for Wj . Since this 2 ) is connected, this is true for all W . is true on W by Corollary 1 and (F 1 Ᏼ|U et i Corollary 2 is proved. We conclude with the following corollary. 2 be such that U is irreducible reduced, and Corollary 3. Let λu ∈ F 1 Ᏼ|U λu generically finite over B via p. Then if (∗) is satisfied, for any h ∈ TUλ ,λu , such that

P∗ (h) = 0 in TB,p(u) , we have ds(h) = 0 in H 3 (Xu )/F 2 H 3 (Xu ).

u

This is immediate since P : Uλ u → B is a generic isomorphism, that is, (P , φ) is étale at the generic point of Uλ u . We then can apply the previous corollary and conclude that for some k ∈ K, we have P ∗ (k) = s on some open set of Uλ u . Hence the equality is true everywhere by irreducibility, and it follows that the vertical derivatives of s|Uλ vanish. Proof of Proposition 1. We now show that the hypotheses of Proposition 1 contradict the conclusion of Corollary 3. The hypotheses are as follows: S (i) the map ∇ (η) : H 1 (TSu ) → H 1 (*Su )0 is injective for generic u and η ∈ 0 H (KSu ); S (ii) for generic λ ∈ H 1 (*Su )0 , such that ∇ (λ)(η) = 0, we have that S Ker ∇ (λ) : H 0 Lu |Su −→ H 2 ᏻSu is generated by η.


167

Recall from Lemma 6 that the transposed map t

satisfies

∗ F 1 H 3 (Yu ) S ∼ ∇ (η) : H 1 *Su 0 −→ H 1 TSu = 2 3 F H (Yu ) S S Res ◦t ∇ (η) = ∇ η : H 1 *Su 0 −→ H 2 ᏻSu . S

Now hypothesis (i) says that t (∇ (η)) is surjective; furthermore, the condition dim B > S 0 implies dim F 1 H 3 (Xu )/F 2 H 3 (Xu ) > 0. It follows that for generic λ ∈ Ker ∇ η , we S

have t (∇ (η))(λ) = 0 in

F 1 H 3 (Yu ) F 1 H 3 (Xu ) 2 = Ker Res : −→ H ᏻ . Su F 2 H 3 (Xu ) F 2 H 3 (Yu ) S

S

Note also that by definition ∇ η (λ) = ∇ (λ)(η) ∈ H 2 (ᏻSu ), so we conclude from assumption (ii) that we can find λ such that S (a) Ker ∇ (λ) is generated by η, with η generic in H 0 (Lu |Su ); S

(b) t (∇ (η))(λ) = 0 in F 1 H 3 (Xu )/F 2 H 3 (Xu ). S Now recall Lemmas 6 and 7, which say that for η ∈ Ker ∇ (λ), so that η is tangent 1 2 to Vλ˜ at u for any λ˜ ∈ F H (Su )0 over λ, and η is annihilated by p∗ , t

F 1 H 3 (Xu ) S ∇ (η) (λ) ∈ 2 3 F H (Xu )

is equal to ds|Vλ˜ (η). So the hypotheses imply that for any λ˜ ∈ F 1 H 2 (Su )0 over λ, the vertical derivative of s|Vλ˜ is nonzero. In order to contradict Corollary 3, it suffices now to show that we can choose λ˜ so that Vλ˜ is smooth at u and generically finite over B. The first statement follows easily from (a) and (b): indeed, to prove the smoothness of Vλ˜ at u, it suffices to show that S ∇ (λ) : H 1 TSu −→ H 2 ᏻSu is surjective or that its dual t

∗ F 1 H 3 (Yu ) S ∇ (λ) : H 0 Lu |Su −→ H 1 TSu = 2 3 F H (Yu )

is injective. S S But one sees easily, as in Section 2, that Res ◦t (∇ (λ)) is equal to ∇ (λ)|H 0 (Lu|S ) , u and hence its kernel is generated by η by (a). Furthermore, one has the equality t

S F 1 H 3 (Xu ) S , ∇ (λ) (η) =t ∇ (η) (λ) ∈ 2 3 F H (Xu )

168

CLAIRE VOISIN S

and by (b) this is nonzero. So t (∇ (λ)) is injective, as we wanted to prove. What remains is to show that for general λ˜ lifting λ, the variety Vλ˜ is generically finite over B via P . Recall from (2.1) that on Vλ˜ we have a natural section λ˜ 1,1 of the bundle Ᏼ1,1 . Now on the total space of Ᏼ1,1 , let Ᏸ be the discriminant hypersurface; that is, for any u, S Ᏸu = λ ∈ Ᏼu1,1 , ∇ (λ) : H 0 Lu |Su −→ H 2 ᏻSu is not an isomorphism . If q : F 1 Ᏼ2 → Ᏼ1,1 is the natural projection, it follows from Lemma 4 that q −1 (Ᏸ) is equal to the ramification locus of the map (P , φ). This implies that the ramification locus of the map P|Vλ˜ is equal to (λ˜ 1,1 )−1 (Ᏸ). Hence P|Vλ˜ is generically finite if and only if λ˜ 1,1 (Vλ˜ ) is not contained in Ᏸ. Now since η is contained in the vertical tangent space of Vλ˜ at u, it suffices to prove that λ˜ 1,1 ∗ (η) is not tangent to Ᏸu at λ. But the symmetric maps S ∇ (µ) : H 0 Lu |Su −→ H 2 ᏻSu can be viewed as quadrics qµ on P(H 0 (Lu |Su )). Then the assumption on λ means that qλ has η as its only singular point. It follows that the tangent space to Ᏸu at λ is the set {µ ∈ H 1 (*Su )0 , qµ (η) = 0}. Now we use Lemma 3 and conclude that if λ˜ 1,1 ∗ (η) was tangent to Ᏸu at λ for any ˜λ lifting λ, the subspace ∇ Sη (H 0 (KSu )) of H 1 (*Su )0 would be tangent to Ᏸu at λ. Hence we would have the following: For any ω ∈ H 0 (KSu ),

η, ∇

S

S ∇ η (ω) (η) = 0.

(3.9)

This cannot hold for generic η and sufficiently ample L for the following reason: One can show (and this is done in the next section) by describing the variation of Hodge structure of the family of surfaces Su (with fixed Xu ) in terms of the Jacobian ring associated to Su ⊂ Xu (see [7] and [10]) that there is a natural surjective map ψ : H 0 3Lu |Su −→ H 2 ᏻSu and that (3.9) would mean exactly that ψ(η3 ) = 0. But if L is sufficiently ample, the multiplication map S 3 H 0 (Lu ) → H 0 (3Lu ) is surjective, so that ψ(η3 ) = 0 for any η would imply that ψ = 0, which is absurd since H 2 (ᏻSu ) = 0. So we have obtained the desired contradiction with the conclusion of Corollary 3, and this shows that the finiteness assumption (∗) is absurd. The proof of Proposition 1 is now complete.

4. Checking the infinitesimal criterion for any Calabi-Yau threefold. In this section we prove that conditions (i) and (ii) of Proposition 1 are satisfied for a sufficiently large multiple of an ample line bundle on X. This will conclude the proof of Theorem 4. We start with the proof of (i).


169

Proposition 2. Let X be a Calabi-Yau threefold and L1 be a line bundle on X. If L1 is sufficiently ample, any sufficiently large multiple L of L1 satisfies the property (i): that is, for generic u ∈ |L| and generic α ∈ H 0 (KSu ), the map S ∇ (α) : H 1 TSu −→ H 1 *Su 0 is injective. Proof. It is known from [9] that the composition of this map with the inclusion H 1 *Su 0 ⊂ H 1 *Su is nothing but the multiplication map by α: H 1 TSu −→ H 1 TSu ⊗ KSu ∼ = H 1 *Su . So the transposed map ∗ ∼ H 1 *Su 0 −→ H 1 TSu = H 1 *Su ⊗ KSu is also the multiplication by α, and we have to show that it is surjective for generic α. We know from [7] and [10] that for sufficiently ample L and smooth S ∈ |L|, the residues on S of the classes of the 3-forms P ω/s 2 generate F 1 H 2 (S)0 , so that their projections modulo H 2,0 (S) generate H 1 (*S )0 , where ω is a generator of H 0 (KX ), P varies in H 0 (2L), and s ∈ H 0 (L) is an equation for S. Hence we have a surjective map H 0 (2L) −→ H 1 (*S )0 .

(4.10)

Similarly, considering residues of meromorphic forms P ω/s 3 , where P ∈ H 0 (3L), we get a surjective map H 0 (3L) −→ H 2 (ᏻS ).

(4.11)

One then shows exactly as in [2] that for α ∈ H 0 (L), one has the commutative diagram α : H 0 (2L)

/ H 0 (3L)

S ∇ α : H 1 (*S )0

/ H 2 (ᏻ ). S

(4.12)

These maps can be obtained as well by looking at the exact sequence 0 −→ ᏻS (−L) −→ *X|S −→ *S −→ 0,

(4.13)

170

CLAIRE VOISIN

which, by taking the second exterior power and tensoring with L, gives 0 −→ *S −→ *X 2|S (L) −→ KS (L) −→ 0.

(4.14)

Using the isomorphism KS (L) ∼ = 2L|S given by ω and the fact that the induced map H 1 *Su −→ H 1 *2X|S (L) ∼ = H 2 *2X is equal to jS ∗ , we get by the long exact sequence induced by (4.14) the desired map H 0 (2L|S ) → H 1 (*S )0 , with kernel H 0 (*X 2|S (L)). Tensoring (4.14) by any line bundle L , we also get maps H 0 2L + L|S −→ H 1 *S (L ) , and in particular H 0 3L|S −→ H 1 *S (L) .

(4.15)

The map (4.11) is then simply obtained by composing the map (4.15) with the map (4.16) δ : H 1 *S (d) −→ H 2 (ᏻS ) deduced from the exact sequence (4.13) twisted by L. It is then obvious that the following diagram is commutative: H 0 2L|S H 1 (*S )0

α

α

/ H 0 3L|S / H 1 *S (L) .

(4.17)

Furthermore, it also follows from the commutativity of diagrams (4.12) and (4.17) that for λ ∈ H 1 (*S )0 , α ∈ H 0 (ᏻS (L)), one has S

∇ α (λ) = δ(αλ).

(4.18)

In order to show the surjectivity of α : H 1 (*S )0 −→ H 1 *S (L) for generic S and α, we do the following. Let L1 = ᏻX (1) be sufficiently ample on X and let φ0 , . . . , φ3 ∈ H 0 (L1 ) define a map φ : X → P3 . For d sufficiently large, let @ ⊂ P3 be defined by σ ∈ H 0 (ᏻP3 (d)) and let S = φ −1 (@) be defined by s = φ ∗ (σ ) ∈ H 0 (ᏻX (d)). Let R ∈ |ᏻX (4)| be the ramification locus of φ. For @ we have the exact sequences analogous to (4.14): 2 0 −→ *@ (k) −→ *P 3 |@ (d + k) −→ K@ (d + k) −→ 0,

(4.19)


171

which can be pulled back to S and which give rise to maps (taking into account the isomorphism K@ ∼ = ᏻ@ (d − 4)) (4.20) H 0 ᏻS (2d − 4 + k) −→ H 1 φ ∗ *@ (k) . We have the following lemma. Lemma 11. The diagram H 0 ᏻS (2d − 4 + k)

/ H 1 φ ∗ *@ (k) φ∗

r

H 0 ᏻS (2d + k)

/ H 1 *S (k)

(4.21)

is commutative for an adequate choice of equation r ∈ H 0 (ᏻX (4)) for R. This follows easily from the fact that the composite φ∗ φ∗ TX −−→ φ ∗ (TP3 ) ∼ = φ ∗ *2P3 (4) −−→ *2X (4) ∼ = TX (4) is the multiplication by r, where the choice of r is determined by the isomorphisms KX ∼ = ᏻX and KP3 ∼ = ᏻP3 (−4). As a consequence of Lemma 11, we get the following. Lemma 12. Let d be sufficiently large, and let @, t ∈ H 0 (ᏻ@ (d − 4)) satisfy the following condition: the multiplication map t : H 1 *@ (−4) −→ H 1 *@ (d − 8) is surjective. Then the multiplication map φ ∗ t : H 1 (*S )0 −→ H 1 *S (d − 4) satisfies that Im φ ∗ t contains rH 1 (*S (d − 8)). Proof. From Lemma 11 we conclude that the image of the map φ ∗ : H 1 φ ∗ *@ (d − 4) −→ H 1 *S (d − 4) contains rH 1 (*S (d − 8)). Indeed, we have the commutative diagrams H 0 ᏻS (3d − 8) r

H 0 ᏻS (3d − 4)

/ H 1 φ ∗ *@ (d − 4) φ∗

/ H 1 *S (d − 4)

(4.22)

172

CLAIRE VOISIN

and H 0 ᏻS (3d − 8) r

H 0 ᏻS (3d − 4)

/ H 1 *S (d − 8) r

/ H 1 *S (d − 4) ,

(4.23)

where the surjectivity of the first horizontal map is easy to check. So it suffices to prove that if d is large enough, the assumption on @, t implies that the multiplication map φ ∗ t : H 1 (φ ∗ *@ )0 −→ H 1 φ ∗ *@ (d − 4) is surjective. Now consider the exact sequences 0 −→ J 2d−8 −→ H 0 ᏻ@ (2d − 8) −→ H 1 *@ (−4) −→ 0, 0 −→ J 3d−12 −→ H 0 ᏻ@ (3d − 12) −→ H 1 *@ (d − 8) −→ 0 constructed above, where J ∗ ⊂ H 0 (ᏻ@ (∗)) is the Jacobian ideal of @, that is, the image of H 0 (*2P3 (−4 + ∗ − d)|@ ) under the map induced by (4.19). The hypothesis on t means exactly that J 3d−12 + tH 0 ᏻ@ (2d − 8) = H 0 ᏻ@ (3d − 12) . Now if d is large enough, the multiplication map H 0 ᏻS (4) ⊗ φ ∗ H 0 ᏻ@ (3d − 12) −→ H 0 ᏻS (3d − 8) is surjective. It follows that H 0 ᏻS (4) · φ ∗ J 3d−12 + φ ∗ tH 0 ᏻS (2d − 4) = H 0 ᏻS (3d − 8) . Since H 0 (ᏻS (4)) · φ ∗ J 3d−12 vanishes in H 1 (φ ∗ *@ (d − 4)) and the map H 0 ᏻS (3d − 8) −→ H 1 φ ∗ *@ (d − 4) is surjective, it follows that φ ∗ t : H 1 (φ ∗ *@ )0 −→ H 1 φ ∗ *@ (d − 4) is surjective. We now conclude the proof of Proposition 2. It is quite easy to verify that for generic @, t the condition of Lemma 12 is satisfied. So we have (@, t) such that Im φ ∗ t contains rH 1 (*S (d − 8)). We want to conclude that φ ∗ t : H 1 (*S )0 −→ H 1 *S (d − 4)


173

is in fact surjective. Consider the surjective map H 0 (ᏻS (3d − 4)) → H 1 (*S (d − 4)). It has for kernel the space JS3d−4 = ds(u), u ∈ H 0 TX (2d − 4)|S . The fact that Im φ ∗ t contains rH 1 (*S (d − 8)) means then that rH 0 (ᏻS (3d − 8))|φ ∗ t=s=0 is contained in JS3d−4 |φ ∗ t=s=0 . Now let s/ = s + /rφ ∗ t. We first show that for generic t, σ , and φ rH 1 *S/ (d − 8) = H 1 *S/ (d − 4) .

(4.24)

Equivalently, we have to show that the multiplication map r : H 1 TS/ (4) → H 1 TS/ (8)

(4.25)

is injective. Looking at the exact sequences 0 −→ TS/ (4) −→ TX (4)|S/ −→ ᏻS/ (d + 4) −→ 0, 0 −→ TS/ (8) −→ TX (8)|S/ −→ ᏻS/ (d + 8) −→ 0, and using the fact that ᏻX (1) is sufficiently ample, we find that the kernel of the map (4.25) identifies to the set u ∈ H 0 TX (8)|R , ds/ (u)|r=s/ =0 = 0 . Now we have the equality ds/ (u)|r=s/ =0 = ds(u)|r=s/ =0 + /tdr(u)|r=s/ =0 , where the curves {r = s = 0} and {r = s/ = 0} coincide. (Notice that all these derivatives only make sense when restricted to the vanishing locus of the considered equation.) Now clearly for sufficiently large d, general t, and any u in the fixed vector space H 0 (TX (8)|R ), the right-hand side vanishes if and only if ds(u)|r=s/ =0 = 0

and

dr(u)|r=s/ =0 = 0.

But if φ is generic, the surface R is reduced, and the second condition means that u is tangent to it. Then clearly there is at most for each such u a one-dimensional family of curves {r = s = 0} on the surface R which are tangent to u; that is, s satisfies the first condition. Since u varies in the fixed subspace of H 0 (TX (8)|R ) of elements tangent to R, it follows that for d large enough and generic σ , the two conditions above imply that u = 0, so that the map (4.25) is injective.

174

CLAIRE VOISIN

This means, as above, that we have JS3d−4 / |r=s

/ =0

= H 0 ᏻS/ (3d − 4) |r=s

/ =0

,

so that, in particular, JS3d−4 / |φ∗t=r=s

/ =0

= H 0 ᏻS/ (3d − 4) |φ ∗ t=r=s

/ =0

.

But the curve defined by s = φ ∗ t = 0 is equal to the curve defined by s/ = φ ∗ t = 0; the restriction map H 0 TX (2d − 4) −→ H 0 TX (2d− 4)|S is surjective, and for u ∈ H 0 (TX (2d − 4)), we have ds/ (u)|r=s=φ ∗ t=0 = ds(u)|r=s=t=0 .

(4.26)

It follows that we have as well JS3d−4 |φ∗t=r=s=0 = H 0 ᏻS (3d − 4) |φ ∗ t=r=s=0 . Since JS3d−4 |φ∗t=s=0 contains rH 0 (ᏻS (3d − 8))|φ ∗ t=s=0 , this implies that JS3d−4 |φ∗t=s=0 = H 0 ᏻS (3d − 4) |φ ∗ t=s=0 ,

which is equivalent to the fact that φ ∗ t : H 1 (*S )0 → H 1 (*S (d − 4)) is surjective. Finally, it is easy to check that for generic t ∈ H 0 (ᏻ@ (4)), the multiplication map φ ∗ t : H 1 *S (d − 4) −→ H 1 *S (d) is surjective, so we have proved that for generic t ∈ H 0 (ᏻ@ (d)) the multiplication map φ ∗ t : H 1 (*S )0 −→ H 1 *S (d) is surjective. Thus Proposition 2 is proved. It remains now to check condition (ii) in Proposition 1. Proposition 3. Let L1 be ample on the Calabi-Yau threefold X. Then for any sufficiently large multiple L of L1 and any generic S ∈ |L|, α ∈ H 0 (L|S ), and λ ∈ S

H 1 (*S )0 such that ∇ (λ)(α) = 0 in H 2 (ᏻS ), we have that S Ker ∇ (λ) : H 0 (L|S ) −→ H 2 (ᏻS ) is generated by α. We follow this strategy: We again consider a generic map φ : X → P3 , with L1 = ᏻX (1) = φ ∗ (ᏻP3 (1)) sufficiently ample, and surfaces S = φ −1 (@) for generic

175


@ ⊂ P3 of degree d sufficiently large. So S = V (s), @ = V (σ ) with s = φ ∗ (σ ). S Next let t ∈ H 0 (ᏻ@ (d)) be generic. Then for α = φ ∗ t, we know that ∇ α : S H 1 (*S )0 → H 2 (ᏻS ) is surjective, so the set of λ ∈ H 1 (*S )0 such that ∇ (λ)(α) = 0 S in H 2 (ᏻS ), which is equal to Ker ∇ α , has the minimal generic dimension. Hence it suffices to prove Proposition 3 for such (S, α). S First, we show that for generic λ ∈ H 1 (φ ∗ *@ )0 such that ∇ (λ)(α) = 0 in H 2 (ᏻS ), S that is, λ ∈ Ker ∇ α , the kernel of S ∇ (λ) : H 0 ᏻS (d) −→ H 2 (ᏻS ) is generated by α and J˜@ := Ker H 0 ᏻS (d) −→ H 1 (T@ ) = Im H 0 φ ∗ (TP3 ) −→ H 0 ᏻS (d) . S

Then we conclude that for generic λ ∈ Ker ∇ α , the kernel of S ∇ (λ) : H 0 ᏻS (d) −→ H 2 (ᏻS ) is generated by α, by showing that the set of quadrics qλ on P(H 0 (ᏻS (d))) for S λ ∈ Ker ∇ α has no base point on P(J˜@ ). Let us introduce the notation Rσ = C[X0 , . . . , X3 ]/Jσ , where Jσ is the ideal gen∂σ . Using (4.19), we get isomorphisms erated by the partial derivatives ∂X i Rσ2d−4 ∼ = H 1 (*@ )0 ,

Rσ2d−4+k ∼ = H 1 *@ (k) ,

(4.27)

for any integer k = 0. Furthermore, Rσ is Gorenstein: we have Rσ4d−8 ∼ = C, and the pairings Rσ2d−4−k × Rσ2d−4+k −→ Rσ4d−8

(4.28)

are perfect. We first show the following. Proposition 4. Assume that ᏻX (1) is sufficiently ample, that φ is generic, and that d is sufficiently large. Let t ∈ H 0 (ᏻ@ (d)) be generic and assume that there exist λ1 ∈ Rσ2d−8 ∼ = H 1 (*@ (−4)), A ∈ H 0 (ᏻ@ (2)) satisfying the following properties: (a) Ker λ1 : Rσd → Rσ3d−8 ∼ = (Rσd )∗ is generated by the image t of t in Rσd ; (b) Aλ1 : Rσd−1 → Rσ3d−7 ∼ = (Rσd−1 )∗ is an isomorphism; 2 d−2 3d−6 ∼ (c) A λ1 : Rσ → Rσ = (Rσd−2 )∗ is an isomorphism; 3 d−3 3d−5 ∼ (d) A λ1 : Rσ → Rσ = (Rσd−3 )∗ is an isomorphism; 4 d−4 3d−4 ∼ (e) A λ1 : Rσ → Rσ = (Rσd−4 )∗ is an isomorphism. 0 Then for ψ ∈ H (ᏻX (4)) generic and Q ∈ H 0 (ᏻX (2)) generic, the element λ = ψφ ∗ (λ1 ) + Qφ ∗ (Aλ1 ) + φ ∗ A2 λ1

176

CLAIRE VOISIN S

of H 1 (φ ∗ *@ )0 satisfies that ∇ (λ)(α) = 0 in H 2 (ᏻS ), where α = φ ∗ (t), and the kernel of S ∇ (λ) : H 0 ᏻS (d) −→ H 2 (ᏻS ) is generated by α and J˜@ . S

It is clear that α is in the kernel of ∇ (λ), since we have λ = f φ ∗ (λ1 ), where λ1 ∈ H 1 (*@ (−4)) satisfies tλ1 = 0 in H 1 (*@ (d − 4)). This implies that αλ = 0 in S H 1 (*S (d)) and a fortiori ∇ (λ)(α) = 0 in H 2 (ᏻS ), since we have by (4.18) S

∇ (λ)(α) = δ(αλ). S

Also J˜@ is contained in the kernel of ∇ (λ). Indeed, since λ ∈ H 1 (φ ∗ *@ )0 , the map S

∇ (λ) : H 1 (TS ) −→ H 2 (ᏻS ), which is given by interior product, clearly factors through H 1 (φ ∗ (T@ )). Let us first prove the following lemma. Lemma 13. Let λ = A2 λ1 ∈ Rσ2d−4 . Assumptions (a),. . . ,(e) on λ1 , A imply the following: (i) λ : Rσd−2 → Rσ3d−6 ∼ = (Rσd−2 )∗ is an isomorphism; (ii) Aλ1 : (Ker λ )d−1 → (Coker λ )3d−7 is an isomorphism; (iii) λ1 : (Ker λ )d → (Coker λ )3d−8 has its kernel generated by t. λ

Here we denote by (Ker λ )∗ (resp., (Coker λ )∗ ) the kernel of the multiplication by : Rσ∗ → Rσ2d−4+∗ (resp., the cokernel of the multiplication by λ : Rσ∗−2d+4 → Rσ∗ ).

Proof. (i) is assumption (c). (ii) Let u ∈ (Ker λ )d−1 and assume Aλ1 u = 0 in (Coker λ )3d−7 . This means that Aλ1 u = A2 λ1 v in Rσ3d−7 for some v ∈ Rσd−3 . By assumption (b), it follows that u = Av. Then A2 λ1 u = 0 implies that A3 λ1 v = 0; by (d), v = 0, so u = 0. We prove (iii) in the same way. In order to prove Proposition 4, we first study the map µλ : H 1 φ ∗ (T@ ) −→ H 1 φ ∗ *@ (d) , which is the factorization of the multiplication map by φ ∗ (λ ) ∈ H 1 (φ ∗ *@ ): µλ : H 0 ᏻS (d) −→ H 1 φ ∗ *@ (d) , using the surjective map

H 0 ᏻS (d) −→ H 1 φ ∗ (T@ ) .

(We use the fact that H 1 (φ ∗ (TP3 )|S ) = 0.) Notice that from (4.18), the composition of µλ with the map δ : H 1 (φ ∗ *@ ) → H 2 (ᏻS ) of (4.16) is equal to the factorization S through H 1 (φ ∗ (T@ )) of ∇ (φ ∗ (λ ))|H 0 (ᏻS (d)) . We have the following lemma.


Lemma 14. Let K = (H 0 (ᏻX (1))/H 0 (ᏻP3 (1)))∗ . Choose a splitting H 0 ᏻX (1) ∼ = K ∗ ⊕ H 0 ᏻP3 (1) ;

177

(4.29)

then Ker µλ is naturally isomorphic to (Ker λ )d ⊕ K ∗ ⊗ (Ker λ )d−1 , and Coker µλ is naturally isomorphic to (Coker λ )3d−8 ⊕ K ⊗ (Coker λ )3d−7 . Notice that the map from (Ker λ )d ⊕K ∗ ⊗(Ker λ )d−1 to Ker µλ is the natural one: indeed, (Ker λ )d identifies to the kernel of µλ |φ ∗ (H 1 (T@ )) while (Ker λ )d−1 identifies to the kernel of the analogous map H 1 φ ∗ (T@ )(−1) −→ H 1 φ ∗ *@ (d − 1) restricted to φ ∗ (H 1 (T@ (−1))). Notice also that both statements are dual to each other: indeed, the map µλ is symmetric with respect to the Serre duality isomorphism 1 ∗ ∗ H 1 φ ∗ (T@ ) ∼ = H φ *@ (d) ; so (Ker λ )d is dual to (Coker λ )3d−8 and (Ker λ )d−1 is dual to (Coker λ )3d−7 by the pairings (4.28). So it suffices to prove the second statement, and for this we can replace µλ by µλ . To prove it we first prove the following. Lemma 15. Let Ᏹ be the vector bundle φ∗ ᏻX (2) on P3 , and let ᏷ be the cokernel of the natural map H 0 ᏻX (2) ⊗ ᏻP3 −→ Ᏹ; then the splitting (4.29) gives an isomorphism ᏷∼ = ᏻP3 (−2) ⊕ K ⊗ ᏻP3 (−1) . Proof. Let ⊂ X × P3 be the graph of φ. Then ᏷ = R 1 pr2 ∗ Ᏽ ⊗ pr1∗ ᏻX (2) . Now if Q is defined by the exact sequence 0 −→ Q −→ H 0 ᏻP3 (1) ⊗ ᏻP3 −→ ᏻP3 (1) −→ 0, Ᏽ has the resolution

0

3

2

∗ ∗ −→ pr2 Q ⊗ pr1 ᏻX (−1) −→ pr2 Q ⊗ pr1∗ ᏻX (−1) −→ pr2∗ Q ⊗ pr1∗ ᏻX (−1) −→ Ᏽ −→ 0. ∗

One concludes from this that ᏷ is isomorphic to Ker β :

3

2

Q ⊗ H 3 ᏻX (−1) −→ Q ⊗ H 3 (ᏻX ).

(4.30)

178

CLAIRE VOISIN

Now the dual of the map β is simply the natural map α : Q ⊗ ᏻP3 (1) −→ H 0 ᏻX (1) ⊗ ᏻP3 (1), from which it follows easily that

᏷ = (Coker α)∗ ∼ = ᏻP3 (−2) ⊕ K ⊗ ᏻP3 (−1) .

Tensorizing the exact sequence H 0 ᏻX (2) ⊗ ᏻP3 −→ Ᏹ −→ ᏷ −→ 0

(4.31)

with ᏻ@ (d − 2), we deduce easily from Lemma 15 the following. Corollary 4. The splitting (4.29) gives an isomorphism H 0 ᏻS (d) ∼ = H 0 ᏻ@ (d − 4) ⊕ K ⊗ H 0 ᏻ@ (d − 3) . 0 0 H ᏻX (2) H ᏻ@ (d − 2) Similarly, tensorizing the exact sequence (4.31) with *@ (d − 2) and using Lemma 15, we easily get the following. Corollary 5. The splitting (4.29) gives an isomorphism H 1 φ ∗ *@ (d) 1 1 ∼ H (d − 4) ⊕ K ⊗ H (d − 3) . * * = @ @ H 0 ᏻX (2) φ ∗ H 1 *@ (d − 2) Proof of Lemma 14. By assumption (i) on λ , it follows that Im µλ contains H 0 (ᏻX (2)φ ∗ (H 1 (*@ (d − 2)), since it means that the map λ : H 1 (T@ (−2)) → H 1 (*@ (d − 2)) is surjective. So it suffices to study the cokernel of the induced map H 1 φ ∗ *@ (d) H 0 ᏻS (d) −→ . ρλ : 0 H ᏻX (2) H 0 ᏻ@ (d − 2) H 0 ᏻX (2) φ ∗ H 1 *@ (d − 2) But applying Corollaries 4 and 5, ρλ gives a map H 0 ᏻ@ (d− 4) ⊕ K⊗H 0 ᏻ@ (d− 3) −→ H 1 *@ (d− 4) ⊕ K⊗H 1 *@ (d− 3) . This last map is now easily seen to be the direct sum of the multiplication map by λ ∈ H 1 (*@ )0 , from which we conclude that Coker µλ ∼ = Coker ρλ ∼ = (Coker λ )3d−8 ⊕ K ⊗ (Coker λ )3d−7 , using the isomorphisms H 1 *@ (d − 4) ∼ = Rσ3d−8 , of (4.27).

H 1 *@ (d − 3) ∼ = Rσ3d−7 ,


179

Next for Q ∈ H 0 (ᏻX (2)), let λ2 = Qφ ∗ (Aλ1 ) ∈ H 1 (φ ∗ *@ )0 . Again, the multiplication map µλ2 : H 0 ᏻS (d) −→ H 1 φ ∗ *@ (d) induces a symmetric map H 1 φ ∗ (T@ ) −→ H 1 φ ∗ *@ (d) , and hence a symmetric map ∗

∗

∗

µλ2 : φ H (T@ ) ⊕ K ⊗ φ H T@ (−1) 1

1

H 1 φ ∗ *@ (d) , −→ 0 H ᏻX (2) H 1 φ ∗ *@ (d − 2)

that is, by Lemma 14 a map µλ2 : Rσd ⊕ K ∗ ⊗ Rσd−1 −→ Rσ3d−8 ⊕ K ⊗ Rσ3d−7 . We have the following lemma. Lemma 16. The map µλ2 vanishes on Rσd , and on K ∗ ⊗ Rσd−1 it is computed as follows: There is a natural map E : H 0 ᏻX (2) −→ Hom(K ∗ , K) such that µλ2 : K ∗ ⊗ Rσd−1 −→ K ⊗ Rσ3d−7 is equal to E(Q) ⊗ Aλ1 . Proof. The first statement is obvious, since µλ2 (φ ∗ (H 0 (ᏻ@ (d)))) is contained in As for the second one, consider the commutative diagram Qφ ∗ (Aλ1 ) / H 0 ᏻS (3d − 4) H 0 ᏻS (d) H 0 (ᏻX (2)) · φ ∗ (H 1 (*@ (d − 2))).

H 0 ᏻS (d)

µλ2

/ H 1 φ ∗ *@ (d) ,

where λ1 is any lifting of λ1 in H 0 (ᏻ@ (2d −8)); the second vertical map was defined in (4.20). The commutative diagram shows that it suffices to prove more generally the following lemma: Consider the multiplication map H 0 ᏻX (d) H 0 ᏻX (d + k + 2) ∗ −→ Qφ P : 0 H ᏻP3 (2) H 0 ᏻX (d − 2) H 0 ᏻP3 (2) H 0 ᏻX (d + k)

180

CLAIRE VOISIN

for any P ∈ H 0 (ᏻP3 (k)). Using the isomorphism deduced from Lemma 15, H 0 ᏻX (d + k + 2) ∼ = H 0 ᏻP3 (d − 2 + k) ⊕ K ⊗ H 0 ᏻP3 (d − 1 + k) , 0 0 H ᏻP3 (2) H ᏻX (d + k) Qφ ∗ P induces a map K ∗ ⊗ H 0 ᏻP3 (d − 1) −→ K ⊗ H 0 ᏻP3 (d − 1 + k) ; then we have the following. Lemma 17. There is a natural map E : H 0 ᏻX (2) −→ Hom(K ∗ , K), such that this map is equal to E(Q) ⊗ P . Proof. We construct the map E as follows: Let ᏸ be the cokernel of the natural map H 0 ᏻX (1) ⊗ ᏻP3 −→ φ∗ ᏻX (1). Using the equality

ᏸ = R1 pr2 ∗ Ᏽ ⊗ pr1∗ ᏻX (1) ,

where the notation is as in the proof of Lemma 15, and the resolution (4.30), we find that ᏸ is isomorphic to the dual of the cokernel of the natural map Q(1) ⊗ H 0 ᏻX (1) −→ H 0 ᏻX (2) ⊗ ᏻP3 (1). In particular, there is a natural inclusion of ᏸ in H 0 (ᏻX (2))∗ ⊗ ᏻP3 (−1). Tensorizing by ᏻP3 (1) and taking global sections, we get a map ∗ χ : H 0 ᏻX (2) −→ H 0 ᏻX (2) whose image is the set of linear forms vanishing on H 0 (ᏻP3 (1))·H 0 (ᏻX (1)). Recalling that K ∗ = H 0 (ᏻX (1))/H 0 (ᏻP3 (1)), such a linear form χ(Q) obviously induces a symmetric bilinear form on K ∗ and, hence, a map E(Q) : K ∗ → K. The statement concerning the multiplication is then clear: in fact, it clearly suffices to do the case d = l = 0, and then this results from the definition of E. So Lemma 17 (hence also Lemma 16) is proved. We also need the following lemma. Lemma 18. If φ is generic, for generic Q ∈ H 0 (ᏻX (2)), the map E(Q) : K ∗ → K is an isomorphism.


181

Proof. Notice that each E(Q) is symmetric and hence defines a quadric qQ on K ∗ . In fact, qQ (k) = χ (Q)(k 2 ), with the notation of the above proof. But we know that the map χ has for image the set of linear forms vanishing on H 0 (ᏻP3 (1))·H 0 (ᏻX (1)). So to prove the lemma, it suffices to show that this set, viewed as a set of quadrics on H 0 (ᏻX (1)), has exactly for base locus H 0 (ᏻP3 (1)). But the base locus of this set of quadrics is exactly the set k ∈ H 0 ᏻX (1) , k 2 ∈ H 0 ᏻP3 (1) · H 0 ᏻX (1) . So we have to prove that for generic φ = (φ0 , . . . , φ3 ) the condition k 2 ∈ φ0 , . . . , φ3 implies that k ∈ φ0 , . . . , φ3 . This is easy: It suffices to degenerate (φ0 , . . . , φ3 ) to the linear system of elements of H 0 (ᏻX (1)) vanishing on a certain number of points of X, and verify that one can do this while keeping the dimension of φ0 , . . . , φ3 ⊂ H 0 (ᏻX (2)) constant. Then for the degenerated system (φ0 , . . . , φ3 ), the result is obvious; this implies the same thing for the generic system. Similarly, let λ3 = ψφ ∗ (λ1 ) ∈ H 1 (φ ∗ *@ )0 , for any ψ ∈ H 0 (ᏻX (4)). Then the multiplication map µλ3 : H 0 ᏻS (d) −→ H 1 φ ∗ *@ (d) induces a map

µλ3 : Rσd ∼ = φ ∗ H 1 (T@ ) −→ H 1 *@ (d − 4) ∼ = Rσ3d−8 ,

where we use Corollary 5 to realize H 1 (*@ (d − 4)) as a quotient of H 1 (φ ∗ *@ (d)). Then we have the following. Lemma 19. There is a nonzero map : H 0 (ᏻX (4)) → C such that µλ3 is equal to (ψ)λ1 . This is not difficult. In fact, ∈ (H 0 (ᏻX (4)))∗ is simply given by the inclusion of C = H 3 (ᏻP3 (−4)) in H 3 (ᏻX (−4)). Proof of Proposition 4. We know from Lemma 13 that Aλ1 : (Ker λ )d−1 → (Coker λ )3d−7 is an isomorphism. By Lemma 18, we also know that for generic Q the map E(Q) : K ∗ → K is an isomorphism. Using Lemmas 14 and 16, we conclude that for generic Q the map induced by µλ2 Ker µλ −→ Coker µλ vanishes on

(Ker λ )d

and induces a (symmetric) isomorphism d−1 3d−7 K ∗ ⊗ Ker λ −→ K ⊗ Coker λ .

Next by Lemma 13, we know that the map λ1 : (Ker λ )d → (Coker λ )3d−8 has for kernel exactly t . Using Lemmas 14 and 19, we conclude that for generic ψ the map induced by µλ3 Ker µλ −→ Coker µλ

182

CLAIRE VOISIN

induces a (symmetric) map

Ker λ

d

3d−8 −→ Coker λ ,

which has for kernel exactly t . But then it follows immediately that for generic Q and ψ and for λ = λ + λ2 + λ3 , the map µλ : H 1 φ ∗ (T@ ) −→ H 1 φ ∗ *@ (d) has its kernel generated by φ ∗ t ∈ H 1 (φ ∗ T@ ). To conclude the proof of Proposition 4, we now simply note that the map δ : H 1 φ ∗ *@ (d) −→ H 2 (ᏻS ) is injective. To see this, it suffices to prove that H 1 (φ ∗ (*P3 (d))|S ) = 0 or that H 2 (φ ∗ (*P3 )) = 0, which is easy. Then we have proved that δ ◦µλ has its kernel generated by φ ∗ (t) and since this map S is equal to the factorization through H 1 (φ ∗ T@ ) of ∇ (λ) : H 0 (ᏻS (d)) → H 2 (ᏻS ), it follows that this last map has its kernel generated by J˜@ and α = φ ∗ t. So Proposition 4 is proved. Next we prove the following lemma. Lemma 20. Assume that for t generic in H 0 (ᏻ@ (d)), there exists λ ∈ H 1 (φ ∗ *@ )0 S such that ∇ (λ) : H 0 (ᏻS (d)) → H 2 (ᏻS ) has its kernel generated by J˜@ and α = S S φ ∗ (t). Then for generic λ ∈ Ker ∇ α ⊂ H 1 (*S )0 the kernel Ker ∇ (λ) : H 0 (ᏻS (d)) → H 2 (ᏻS ) is generated by α. Proof. For any λ ∈ H 1 (*S )0 , the map S ∇ (λ) : H 0 ᏻS (d) −→ H 2 (ᏻS ) is symmetric with respect to Serre duality, so it determines a quadric qλ on P(H 0 (ᏻS (d))). We know by assumption that there is a qλ , which has for singular locus the projective space generated by α and J˜@ , and we want to conclude that the generic qλ singular at α has α as its only singular point. By Bertini, it clearly suffices to prove that the system of quadrics qλ singular at α has no base point on the projective space S P(J˜@ ). Now note that the set Ker ∇ α , which exactly parametrizes this linear system, identifies to S λ ∈ H 1 (*S )0 , λ ⊥ ∇ α H 0 (KS ) , where the symbols ⊥ refer to the pairing on H 1 (*S )0 . Furthermore, by definition, S the condition qλ (u) = 0 is equivalent to λ ⊥ ∇ u (u). Recalling that the map S

∇ u : H 0 (KS ) −→ H 1 (*S )0


183

identifies to the composite of the multiplication by u H 0 (KS ) = H 0 ᏻS (d) −→ H 0 ᏻS (2d) and of the map (4.10)

H 0 ᏻS (2d) −→ H 1 (*S )0 , S

we conclude that u is in the base locus of the system of quadrics Ker ∇ α if and only if the image of u2 in H 1 (*S )0 is contained in the image of αH 0 (ᏻS (d)) in H 1 (*S )0 , which has for kernel the space JS2d , image of H 0 (TX (d)|S ) in H 0 (ᏻS (d)) . So the proof of Lemma 20 is concluded by the following lemma. Lemma 21. For generic σ, t, the condition u2 = φ ∗ t.v mod .JS2d for u ∈ J˜@d , v ∈ H 0 (ᏻS (d)) implies that u = 0. The proof of this last lemma is not very difficult, so we do not give it here. From Lemma 20 and Proposition 4, we conclude now with the following. Corollary 6. If for σ, t generic, there exist A, λ1 satisfying the assumptions of Proposition 4, then Proposition 3 is true. Proof of Proposition 3. It remains only to show the existence of A, λ1 satisfying conditions (a) to (e) of Proposition 4. For any integer k we have the map given by the multiplication in the Jacobian ring of σ Rσ2k −→ Homsym Rσ2d−4−k , Rσ2d−4+k , where the subscript “sym” refers to the perfect pairings (4.28). We denote by Dσ2k ⊂ P(Rσ2k ) the discriminant hypersurface for these families of quadrics. It is easy to show that for generic σ and for any 0 ≤ k ≤ 2d − 4, Dσ2k = P(Rσ2k ). This is what we want to show: For generic σ and generic t ∈ Rσd , there exists λ1 ∈ Dσ2d−8 such that Ker λ1 is generated by t. Furthermore, for generic A ∈ H 0 (ᏻ@ (2)), we have Ak λ1 ∈ Dσ2d−8+2k , for 1 ≤ k ≤ 4. Now notice that the degree of Dσ2k is equal to the rank of Rσ2d−4−k . In particular, we have the following: d 0 Dσ2d−6 = rk Rσd−1 < rk Rσd = d 0 Dσ2d−8 , d 0 Dσ2d−4 = rk Rσd−2 < rk Rσd = d 0 Dσ2d−8 , d 0 Dσ2d−2 = rk Rσd−3 < rk Rσd = d 0 Dσ2d−8 , d 0 Dσ2d = rk Rσd−4 < rk Rσd = d 0 Dσ2d−8 . Furthermore, it is easy to show that for σ generic and A generic we have Ak Rσ2d−8 ⊂ Dσ2d−8+2k for 1 ≤ k ≤ 4.

184

CLAIRE VOISIN

Then we contend that the existence of A, λ1 satisfying conditions (a) to (e) follows from the next lemma. Lemma 22. Let σ be generic, and let ᏾ ⊂ P(Rσd ) × P(Rσ2d−8 ) be defined as ᏾ = (t, λ1 ), tλ1 = 0 in Rσ3d−8 . Then ᏾ has only one component ᏾gen of dimension at least equal to dim P(Rσ2d−8 )−1. Indeed, we know that for generic t ∈ Rσd , the map t : Rσ2d−8 → Rσ3d−8 is surjective. It follows that the principal component of ᏾ (the one that dominates P(Rσd )) is exactly of dimension dim P(Rσ2d−8 ) − 1, and it must be equal to ᏾gen . So ᏾gen is of dimension dim P(Rσ2d−8 ) − 1. But ᏾gen has to dominate Dσ2d−8 by the second projection, since ᏾ has no other component of dimension at least equal to dim Dσ2d−8 . Since dim ᏾gen = dim Dσ2d−8 , the second projection ᏾gen −→ Dσ2d−8

must be birational, and any other component of ᏾ is sent to a proper subset of Dσ2d−8 . It follows that Dσ2d−8 is irreducible, and its generic element λ1 satisfies that Ker λ1 is generated by t for generic t ∈ Rσd . But then Dσ2d−8 is also reduced. For degree reasons, we cannot then have Ak Dσ2d−8 ⊂ Dσ2d−8+2k for 1 ≤ k ≤ 4, and since Dσ2d−8 is irreducible, it follows that for generic λ1 ∈ Dσ2d−8 , we have Ak λ1 ∈ Dσ2d−8+2k for 1 ≤ k ≤ 4. So Lemma 22 implies the existence of A, λ1 satisfying conditions (a) to (e). Proof of Lemma 22. One has to prove that there exists no nonempty proper subset Z ⊂ P(Rσd ) such that for z ∈ Z the multiplication map z : Rσ2d−8 → Rσ3d−8 has cokernel of dimension at least equal to k = codim Z. Equivalently, by duality the map z : Rσd → R 2d has a kernel of dimension at least equal to k = codim Z. Let l ≤ d be such that h0 ᏻP3 (l) ≤ k < h0 ᏻP3 (l + 1) . One first verifies that there exists 0 < / < / < 1 such that for d large enough, σ generic, and Z as above, one has /d < l < / d. This follows from the following facts, which are proved by a dimension count: (a) there exists 0 < / < 1 such that for sufficiently large d, generic σ , and any t = 0 ∈ Rσ[/d] , the multiplication map t : Rσd → Rσd+[/d] is injective; (b) there exists B ∈ Rσd−[/d] such that the multiplication map B : Rσd+[/d] → R 2d is injective. It follows from (a) and (b) that BRσ[/d] does not meet Z, which implies that l +1 ≥ /d. Also it follows from (a) and (b) that for any z ∈ Rσd , we have Ker z ∩ BRσ[/d] = {0}. Hence for z ∈ Z, we have k ≤ dim Ker z ≤ h0 (d) − h0 ([/d]) ≤ h0 ([/ d]),


185

where / is chosen so as to satisfy the last inequality for large d. This gives the other inequality. Now one shows that for any l < / d, d large enough, and for generic σ , there exists C ∈ Rσd−l−2 such that the multiplication map C : Rσd+l+2 −→ Rσ2d is injective. Consider now the map C : Rσl+2 → Rσd . Then for z ∈ Rσl+2 , we have Ker z : Rσd −→ Rσd+l+2 = Ker Cz : Rσd −→ Rσ2d ; hence, in particular, if Cz ∈ Z, we have dim Ker(z : Rσd → Rσd+l+2 ) ≥ h0 (l). Hence we conclude that if Z = CRσl+2 ∩ Z, we have that the codimension of Z in P(Rσl+2 ) is at most equal to h0 (l +1), and for z ∈ Z , dim Ker(z : Rσd → Rσd+l+2 ) ≥ h0 (l). This is absurd because of the following fact (which is proved by looking at the Fermat equation): The dimension of the subspace Z of P(Rσl+2 ) defined by the condition z ∈ Z ⇐⇒ dim Ker z : Rσd −→ Rσd+l+2 ≥ h0 (l) is not greater than 140, for generic σ . This obviously contradicts the fact that Z ⊂ Z and dim Z ≥ h0 (l + 2) − h0 (l + 1), which is strictly greater than 140 for d large enough, since l > /d. So the existence of such Z for generic σ is absurd, and Lemma 22 is proved. The proof of Proposition 3 is now finished, and together with Propositions 1 and 2, it implies Theorem 4. References [1] [2]

[3] [4]

[5] [6]

[7]

A. Albano and A. Collino, On the Griffiths group of the cubic sevenfold, Math. Ann. 299 (1994), 715–726. J. Carlson and P. Griffiths, “Infinitesimal variations of Hodge structure and the global Torelli problem” in Journées de Géométrie Algébrique (Angers, 1979), ed. A. Beauville, Sijthoff & Noordhoff, Alphen aan den Rijn, 1980, 51–76. H. Clemens, Homological equivalence, modulo algebraic equivalence, is not finitely generated, Inst. Hautes Études Sci. Publ. Math. 58 (1983), 19–38. , “Some results about Abel-Jacobi mappings” in Topics in Transcendental Algebraic Geometry (Princeton, 1981/1982), ed. P. Griffiths, Ann. of Math. Stud. 106, Princeton Univ. Press, Princeton, 1984, 289–304. P. Deligne, Théorie de Hodge, II, Inst. Hautes Études Sci. Publ. Math. 40 (1971), 5–57. R. Donagi and E. Markman, “Cubics, integrable systems, and Calabi-Yau threefolds” in Proceedings of the Hirzebruch 65 Conference on Algebraic Geometry (Ramat Gan, 1993), Israel Math. Conf. Proc. 9, Bar-Ilan Univ., Ramat Gan, 1996, 199–221. M. Green, The period map for hypersurface sections of high degree of an arbitrary variety, Compositio Math. 55 (1985), 135–156.

186 [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18]

CLAIRE VOISIN , Griffiths’ infinitesimal invariant and the Abel-Jacobi map, J. Differential Geom. 29 (1989), 545–555. P. Griffiths, Periods of integrals on algebraic manifolds, I, Amer. J. Math. 90 (1968), 568–626; II, 805–865. , On the periods of certain rational integrals, I, Ann. of Math. (2) 90 (1969), 460–495; II, 496–541. , Infinitesimal variations of Hodge structure, III: Determinantal varieties and the infinitesimal invariant of normal functions, Compositio Math. 50 (1983), 267–324. S.-O. Kim, Noether-Lefschetz locus for surfaces, Trans. Amer. Math. Soc. 324 (1991), 369–384. M. Nori, Algebraic cycles and Hodge-theoretic connectivity, Invent. Math. 111 (1993), 349– 373. , Cycles on the generic abelian threefold, Proc. Indian Acad. Sci. Math. Sci. 99 (1989), 191–196. K. Paranjape, Curves on threefolds with trivial canonical bundle, Proc. Indian Acad. Sci. Math. Sci. 101 (1991), 199–213. Z. Ran, Hodge theory and the Hilbert scheme, J. Differential Geom. 37 (1993), 191–198. C. Voisin, Densité du lieu de Noether-Lefschetz pour les sections hyperplanes des variétés de Calabi-Yau de dimension 3, Internat. J. Math. 3 (1992), 699–715. , Sur l’application d’Abel-Jacobi des variétés de Calabi-Yau de dimension trois, Ann. Sci. École Norm. Sup. (4) 27 (1994), 209–226.

Institut de Mathématiques de Jussieu, Centre National de la Recherche Scientifique, Unité Mixte de Recherche 9994, 75251 Paris Cedex 05, France

Vol. 102, No. 2


© 2000

A CHARACTERIZATION OF RATIONAL SINGULARITIES SÁNDOR J. KOVÁCS

The main purpose of this note is to present a characterization of rational singularities in characteristic 0. The essence of the characterization is that it is enough to require less than the usual definition. Theorem 1. Let φ : Y → X be a morphism of varieties over C, and let ρ :

ᏻX → Rφ∗ ᏻY be the associated natural morphism. Assume that Y has rational singularities and there exists a morphism (in the derived category of ᏻX -modules) ρ : Rφ∗ ᏻY → ᏻX such that ρ ◦ ρ is a quasi-isomorphism of ᏻX with itself. Then X

has only rational singularities. If ρ exists, it could be considered similar to a trace operator. In fact, for any finite morphism of normal varieties, ρ exists because of the trace operator. Note that for the first statement of Theorem 1, φ does not need to be birational. In particular, Theorem 1 implies that quotient singularities are rational, including quotients by reductive groups as in [B, Corollaire]. In the latter case, ρ is given by the Reynolds operator. A well-known and widely used theorem states that in characteristic 0, canonical singularities are Cohen-Macaulay and therefore rational (see [E] and [KMM]). The original proofs are based on a very clever use of Grothendieck duality simultaneously for a resolution and its restriction onto the exceptional divisor and on a double loop induction. Kollár gave a simpler proof in [K2, §11] without using derived categories but still relying on a technically hard vanishing theorem. Recently Kollár and Mori found a simple proof allowing nonempty boundaries. They do not use derived categories either, but restrict to the projective case (see [KM, 5.18]). These proofs are ingenious, but one would like to have a simple natural proof (at least in the “classical” case, when the boundary is empty). As an application of Theorem 1 a simple proof is given here in the “classical” case, but without the projective assumption. This proof seems even simpler than that of Kollár and Mori. Derived categories and Grothendieck duality are used, but in such a simple way that one is tempted to say that this proof is the most natural one. Note also that everything used here was already available when the question was raised for the first time. A statement similar to Theorem 1 was given in [K2, 3.12]. Some ideas of the Received 19 May 1999. Revision received 23 June 1999. 1991 Mathematics Subject Classification. Primary 14B05; Secondary 14E15. Author’s work partially supported by an American Mathematical Society Centennial Fellowship and by National Science Foundation grant number DMS-9818357. 187

188

SÁNDOR J. KOVÁCS

present proof already appear there. In addition, the results of [K1] and [K2] can be used to show that for projective varieties the existence of ρ is actually equivalent to X having rational singularities. Note that the assumption of Theorem 1 implies that φ has to be surjective. Theorem 2. Let φ : Y → X be a surjective morphism of projective varieties over C, and let ρ : ᏻX → Rφ∗ ᏻY be the associated natural morphism. Assume that both X and Y have rational singularities. Then there exists a morphism ρ : Rφ∗ ᏻY → ᏻX such that ρ ◦ ρ is a quasi-isomorphism of ᏻX with itself. Finally, as a byproduct of the proof, a partial generalization of Kempf’s criterion for rational singularities (cf. [KKMS, p. 50]) is presented. Theorem 3. Let φ : Y → X be a surjective morphism of projective varieties over C. Let N = dim Y and n = dim X. Assume that Y has rational singularities. Then X has rational singularities if and only if X is Cohen-Macaulay and R N−n φ∗ ωY ωX . Acknowledgements. I would like to thank János Kollár for valuable comments and the editors for their kind flexibility. Definitions and notation. Throughout the article, the ground field is always C, the field of complex numbers. A variety means a separated variety of finite type over C. A divisor D is called Q-Cartier if mD is Cartier for some m > 0. A normal variety X is said to have log-terminal, (resp., canonical) singularities if KX is Q-Cartier. For any resolution of singularities, f : Y → X, with the collection of exceptional prime divisors {Ei }, there exist ai ∈ Q, ai > −1 (resp., ai ≥ 0) such that KY ≡ f ∗ KX + ai Ei (cf. [KMM] and [CKM]). The index of a normal variety X with KX Q-Cartier is the smallest positive integer m such that mKX is Cartier. For a normal variety X with KX Q-Cartier, there exists locally an index-1 cover: that is, a finite surjective morphism σ : X → X such that X has index 1. A log-terminal variety of index 1 is canonical, and an easy computation shows that finite covers of log-terminal (resp., canonical) singularities are log-terminal (resp., canonical). In particular, the index-1 cover of a log-terminal variety is canonical (see [R, 1.7, 1.9] and [CKM, 6.7, 6.8]). Let X be a normal variety and φ : Y → X a resolution of singularities. X is said to have rational singularities if R i φ∗ ᏻY = 0 for all i > 0 or, equivalently, if the natural map ᏻX → Rφ∗ ᏻY is a quasi-isomorphism. · denote the dualizing complex of X; that is, ω· = f ! C, where f : X → Let ωX X Spec C is the structure map (cf. [H]). Main ingredients. Let φ : Y → X be a proper morphism. Then one has the following: • Grothendieck duality [H, VII]: For all G· -bounded complexes of ᏻY -modules, Rφ R Ᏼom (G· , ω· ) R Ᏼom (Rφ G· , ω· ). ∗

Y

Y

X

∗

X

• Adjointness [H, II.5.10]: For all F · -bounded complexes of ᏻX -modules and

A CHARACTERIZATION OF RATIONAL SINGULARITIES

189

G· -bounded complexes of ᏻY -modules, Rφ∗ R ᏴomY (Lφ ∗ F · , G· ) R ᏴomX (F · , Rφ∗ G· ). If φ is a resolution of singularities, then one has • Grauert-Riemenschneider vanishing [GR]: R i φ∗ ωY = 0 for i > 0. This is referred to as “GR vanishing.” Lemma 1 [KKMS, p. 50], [K2, 11.9]. Let X be a normal variety, and let φ : Y → X be a resolution of singularities. If X is Cohen-Macaulay and ωX φ∗ ωY , then X has rational singularities. Proof. By GR vanishing, ω· Rφ ω· , and then ∗ Y

X

· , ω· ) R Ᏼom (Rφ ω· , ω· ) ᏻX R ᏴomX (ωX X ∗ Y X X · · Rφ R Ᏼom (ω , ω ) Rφ ᏻ . ∗

Y

Y

∗ Y

Y

Proof of Theorem 1. Let π : X˜ → X be a resolution of X and σ : Y˜ → Y be a resolution of Y such that φ ◦ σ factors through π: There exists ψ : Y˜ → X˜ such that φ ◦ σ = π ◦ ψ. Then one has the following commutative diagram: ρ

ᏻX α

/ Rφ∗ ᏻY β

Rπ∗ ᏻX˜

γ

/ Rφ∗ Rσ∗ ᏻ ˜ . Y

Now ρ has a left inverse by assumption and β is a quasi-isomorphism since Y has rational singularities. Therefore (ρ ◦ β −1 ◦ γ ) ◦ α is a quasi-isomorphism of ᏻX with itself, so one may assume that φ is a resolution of singularities. · ) to the quasi-isomorphism Next apply R ᏴomX ( , ωX ρ

ρ

ᏻX −→ Rφ∗ ᏻY −−→ ᏻX .

Then

· −→ Rφ ω· −→ ω· ωX ∗ Y X · ) ⊆ R i φ ω· R i+d φ ω . Now is a quasi-isomorphism as well. Hence hi (ωX ∗ Y ∗ Y · ) = 0 for i > −d, R i+d φ∗ ωY = 0 for i > −d by GR vanishing. Therefore hi (ωX so X is Cohen-Macaulay. The above proof also shows that ωX −→ φ∗ ωY −→ ωX is an isomorphism, so ωX φ∗ ωY . Therefore X has rational singularities by Lemma 1. Theorem 4 [E]. Log-terminal singularities are rational. Proof. Let X be a variety with log-terminal singularities. By Theorem 1 it is enough to prove that the index-1 cover of X has rational singularities. Thus one can

190

SÁNDOR J. KOVÁCS

assume that X has canonical singularities and ωX is a line bundle. Let φ : Y → X be a resolution of singularities of X. By assumption there exists a nontrivial morphism ι : Lφ ∗ ωX φ ∗ ωX → ωY . Its adjoint morphism on X is ωX → Rφ∗ ωY , which is a quasi-isomorphism by GR vanishing. Applying R ᏴomY ( , ωY ) (not R ᏴomY ( , ωY· )) to ι, one obtains Rφ∗ R ᏴomO Y (ωY , ωY )

/ Rφ∗ R ᏴomY (Lφ ∗ ωX , ωY )

/ R ᏴomX (ωX , Rφ∗ ωY )

/ ᏻX .

ρ

Rφ∗ ᏻY

The last quasi-isomorphism uses the fact that Rφ∗ ωY ωX and that ωX is a line bundle. It is easy to see that ρ ◦ ρ acts trivially on ᏻX ; hence Theorem 1 can be applied. The following is a simple consequence of [K1, 7.6]. Theorem 5. Let φ : Y → X be a surjective morphism of projective varieties over C, and assume that both X and Y have rational singularities. Let N = dim Y and n = dim X. Then R N−n φ∗ ωY ωX . Proof. Let π : X˜ → X be a resolution of X and σ : Y˜ → Y a resolution of Y such that φ ◦ σ factors through π; that is, there exists ψ : Y˜ → X˜ such that φ ◦ σ = π ◦ ψ. Then one has the following commutative diagram: R(φ ◦ σ )∗ ωY˜ [N]

δ

α

Rφ∗ ωY [N]

/ Rπ∗ ω ˜ [n] X β

γ

/ ωX [n].

Because both X and Y have rational singularities, α and β are quasi-isomorphisms by GR vanishing. Next take −nth cohomology of these complexes. By [K2, 3.4], R −n (φ ◦ σ )∗ ωY˜ [N] R N−n (π ◦ ψ)∗ ωY˜ π∗ R N−n ψ∗ ωY˜ , so one has π∗ R N−n ψ∗ ωY˜

h−n (δ)

h−n (α)

R N−n φ∗ ωY

/ π∗ ωX˜ h−n (β)

h−n (γ )

/ ωX .

Now h−n (δ) is an isomorphism by [K1, 7.6], so h−n (γ ) is an isomorphism as well.

A CHARACTERIZATION OF RATIONAL SINGULARITIES

191

Lemma 2. Let φ : Y → X be a surjective morphism of projective varieties over C and ρ : ᏻX → Rφ∗ ᏻY the associated natural morphism. Let N = dim Y and n = dim X. Assume that Y has rational singularities. If X is Cohen-Macaulay and R N−n φ∗ ωY ωX , then there exists a morphism ρ : Rφ∗ ᏻY → ᏻX such that ρ ◦ρ is a quasi-isomorphism of ᏻX with itself. Proof. Let σ : Y˜ → Y be a resolution of singularities. Since Y has rational singularities, ᏻY Rσ∗ ᏻY˜ and ωY Rσ∗ ωY˜ . Thus one may assume that Y is smooth. · ω [n] R N−n φ ω [n] is a direct summand of Rφ ω [N] By [K2, 3.1] ωX X ∗ Y ∗ Y Rφ∗ ωY· . Therefore there exist morphisms whose composition is a quasi-isomorphism ·. · ωX −→ Rφ∗ ωY· −→ ωX · ) to this quasi-isomorphism, one concludes that Applying R ᏴomX ( , ωX ρ

ρ

ᏻX −→ Rφ∗ ᏻY −→ ᏻX

is a quasi-isomorphism as well. Corollary. Theorems 2 and 3 hold. References [B] [CKM] [E] [GR] [H] [KMM]

[KKMS] [K1] [K2] [KM] [R]

J.-F. Boutot, Singularités rationnelles et quotient par les groupes réductifs, Invent. Math. 88 (1987), 65–68. H. Clemens, J. Kollár, and S. Mori, Higher-Dimensional Complex Geometry, Astérisque 166 (1988). R. Elkik, Rationalité des singularités canoniques, Invent. Math. 64 (1981), 1–6. H. Grauert and O. Riemenschneider, Verschwindungssätze für analytische Kohomologiegruppen auf komplexen Räumen, Invent. Math. 11 (1970), 263–292. R. Hartshorne, Residues and Duality, Lecture Notes in Math. 20, Springer-Verlag, Berlin, 1966. Y. Kawamata, K. Matsuda, and K. Matsuki, “Introduction to the minimal model problem” in Algebraic Geometry (Sendai, 1985), Adv. Stud. Pure Math. 10, NorthHolland, Amsterdam, 1987, 283–360. G. Kempf, F. Knudsen, D. Mumford, and B. Saint-Donat, Toroidal Embeddings, I, Lecture Notes in Math. 339, Springer-Verlag, Berlin, 1973. J. Kollár, Higher direct images of dualizing sheaves, II, Ann. of Math. (2) 124 (1986), 171–202. , “Singularities of pairs” in Algebraic Geometry (Santa Cruz, 1995), Proc. Sympos. Pure Math. 62, Amer. Math. Soc., Providence, 1997, 221–287. J. Kollár and S. Mori, Birational Geometry of Algebraic Varieties, Cambridge Tracts in Math. 134, Cambridge Univ. Press, Cambridge, 1998. M. Reid, “Canonical 3-folds” in Journées de Géométrie Algébrique d’Angers (Juillet 1979), Sijthoff & Noordhoff, Alphen aan den Rijn, 1980, 273–310.

Department of Mathematics, University of Chicago, Chicago, Illinois 60637 USA; [email protected]; [email protected]

Vol. 102, No. 2


© 2000

SMOOTHNESS OF PROJECTIONS, BERNOULLI CONVOLUTIONS, AND THE DIMENSION OF EXCEPTIONS YUVAL PERES and WILHELM SCHLAG n 1. Introduction. The distribution νλ of the random series ∞ 0 ±λ , where the 1 signs are chosen independently with probability 2 , has been studied by many authors since the two seminal papers by Erd˝os in 1939 and 1940. It is immediate that νλ is singular for λ < 21 . In [6], Erd˝os showed that νλ is singular for infinitely many λ ∈ ( 21 , 1), namely those λ such that λ−1 is a Pisot number. In [7] he proved that νλ is absolutely continuous for a.e. λ ∈ (1 − δ, 1), where δ < 0.01. It had been observed before by Jessen and Wintner [21] that νλ is either absolutely continuous or singular with respect to Lebesgue measure for any choice of λ. Works of Alexander and Yorke [2], Przytycki and Urbański [37], and Ledrappier [27] showed the importance of these distributions in several problems in dynamical systems. Solomyak [39] proved a conjecture made by Garsia in 1962: νλ is absolutely continuous for a.e. λ ∈ ( 21 , 1). In fact, Solomyak showed that νλ has density in L2 for a.e. λ ∈ ( 21 , 1). Peres and Solomyak [34] then gave a simpler proof of this. A survey of the results obtained on Bernoulli convolutions from the 1930s to 1999 is presented in [33]. In the present paper, we show that the density of νλ has a fractional derivative in L2 for a.e. λ ∈ ( 21 , 1), and we obtain a bound on the integrated Sobolev norm. We then use this bound to establish that for any closed interval I ⊂ ( 21 , 1], the set of λ ∈ I such that νλ is singular, has Hausdorff dimension less than 1 (see Theorem 5.4 and Corollary 5.6). In order to prove these results, we were led to develop a general method, which can be applied to improve previous results on Hausdorff dimensions of sums of Cantor sets, distance sets, and self-similar sets with deleted digits. (An exposition of our method for the case of Bernoulli convolutions is in [33].) Heuristically, the measures νλ may be viewed as nonlinear projections of the uniform measure on sequence space. Indeed, the analysis of νλ in [39] and [34] employed techniques developed by Kaufman [23] and Mattilla [30] to study orthogonal projections of sets and measures in Euclidean space. The following (informally stated) principles were first discovered by Marstrand [28], Kaufman [23], [24], and Mattila [29], [30] in the study of orthogonal projections, and then they were extended to a variety of other settings by Falconer [12], Hu and Received 6 January 1999. 1991 Mathematics Subject Classification. Primary 42B25; Secondary 28A78. Y. Peres’s work partially supported by National Science Foundation grant number DMS-9404391. W. Schlag’s work supported by a National Science Foundation Postdoctoral Fellowship at the Mathematical Sciences Research Institute, Berkeley, California. 193

194

PERES AND SCHLAG

Taylor [16], Pollicott and Simon [36], Solomyak [39], [40], [41], Peres and Solomyak [34], [35], and Sauer and Yorke [38]: • if an (m − )-dimensional measure is projected to m-dimensional space, then typically its dimension is preserved; • if an (m + )-dimensional measure is projected to m-dimensional space, then typically the projected measure is absolutely continuous, with a density in L2 . In making these principles precise, the appropriate notion of dimension of measures must be used (correlation or information dimension, but not Minkowski or packing dimension; see Jarvenpää [20] and [38]), and the parametrized family of generalized projections considered must be sufficiently rich (at least m-dimensional). A more delicate requirement is a certain “transversality condition” (e.g., [31, Lemma 3.11]). For Bernoulli convolutions and other self-similar measures, this condition involves bounds on the double zeros of certain power series, and it was first made explicit by Pollicott and Simon [36]. Transversality is crucial in the works of Solomyak [39], [40], [41], Peres and Solomyak [34], [35], and in the present paper. The general principles that underly our work can be informally stated as follows: • if an (m − )-dimensional measure µ is projected to an m-dimensional space, then the set of parameters where the projection of µ has lower dimension than µ itself, has codimension at least ; • if an (m + )-dimensional measure is projected to m-dimensional space, then typically the projected measure has a density with a fractional derivative of order

/2 in L2 , and the set of parameters where the projected measure is singular has codimension at least . The notion of Sobolev dimension of a measure, defined in (2.3) below, yields a unification of these two principles. For orthogonal projections, these strengthened principles have been stated precisely, and established as theorems, by Kaufman [24], Falconer [9], and Mattila [29], except that they did not consider fractional derivatives. Their proofs are based on averaging with respect to an appropriate Frostman measure on parameter space. In the setting of Bernoulli convolutions, this averaging approach is not powerful enough to establish the last principle, so we were forced to develop a different technique. Our methods apply to several concrete problems, which we now describe. Throughout the paper, dim (without subscripts) means Hausdorff dimension. Orthogonal projections. To motivate the general formulations, we start by clarifying the important results of Falconer [9] on exceptional directions for projections in Rd (see Sections 2 and 6). As a corollary, we find that for any Borel set E ⊂ Rd with dim E > 2 and for a.e. direction θ in the sphere S d−1 , the orthogonal projection projθ (E) of E to the line through 0 and θ has nonempty interior. More precisely (see Corollary 6.2), dim θ ∈ S d−1 : projθ (E) has empty interior ≤ d + 1 − dim E.

SMOOTHNESS OF PROJECTIONS

195

Bernoulli convolutions. Let νλ be the distribution series ∞ of the random Symmetric ∞ n , so that ν has the Fourier transform ν (ξ ) = n ξ ). Here we cos(λ ±λ λ λ n=0 n=0 sharpen the result of Solomyak [39] that νλ is absolutely continuous for a.e. λ ∈ ( 21 , 1) by showing that the 2, γ -Sobolev norm satisfies ∞ νλ (ξ )2 |ξ |2γ dξ < ∞ νλ 22,γ = −∞

for a.e. λ < 0.649 such that λ1+2γ > 21 . Moreover, for some constant C > 0 and all small > 0, we have

1 + , 1 : νλ is singular < 1 − C . dim λ ∈ 2 p

Bernoulli convolutions. Let νλ be the distribution of the random sum Asymmetric ∞ n , where the signs are chosen independently with probabilities (p, 1 − p). ±λ n=0 p Peres and Solomyak [35] showed that νλ is absolutely continuous for a.e. λ ∈ (p p (1− p p)1−p , 1), but νλ has a density in L2 for a.e. λ ∈ (p 2 + (1 − p)2 , 1) and not for any p smaller λ. The Pisot numbers still provide infinitely many examples of singular νλ . In the present paper, we show that ∞ p 2 2γ νλ (ξ ) |ξ | dξ < ∞ −∞

for a.e. λ < 0.649 such that λ1+2γ > p 2 + (1 − p)2 . Moreover, if p ∈ ( 13 , 23 ), then for some constant C > 0 and all > 0,

p dim λ ∈ p 2 + (1 − p)2 + , 1 : νλ ∈ L2 < 1 − C ,

p dim λ ∈ p p (1 − p)1−p + , 1 : νλ is singular < 1 − C . Intersections of sets with spheres. Suppose E ⊂ Rd is a Borel set with d ≥ 2. Let S0 ⊂ Rd be a strictly convex, closed C ∞ -hypersurface surrounding the origin. Then dim x ∈ Rd : (x + rS0 ) ∩ E = ∅ for a.e. r > 0 ≤ 1 + d − dim E. This statement fails if S0 is the boundary of a cube, or more generally, if S0 contains a piece of a hyperplane. If S0 is the sphere S d−1 , then the estimate above reduces to a strengthened form of Falconer’s result in [10] about distance sets. For any Borel set A ⊂ Rd with Hausdorff dimension dim A > (d +1)/2, there are points x ∈ A such that the pinned distance set {|x − y| : y ∈ A} has positive Lebesgue measure. Moreover, the set of x ∈ Rd where this fails has Hausdorff dimension at most d + 1 − dim A (see Corollary 8.4). Variants involving the Hausdorff dimension of the radii, as well as estimates where x is restricted to a hyperplane, can be found in Section 8.2. We also obtain results if S0 is assumed to be only in C 2,δ for some 1 > δ > 0.

196

PERES AND SCHLAG

Sums of Cantor sets. Let ᏷λ = (1 − λ)

∞

n

an λ : an ∈ {0, 1}

n=0

be the middle-α Cantor set for α = 1 − 2λ. Let K ⊂ R be any compact set. Peres and Solomyak [35] showed that Ᏼ1 (K + ᏷λ ) > 0 for almost every λ ∈ (0, 21 ) such that dim K +dim ᏷λ > 1, and that dim(K + ᏷λ ) = dim K +dim ᏷λ for a.e. λ ∈ (0, 21 ) such that dim K + dim ᏷λ < 1. In Theorem 5.12, we prove that 1 dim λ ∈ λ0 , : Ᏼ1 (K + ᏷λ ) = 0 ≤ 2 − (dim K + dim ᏷λ0 ), 2 dim λ ∈ (0, λ1 ) : dim(K + ᏷λ ) < dim K + dim ᏷λ ≤ dim K + dim ᏷λ1 . We also establish similar estimates for Cantor sets whose symbols belong to the Hölder space C 1,δ for some δ > 0. Keane-Smorodinsky {0, 1, 3}-problem. The Hausdorff dimension of the set ∞ i ai λ : ai ∈ {0, 1, 3} (λ) = i=0

has been studied by several authors, since it is perhaps the simplest example of a parametrized family of self-similar sets with overlap. For λ ≤ 41 , it is easy to see that (λ) is self-similar with Hausdorff dimension log 3/(− log λ). Keane, Smorodinsky, and Solomyak [26] proved that (λ) = [0, 3/(1−λ)] if λ > 25 and that dim (λ) < 1 for infinitely many λ ∈ ( 31 , 25 ). Pollicott and Simon [36] showed that for a.e. λ ∈ ( 41 , 31 ), one still has dim (λ) = log 3/(− log λ), and they found a dense subset of ( 41 , 31 ) where the dimension is strictly less than that fraction. Solomyak [39] finally established that for a.e. λ ∈ ( 13 , 25 ), the set (λ) has positive Lebesgue measure. In Theorem 5.10, we establish that 2 log 3 1 1 dim λ ∈ λ0 , : Ᏼ ( (λ)) = 0 ≤ 2 − for any λ0 > , 5 − log λ0 3 1 log 3 1 log 3 dim λ ∈ , λ1 : dim (λ) < ≤ for any λ1 > . 2 − log λ − log λ1 4 The latter inequality improves an estimate from [36]. Self-similar sets in the plane. Consider the self-similar sets ∞ S n Cλ = an λ : a n ∈ S , n=0


197

where S = {s1 , . . . , s# } ⊂ C is a fixed set of symbols. In [41] Solomyak showed that Ᏼ2 (CλS ) > 0 for a.e. λ > |l|−1/2 in a region of transversality. In Theorem 8.2, we prove that the dimension of the exceptional set for this property has to be strictly less than 2. Typical C 1 maps trade off dimension for smoothness. Hunt, Sauer, and Yorke [18] defined a Borel set A in a Banach space X to be prevalent if some Borel probability measure ν on X satisfies ν(A + x) = 1 for all x ∈ X. Sauer and Yorke [38] and Hunt and Kaloshin [17] showed that if µ is a Borel probability measure on Rd with correlation dimension at most m, then the set Aµ (d, m) of C 1 maps f : Rd → Rm that map µ to a measure with the same correlation dimension is prevalent. To prove this, they showed that the definition of prevalence holds with ν equal to Lebesgue measure on the set of m×d real matrices with entries bounded by 1 in absolute value. (These matrices are considered as linear maps from Rd to Rm .) This result can be obtained as a special case of Theorem 7.3, which also yields the following complement (see Remark 7.4): Let µ be a Borel probability measure on Rd with correlation dimension greater than m + 2γ . Then for a prevalent set of C 1 maps f : Rd → Rm , the image of µ under f has a density with at least γ fractional derivatives in L2 (Rm ). Consequently, if E ⊂ Rd has dim E > m (respectively, dim E > 2m), then for a prevalent set of maps f : Rd → Rm , the image f (E) has positive Lebesgue measure (respectively, nonempty interior) in Rm . All these examples are special cases of our main result, which is formulated in the following general setting. Given a measure µ on a compact metric space (', d) and a family of maps (λ : ' → Rm parameterized smoothly by λ ∈ Rn with n ≥ m, we show that for a.e. λ the projection of µ under (λ has as much “smoothness” as µ and that the Hausdorff dimension of the exceptional parameters decreases with the loss of smoothness. This requires the family (λ to satisfy some nondegeneracy assumption. In this paper, we impose the aforementioned transversality condition. In n where ω ∈ ' = {−1, +1}N the case of Bernoulli convolutions, (λ (ω) = ∞ ω λ n=0 n (N denotes the nonnegative integers). Here transversality means that the power series (λ (ω) − (λ (τ ) do not have double zeros. Solomyak showed in [39] that this is the case if λ < 0.649. Extending our smoothness results to the entire unit interval then requires arguments that are special to Bernoulli convolutions, namely, breaking the series up into various subseries. The paper is organized as follows. In Section 2, which is still mainly expository, we discuss transversality and state the general projection theorem in one dimension. Our estimate on the Hausdorff dimension of the set of λ for which νλ is a singular measure does not rely on Frostman’s lemma. Instead we use the following fact: Let {hj }∞ j =0 be a family of nonnegative C ∞ -functions on [0, 1] whose derivatives grow at most 1 j h (x) dx < ∞, then one can bound the dimension of exponentially in j . If 0 ∞ j 0 R j the set of x ∈ [0, 1] for which ∞ 0 r hj (x) = ∞ for any 1 ≤ r < R. This is proved in Section 3, see Lemma 3.1. In Section 3 we also deal with the case where {hj }∞ j =0 is assumed to have only finitely many derivatives. This is applied later to analyze

198

PERES AND SCHLAG

sums of Cantor sets with symbols in the Hölder space C 1,δ and to the geometric problem about intersections of sets with dilates of convex surfaces. It turns out that it suffices to assume some finite degree of smoothness on the surfaces. For Bernoulli convolutions, however, the C ∞ case suffices. In Section 4 we prove the general projection theorem in case m = n = 1. Our methods rely on the dyadic decomposition of frequency space (i.e., the LittlewoodPaley decomposition from harmonic analysis), which is recalled at the beginning of Section 4. To make the connection 3.1, which we sketched in the with Lemma previous paragraph, let hj (λ) = 2−j |ξ |∼2j | νλ (ξ )|2 dξ , where νλ is the distribution n j (1+2γ ) h (λ) dλ is controlled by the square of the of, say, ±λ . Then J ∞ j j =0 2 j 2, γ -Sobolev norm of νλ averaged in λ, whereas ∞ j =0 2 hj (λ) = ∞ characterizes 2 those λ ∈ J so that νλ does not have an L -density. Section 4 is split into two subsections. In the first subsection, we discuss the case where the projections have infinitely many derivatives with respect to the parameter, whereas the second subsection deals with the case where the dependence on the parameter has only some finite degree of smoothness. Applications of the one-dimensional scheme are discussed in Section 5. We start with classical Bernoulli convolutions, then consider asymmetric Bernoulli convolutions, the {0, 1, 3}-problem, and finally sums of Cantor sets. In all of these applications, the main new results concern smoothness of the densities of certain measures and the dimension of the set of those parameters for which some generic property fails. Section 6 discusses orthogonal projections in Euclidean space. We give a short proof of the projection theorem in this special case, which is simpler than the available proofs in the literature. Moreover, we obtain sufficient conditions for sets to have a.e. projection with nonempty interior. This section also serves to motivate the statement of the general projection theorem in higher dimensions, which is proved in Section 7. In Section 8, this result is applied to self-similar sets in the complex plane and to distance sets in arbitrary dimensions. We conclude the paper by stating some unsolved problems related to our work. Acknowledgments. We are grateful to P. Mattila and B. Solomyak for useful discussions. 2. A general scheme for families of projections. To motivate this section, we review some simple facts about orthogonal projections in Euclidean space. Definition 2.1. Let µ be a finite measure on Rn , n ≥ 2, with compact support. n−1 , its projection ν onto the line {tθ : t ∈ R} is given by f dν = For any θ ∈ S θ θ f ((x · θ )θ ) dµ(x) for any continuous f . For any α ∈ (0, n), the α-energy of µ is defined to be Ᏹα (µ) =

Rn

Rn

dµ(x)dµ(y) . |x − y|α

199


We measure smoothness of measures ν in Rn in terms of the homogeneous Sobolev norm 2 νˆ (ξ )2 |ξ |2γ dξ. ν 2,γ = Rn

Finiteness of ν 2,γ for some γ > 0 means that ν has γ (fractional) derivatives in L2 . The following proposition relates the smoothness of the projected measures to the energy of the original measure. If a, b > 0, then the notation a b means that C −1 a < b < Ca for some absolute constant C. Proposition 2.2. Let µ be a finite measure on Rn , n ≥ 2, with compact support, and let νθ be as in Definition 2.1. Suppose 0 < 1 + 2γ < n. Then S n−1 νθ 22,γ dθ Ᏹ1+2γ (µ). ˆ · θ)θ). Hence Proof. The projected measures νθ clearly satisfy νθ (ξ ) = µ((ξ ∞ ∞ 2 2γ νθ (tθ)2 |t|2γ dt dθ = νθ 22,γ dθ = |µ(tθ)| ˆ |t| dt dθ S n−1

S n−1 −∞

=2

Rn

S n−1 −∞

|µ(ξ ˆ )|2 |ξ |1+2γ −n dξ = cγ ,n Ᏹ1+2γ (µ).

The final equality follows from Plancherel’s theorem and the definition of energy; see Lemma 12.12 in [31]. The main purpose of this section is to study regularity properties of projected measures in a more general setting. Moreover, we bound the dimension of the set of those parameters for which the projected measure has less regularity than generically. In case of projections in the plane, the latter question was considered by Kaufman [24] and Falconer [9]; see also Section 6. For the exceptional set in Proposition 2.2 with γ = 0, Falconer found the estimate dim θ ∈ S 1 : νθ ∈ L2 (R) ≤ 2 − α if Ᏹα (µ) < ∞. However, his method does not apply to Bernoulli convolutions or other examples considered in this paper. Our approach is based on the Littlewood-Paley decomposition from harmonic analysis, which provides a simple characterization of various degrees of smoothness; see Section 4. Moreover, the analogue of Falconer’s bound is obtained without Frostman measures. Definition 2.3. For any finite measure ν on Rn , let its Sobolev dimension be defined as (2.1)

dims (ν) = sup α ∈ R :

|ˆν (ξ )| (1 + |ξ |) 2

Rn

α−n

dξ < ∞ .

Clearly, if 0 < dims (ν) < n, then dims (ν) = sup{α : Ᏹα (ν) < ∞}. In particular, if a Borel set E ⊂ Rn supports a probability measure µ with dims (µ) ≤ n, then

200

PERES AND SCHLAG

dim E ≥ dims (µ). If dims (ν) < n, then dims (ν) is also known as the correlation dimension of the measure ν. If dims (ν) = σ > n, then ν is absolutely continuous and its density has fractional derivatives of order (σ − n)/2 in L2 (Rn ). Throughout this paper, dim (without sub- or superscripts) means Hausdorff dimension. The following definition introduces the general framework we are working with. Definition 2.4. Let (', d) be a compact metric space, let J ⊂ R be an open interval, and let ( : J × ' → R be a continuous map. We assume that for any compact I ⊂ J and # = 0, 1, . . . , there exists a constant C#,I such that d# (2.2) # ((λ, ω) ≤ C#,I dλ for all λ ∈ I and ω ∈ '. Given any finite measure µ on ', let νλ = µ ◦ (−1 λ where (λ (·) = ((λ, ·). The α-energy of µ is defined as Ᏹα (µ) = dµ(ω1 ) dµ(ω2 )/d(ω1 , ω2 )α ' '

.

Informally, one can think of (λ (·) = ((λ, ·) as a family of projections parameterized by λ. We state two results on the smoothness of a typical νλ in terms of Ᏹα (µ) assuming certain transversality conditions. Definition 2.5. Let ', J , and ( be as in Definition 2.4. For any distinct ω1 , ω2 ∈ ' and λ ∈ J , let ((λ, ω1 ) − ((λ, ω2 ) . 3λ (ω1 , ω2 ) = d(ω1 , ω2 ) J is an interval of strong transversality for ( if there exist positive constants C, C# so that for all λ ∈ J and ω1 , ω2 ∈ ', # (i) |(d # /dλ# )3λ (ω1 , ω2 )| ≤ C# |(d/dλ)3λ (ω1 , ω2 ) for # = 2, 3, . . . , (ii) |(d/dλ)3λ | ≥ C. Theorem 2.6. Let J and ( be as in Definition 2.4 and suppose that J is an interval of strong transversality for (. Let µ be a finite positive measure on ' with finite α-energy for some α > 0. Then for any compact I ⊂ J , νλ (ξ )2 dλ ≤ Cα |ξ |−α Ᏹα (µ). (2.3) I

Moreover, dims (νλ ) ≥ α for a.e. λ ∈ J . More precisely, for any σ ∈ (0, α], dim λ ∈ J : dims (νλ ) < σ ≤ σ + min(1 − α, 0). (2.4) The condition of strong transversality turns out to be too restrictive for most applications. Since the cosine function has vanishing derivative at 0, π, it fails for projections onto lines. More importantly, (2.3) fails for Bernoulli convolutions because of Pisot numbers; See Lemma 5.7. The correct notion of transversality in the context of Bernoulli convolutions turns out to be the following one.


201

Definition 2.7. Let 3λ be as in Definition 2.5. For any β ∈ [0, 1), we say that J is an interval of transversality of order β for ( if there exists a constant Cβ so that for all λ ∈ J and ω1 , ω2 ∈ ', the condition |3λ (ω1 , ω2 )| ≤ Cβ d(ω1 , ω2 )β implies d 3λ ≥ Cβ d(ω1 , ω2 )β . dλ

(2.5)

In addition, we say that (λ is L-regular on J for some positive integer L or L = ∞, if under the same condition and for some constants Cβ,# , (2.6)

# d −β# dλ# 3λ (ω1 , ω2 ) ≤ Cβ,# d(ω1 , ω2 )

for # = 1, 2, . . . , L.

Our main result in this section is the following theorem. Theorem 2.8. Let ', J , and ( be as in Definition 2.4, and suppose that J is an interval of transversality of order β for ( for some β ∈ [0, 1) and that (λ is ∞-regular on J . Let µ be a finite positive measure on ' with finite α-energy for some α > 0. Then for any compact I ⊂ J , 2 νλ dλ ≤ Cγ Ᏹα (µ) if 0 < (1 + 2γ )(1 + a0 β) ≤ α, (2.7) 2,γ I

where a0 is some absolute constant. Moreover, for any σ ∈ (0, α], (2.8)

dim λ ∈ J : dims (νλ ) ≤ σ ≤ 1 + σ −

α , 1 + a0 β

and for any σ ∈ (0, α − 3β], (2.9)

dim λ ∈ J : dims (νλ ) < σ ≤ σ.

In all our applications of this theorem, we are able to take β arbitrarily small. Moreover, the geometric problems we consider in this paper satisfy transversality with β = 0; see Sections 6 and 8.2. Some of these geometric estimates are known to be optimal. Since they are covered by our general theory, it follows that the corresponding cases of Theorem 2.8 are sharp; see Section 6 for more discussion. However, we do not know whether Theorem 2.8 is optimal in all cases. The basic idea behind the proof of (2.7) is that for any finite measure ν on R and γ ∈ (− 21 , 1),  1/2 ∞ 2

 (2.10) ν 2,γ 22j (γ +1) ν x − 2−j , x − ν x, x + 2−j  j =−∞ 2 L (dx)

202

PERES AND SCHLAG

as one can easily check by applying Plancherel’s theorem to the right-hand side. However, it is difficult to work with the square-function in (2.10) because of the singularities of the kernel χ[−1,0] − χ[0,1] . As is standard in harmonic analysis, one circumvents this difficulty by considering a smoother kernel. The most convenient form is given in terms of the Littlewood-Paley decomposition; see Lemma 4.1. Theorem 2.8 has a simple corollary concerning the dimension of the exceptional set with respect to absolute continuity without making any assumptions on the energy of µ. The hypothesis is in terms of the upper and lower information dimension of µ. Recall that the lower pointwise dimension is given by πµ− (ω) = lim inf r→0+

log µ(B(ω, r)) , log r

and set

dimI (µ) = πµ− L∞ (dµ)

and

dimI (µ) = essinf dµ(ω) πµ− (ω) .

Notice that Ᏹα (µ) < ∞ implies dimI (µ) ≥ α, but the converse is clearly false. Corollary 2.9. Suppose J is an interval of transversality of order β for ( and that (λ is ∞-regular on J . Then (2.11) (2.12)

dimI (µ) dim λ ∈ J : νλ is singular ≤ 2 − , 1 + a0 β dimI (µ) dim λ ∈ J : νλ is not absolutely continuous ≤ 2 − , 1 + a0 β

where a0 is the constant from Theorem 2.8. Proof. Let α = dimI (µ) and fix some α˜ < α and > 0. Then

log µ B(ω, r) ˜ ' = ω : lim inf > α˜ r→0+ log r ˜ > 0. By Egoroff’s theorem, there exists ' ⊂ ' ˜ so that µ(' ˜ \ ' ) < satisfies µ(')

and

log µ B(ω, r) ≥ α˜ uniformly on ' . lim inf r→0+ log r ( )

Let µ = µ ' and νλ = µ ◦ (−1 ˜ (µ ) < ∞, and thus by λ . By definition, Ᏹα− Theorem 2.8, ( ) dim λ ∈ J : dims νλ ≤ σ ≤ 1 + σ − (α˜ − )(1 + a0 β)−1 .

In particular, (2.13)

( ) dim λ ∈ J : νλ is singular ≤ 2 − (α˜ − )(1 + a0 β)−1 .


Because

203

(2−j ) λ ∈ J : νλ is singular ⊂ lim sup λ ∈ J : νλ is singular , j →∞

(2.11) follows by letting α˜ → α and → 0+ in (2.13). The proof of (2.12) is very similar and is omitted. 3. Exceptional parameters for convergence of power series 3.1. The C ∞ case. The dimension bounds (2.4) in case α ≤ 1 and (2.9) are analogous to Kaufman’s result for orthogonal projections and involve Frostman measures as in his original proof. However, if α > 1, estimates (2.4) and (2.8) are derived from the smoothness bounds by means of the following lemma, which we state for parameters in Rn for an arbitrary n ≥ 1. Recall that the length of a multi-index η = (η1 , η2 , . . . , ηn ) ∈ Nn is defined to be |η| = η1 + · · · + ηn , and that ∂ η = ∂ |η| /(∂λ1 )η1 · . . . ·(∂λn )ηn , where λ = (λ1 , . . . , λn ). For applications to Bernoulli convolutions, it suffices to read the following lemma with n = 1. ∞ Lemma 3.1. Let Q ⊂ Rn be a nonempty open set. Suppose {hj }∞ 0 ∈ C (Q) satisfy sup A−j |η| ∂ η hj ∞ ≤ Cη for all multi-indices η ∈ Nn

(3.1)

j ≥0

sup

j ≥0 Q

R j |hj (λ)| dλ ≤ C∗ < ∞,

where A > 1. Suppose that R > rj ≥ 1. (i) If An < R/r, then ∞ j =0 r |hj (λ)| < ∞ for all λ ∈ Q. (ii) If Aα ≤ R/r ≤ An , then (3.2)

∞ dim λ ∈ Q : r j |hj (λ)| = ∞ ≤ n − α. j =0

Proof. Fix some 0 < r < R with An ≥ R/r. It suffices to prove (3.2) for any compact cube Q ⊂ Q. Fix such a Q and define Ej = {λ ∈ Q : |hj (λ)| > (1/j 2 )r −j }. Then   ∞   (3.3) r j |hj (λ)| = ∞ ⊂ lim sup Ej . λ ∈ Q :   j →∞ j =0

We estimate the (n − α)-Hausdorff measure of lim sup Ej by covering each Ej with cubes of side length A−j . The idea is that any point in Ej has a neighborhood of size A−j on which the average of |hj | is at least C(1/j 2 )r −j . More precisely, for

204

PERES AND SCHLAG

any positive integer N , N N

N (3.4) (−1)i hj (λ + iy) ≤ |y|N sup ∂ η hj ∞ ≤ CN |y|Aj . i |η|=N i=0

For any j = 0, 1, . . . and λ0 ∈ Q , let hj (λ0 + λ)dλ, Ij,N (λ0 ) = [−N Lj,N ,NLj,N ]n where Lj,N > 0 is determined below. In view of (3.4), with some dimensional constant bn , N N bn CN N+n j N (−1)i hj (λ0 + iy) dy Lj,N A ≥ n N +n [−Lj,N ,Lj,N ] i=0 i (3.5) N N 1 ≥ (2Lj,N )n hj (λ0 ) − Ij,N (λ0 ). i in i=1

j 2 r j )−1/N with a suitable constant C , one has In particular, setting Lj,N = A−j (CN N

n 1 2 j −1 j r 2Lj,N ≥ (2Lj,N )n |hj (λ0 )| − 2N Ij,N (λ0 ), 2 and therefore, for any λ0 ∈ Ej , (3.6) Ij,N λ0 ≥ 2−N j −2 r −j (2Lj,N )n if j is sufficiently large (depending on dist(Q , ∂Q)). Fix some positive integer N, and let {Uij : i = 1, . . . , Mj,N } be a covering of Ej with disjoint cubes of side length 2NLj,N . Pick any λij ∈ Uij ∩ Ej . Setting λ0 = λij in (3.6) and summing over i = 1, . . . , Mj,N yields

n Mj,N 2−N j −2 r −j 2Lj,N ≤ 2n hj (λ)dλ ≤ 2nC∗ R −j . Q

Therefore, by the definition of Lj,N , Mj,N

(3.7)

"j ! n 1+(n/N) 2(1+(n/N)) A r ˜ ≤ CN j . R

Let α ∈ (0, 1) satisfy Aα r α/N < R/r. In view of (3.7), Ᏼ

n−α

lim sup Ej

≤ C˜ N lim

k→∞

= C˜ N lim

k→∞

∞

n−α Mj,N 2NLj,N

j =k ∞ 1+(α/N) r j =k

R

α

A

j

j 2(1+(α/N)) = 0.


205

Thus (3.2) follows from (3.3) by letting N → ∞. To prove assertion (i), assume that An < R/r and let N be a fixed positive integer such that r n/N An < R/r. Let λ0 ∈ Ej for some sufficiently large j . It follows from (3.6) that

j R , C∗ ≥ R j Ij,N (λ0 ) ≥ C˜ N j −2(1+(n/N)) Ar 1+(n/N) which is a contradiction for large j . Thus lim supj →∞ Ej = ∅, and (i) follows from (3.3). L 3.2. The case of limited regularity. Suppose Q ⊂ Rn and {hj }∞ 0 ⊂ C (Q) for L some positive integer L. Here C denotes the space of functions that are L times continuously differentiable. Under assumption (3.1) for |η| ≤ L, the previous proof with N = L yields (3.2), provided Aα r α/L ≤ R/r ≤ An r n/L . Below we show how to weaken this condition on α under a slightly different assumption. For a positive integer L and δ ∈ [0, 1), we let C L,δ denote the space of functions that are L times continuously differentiable and such that the derivatives of order L satisfy a Hölder condition of order δ (we write C 0,δ = C δ ).

Lemma 3.2. Let Q ⊂ Rn be an open set and suppose {hj }∞ 0 are nonnegative j and uniformly bounded functions satisfying supj ≥0 R Q hj (λ) dλ < C∗ < ∞. Let j Hj = i=0 B i−j hi for some fixed B ≥ A > 1. 1 (i) Let 1 ≤ r < R and r ≤ B. Assume that {hj }∞ 0 ⊂ C (Q) and ∇Hj (λ) ≤ C1 Aj Hj (λ) for all j = 0, 1, 2 . . . , and all λ ∈ Q. j If An < R/r, then ∞ j =0 r hj < ∞ on Q. Otherwise,   ∞   dim λ ∈ Q : (3.8) r j hj (λ) = ∞ ≤ n − α,   j =0

Aα

≤ R/r. provided (ii) Let B < r < R. Assume that, for some integer L ≥ 1 and δ ∈ [0, 1), one has L,δ (Q) and {hj }∞ 0 ⊂C ∇Hj (λ) ≤ C1 Aj Hj (λ) for all j = 0, 1, 2 . . . and all λ ∈ Q, (3.9)

$ # sup ∂ η Hj (λ1 ) − ∂ η Hj (λ2 ) ≤ CL,δ |λ1 − λ2 |δ Aj (L+δ) Hj (λ1 ) + Hj (λ2 ) .

|η|=L

j If An (r/B)n/(L+δ) < R/r, then ∞ j =0 r hj < ∞ on Q. Otherwise,   ∞   dim λ ∈ Q : (3.10) r j hj (λ) = ∞ ≤ n − α,   j =0

provided Aα (r/B)n/(L+δ) ≤ R/r.

206

PERES AND SCHLAG

Proof. It suffices to prove these statements for an arbitrary cube Q ⊂ Q. We start with the proof of (3.8). Fix 0 < r0 < r1 ≤ B and T ∈ (0, ∞), and let ∞ (T ) −2 −j i Ej = λ ∈ Q : hj (λ) > j r1 , (3.11) r0 hi (λ) ≤ T i=0

for j = 0, 1, . . . . Clearly, ∞ ∞ ∞ % (T ) i i (3.12) r1 hi (λ) = ∞, r0 hi (λ) < ∞ ⊂ lim sup Ej . λ∈Q : i=0

T =1 j →∞

i=0

M

(T )

j be a covering of Ej with disjoint Fix some positive integer T , and let {Uij }j =1 cubes of side length Wj < A−j for j = 0, 1, . . . . To make an appropriate choice of Wj , we bound the variation of hj on cubes. By the bound on |∇Hj |, d Hj (λ + te) ≤ C1 Aj Hj (λ + te) dt

for any unit vector e and λ, λ + te ∈ Q . By Gronwall’s inequality (see, e.g., [14, Chapter 10]), Hj (λ) ≤ Hj (λ0 ) exp(C1 Aj |λ − λ0 |) for any λ, λ0 ∈ Q . Hence, for all j = 0, 1, . . . , Hj (λ) − Hj (λ0 ) ≤ CAj |λ − λ0 |Hj (λ0 ), (3.13) provided |λ − λ0 | < A−j . Since r0 < B, ∞ j =0

j

r0 Hj (λ0 ) ≤

j i ∞ r0 j B j =0

B

i=0

r0

∞

−1 r0i hi (λ0 ) ≤ 1 − r0 /B r0i hi (λ0 ). i=0

Combining this with (3.13) yields (3.14) # $ $ # hj (λ) − hj (λ0 ) = B −j B j Hj (λ) − Hj (λ0 ) − B j −1 Hj −1 (λ) − Hj −1 (λ0 ) j ∞

−1 A ≤ C 1 − r0 /B |λ − λ0 | r0i hi (λ0 ) r0 i=0

(T ) Uij ∩ Ej

if |λ − λ0 | < A−j . Applying this to any λ0 ∈ and λ ∈ Uij , one concludes that

−1 A j 1 −j 1 −j |λ − λ0 | ≥ 2 r1 hj (λ) ≥ 2 r1 − CT 1 − r0 /B r0 j 2j 2 j −j ˜ if Wj = C1/j (r0 /r1 ) A , where C˜ depends only on T , B, and r0 . Therefore, 1 1 −j −j −j C∗ R ≥ (3.15) hj (λ) dλ ≥ &Mj r1 dλ = 2 Mj Wjn r1 , 2 2j 2j Q i=1 Uij

207


and thus, by choice of Wj , Mj ≤ 2C∗ j 2(1+n)

r

1

R

An

j r j n 1

r0

.

Hence,

(T )

Ᏼn−α lim sup Ej

≤ lim

k→∞

j →∞

∞ j =k

Mj Wjn−α

r1 n j r1 j n A 2C∗ j ≤ lim k→∞ R r0 j =k j (n−α) r0 × C˜ n−α j −2(n−α) A−j (n−α) r1 ∞ r

1 α j r1 j α A = C lim j 2(1+α) = 0, k→∞ R r0

∞

2(1+n)

j =k

)α .

Aα

< R/r1 (r0 /r1 provided Therefore, in view of (3.12), ∞ ∞ i i (3.16) r1 hi (λ) = ∞, r0 hi (λ) < ∞ ≤ n − α dim λ ∈ Q : i=0

i=0

)α

if Aα

≤ R/r1 (r0 /r1 and 0 < r0 < r1 ≤ A. To prove (3.8), choose r = ρm > ρm−1 > · · · > ρ2 > 1 > ρ1 . Then

(3.17) λ∈Q:

∞

i

r hi (λ) = ∞ =

i=0

and (3.16) imply that

m %

λ∈Q:

k=2

dim λ ∈ Q :

∞ i=0

∞

ρki hi (λ) = ∞,

∞ i=0

i ρk−1 hi (λ) < ∞

r i hi (λ) = ∞ ≤ n − α

i=0

Aα

)α .

≤ R/r mini=2,...,m (ρi−1 /ρi Letting max ρi /ρi−1 → 1 finishes the proof. If An < R/r1 (r0 /r1 )n , then (3.15) with Mj = 1 leads to a contradiction if j is sufficiently large. Subdividing as in (3.17) therefore shows that the left-hand side of (3.17) is empty if R/r > An , and the proof of (i) is complete. (T ) For the proof of (ii), let r > r0 = B and T ∈ (0, ∞), and define Ej as in (3.11).

if

(T )

The definition of Hk implies that supk≥0 B k Hk (λ0 ) ≤ T for all λ0 ∈ Ej Therefore, by (3.13), |Hk (λ) − Hk (λ0 )| ≤ CT |λ − λ0 |, and thus (3.18)

Hk (λ) ≤ CT B −k

and all j .

208

PERES AND SCHLAG (T )

provided |λ0 − λ| < B −k for all λ0 ∈ Ej , and all j and k. Let N be the smallest integer greater than or equal to L + δ. In view of the definition of Hj , the Hölder estimate (3.9) on ∂ η Hj for |η| = L, and (3.18) with k = j and k = j − 1, N N i (−1) hj (λ0 + iy) i i=0 N N ( ' = B −j (−1)i B j Hj (λ0 + iy) − B j −1 Hj −1 (λ0 + iy) i (3.19) i=0

L+δ ≤ CL,δ Aj |y| max Hj (λ0 + ty) + max Hj −1 (λ0 + ty) 0≤t≤N

0≤t≤N

L+δ

≤ CL,δ T Aj |y| (T )

if λ0 ∈ Ej

B −j

1 ≤ 2 r −j 2j

and |y| ≤ Cj −2/(L+δ) A−j (B/r)j/(L+δ) = Wj , where C = C(L, δ, T ). M

(T )

j Now let {Uij }i=1 be a covering of Ej

(T ) some λij ∈ Uij ∩ Ej Wj , λij + Wj ]n yields

with cubes of side length 2NWj , and fix

for all i and j . Integrating (3.19) with λ0 = λij over [λij −

N

N 1 1 −j n n r Wj ≥ hj (λij )Wj − hj λij + λ dλ i i n [−N Wj ,NWj ] 2j 2 i=1

and thus (3.20)

[−NWj ,N Wj ]

−1 n Wj . hj λij + λ dλ ≥ 2−N 2j 2 r j

Summing (3.20) over i = 1, . . . , Mj implies, in view of our assumption C∗ R −j ≥ Q hj (λ) dλ, that 1 r −j dλ 2nC∗ R −j ≥ 2n hj (λ) dλ ≥ &Mj N +1 j 2 2 Q i=1 Uij (3.21)

n −j 1 = N +1 2 Mj 2NWj r . 2 j Therefore, by definition of Wj , j j n/(L+δ) r r Mj ≤ Cj 2n/(L+δ) Anj . R B Hence,

α/(L+δ) R r (3.22) Ᏼ < = 0 if A B r ∞ i for all T . Since for any r > B, the set {λ ∈ Q : i=0 r hi (λ) = ∞} is contained in n−α

(T ) lim sup Ej j →∞

α

209


(3.23) λ∈Q:

∞

r i hi (λ) = ∞,

i=0

∞

B i hi (λ) < ∞ ∪ λ ∈ Q :

i=0

∞

B i hi (λ) = ∞ ,

i=0

(3.10) follows from (3.12), (3.22), and (3.8). To apply (3.8) to (3.23), one needs to check that Aα (r/B)α/(L+δ) < R/r implies that Aα ≤ R/B. However, this is an immediate consequence of B < r. (T ) Finally, if An (r/B)α/(L+δ) < R/r, then (3.21) with Mj = 1 shows that Ej = ∅ j if j is sufficiently large and thus r hj < ∞ on Q, as claimed. 4. Proofs of the general projection theorems 4.1. The C ∞ case. To estimate Sobolev norms, it is convenient to decompose frequency space dyadically. For the sake of completeness, we first recall the construction of such a Littlewood-Paley decomposition; see Stein [43] and Frazier, Jawerth, and Weiss [13]. ᏿(Rn ) is the Schwartz space of smooth functions all of whose derivatives decay faster than any power. It is a basic property of the Fourier transform that it preserves ᏿. Lemma 4.1. There exists ψ ∈ ᏿(Rm ) so that ψˆ ≥ 0, (4.1)

supp ψˆ ⊂ ξ ∈ Rm : 1 ≤ |ξ | ≤ 4 ,

∞

ψˆ 2−j ξ = 1

if ξ = 0.

j =−∞

Moreover, given any finite measure ν on Rm and any γ ∈ R, ∞

(4.2) ν 22,γ ψ2−j ∗ ν (x) dν(x), 22j γ j =−∞

Rm

where ψ2−j (x) = 2j m ψ(2j x). ˆ ) = 1 for |ξ | ≤ 1, and φ(ξ ˆ )=0 Proof. Choose φ ∈ ᏿(Rm ) with φˆ ≥ 0, φ(ξ ˆ ˆ ˆ ˆ for |ξ | > 2. Define ψ via ψ(ξ ) = φ(ξ/2) − φ(ξ ). It is clear that ψ(ξ ) ≥ 0 and ˆ ) = 0 if |ξ | < 1 or |ξ | > 4. Equation (4.1) holds since the sum telescopes. that ψ(ξ Moreover, it is clear from (4.1) that there exists some constant Cγ depending only on γ so that for any ξ = 0, Cγ−1 |ξ |2γ ≤

∞

22j γ ψˆ 2−j ξ ≤ Cγ |ξ |2γ .

j =−∞

ˆ −j Since ψ) 2−j (ξ ) = ψ(2 ξ ), Plancherel’s theorem implies

ψ2−j ∗ ν (x) dν(x) = ψˆ 2−j ξ |ˆν (ξ )|2 dξ, Rm

and (4.2) follows. Notice that (4.2) is closely related to (2.10).

210

PERES AND SCHLAG

The following proposition shows how to obtain a dimension bound from a suitable Sobolev estimate. This depends on Lemma 3.1. Proposition 4.2. Let J and νλ be as in Definition 2.4. If I νλ 22,γ dλ < ∞ with some I ⊂ J and γ > − 21 , then for all σ ∈ [0 ∧ 2γ , 1 + 2γ ), one has dim{λ ∈ I : dims (νλ ) ≤ σ } ≤ σ − 2γ . Proof. Let ψ be the Littlewood-Paley function from Lemma 4.1. For any j = 0, 1, 2, . . . , define hj (λ) = 2−j ψ) νλ (ξ )|2 dξ 2−j (ξ )| (4.3) # $

= ψ 2j ((λ, ω1 ) − ((λ, ω2 ) dµ(ω1 ) dµ(ω2 ). ' '

Lemma 4.1 implies that C νλ 22,γ ≥

(4.4)

∞

2j (1+2γ ) hj (λ).

j =0

Moreover, for any > 0, (4.5)

λ ∈ I : dims (νλ ) ≤ σ ⊂

 

λ∈I :



∞

 

j 2σ + hj (λ) = ∞

j =0



by (2.1) and Lemma 4.1. Since (2.2) implies that for any j, # = 0, 1, . . . , # j# $ (#) d ψ 2 ((λ, ω1 ) − ((λ, ω2 ) dµ(ω1 ) dµ(ω2 ) hj (λ) = # dλ ' ' l i d ( ≤ C# 2 j # , ≤ C# 2 j # dλi ∞ L (I ×') i=0

the proposition follows from Lemma 3.1 with n = 1, A = 2, r = 2σ + , and R = 22γ +1 as → 0+. Proposition 4.4 below deals with the case α ≤ 1 in Theorems 2.6 and 2.8, see (2.4) and (2.8). In that case it is more efficient to rely on Frostman measures than on the previous proposition. The idea of using Frostman measures appears repeatedly in geometric measure theory; see [31]. First we prove a simple technical lemma that describes the location of the zeros of 3λ . Lemma 4.3. Let J and ( be as in Definition 2.4. Suppose J = (λ0 , λ1 ) is an interval of transversality of order β for ( and that (λ is 1-regular on J . Let Cβ be the constant from (2.5) and set r = d(ω1 , ω2 ). Then


211

Nβ % λ ∈ J : |3λ | < Cβ r β = Ij ,

(4.6)

j =1

where Ij are disjoint open intervals of length C −1 r 2β ≤ |Ij | ≤ C and Nβ ≤ Cr 2β . With the possible exception of the at most two intervals touching ∂J , each Ij contains a unique zero λj of 3λ . If λ0 ∈ I1 and I1 does not contain a zero of 3λ , then we let λ1 = λ0 and similarly with λ1 . All constants depend only on β and the constants in Definition 2.7. Proof. Let the intervals Ij be defined by (4.6). Since Cβ r β ≤ |(d/dλ)3λ | ≤ Cβ,1 r −β on each Ij by Definition 2.7, their lengths are as claimed. The other statements are now clear. Proposition 4.4. Let ', J , and ( be as in Definition 2.4. Assume that J is an interval of transversality of order β for (, that (λ is 1-regular on J , and that the measure µ on ' has finite α-energy for some α ∈ (0, 1]. Then νλ = µ ◦ (−1 λ satisfies (4.7) Ᏼσ λ ∈ J : dims (νλ ) < σ = 0 for any σ ∈ (0, α − 3β]. If J is an interval of strong transversality, then (4.7) holds for all σ ∈ (0, α]. Proof. Suppose (4.7) fails for some σ . By outer regularity of Hausdorff measure, there exists 0 > 0 so that

Ᏼσ {λ ∈ J : dims (νλ ) ≤ σ − 0 } > 0. By Frostman’s lemma, there exists a nonzero measure ρ on J so that ρ(I ) ≤ |I |σ for all intervals I and

ρ {λ ∈ J : dims (νλ ) > σ − 0 } = 0. (4.8) Frostman’s lemma can be applied since {λ ∈ J : dims (νλ ) > κ} is an Ᏺσ -set for any κ > 0. Indeed, ∞ ∞ 2 κ+δ−1 % % λ∈J : νλ (ξ ) |ξ | (4.9) λ ∈ J : dims (νλ ) > κ = dξ ≤ M . δ>0 M=1

Since

−∞

iξ (λ (ω) − eiξ (λ0 (ω) dµ(ω) e ' ≤ |ξ | (λ (ω) − (λ0 (ω) dµ(ω) ≤ C|ξ ||λ − λ0 |,

νλ (ξ ) − ν* λ (ξ ) ≤ 0

'

the sets on the right-hand side of (4.9) are closed, as claimed. Thus for small > 0 and with r = d(ω1 , ω2 ),

212

PERES AND SCHLAG

J

(4.10)

∞

∞

dνλ (x) dνλ (y) dρ(λ) σ − −∞ −∞ |x − y| dµ(ω1 ) dµ(ω2 ) dρ(λ) = |((λ, ω1 ) − ((λ, ω2 )|σ − J ' ' dµ(ω1 ) dµ(ω2 ) = χ[|3λ |≥Cβ r β ] |3λ (ω1 , ω2 )|−(σ − ) dρ(λ) d(ω1 , ω2 )σ − ' ' J dµ(ω1 ) dµ(ω2 ) + χ[|3λ |≤Cβ r β ] |3λ (ω1 , ω2 )|−(σ − ) dρ(λ) d(ω1 , ω2 )σ − ' ' J dµ(ω1 ) dµ(ω2 ) ≤C (σ − )(1+β) d(ω 1 , ω2 ) ' ' N β dρ(λ) dµ(ω1 ) dµ(ω2 ) C + σ − λ − λj d(ω1 , ω2 )(σ − )(1+β) ' '

(4.11)

≤C

'

j =1

dµ(ω1 ) dµ(ω2 ) , d(ω1 , ω2 )α '

since σ + 3β ≤ α. The sum is obtained by applying Lemma 4.3 to (4.10). However, (4.11) contradicts (4.8) if < 0 . The easier case of strong transversality is implicit in the above and is omitted. Proof of Theorem 2.6. Fix some compact I ⊂ J , and let ρ ∈ C ∞ (R) be nonnegative so that ρ = 1 on I and supp(ρ) ⊂ J . Fix distinct ω1 , ω2 ∈ ' and let r = d(ω1 , ω2 ). Then for any nonnegative integer N, ∞ ∞ eiξ [((λ,ω1 )−((λ,ω2 )] ρ(λ) dλ = eiξ r3λ (ω1 ,ω2 ) ρ(λ) dλ −∞ −∞ (4.12) ∞ eiξ r3λ (ω1 ,ω2 ) LN (ρ)(λ) dλ, = −∞

where L is the differential operator L(·) = (d/dλ)((−iξ r3λ (ω1 , ω2 ))−1 ·). It follows from Definition 2.5 that |Lk (f )| ≤ Ck |ξ r|−k ki=0 f (i) ∞ for all k. In particular, applying (4.12) with N = 0 and N ≥ α yields ∞

−α iξ [((λ,ω1 )−((λ,ω2 )] e ρ(λ) dλ ≤ Cα |ξ |d(ω1 , ω2 ) . −∞

Therefore, ∞ ∞ νλ (ξ )2 ρ(λ) dλ = eiξ [((λ,ω1 )−((λ,ω2 )] ρ(λ) dλ dµ(ω1 ) dµ(ω2 ) −∞ ' ' −∞ dµ(ω1 ) dµ(ω2 ) ≤ C|ξ |−α . d(ω1 , ω2 )α ' '


213

This proves (2.3) and also that I νλ 22,(α−1)/2− dλ < ∞ for any small > 0. The dimension bound (2.4) in case α ≥ 1 thus follows from Proposition 4.2, whereas (2.4) with α ≤ 1 is covered by Proposition 4.4. In view of Propositions 4.2 and 4.4, Theorem 2.8 follows from the Sobolev estimate (2.7). This is proved using the characterization (4.2). The main technical statement is Lemma 4.6 below. The following routine calculus lemma is needed in the proof of that lemma. Lemma 4.5. Let L be a positive integer and suppose h ∈ C L (I ) satisfies h = 0 on some interval I ⊂ R. Let H denote the inverse of h. Then for any positive integer # ≤ L, (4.13)

H (#) =

#−1 t=0 #, $ p$

p

p

−(#+p1 +···+pt ) (# ) h 1 ◦ H 1 · · · · · h(#t ) ◦ H t a#, $ p$ h ◦ H

$ with some integer coefficients a#, $ p$ . The inner sum runs over vectors # = (#1 , . . . , #t ) and p$ = (p1 , . . . , pt ) with integer coordinates. Moreover, if # ≥ 2, one has a#, $ p$ = 0 unless # ≥ #i ≥ 2 for all i, min pj ≥ 1, and #1 p1 + · · · + #t pt ≤ 2(# − 1). Proof. If # = 1, then (4.13) holds with t = 0. The general case now follows easily by induction. Lemma 4.6. Let J and ( be as in Definition 2.4. Assume that J is an interval of transversality of order β for ( and that (λ is ∞-regular on J . Suppose ρ ∈ C ∞ (R) is supported on J . Let ψ be the Littlewood-Paley function from Lemma 4.1. Then for any distinct ω1 , ω2 ∈ ', any integer j , and any positive integer q, (4.14)

∞

−∞

−q ρ(λ)ψ 2 [((λ, ω1 ) − ((λ, ω2 )] dλ ≤ Cq 1 + 2j d(ω1 , ω2 )1+a0 β ,

j

where Cq depends only on q, ρ, and β, and a0 is some absolute constant. Proof. Fix distinct ω1 , ω2 ∈ ' and j, q as above. We may assume that 2j r > 1 where r = d(ω1 , ω2 ). Let φ ∈ C ∞ be nonnegative with φ = 1 on [− 21 , 21 ] and supp(φ) ⊂ [−1, 1]. Then ∞ # $ ρ(λ)ψ 2j ((λ, ω1 ) − ((λ, ω2 ) dλ −∞

(4.15) = ρ(λ)ψ 2j r3λ (ω1 , ω2 ) φ Cβ−1 r −β 3λ dλ

#

$ + ρ(λ)ψ 2j r3λ (ω1 , ω2 ) 1 − φ Cβ−1 r −β 3λ dλ. Here Cβ is the constant from Definition 2.7. By the rapid decay of ψ,

214 PERES AND SCHLAG

#

$ ρ(λ)ψ 2j r3λ (ω1 , ω2 ) 1 − φ C −1 r −β 3λ dλ β

−q

−q ≤ Cq,β |ρ(λ)| 1 + 2j r 1+β dλ ≤ Cq,β 1 + 2j r 1+β . Thus it suffices to estimate the first integral in (4.15). By Lemma 4.3, there exists χ ∈ C ∞ (R) depending only on β so that supp(χ) is compact, χ = 1 on a neighborhood of the origin, and such that

χ r −2β λ − λj = χ r −2β λ − λj φ Cβ−1 r −β 3λ , where these functions have disjoint supports for distinct j . We can therefore write the first integral in (4.15) as

ρ(λ)ψ 2j r3λ φ Cβ−1 r −β 3λ dλ =

Nβ i=1

+

ρ(λ)χ r −2β λ − λj ψ(2j r3λ ) dλ

 Nβ −2β

χ r λ − λj  ψ 2j r3λ φ Cβ−1 r −β 3λ dλ ρ(λ) 1 − 

i=1

Nβ

=

A(j ) + B.

i=1

To estimate the second term B, notice that |3λ | ≥ Cr 3β on the support of the integrand. The rapid decay of ψ therefore implies

−q |B| ≤ Cq,β 1 + 2j r 1+3β . For simplicity, we assume henceforth that i = 1 and set λ = λ1 . By choice of χ, we have |(d/dλ)3λ | ≥ Cβ r β on the support of the integrand of A(1) . In order to change variables in A(1) , we define H via

3λ = u ⇐⇒ λ = λ + H (u), provided χ r −2β λ − λ = 0. (4.16) Let F (u) = ρ(λ + H (u))χ (r −2β H (u))H (u). Thus

A(1) = F (u)ψ 2j ru du   2(q−1) F (#) (0)

 u# + O F (2q−1) (u)u2q−1 du ψ(2j ru) (4.17) =   #! |u|(2j r)−1/2


215

Let |H (u)| ≤ Cβ r −β by the choice of χ, and thus |F (u)| ≤ Cr −β . In particular, (1) |A2 | ≤ Cβ,q (2j r)−q−1 r −β . Since ψ has vanishing moments of all orders,

(1)

A1 = −

2(q−1) −q

F (#) (0) # u du + O F (2q−1) ∞ 2j r ψ 2j ru . #! |u|>(2j r)−1/2 #=0

It remains to estimate F (#) (u). Suppose λ and u are related via (4.16). By Lemma 4.5 (with L = ∞) and (2.6) in Definition 2.7, #−1 (#) H (u) ≤ Cβ,# a $ 3 −(#+p1 +···+pt ) r −β(#1 p1 +···+#t pt ) ≤ Cβ,# r −β(4#−3) . λ #,p$ t=0 #, $ p$

Therefore, by Leibnitz’s rule, (#)

F ≤ Cβ,# r −2β# r −β(4#−3) + r −β(4l+1) ≤ Cβ,# r −6β# . ∞ Thus 2(q−1) (1) A ≤ Cβ,q r −6β# 1

#=0

∞ (2j r)−1/2

−2q−#−1

2j ru

u# du + O r −6(2q−1)β (2j r)−q

−q . ≤ Cβ,q r −6(2q−1)β 2j r Since Nβ ≤ Cβ r −2β , we finally obtain (4.14) with a0 = 12. Proof of (2.7). Fix some γ with 0 < (1 + 2γ )(1 + a0 β) ≤ α. Let ρ be a smooth nonnegative function on the line so that supp(ρ) ⊂ J . Fix any q > 1+2γ . In view of (4.2), the definition of νλ , and Lemma 4.6, (4.18) ∞ νλ 22,γ ρ(λ) dλ −∞

≤

∞

22j γ

j =−∞ ∞

' ' j =−∞

≤ Cβ,γ

≤ Cβ,γ

' ' −∞

as claimed.

'

−∞

ψ2−j ∗ νλ (x) dνλ (x)ρ(λ) dλ

$

# 2j (1+2γ ) ψ 2j ((λ, ω1 ) − ((λ, ω2 ) ρ(λ) dλ dµ(ω1 ) dµ(ω2 )

∞

∞

−q 2j (1+2γ ) 1 + 2j d(ω1 , ω2 )1+a0 β dµ(ω1 ) dµ(ω2 )

dµ(ω1 ) dµ(ω2 ) ≤ Cβ,γ Ᏹα (µ) < ∞, (1+a0 β)(1+2γ ) d(ω 1 , ω2 ) '

216

PERES AND SCHLAG

4.2. The case of limited regularity. We now discuss the case where ( : J ×' → R only has a finite degree of smoothness as a mapping λ ( → (λ . Definition 4.7. Let ' and J be as in Definition 2.4. Suppose ( : J × ' → R is continuous, and let L be a positive integer and δ ∈ [0, 1). We write (λ ∈ C L,δ (J ) if the following conditions are satisfied: given any compact I ⊂ J , we assume that the bounds (2.2) hold for all # = 0, 1, . . . , L and that (4.19) L L d d δ dλL ((λ1 , ω) − dλL ((λ2 , ω) ≤ Cδ,I |λ1 − λ2 |

for all λ1 , λ2 ∈ I and ω ∈ '

with some suitable constant CI,δ . The following definition is similar to Definition 2.7, only here we also allow for the slightly more general case of Hölder continuity. Definition 4.8. Suppose (λ ∈ C L,δ (J ) as in Definition 4.7 and let β ∈ (0, 1). We say that J is an interval of transversality of order β for ( if there exists a constant Cβ so that for all λ1 , λ2 ∈ J and ω1 , ω2 ∈ ', the condition 3λ (ω1 , ω2 ) + 3λ (ω1 , ω2 ) ≤ Cβ d(ω1 , ω2 )β 2 1 implies that (4.20)

d 3λ ≥ Cβ d(ω1 , ω2 )β . dλ 1

In addition, we say that (λ is L, δ-regular on J if under the same condition, with some constants Cβ,# , Cβ,L,δ , (4.21) # d −β# for # = 1, 2, . . . , L, dλ# 3λ1 (ω1 , ω2 ) ≤ Cβ,# d(ω1 , ω2 ) L d dL δ −β(L+δ) . dλL 3λ1 (ω1 , ω2 ) − dλL 3λ2 (ω1 , ω2 ) ≤ Cβ,L,δ |λ1 − λ2 | d(ω1 , ω2 ) Under these conditions, we have the following analogue of Theorem 2.8. Since Lemma 4.3 and Proposition 4.4 only require that (λ is 1-regular on J , we restrict ourselves to a discussion of the Sobolev estimate (2.7) and the dimension bound (2.8) from Theorem 2.8. Notice that the following statements reduce to those estimates if L = ∞. Theorem 4.9. Let ( : J × ' → R be as in Definition 4.7. Assume that J is an interval of transversality of order β for some β ∈ [0, 1) in the sense of Definition 4.8,

217


and assume that (λ is L, δ-regular on J with L + δ > 1. Let µ be a finite positive measure on ' with finite α-energy for some α > 1. Then on any compact I ⊂ J , the family of measures νλ = µ ◦ (−1 λ satisfies (4.22) νλ 22,γ dλ ≤ Cγ Ᏹα (µ), I

provided 0 < 1 + 2γ < L + δ and 1 + 2γ ≤

α , 1 + a0 β

where a0 is some absolute constant. Moreover, if σ ∈ (0, 1], then (4.23)

dim λ ∈ J : dims (νλ ) ≤ σ ≤ 1 + σ − min L + δ,

α . 1 + a0 β

If 1 < σ ≤ α, then (4.24)

dim λ ∈ J : dims (νλ ) ≤ σ ≤ 1 − min L + δ,

σ − 1 −1 α −σ 1+ . 1 + a0 β L+δ

Finally, if σ ∈ (0, α − 3β], then dim λ ∈ J : dims (νλ ) < σ ≤ σ. As already mentioned above, the final statement holds since Proposition 4.4 requires only C 1 parameterizations. As in the C ∞ -case, we exploit the Sobolev bound in order to obtain the dimension estimates (4.23) and (4.24). This is accomplished by means of the following proposition, which is based on Lemma 3.2 (cf. Proposition 4.2). Proposition 4.10. Let ', J and ( : J × ' → R be as in Definition 4.7 for some positive integer L and δ ∈ [0, 1). Suppose νλ as in 4.7 satisfies I νλ 22,γ dλ < ∞ with some I ⊂ J and γ > − 21 . (i) If σ ∈ [0 ∧ 2γ , 1 ∧ (1 + 2γ )], then

dim λ ∈ I : dims (νλ ) ≤ σ ≤ σ − 2γ .

(4.25)

(ii) If σ ∈ [1, 1 + 2γ ), then (4.26)

σ − 1 −1 . dim λ ∈ I : dims (νλ ) ≤ σ ≤ 1 − (1 + 2γ − σ ) 1 + L+δ

Proof. Let {hj }∞ j =1 be defined as in (4.3) and set h0 = 1. We verify the hypotheses j of Lemma 3.2 with A = 2 and Hj = i=0 2i−j hi . Recall that the Littlewood-Paley ˆ ) = φ(ξ/2) ˆ ˆ ), where φˆ ≥ 0, φ(ξ ˆ ) = 1 for |ξ | ≤ 1, function ψ is given by ψ(ξ − φ(ξ

218

PERES AND SCHLAG

ˆ ) = 0 for |ξ | > 2; see the proof of Lemma 4.1. In particular, and φ(ξ ∞ $ # −j −1 j ˆ ) νλ (ξ )2 dξ 2 Hj (λ) = 1 + ξ − φ(ξ φˆ 2 −∞ $ j +1 j +1 # (4.27) = 1+ (λ (ω) − (λ (τ ) 2 φ 2 ' ' # $ − φ (λ (ω) − (λ (τ ) dµ(ω) dµ(τ ). ∞ Let χ ∈ ᏿(R) satisfy |φ | ≤ χ. (For example, set χ(x) = C −∞ max|y−z|≤1 |φ (z)| η(x − y) dy, where η ∈ C ∞ is a standard bump function.) Thus # $ H (λ) ≤ C + C2j +1 χ 2j +1 (λ (ω) − (λ (τ ) j ' ' × (λ (ω)| + |(λ (τ ) dµ(ω) dµ(τ ) (4.28) ∞ 2

≤ C +C χˆ 2−j −1 ξ νλ (ξ ) dξ ≤ C2j Hj (λ). −∞

To obtain the final inequality in (4.28), notice that

ˆ ∞ φˆ 2−j −1 ξ χˆ 2−j −1 ξ ≤ χ

if |ξ | ≤ 2j +1 .

Hence, (4.28) follows from (4.27) and the rapid decay of χ. ˆ In view of (4.4) and (4.5), inequality (4.25) follows from (3.8) in Lemma 3.2 with n = 1, A = B = 2, R = 21+2γ , and r = 2σ + as → 0+. Estimate (4.26) follows from Lemma 3.2(ii) with the same choice of parameters. It remains to verify the Hölder estimate (3.9). For simplicity we only carry this out for L = 1. The general case is only slightly more technically involved. Let (λ (ω, τ ) = (λ (ω) − (λ (τ ). Hj , as defined above, clearly satisfies (4.29) H (λ1 ) − H (λ2 ) j j j +1

j +1 φ 2 (λ (ω, τ ) ( (ω, τ ) − ( (ω, τ ) dµ(ω) dµ(τ ) ≤ C +2 λ1 λ2 1 ' '

(4.30) +2

j +1

' '

j +1

φ 2 (λ (ω, τ ) − φ 2j +1 (λ (ω, τ ) ( (ω, τ ) dµ(ω) dµ(τ ). λ2 1 2

In view of (4.19), the same argument as in the first part of the proof shows that the right-hand side of (4.29) is bounded by C2j Hj (λ1 )|λ1 − λ2 |δ . If |λ1 − λ2 | > 2−j , then the integral in (4.30) can be again bounded by

C2j Hj (λ1 ) + Hj (λ2 ) ≤ C2j (1+δ) |λ1 − λ2 |δ Hj (λ1 ) + Hj (λ2 ) .


219

It remains to estimate (4.30) if |λ1 −λ2 | ≤ 2−j . As above, one constructs χ1 ∈ ᏿ such that |φ | ≤ χ1 and Cχ1 (x) ≥ max|x−y|≤1 χ1 (y) for all x ∈ R. The integral in (4.30) is therefore bounded by 1 j +1

2j φ 2 (λ (ω, τ ) + 2j +1 t (λ − (λ (ω, τ ) dt C2 1 2 1 ' ' 0

× (λ2 − (λ1 (ω, τ ) dµ(ω) dµ(τ )

χ1 2j +1 (λ1 (ω, τ ) dµ(ω) dµ(τ ) ≤ 22j |λ2 − λ1 |

(4.31)

= C2j |λ2 − λ1 |

' ' ∞

−∞

2

χ1 2−j −1 ξ νλ (ξ ) dξ

≤ C2 |λ2 − λ1 |Hj (λ1 ) ≤ C2j (1+δ) |λ2 − λ1 |δ Hj (λ1 ). 2j

To pass to (4.31), one uses 2j |(λ1 (ω, τ ) − (λ2 (ω, τ )| ≤ C2j |λ2 − λ1 | ≤ C and the properties of χ1 . Proof of Theorem 4.9. By Proposition 4.10, the dimension bounds (4.23) and (4.24) follow from (4.25) and (4.26), respectively, if (4.22) holds. (4.18) shows that this is the case as soon as the inequality (4.14) is valid for some (real) q > 1 + 2γ . In fact, inspection of the proof of Lemma 4.6 reveals that under the conditions of Definition 4.8, one has (4.14) for any q < L + δ. More precisely, the only term in the proof of Lemma 4.6 that does not involve the rapid decay of ψ is the O-part of (1) A1 (see (4.17)). Thus the proof carries over unchanged up to (4.16). Now fix a small

> 0 and some M > 0. Since F ∈ C L−1,δ , as can be seen from the definition of F , Definition 4.7, and Lemma 4.5, (4.32) A(1) =

F (u)ψ 2j ru du

=

|u|(2j r)−1+

(1)

Taking M = M( , q) large enough, one obtains |A2 | ≤ Cβ,q (2j r)−q r −β for any q > 0. Estimating the contribution of the Taylor polynomial in (4.32) again involves the rapid decay of ψ. To bound the error term in (4.32), one uses the Hölder condition on F (L−1) :

220

PERES AND SCHLAG

|u| 0. We say that J ⊂ R is an interval of δ-transversality for the class of power series (5.1)

g(x) = 1 +

∞

bn x n ,

with bn ∈ {−1, 0, 1}

n=1

if g(x) < δ implies g (x) < −δ for any x ∈ J . Lemma 5.3 establishes the connection between Definitions 2.7 and 5.1. A useful criterion for checking δ-transversality was found in [34]. A power series h(x) is called a (∗)-function if for some k ≥ 1 and ak ∈ [−1, 1], h(x) = 1 −

k−1

x i + ak x k +

∞

xi .

i=k+1

i=1

In [39] Solomyak showed that among all convex combinations of series of the form (5.1), the power series with the smallest double zero must be a (∗)-function. The following lemma from [34] bypasses this fact and reduces the search for intervals of transversality to finding a suitable (∗)-function. Lemma 5.2. Suppose that a (∗)-function h satisfies h(x0 ) > δ

and

h (x0 ) < −δ

for some x0 ∈ (0, 1) and δ ∈ (0, 1). Then [0, x0 ] is an interval of δ-transversality for the class of power series (5.1). In [34] a particular (∗)-function was found that satisfies h(2−2/3 ) > 0.07 and h(2−2/3 ) < −0.09, so transversality in the sense of Definition 5.1 holds on [0, 2−2/3 ] by this lemma. On the other hand, in [39] Solomyak proved that there is a power

221


series of the form (5.1) with a double zero at roughly 0.68, whereas 2−2/3 0.63. We return to this issue below. It is easy to see that Bernoulli convolutions are a special case of the general results in Section let ' = {−1, +1}N be equipped with the product measure ∞ 1 2. Indeed, 1 µ = 0 ( 2 δ−1 + 2 δ1 ). For any distinct ω, τ ∈ ', we define |ω ∧ τ | = min{i ≥ 0 : ωi = τi }. Fix some interval J = (λ0 , λ1 ) ⊂ (0, 1) and define ( : J × ' → R via (λ (ω) = ∞ |ω∧τ | n . One n=0 ωn λ . The metric on ' (depending on J ) is given by d(ω, τ ) = λ1 1 α checks that Ᏹα (µ) < ∞ if and only if λ1 > 2 . Lemma 5.3. Suppose J = (λ0 , λ1 ) is an interval of δ-transversality. Then J is an 1+β interval of transversality of order β for ( if λ0 > λ1 . |ω ∧ω |

Proof. Since d(ω1 , ω2 ) = λ1 1 2 , 3λ (ω1 , ω2 ) = 2(λk /λk1 )g(λ) with k = |ω1 ∧ ω2 | and a power series g of the form (5.1). Therefore, " ! k−# k (#) λ λ # (#) 3 ≤ C# k g ∞ + k g ∞ λ λk1 λ1

−βlk −1 −#−1 ≤ C#,β λ1 , ≤ C# k # λ−# 0 (1 − λ1 ) + (1 − λ1 ) since λ1 < 1 and β > 0. Thus condition (2.6) in Definition 2.7 holds. To check (2.5), βk assume |3λ | ≤ δbβ λ1 , where the constant bβ ∈ (0, 1) is determined below. Then 2

λ0 λ1

k

implies |g(λ)| ≤ 21 δbβ (λ−1 0 λ1

|g(λ)| ≤ 2

1+β k ) .

λ λ1

k

βk

|g(λ)| ≤ δbβ λ1

Hence

3 ≥ 2 λ0 λ−1 k |g (λ)| − k |g(λ)| ≥ 2λβk δ − δbβ k λ−1 λ1+β k ≥ δλβk , λ 1 1 1 λ0 2λ0 0 1 provided bβ ≤ 21 [1+supk≥0 k/2λ0 (λ−1 0 λ1 2.7 holds with Cβ = δbβ .

1+β k −1 ) ] .

Thus condition (2.5) in Definition

Theorem 2.8 now implies the following theorem. Theorem 5.4. Suppose J = [λ0 , λ0 ] ⊂ ( 21 , 1) is an interval of δ-transversality for the power series (5.1). Then dims (νλ ) ≥ log 2/(− log λ) for a.e. λ ∈ J . Furthermore, log 2 dim λ ∈ J : νλ ∈ L2 (R) ≤ 2 − . − log λ0

222

PERES AND SCHLAG

Proof. Fix any small β > 0, and partition J into subintervals Ji = [λi , λi+1 ] for 1+β i = 0, . . . , m so that λi ≥ (1 + β)λi+1 . By the previous lemma, all Ji are intervals of transversality of order β for (. Notice that the metric on ' depends on i; in |ω1 ∧ω2 | . In particular, µ has finite αi -energy with respect to di if fact, di (ω1 , ω2 ) = λi+1 i > 21 . Theorem 2.8 therefore implies that λαi+1 dims (νλ ) ≥

log 2 (1 + a0 β)−1 − log λi+1

for a.e. λ ∈ Ji

and dim λ ∈ Ji : νλ ∈ L2 (R) ≤ 2 −

log 2 log 2 (1 + a0 β)−1 . (1 + a0 β)−1 ≤ 2 − − log λi+1 − log λ0

Letting β → 0+ finishes the proof. It is well known that for 0 < λ < 21 , the support of νλ is a Cantor set of dimension log 2/(− log λ). In fact, νλ is a Frostman measure on that set, which implies that dims (νλ ) = log 2/(− log λ) for 0 < λ < 21 . In [39] Solomyak showed that the first double zero for a power series of the form (5.1) lies in the interval [0.649, 0.683]. In particular, the previous theorem applies only up to some point in this interval. It seems natural to conjecture that dims (νλ ) ≥ log 2/(− log λ) for a.e. λ ∈ ( 21 , 1), but our methods do not yield this estimate. Nevertheless, one can show that νλ has some smoothness for a.e. λ ∈ ( 21 , 1). This follows from Theorem 5.4 by “thinning and convolving” (see [39] and [34]). As one expects, the number of derivatives tends to ∞ as λ → 1. Lemma 5.5. For any > 0, there exists γ = γ ( ) > 0 so that 1/√2 (5.2) νλ 22,γ dλ < ∞. (1/2)+

Furthermore, there exists some positive constant γ0 so that (5.3)

2−2

2−2

−k−1

−k

−k+1 νλ 22,2k γ + νλ 22,γ0 dλ < ∞ 0

for k = 1, 2, . . . .

Proof. As mentioned above, [0, λ1 ] is an interval of transversality for the power series (5.1) for some λ1 > 2−2/3 . Fix any λ0 ∈ ( 21 , 2−2/3 ]. Partitioning the interval [λ0 , λ1 ] as in the proof of Theorem 5.4, one obtains from (2.7) that λ1 1 1+2γ (5.4) νλ 22,γ dλ < ∞, provided λ0 > . 2 λ0 , we remove every third term from the original series. More preTo go beyond 2−2/3 / cisely, let (λ (ω) = 3 | n ωn λn and denote the distribution of this series by ν0λ . It was

223


shown in [39] and [34] that the class of power series (5.1) that satisfy either b3j +1 = 0 for all j ≥ 0 or √ b3j +2 for all j ≥ 0 have [0, λ3 ] as an interval of δ-transversality for some λ3 > 1/ 2. As for the full series, one easily deduces from Theorem 2.8 that λ3 1+2γ (5.5) > 2−2/3 . 0 νλ 22,γ dλ < ∞, provided λ2 λ2

0λ | and λ1 > 2−2/3 , (5.2) follows from (5.4) and (5.5). Moreover, Since | νλ | ≤ |ν we have shown that there exists some #0 ∈ (2−1/2 , 2−1/4 ) and a γ0 > 0 so that #0 νλ 22,γ0 dλ < ∞. Using νλ (ξ ) = ν* λ2 (ξ )ν* λ2 (λξ ), we conclude that #2 0

#0

1/2

#0

νλ 2,2γ0 dλ =

1/2

#0

≤C

#0

≤C

(5.7)

λ

1/2 dλ

λ

νλ (ξ )4 |ξ |4γ0 dξ

#20

#0

ν*2 (ξ )ν*2 (λξ )2 |ξ |4γ0 dξ

∞

−∞

#0

(5.6)

1/2 dλ

νλ (ξ )2 |ξ |2γ0 dξ dλ < ∞.

#20

Equation (5.6) follows from Cauchy-Schwarz and a change of variables. To pass from (5.6) to (5.7), we basically observe that the inner integral in (5.6) behaves like a sum over ξ ∈ Z. More precisely, we have νλ = νλ χ for some χ ∈ C ∞ (R) with compact support. Consequently, ∞ ∞ νλ (ξ )4 |ξ |4γ0 dξ = νλ ∗ χˆ 4 (ξ )|ξ |4γ0 dξ −∞

= = =

−∞ ∞ ∞ −∞

−∞ #+1 ∞

#∈Z #

1 0 #∈Z

≤

νλ (ξ − η)χ(η) ˆ dη

−∞ ∞ −∞

1 ! 0

#∈Z

≤C

0 ∞ −∞

|ξ |4γ0 dξ

νλ (ξ − η)χ(η) ˆ dη

4

νλ (ξ + # − η)χ(η) ˆ dη

∞ −∞

|ξ |4γ0 dξ 4

νλ (ξ + # − η)χ(η) ˆ dη

! 2 1 ≤χˆ 1 L

4

∞

#∈Z −∞

|ξ + #|4γ0 dξ "2

2 |ξ + #|2γ0

dξ

"2 2 2γ νλ (η) χ(ξ ˆ + # − η)|ξ + #| 0 dη dξ

νλ (η)2 (1 + |η|)2γ0 dη

2 < ∞.

224

PERES AND SCHLAG

Finally, since | νλ (ξ )| ≤ 1,

#0

1/2

#0

νλ 22,γ0 dλ =

1/2

#0 #0

≤C

ν*2 (ξ )ν*2 (λξ )2 |ξ |2γ0 dξ dλ λ

#0 #20

λ

νλ (ξ )2 |ξ |2γ0 dξ dλ < ∞.

We have shown that (5.3) holds for k = 1. The general case now follows easily by induction along the same lines. Corollary 5.6. Let S(λ0 ) = essinf λ∈[λ0 ,1] dims (νλ ). Then S(λ0 ) > 1 for any λ0 > 21 and S(λ0 ) → ∞ as λ0 → 1. Furthermore, for any λ0 > 21 , there exists

(λ0 ) > 0 such that dim λ ∈ (λ0 , 1) : νλ does not have L2 -density < 1 − (λ0 ). Proof. This follows immediately from the previous lemma and Proposition 4.2. As observed by Kahane [22], Erd˝os’s argument yields that (λ0 ) → 1 as λ0 → 1−. To conclude the discussion of classical Bernoulli convolutions, we prove that (2.3) fails. Lemma 5.7. Suppose 1/λ0 is a Pisot number. Then 2 lim sup |ξ | νλ (ξ ) dλ > 0 |ξ |→∞

J

for any interval J containing λ0 . Proof. Recall that lim sup|ξ |→∞ |ν* λ0 (ξ )| > 0; see [6]. Also, iξ ( (ω) e λ − eiξ (λ0 (ω) dµ(ω) νλ (ξ ) − ν* λ0 (ξ ) ≤ ' ≤ |ξ | (λ (ω) − (λ0 (ω) dµ(ω) ≤ C|ξ ||λ − λ0 |. '

Therefore, if C is a sufficiently large constant, νλ (ξ )2 dλ > 0 lim sup |ξ | |ξ |→∞

|λ−λ0 |≤(C|ξ |)−1

as claimed. 5.2. Asymmetric Bernoulli convolutions. In [35] Peres and Solomyak studied n p the distribution νλ of the sum ±λ where the signs are chosen independently p with probabilities (p, 1 − p). For p ∈ [ 13 , 23 ], they showed that νλ is absolutely

225


continuous for a.e. λ ∈ [p p (1 − p)1−p , 1) and has Lq -density for a.e. λ ∈ [(p q + (1 − p)q )1/(q−1) , 1), where q ∈ (1, 2]. The results from Section 2 give bounds on the dimension of the exceptional parameters for the absolutely continuous case and if q = 2. As in the symmetric case, ' = {−1, 1}N . Here µ is the product measure that assigns weights p and 1 − p to +1 and −1, respectively. Theorem 5.8. Let p ∈ [ 13 , 23 ]. For all λ0 ∈ (p 2 + (1 − p)2 , 1], there exists =

(λ0 ) such that p dim λ ∈ (λ0 , 1) : νλ does not have L2 -density < 1 − (λ0 ) (5.8) and such that for all λ0 ∈ (p p (1 − p)1−p , 1] p dim λ ∈ (λ0 , 1) : νλ is singular < 1 − (λ0 ). (5.9) Proof. Since [0, 0.649] is an interval of transversality for the power series (5.1), Theorem 2.8 implies, via Lemma 5.3 and a subdivision of the interval as in the proof of Theorem 5.4, that p dim λ ∈ (λ0 , 0.649) : νλ does not have L2 -density < 1 for any λ0 > p2 + (1 − p)2 . To extend this up to 1, we follow [35]. The measure p p n νλ ∗ νλ is the distribution of the random series ∞ 0 an λ , where an ∈ {−2, 0, 2} 2 2 with probabilities p , 2p(1 − p), (1 − p) . By Lemma 5.2 and an explicit choice of (∗)-function, Peres and Solomyak showed that [0, 0.5] is an interval of transversality for the power series (5.1) with bn ∈ {−4, −2, 0, 2, 4}. Since (0.649)2 > 11/27 = maxp∈[1/3,2/3] (p 4 +(2p(1−p))2 +(1−p)4 ), Theorem 2.8 yields in the same fashion as before that 1 p p 2 dim λ ∈ 0.649, √ : νλ ∗ νλ ∈ L (R) 2

p = dim λ ∈ (0.649)2 , 0.5 : ν ∈ L4 (R) < 1. Splitting the random power series

λ

±λn into odd and even indices, one obtains

p p p (ξ )ν* (λξ ). νλ (ξ ) = ν* λ2 λ2

(5.10) Therefore,

1 dim λ ∈ 0.649, √ 2

p 2 : νλ ∈ L (R) < 1.

p p p n The measure νλ ∗ νλ ∗ νλ is the distribution of the random series ∞ 0 an λ , where an ∈ {−3, −1, 1, 3} with probabilities p 3 , 3p 2 (1 − p), 3p(1 − p)2 , (1 − p)3 . It was shown in [35] that [0, 0.415] is an interval of transversality for the corresponding

226

PERES AND SCHLAG

power series. Since 2−3/2 > 245/729 = maxp∈[1/3,2/3] (p 6 +(3p2 (1−p))2 +(3p(1− p)2 )2 + (1 − p)6 ), Theorem 2.8 again implies that

p dim λ ∈ 2−3/2 , 0.415 : νλ ∈ L6 (R) < 1. Splitting the original series modulo 3, one concludes that 1 p 1/3 2 : νλ ∈ L (R) < 1. dim λ ∈ √ , (0.415) 2 By this procedure we have estimated the dimension of λ inside the interval [λ0 , (0.415)1/3 ] for which νλ does not have L2 -density. Since maxp∈[1/3,2/3] (p 2 + (1 − √ p)2 ) = 59 and (0.415)1/3 > 5/9, inequality (5.8) follows from this estimate by applying (5.10) repeatedly. By the strong law of large numbers, the lower pointwise dimension of µ is

πµ− (ω) = − p log p + (1 − p) log(1 − p) µ-a.e. It therefore follows immediately from Corollary 2.9 that p dim λ ∈ (λ0 , 0.649) : νλ is singular < 1 for any λ0 > exp[−(p log p + (1 − p) log(1 − p))]. Since (5.8) is a stronger assertion on the interval [0.649, 1], we finally obtain (5.9). 5.3. {0, 1, 3}-problem. This problem concerns the Hausdorff dimension of the set ∞ ai λi : ai ∈ {0, 1, 3} . (λ) = i=0

Pollicott and Simon [36] showed that for a.e. λ ∈ ( 41 , 13 ), we have dim (λ) = log 3/(− log λ), and Solomyak [39] established that for a.e. λ ∈ ( 13 , 25 ), the set (λ) has positive Lebesgue measure. In this section, we estimate the Hausdorff dimension of the exceptional set of λ with respect to these properties. It is again clear that the general results in Section 2 apply to this case. Indeed, let ' = {0, 1, 3}N be equipped with uniform product measure. For any interval J = (λ0 , λ1 ) ⊂ (0, 1), define the projections ( and the metric d on ' as in Section 5.1. In [39] Solomyak showed that the smallest double zero of the class of power series (5.11)

g(x) = a +

∞ n=1

b n λn

with |bn | ≤ 1, |a| ≥

1 3

occurs in the interval [0.418, 0.437]; see also Corollary 5.2 in [35]. In particular, a simple compactness argument shows that [0, 25 ] is an interval of δ-transversality for the class (5.11) for some δ > 0 in the sense that |g(λ)| < δ implies |g (λ)| > δ for any λ in that interval. As in Lemma 5.3, we establish the connection with Definition 2.7.


227

Lemma 5.9. Suppose J = (λ0 , λ1 ) is an interval of δ-transversality for the class of power series (5.11). Then J is an interval of transversality of order β for (, provided 1+β λ0 > λ1 . |ω1 ∧ω2 | g(λ), where g is of Proof. It suffices to notice that 3λ (ω1 , ω2 ) = 3(λλ−1 1 ) the form (5.11). Otherwise the proof is identical with that of Lemma 5.3.

Theorem 2.8 now easily leads to the following result. Let M(λ) = log 3/(− log λ). Theorem 5.10. For any λ0 ∈ (0.25, 0.4) (5.12) (5.13)

dim λ ∈ (0.25, λ0 ) : dim (λ) < M(λ) ≤ M(λ0 ), dim λ ∈ (λ0 , 0.4) : Ᏼ1 ( (λ)) = 0 ≤ 2 − M(λ0 ).

Proof. Suppose λ0 ∈ ( 41 , 13 ). Fix any β ∈ (0, 1). As in the proof of Theorem 5.4 & 1+β we write [ 41 , λ0 ] = m ≤ λi+1 . Applying i=0 Ji with Ji = [λi+1 , λi ] and (1 + β)λi Lemma 5.9 to each Ji , one concludes from Theorem 2.8 that dim λ ∈ Ji : dims (νλ ) < M(λi ) − 3β ≤ M(λi ) ≤ M(λ0 ). Equation (5.12) follows by letting β → 0+. If λ0 ∈ ( 13 , 25 ), one partitions [λ0 , 25 ] in the same fashion. Theorem 2.8 then implies dim{λ ∈ Ji : νλ is singular} ≤ 2 −

M(λ0 ) M(λi+1 ) ≤ 2− . 1 + a0 β 1 + a0 β

Equation (5.13) now follows again by letting β → 0+. 5.4. Sums of Cantor sets. Following [35] we consider homogeneous self-similar sets in R ∞ n Ꮿλ = sn (λ)λ : sn ∈ Ᏸ, λ ∈ J , n=0

where Ᏸ = {s1 , s2 , . . . , sm } is a set of C 1 (J )-functions and J = [λ0 , λ1 ] ⊂ (0, 1). As in [35], we assume the strong separation condition (5.14)

min

min dist si (λ) + λᏯλ , sj (λ) + λᏯλ > h0 > 0.

1≤i<j ≤m λ0 ≤λ≤λ1

This condition implies dim Ꮿλ = log m/(− log λ) < 1, so λ < m−1 . The usual middleα Cantor sets are clearly special cases of the Ꮿλ , and we refer the reader to [35] for more motivation and background. In this section, we are concerned with the problem of estimating the dimension of the sum of two Cantor sets, or more generally, of

228

PERES AND SCHLAG

the sum of any compact set with one of the Ꮿλ . For technical reasons, we split Ꮿλ into the disjoint union of its cylinder sets of a certain fixed length. More precisely, let M0 = maxi=j si − sj L∞ (J ) and M1 = maxi=j si − sj L∞ (J ) and define k0 = *2(M0 + M1 )λ1 / h0 (1 − λ1 )2 +. Then every Ꮿλ is the union of mk0 congruent subsets that are obtained by fixing the first k0 symbols sn . Slightly abusing notation, we use Ꮿλ to denote one of these subsets. Since we are only concerned about questions relating to dimension, this makes no difference. To see that the case at hand is again a special case of the results in Section 2, fix a compact set K ⊂ R. Define ' = K × {1, 2, . . . , m}N and denote a typical point of ' n by (x, ω). Let (λ (x, ω) = x + ∞ n=0 sωn (λ)λ be the projections ( : J ×' → R. Let ρ be a Frostman measure on K. Define the measure µ on ' to be µ = ρ ×µ0 , where µ0 is the uniform product measure on the sequence space {1, 2, . . . , m}N . Finally, we let

|ω∧τ | d (x, ω), (y, τ ) = |x − y| + λ1 be the metric on '. By definition, ( : J × ' → R is continuous and C 1 in λ. In the following lemma, we show that the conditions in Definition 2.7 are satisfied. Lemma 5.11. Suppose s1 , s2 , . . . , sm ∈ C L,δ (J ) for some positive integer L and some δ ∈ [0, 1). Then J = [λ0 , λ1 ] is an interval of transversality of order β for ( 1+β (in the sense of Definition 4.8), provided λ1 > λ0 . Proof. Fix some (x, ω) and (y, τ ) ∈ '. Let k = |ω ∧ τ | and r = d((x, ω), (y, τ )). Notice that k > k0 by our assumption above. Then

x − y + λk g(λ) 3λ (x, ω), (y, τ ) = , |x − y| + λk1 where |g(λ)| > h0 by (5.14). Now suppose

β |3λ | ≤ Cβ |x − y| + λk1 = Cβ r β , (5.15) where the constant Cβ is to be determined. In fact, Cβ can be chosen so that |x − y| ≤ C1 λk1 with some constant C1 that depends only on Ᏸ and λ1 . More precisely, suppose that |x − y| ≥

2λk1 2λk1 max di − dj L∞ (J ) = M0 . 1 − λ1 i=j 1 − λ1

Then |3λ | ≥

|x − y| − λk1 M0 (1 − λ1 )−1 |x − y| + λk1

≥ C,

which contradicts (5.15) if Cβ is sufficiently small. Consequently, r λk1 and |3λ | ≤ kβ −βk (#) Cβ λ1 . It is then easy to see that |3λ | ≤ Cβ,# λ1 for suitable constants Cβ,# and (L) all # = 1, 2, . . . , L and also that a Hölder condition of order δ on 3λ holds if δ > 0.


229

Hence, (4.21) in Definition 4.8 is satisfied. Moreover, λk k

3 = g(λ) + g (λ) ≥ C λ0 λ−1 k kh0 − M0 + M1 ≥ Cλkβ ≥ Cβ r β λ 1 1 2 r λ λ1 (1 − λ1 ) 1+β

if Cβ is sufficiently small, since λ0 > λ1

and k > k0 . Thus (4.20) also holds.

Under the strong separation assumption (5.14), Peres and Solomyak showed in [35] that dim(K + Ꮿλ ) = dim K + dim Ꮿλ for a.e. λ such that dim K + dim Ꮿλ < 1. Furthermore, they also showed that K + Ꮿλ has positive Lebesgue measure for a.e. λ such that dim K + dim Ꮿλ > 1. The following theorem estimates the dimension of the exceptional values of λ in these statements. This is a simple consequence of Theorem 4.9. Theorem 5.12. Suppose K ⊂ R is compact and the sets Ꮿλ satisfy the strong separation condition (5.14) on J = [λ0 , λ0 ] ⊂ (0, 1). Then (5.16) dim λ ∈ J : dim(K + Ꮿλ ) < dim K + dim Ꮿλ ≤ dim K + dim Ꮿλ0 . If the symbols s1 , . . . , sm are in C L,δ with L + δ > 1, then dim λ ∈ J : Ᏼ1 (K + Ꮿλ ) = 0 ≤ 2 − min(dim K + dim Ꮿλ0 , L + δ). (5.17) Proof. Fix some > 0 and let ρ be a Frostman measure on K with exponent dim K − . Let the measure µ on ' be defined as above. Fix β ∈ (0, 1) and partition & 1+β J= N n=1 Ji , where each Ji = [λi , λi+1 ] satisfies λi ≥ (1 + β)λi+1 . It is easy to see that µ has finite αi -energy with respect to the metric di ((x, ω), (y, τ )) = |x − y| + |ω∧τ | λi+1 on ' if αi < dim K − + dim Ꮿλi+1 . Indeed, setting σi = dim Ꮿλi+1 , which is i the same as λσi+1 m = 1, one obtains dρ(x)dρ(y) dµ(ω) dµ(τ )

Ᏹαi (µ) = |ω∧τ | αi ' ' K K |x − y| + λi+1 ∞ dρ(x) dρ(y)

αi k ≤ m−k K K λi+1 + |x − y| k=0 dρ(x) dρ(y) ≤C < ∞, α −σ K K |x − y| i i provided 0 < αi − σi < dim K − . Therefore, by Lemma 5.11 and (2.9), dim λ ∈ Ji : dim(K + Ꮿλ ) < dim K − + σi − 3β ≤ dim λ ∈ Ji : dims (νλ ) < dim K − + σi − 3β ≤ dim K − + dim Ꮿλi+1 ≤ dim K − + dim Ꮿλ0 , and (5.16) follows by letting β → 0+ and then → 0+. The second statement (5.17) follows in a similar fashion from (4.23).

230

PERES AND SCHLAG

6. Orthogonal projections onto planes in Euclidean space. It is easy to see that Theorem 2.8 covers various classical results on the projection of planar sets onto lines (see [28], [23], [24], [9], [31]). Let ' = R2 and ((θ, x) = projθ (x) = (x ·θ)θ, where we consider J ⊂ S 1 as an interval of angles. Here projθ denotes the projection onto the line {tθ : t ∈ R}. Note that 3θ (x, y) = cos((θ, x − y)) for any x = y so that Definition 2.7 holds with β = 0. Suppose E ⊂ R2 is a Borel set. Applying Theorem 2.8 to a suitable Frostman measure supported on a compact subset of E, we therefore obtain Kaufman’s [24] and Falconer’s [9] theorems in the plane, that is, dim θ ∈ S 1 : dim projθ (E) ≤ t ≤ t and dim θ ∈ S 1 : Ᏼ1 (projθ (E)) = 0 ≤ 2 − dim E. Kaufman and Mattila [25] proved that the first bound is optimal. As noted by Falconer [9], their example can be easily modified to show that the second bound is sharp, too; see also Falconer [11, Theorem 8.17]. In particular, this implies that the estimates stated in Theorem 2.8 are optimal if β = 0, α ≤ 2, and σ ≤ 1. Moreover, it is easy to see that in this case (2.7) holds with β = 0 and equality; see also Proposition 2.2. It turns out that the calculation from the proof of that proposition can be generalized to higher dimensions. G(d, k) denotes the Grassmann manifold of all k-planes in Rd passing through the origin. Recall that dim G(d, k) = k(d −k); see [31]. For any finite measure µ on Rd , we define its projections νπ onto any k-plane π through the origin as usual, that is, given any continuous f ,

f dνπ = f projπ (x) dµ(x). Proposition 6.1. Let d ≥ 2 and k be positive integers. Suppose µ is a compactly supported measure in Rd . Then (6.1) dim π ∈ G(d, k) : dims (νπ ) < σ ≤ k(d − k) + σ − dims (µ). Proof. Let α = dims (µ). Suppose (6.1) fails. By Frostman’s lemma, there exists a nonzero positive measure ρ so that for some > 0,

ρ B(π, r) ≤ r k(d−k)+σ −α+ for any ball B(π, r) ⊂ G(d, k), (6.2)

ρ {π ∈ G(d, k) : dims (µπ ) ≥ σ } = 0. The measurability hypothesis in Frostman’s lemma can be verified as in Proposition 4.4, and we skip the details. We may assume that supp(µ) ⊂ B(0, 1). Fix a ˆ Clearly, φ ∈ ᏿ so that φ = 1 on B(0, 1). Hence, µ = φµ and thus µˆ = φˆ ∗ µ. 2 ˆ ˆ ∗ |µ| ˆ ˆ ≤ φ L1 |φ| ˆ 2 νπ (ξ ) = µ(proj π (ξ )) and supp(νπ ) = projπ (supp(µ)). Also, |µ| by Cauchy-Schwarz. Therefore,


(6.3)

G(d,k) π

231

νπ (ξ )2 (1 + |ξ |)σ −k d Ᏼk (ξ ) dρ(π)

≤C

G(d,k) π

(6.4)

≤ CN

(6.5)

≤ Cα,d

Rd

φˆ ∗ |µ| ˆ 2 (ξ )(1 + |ξ |)σ −k d Ᏼk (ξ ) dρ(π)

2 µ(η) ˆ

Rd

G(d,k) π

(1 + |ξ − η|)−N (1 + |ξ |)σ −k d Ᏼk (ξ ) dρ(π) dη

2 µ(η) ˆ (1 + |η|)α−d− dη < ∞.

Here N is a sufficiently large integer. To pass from (6.4) to (6.5), first notice that for any r < |η|, the set {π ∈ G(d, k) : dist(π, η) ≤ r} is an (r|η|−1 )-neighborhood of a smooth manifold of codimension d − k in G(d, k) and is therefore contained in the union of no more than (|η|r −1 )(k−1)(d−k) balls of radius r|η|−1 . Thus by (6.2),

ρ {π ∈ G(d, k) : dist(π, η) ≤ r} ≤ C(|η|r −1 )k−σ +α−d− . Considering the contributions from the dyadic shells {ξ ∈ π : 2j ≤ |ξ − η| ≤ 2j +1 } separately, the inner integrals in (6.4) can therefore be estimated as (1 + |ξ − η|)−N (1 + |ξ |)σ −k d Ᏼk (ξ ) dρ(π) G(d,k) π

≤ C(1 + |η|)σ −k ρ {π ∈ G(d, k) : dist(π, η) ≤ 1}

2−j (N −k) (1 + |η|)σ −k ρ π ∈ G(d, k) : dist(π, η) ≤ 2j +C 1≤ 2j 2k. Then for a.e. π ∈ G(d, k) the projection of E onto π has nonempty interior. More precisely, dim π ∈ G(d, k) : projπ (E) has empty interior ≤ k(d − k) + 2k − dim E. Proof. Let µ be a Frostman probability measure on E, that is, µ is supported on a compact subset of E and dims (µ) = dim E. If νπ = projπ (µ) has a continuous density, its support has nonempty interior and therefore so does projπ (E). Applying Cauchy-Schwarz to Rk |fˆ(ξ )| dξ shows that for any f ∈ L2 (Rk ), f 2,(1/2)(k+ ) < ∞ ,⇒ f ∈ C Rk ;

232

PERES AND SCHLAG

see Stein [42, Chapter 5] for more precise estimates. The corollary therefore follows by letting σ → 2k+ in (6.1). The following example, described to us by Mattila, shows that there are Borel sets E of dimension two in R3 so that dim{θ ∈ S 2 : projθ (E) contains an interval} ≤ 1. Take a Besicovitch set A in R2 , that is, a set of measure zero that contains a line in every direction; see [31, Theorem 18.11]. Define E = R2 \ ∪r∈Q2 (r + A) as a subset of the (x1 , x2 )-coordinate plane of R3 . It clearly has dimension two, but its projections onto lines that do not lie in the (x1 , x2 )-plane do not contain intervals. If k > 1, we do not know of an example of a 2k-dimensional set E with the property that γd,k ({π ∈ G(d, k) : projπ (E) has empty interior}) > 0, where γd,k denotes Haar measure on G(d, k). It seems unlikely that the Sobolev embedding theorem would give the sharp bound for k > 1. On the other hand, it is easy to see that Corollary 6.2 allows us to recover Falconer’s theorem [8] about the nonexistence of Besicovitch (d, k)-sets for k > d/2. Recall that a (d, k)-set is defined to be a set of measure zero that contains some translate of every k-plane. Suppose that A ⊂ Rd is such a set for some choice of d ≥ 3 and k. Then E = Rd \ ∪r∈Qd (A + r) has dimension d but the projection of E onto any (d − k)-dimensional plane has empty interior. This contradicts Corollary 6.2 if d > 2(d − k), as claimed. However, Bourgain [3] proved a much stronger result, namely, that (d, k)-sets do not exist for k + 2k−1 ≥ d. It therefore seems that the case k > 1 in Corollary 6.2 is not optimal. 7. The general projection theorems in higher dimensions. In this section we obtain a higher-dimensional version of Theorem 4.9. The proof closely resembles that of Theorem 4.9 in Section 4. In particular, we need to prove higher-dimensional versions of various technical lemmas. We first introduce some terminology. Definition 7.1. Let (', d) be a compact metric space, let Q ⊂ Rn be an open connected set, and let ( : Q × ' → Rm be a continuous map with n ≥ m. The length of a multi-index η = (η1 , η2 , . . . , ηn ) ∈ Nn is defined to be |η| = η1 + · · · + ηn and ∂ η = ∂ |η| /(∂λ1 )η1 · · · · · (∂λn )ηn where λ = (λ1 , . . . , λn ). Let L be a positive integer and δ ∈ [0, 1). We assume that for any compact Q ⊂ Q and any multi-index η = (η1 , η2 , . . . , ηn ) ∈ Nn , there exist constants Cη,Q and Cδ,Q such that η ∂ ((λ1 , ω) ≤ Cη,Q , provided |η| ≤ L, and sup ∂ η ((λ1 , ω) − ∂ η ((λ2 , ω) ≤ Cδ,Q |λ1 − λ2 |δ

|η|=L

for all λ1 , λ2 ∈ Q and ω ∈ '. We denote these conditions by (λ ∈ C L,δ (Q). Given any finite measure µ on ', let νλ = µ ◦ (−1 λ for all λ ∈ Q, where (λ (·) = ((λ, ·). The α-energy of µ is Ᏹα (µ) = ' ' dµ(ω1 )dµ(ω2 )/d(ω1 , ω2 )α .


233

As in the previous sections, we need a transversality condition. Definition 7.2. Let (λ ∈ C L,δ be as in Definition 7.1 for some positive integer L and some δ ∈ [0, 1). Let 3λ (ω1 , ω2 ) =

((λ, ω1 ) − ((λ, ω2 ) . d(ω1 , ω2 )

For any β ∈ [0, 1), we say that Q is a region of transversality of order β for ( if there exists a constant Cβ so that for all λ1 , λ2 ∈ Q and distinct ω1 , ω2 ∈ ', the condition |3λ1 (ω1 , ω2 )| + |3λ2 (ω1 , ω2 )| ≤ Cβ d(ω1 , ω2 )β implies that $ # (7.1) det D3λ1 (D3λ1 )t ≥ Cβ2 d(ω1 , ω2 )2β . In addition, we say that (λ is L, δ-regular on Q if under the same condition, for some constants Cβ,η , Cβ,L,δ , (7.2) |∂ η 3λ1 (ω1 , ω2 )| ≤ Cβ,η d(ω1 , ω2 )−β|η| for any nonzero multi-index η of length |η| ≤ L sup ∂ η 3λ1 (ω1 , ω2 ) − ∂ η 3λ2 (ω1 , ω2 ) ≤ Cβ,L,δ |λ1 − λ2 |δ d(ω1 , ω2 )−β(L+δ) .

|η|=L

D3λ above denotes the Jacobi matrix of all first derivatives of 3λ with respect to λ (the number of rows of D3λ is m). Our main result in this section is the following theorem. Theorem 7.3. Let (λ ∈ C L,δ be as in Definition 7.1 with L + δ > 1. Assume that Q ⊂ Rn is a region of transversality of order β for ( and that (λ is L, δ-regular on Q in the sense of Definition 7.2. Suppose µ is a finite positive measure on ' with m finite α-energy for some α > 0, and let νλ = µ ◦ (−1 λ be the projection of µ onto R under ( for any λ ∈ Q. Then for any compact Q ⊂ Q, (7.3) νλ 22,γ dλ ≤ Cγ Ᏹα (µ), Q

provided 0 < (m+2γ )(1+a0 β) ≤ α and 2γ < L+δ −1. Moreover, if σ ∈ (0, α ∧m], then α dim λ ∈ Q : dims (νλ ) ≤ σ ≤ n + σ − min (7.4) ,L+δ ; 1 + a0 β if σ ∈ (m, α], then (7.5)

dim{λ ∈ Q : dims (νλ ) ≤ σ } ≤ n − min

σ − m −1 α ,L+δ −σ 1+ . 1 + a0 β L+δ

234

PERES AND SCHLAG

Finally, if σ ∈ (0, α − a0 β], then (7.6)

dim{λ ∈ Q : dims (νλ ) < σ } ≤ n + σ − m.

The constant a0 depends only on m, n, and δ. It is easy to check that projections onto planes in Rd satisfy Definition 7.2 with Q ⊂ G(d, k) being a coordinate chart, (λ the Euclidean projection from ' = supp(µ) onto the plane given by λ, and L = ∞, β = 0. Hence, Proposition 6.1 follows from Theorem 7.3 with n = k(d − k) and m = k. If β = 0 and L = ∞, Proposition 2.2 shows that (7.3) is sharp. Remark 7.4. Equation (7.3) implies the following statement from the introduction: Let µ be a Borel probability measure on Rd with correlation dimension greater than m + 2γ . Then for a prevalent set of C 1 maps f : Rd → Rm , the image of µ under f has a density with at least γ fractional derivatives in L2 (Rm ). Let A = {f ∈ C 1 (Rd , Rm ) : µ ◦ f −1 have a density in L2,γ (Rm )}, and let ᏸ be Lebesgue measure on the matrices M d×m (R) with entries in [− 21 , 21 ] (we identify linear maps Rd → Rm with their matrices). We need to check that ᏸ(f + A) = 0 for any f ∈ C 1 (Rd , Rm ). Fix some f ∈ C 1 (Rd , Rm ). To apply Theorem 7.3, we may assume that supp(µ) = ' is compact. Let Q = [−1, 1]dm ⊂ M d×m (R) and (λ (ω) = −f (ω) + λω. Since Ᏹm+2γ (µ) < ∞ by assumption, one has νλ 22,γ d ᏸ(λ) < ∞, as desired, provided transversality holds with β = 0 and regularity holds with L = ∞. Regularity is obvious. To check transversality, notice that Dλ 3λ (ω, τ ) · λ0 = λ0 ·

ω−τ |ω − τ |

for all λ0 ∈ M d×m (R). In particular, the linear map λ0 ( → Dλ 3λ (ω, τ ) · λ0 is surjective and therefore rank[Dλ 3λ (ω, τ )] = m. In fact, estimate (7.1) holds with β = 0. The following proposition is a higher-dimensional analogue of Proposition 4.10. Proposition 7.5. Let (λ ∈ C L,δ (Q) be as in Definition 7.1 for some positive integer L and δ ∈ [0, 1). Suppose the family of measure {νλ }λ∈Q on Rm , as given by Definition 7.1, satisfies Q νλ 22,γ dλ < ∞ with some Q ⊂ Q and γ > −m/2. (i) If σ ∈ [0 ∧ 2γ , m ∧ (m + 2γ )], then dim λ ∈ Q : dims (νλ ) ≤ σ ≤ n + σ − (m + 2γ ). (7.7) (ii) If σ ∈ [m, m + 2γ ), then (7.8)

σ − m −1 . dim λ ∈ Q : dims (νλ ) ≤ σ ≤ n − (m + 2γ − σ ) 1 + L+δ


235

Proof. Let ψ be the Littlewood-Paley function from Lemma 4.1. For any j = 1, 2, . . . , we define 2 −j m hj (λ) = 2 ψ) 2−j (ξ ) νλ (ξ ) dξ =

' '

# $

ψ 2j ((λ, ω1 ) − ((λ, ω2 ) dµ(ω1 ) dµ(ω2 ),

j whereas h0 = 1. Let Hj = i=0 2m(j −i) hi . As in the proof of Proposition 4.10, we check that these functions satisfy the hypotheses of Lemma 3.2 with A = 2 and B = 2m . In fact, up to replacing 2 with 2m in various places, the argument remains the same and we omit the details. In view of (4.4) and (4.5), the proposition now follows from Lemma 3.1 with r = 2σ + and R = 22γ +m as → 0+. Roughly speaking, transversality means that 3λ can be inverted where it is small. However, one needs to interpret this statement more carefully in the higher-dimensional case. First, we are dealing with maps from Rn → Rm with n ≥ m. Second, even if m = n, we cannot expect to break up a region of transversality into disjoint cubes on which either 3λ is large or invertible as in the one-dimensional case; see Lemma 4.3. Third, any decomposition of Q has to provide estimates involving the parameters in Definition 7.2. The following quantitative version of the inverse function theorem is used for that purpose. It is undoubtedly well known, but we provide a proof for the sake of completeness. Lemma 7.6. Suppose f : U ⊂ Rm → Rm is C 1 and that Df (x) − I ≤ 21 on B(x0 , r) ⊂ U for some r > 0. Then f : B(x0 , r/3) → f (B(x0 , r/3)) is a diffeomorphism, and f (B(x0 , ρ)) ⊃ B(f (x0 ), ρ/2) for any 0 < ρ ≤ r. Proof. Without loss of generality, x0 = f (x0 ) = 0. Define Ty (x) = y − f (x) + x and fix some ρ ∈ (0, r]. We claim that Ty : B(0, ρ) → B(0, ρ) is a contraction, provided |y| ≤ ρ/2. Indeed, if x, x ∈ B(0, ρ), then 1 Ty (x) ≤ 1 ρ + I − Df (sx) ds |x| ≤ ρ, 2 0

Ty (x) − Ty (x ) = x − x − f (x) − f (x )

1

≤ 0

I − Df x + s(x − x ) ds |x − x | ≤ 1 |x − x |. 2

Hence, for any y ∈ B(0, ρ/2), there is a unique x ∈ B(0, ρ) so that f (x) = y. In particular, f is one-to-one on B(0, ρ) if f (B(0, ρ)) ⊂ B(0, r/2). Since |f (x)| ≤ 0

1

Df (sx) ds |x| ≤ 3 |x|, 2

236

PERES AND SCHLAG

it suffices to take ρ = r/3, as claimed. The following lemma is a precise statement to the effect that 3λ is invertible where it is small. This is true only locally. However, we can cover the set of small values of 3λ by a collection of balls {Bj } whose size and number can be controlled and on each of which 3λ can be inverted in the following sense: there exist m coordinate directions (depending on Bj ) so that the restriction of 3λ to the intersection of Bj with any hyperplane in those directions is invertible. Moreover, we have uniform bounds on the derivatives of the inverse. Lemma 7.7. Let (λ ∈ C 1,δ (Q) for some 0 < δ < 1. Suppose that Q ⊂ Rn is a region of transversality of order β for ( and that (λ is 1, δ-regular on Q; see Definition 7.2. Let U ⊂ Q be open and bounded with dist(U, ∂Q) > 0. Then there exist constants C0 , C1 , C2 depending only on β, n, m, U, Q so that for any distinct ω1 , ω2 ∈ ', there exist λ1 , . . . , λN ∈ Q with the following properties. Writing r = d(ω1 , ω2 ) for simplicity, we have N

% B λj , C 1 r b 0 β , λ ∈ U : |3λ | ≤ C0 r b0 β ⊂

(7.9)

j =1

(7.10)

N %

B λj , 2C1 r b0 β ⊂ λ ∈ Q : |3λ | ≤ Cβ r β ,

j =1

where b0 = (2 + m)δ −1 and N ≤ C2 r −b0 n . Cβ is the constant from Definition 7.2. Moreover, on each ball Bj = B(λj , 2C1 r b0 β ), we can select n − m coordinate directions 1 ≤ i1 < · · · < in−m ≤ n so that for any choice of y$ = (y1 , . . . , yn−m ), Fy$ = 3λ λ ∈ Bj : λi1 = y1 , . . . , λin−m = yn−m is a diffeomorphism satisfying | det(DFy$ )−1 | ≤ C2 r −β and

−1 (7.11) DFy$ ≤ C2 r −mβ . Proof. Fix distinct ω1 , ω2 ∈ '. By Definition 7.2, we can choose C0 and C3 so that E = λ ∈ U : |3λ | ≤ C0 r (2+m)β/δ satisfies (7.12)

E = λ ∈ Q : dist(E, λ) ≤ C3 r (2+m)β/δ ⊂ λ ∈ Q : |3λ | ≤ Cβ r β .

Now fix any λ0 ∈ E. By Definition 7.2, (7.1), and the Cauchy-Binet formula, there exist m coordinate directions, say, the first m, so that the determinant of the first m×m-minor of D3λ0 is bounded below by Cr β . Moreover, by (7.12) and the Hölder bounds on D3λ in (7.2), this continues to hold on all of B(λ0 , C4 r (2+m)β/δ ) for an

237


appropriate choice of C4 . Let sm (λ) ≥ sm−1 (λ) ≥ · · · ≥ s1 (λ) > 0 be the singular values of the first m × m-minor of D3λ . We have shown that sm (λ) ≤ Cr −β ,

(sm sm−1 · · · · · s1 )(λ) ≥ Cr β

on B(λ0 , C4 r b0 β ). Thus s1 ≥ Cr mβ on B(λ0 , C4 r b0 β ). As above, let

Fy$ (λ1 , . . . , λm ) = 3λ λ ∈ B λ0 , C4 r b0 β : λm+1 = y1 , . . . , λn = yn−m for any choice of y$ = (y1 , . . . , yn−m ). By the lower bound on the singular values, (DFy$ )−1 ≤ Cr −mβ on the domain of Fy$ . In particular, with λ = (λ1 , . . . , λm ),

DFy$ λ −1 ◦ DFy$ λ − Im×m ≤ Cr −(m+1+δ)β λ − λ δ . 0

0

In view of Lemma 7.6, there exists a constant C1 so that Fy$ is a diffeomorphism on

λ ∈ B λ0 , 2C1 r b0 β : λm+1 = y1 , . . . , λn = yn−m

for any choice of y$. The lemma follows by applying Wiener’s covering lemma to a covering of E with balls of size 13 C1 r b0 β . We now prove the higher-dimensional version of Proposition 4.10. Proposition 7.8. Let (λ ∈ C 1,δ (Q) for some 0 < δ < 1. Suppose that Q ⊂ Rn is a region of transversality of order β for ( and that (λ is 1, δ-regular on Q; see Definition 7.2. Assume that µ has finite α-energy for some α ∈ (0, m]. Then (7.13)

Ᏼσ +n−m {λ ∈ Q : dims (νλ ) < σ } = 0

for any σ ∈ (0, α − a0 β] with a constant a0 depending only on m, n, and δ. Proof. It suffices to prove (7.13) for any subcube Q ⊂ Q with dist(Q , ∂Q) > 0. As in the proof of Proposition 4.4, we assume that (7.13) fails. By Frostman’s lemma there exists a nonzero measure ρ on Q so that ρ(V ) ≤ | diam(V )|σ +n−m for all Borel sets V ⊂ Q and (7.14)

ρ λ ∈ Q : dims (νλ ) > σ − 0 = 0

for an appropriate choice of 0 > 0. Fix any small > 0 and let r = d(ω1 , ω2 ). In view of Lemma 7.7,

238

PERES AND SCHLAG

Q

(7.15)

(7.16)

(7.17)

dνλ (x) dνλ (y) dρ(λ) σ − Rm Rm |x − y| dµ(ω1 ) dµ(ω2 ) = σ − dρ(λ) Q ' ' ((λ, ω1 ) − ((λ, ω2 ) −(σ − ) dµ(ω1 )dµ(ω2 ) χ[|3λ |≥C0 r b0 β ] 3λ (ω1 , ω2 ) dρ(λ) = d(ω1 , ω2 )σ − ' ' Q −(σ − ) dµ(ω1 )dµ(ω2 ) dρ(λ) + χ[|3λ |≤C0 r b0 β ] 3λ (ω1 , ω2 ) d(ω1 , ω2 )σ − ' ' Q dµ(ω1 )dµ(ω2 ) ≤C (σ − )(1+b0 β) ' ' D(ω1 , ω2 ) N dρ(λ) dµ(ω1 ) dµ(ω2 ) +C σ − d(ω , ω )σ − b β 1 2 ' ' j =1 B (λj ,C1 r 0 ) |3λ | dµ(ω1 ) dµ(ω2 ) ≤C . d(ω1 , ω2 )α ' '

To pass from (7.15) to (7.16), first notice that

∞

dρ(λ) ≤ 2i(σ − ) ρ λ ∈ B λj , C1 r b0 β : |3λ | ≤ 2−i . σ − b β |3 | λ B (λj ,C1 r 0 ) i=−∞

To bound the measure on the right-hand side, we use (7.11) and (7.2). In fact, those estimates imply that the set {λ ∈ B(λj , C1 r b0 β ) : |3λ | ≤ 2−i } is the union of (1 + r b0 β /2−i r β )n−m many balls of diameter no larger than C min(r b0 β , 2−i r −mβ ). Thus dρ(λ) σ − b β B (λj ,C1 r 0 ) |3λ | ≤C

∞

σ +n−m

n−m ˜ 1 + r (b0 −1)β 2i 2i(σ − ) min r b0 β , 2−i r −mβ r −Cβ ,

i=−∞

˜ where C˜ = C(m, n, σ, δ). Inserting this bound into (7.16) and using the bound on N given by Lemma 7.7 implies (7.17) for an appropriate choice of a0 . However, (7.17) contradicts (7.14) if < 0 . In view of Propositions 7.5 and 7.8, Theorem 7.3 follows once the Sobolev estimate (7.3) is established. As in Section 4, this requires a technical lemma about averages involving Littlewood-Paley functions; see Lemmas 4.6 and 7.10. The proof of that lemma uses the following calculus fact. Lemma 7.9. Suppose h = (h1 , . . . , hm ) : U ⊂ Rm → Rm is a C L -diffeomorphism on the open set U , and let H denote the inverse of h. For any nonzero multi-index

239


η = (η1 , . . . , ηm ) ∈ Nm with |η| ≤ L, (7.18) ∂ η H =

|η|−1 t=0

−|η|−p

Dh ◦ H

p σ ,...,σ ,#$ p 1

p

1 σ

v$η,p σ1 , . . . , σp , #$ ∂ i h#i ◦ H, i=1

where the third sum runs over multi-indices σi ∈ Nm and #$ ∈ {1, 2, . . . , m}p . If |η| ≥ 2, $ ∈ Rm vanish unless |η| ≥ |σi | ≥ 2 for all i and the vectors v$η,p (σ1 , . . . , σp , #) p i=1 |σi | ≤ 2(|η| − 1). Proof. If η = (1, 0, . . . , 0) = e1 , we clearly have ∂ η H = (Dh ◦ H )e1 . Hence the lemma holds with |η| = 1. For the inductive step, we differentiate (7.18): ∂ η+e1 H =

|η|−1 t=0

p σ ,...,σ ,#$ p 1

 

−|η|−p−1 e ∂ 1 Dh ◦ H v$η,p σ1 , . . . , σp , #$ −(|η|+ p) Dh ◦ H  ×

−|η|−p v$η,p σ1 , . . . , σp , #$ ∂ σi h#i ◦ H + Dh ◦ H

p 1 i=1

×

p j =1

 

p 1



∂ σ i h # i ◦ H  ∂ e1 ∂ σ j h # j

i=1,i=j

Since for any multi-index σ

−1 ∂ e1 ∂ σ h ◦ H = Dh ◦ H



 ◦H . 

aτ ∂ τ h ◦ H

|τ |=|σ |+1

Rm ,

(7.18) holds for all multi-indices η with with suitable constant vectors a τ ∈ p |η | = |η| + 1. Also notice that i=1 |σi | has increased by at most two, as claimed. Lemma 7.10. Let (λ ∈ C L,δ (Q) for some positive integer L and some δ ∈ [0, 1). Suppose that Q ⊂ Rn is a region of transversality of order β for ( and that (λ is L, δ-regular on Q; see Definition 7.2. Let ρ ∈ C ∞ (Rn ) be supported inside Q. If ψ is the Littlewood-Paley function in Rm from Lemma 4.1, then for any distinct ω1 , ω2 ∈ ', any integer j , and any 0 ≤ q < L + δ + m − 1, $

−q j# (7.19) ρ(λ)ψ 2 ((λ, ω1 ) − ((λ, ω2 ) dλ ≤ Cq 1 + 2j d(ω1 , ω2 )1+a0 β , where Cq depends only on q, m, n, ρ, β, L, δ and a0 only depends on m, n, and L, δ. Proof. Fix distinct ω1 , ω2 ∈ ' and j, q as above. We may assume that 2j r > 1, where r = d(ω1 , ω2 ). Let φ ∈ C ∞ be nonnegative with φ = 1 on [−1, 1] and supp(φ) ⊂ [−2, 2]. Then

240

PERES AND SCHLAG

(7.20)

$ # ρ(λ)ψ 2j ((λ, ω1 ) − ((λ, ω2 ) dλ

= ρ(λ)ψ 2j r3λ (ω1 , ω2 ) φ Cβ−1 r −β 3λ dλ

$

# + ρ(λ)ψ 2j r3λ (ω1 , ω2 ) 1 − φ Cβ−1 r −β 3λ dλ.

Here Cβ is the constant from Definition 7.2. By the rapid decay of ψ,

#

$ ρ(λ)ψ 2j r3λ (ω1 , ω2 ) 1 − φ C −1 r −β 3λ dλ β

−q

−q ≤ Cq,β |ρ(λ)| 1 + 2j r 1+β . dλ ≤ Cq,β 1 + 2j r 1+β Thus it suffices to estimate the first integral in (7.20). To this end we introduce a partition of unity subordinate to the cover given by Lemma 7.7 with U = {Q : ρ > 0}. More precisely, by the standard construction of a partition of unity, there exist χj ∈ C ∞ (Rn ) so that supp(χj ) ⊂ Bj = B(λj , 2C1 r b0 β ) for j = 1, . . . , N and so that N b0 β }; see (7.9). Moreover, i=1 χj = 1 on {λ ∈ supp(ρ) : |3λ | ≤ C0 r (7.21) sup ∂ η χj ∞ ≤ Cη r −|η|b0 β j

for all multi-indices η ∈ Nn . By (7.10), χj (λ) = χj (λ)φ(Cβ−1 r −β 3λ ) for all j . We can therefore write the first integral in (7.20) as

ρ(λ)ψ 2j r3λ φ Cβ−1 r −β 3λ dλ =

N i=1

+

ρ(λ)χj (λ)ψ 2j r3λ dλ 2

ρ(λ) 1 −

N i=1

=

N

3

χj (λ) ψ 2j r3λ φ Cβ−1 r −β 3λ dλ

A(j ) + B.

i=1

To estimate the second term B, notice that |3λ | ≥ C0 r b0 β on the support of the integrand. The rapid decay of ψ therefore implies |B| ≤ Cq (1 + 2j r 1+b0 β )−q . For simplicity, we assume henceforth that i = 1. By Lemma 7.7, the map λ ( → 3(λ ,λ ) = Fλ (λ ) is a diffeomorphism on {λ ∈ Rm : (λ , λ ) ∈ Bj }, where λ = (λ1 , . . . , λm ) and λ = (λm+1 , . . . , λn ) (possibly after a permutation of the coordinates). Let Hλ denote the inverse of Fλ . Assuming as we may that ρ(λ) = ρ1 (λ )ρ2 (λ ), we let

Gλ (u) = ρ1 Hλ (u) χ1 Hλ (u) det DHλ (u) .


241

Clearly, Gλ ∈ C L−1,δ . Since q < L + δ + m − 1, there exists an integer M so that q −m−δ < M ≤ L−1. Fix > 0 such that (M +m+δ)(1− ) > q and rewrite A(1) as A(1) =

Rn−m

=

Rn−m

ρ2 λ

ρ2 λ

Rm

Gλ (u)ψ 2j ru du dλ

ψ 2j ru

|u|(2j r)−1+

O

−2q−m Gλ (u)du dλ

2j r|u|

(1)

= A 1 + A2 . (1)

By Lemma 7.7, Gλ ∞ ≤ Cr −β . In particular, |A2 | ≤ Cq (2j r)−q−m r −β . Since ψ has vanishing moments of all orders, (1)

(7.22)

∂ η Gλ (0) η u du dλ ψ 2j ru η! Rn−m |u|>(2j r)−1+ |η|≤M " ! η j −(M+m+δ)(1− ) dλ . ρ2 λ O sup ∂ Gλ δ 2 r + ρ2 λ

A1 = −

Rn−m

|η|=M

C

It remains to estimate ∂ η Gλ (u), which can be done uniformly in λ . Fix any λ and let u = Fλ (λ ). By Lemma 7.9, (7.2) in Definition 7.2, and Lemma 7.7, |η|−1 η ∂ Hλ (u) ≤ Cη t=0

p σ ,...,σ ,#$ p 1

−1 |η|+p −2β(|η|−1) r v$η,p σ1 , . . . , σp , #$ DFλ

≤ Cη r −2|η|(m+1)β . Therefore, by Leibnitz’s rule and (7.21), ∂ η Gλ ∞ ≤ Cη r −C|η|β where C = C(b0 , (1) m). Plugging this into (7.22) and exploiting the rapid decay of ψ, we obtain |A1 | ≤ Cq r −a0 Mβ (2j r)−(M+m+δ)(1− ) . Since N ≤ Cr −b0 nβ and (M +m+δ)(1− ) > q, this finally yields (7.19). Proof of (7.3). Fix some γ with 0 < (m + 2γ )(1 + a0 β) ≤ α and 2γ < L + δ − 1. Let ρ be a smooth nonnegative function on Rn so that supp(ρ) ⊂ Q and fix some

242

PERES AND SCHLAG

q ∈ (m + 2γ , L + δ + m − 1). In view of (4.2), the definition of νλ , and Lemma 7.10, νλ 22,γ ρ(λ) dλ

∞

22j γ

j =−∞ ∞

≤

2

' ' j =−∞ ∞

≤ Cγ

' ' −∞

∞ −∞

(ψ2−j ∗ νλ )(x) dνλ (x)ρ(λ) dλ $ j# dµ(ω1 )dµ(ω2 ) ) − ((λ, ω ) ρ(λ) dλ ((λ, ω ψ 2 1 2

j (m+2γ )

−q 2j (m+2γ ) 1 + 2j d(ω1 , ω2 )1+a0 β dµ(ω1 )dµ(ω2 )

≤ Cγ

'

dµ(ω1 ) dµ(ω2 ) ≤ C Ᏹα (µ) < ∞, (1+a0 β)(m+2γ ) ' d(ω1 , ω2 )

as claimed. 8. Applications of the higher-dimensional projection theorems 8.1. Self-similar sets in the complex plane. In [41], Solomyak considered sets of the form ∞ S n (8.1) an λ : a n ∈ S , Cλ = n=0

where S = {s1 , . . . , s# } is a set of digits and λ ∈ D = {z ∈ C : |z| < 1} (λn of course denotes complex powers). Let R = S − S, ∞ n bn λ : b n ∈ R ᏮR =

n=0

ᏹR = λ ∈ D : there exists a nonzero f ∈ ᏮR with f (λ) = 0

0 R = λ ∈ D : there exists a nonzero f ∈ ᏮR with f (λ) = f (λ) = 0 . ᏹ If λ ∈ D \ ᏹR , it is easy to see that CλS satisfies a strong separation condition as in (5.14) and that therefore dim CλS = log #/(− log |λ|). The case λ ∈ ᏹR was studied in 0 R . It was shown there that for any [41], assuming the transversality condition λ ∈ ᏹ 0 R satisfies fixed choice of digit set, S ⊂ C a.e. λ ∈ ᏹR \ ᏹ dim CλS =

log # − log |λ|

if |λ| < #−1/2

and

Ᏼ2 CλS > 0

if |λ| > #−1/2 .

It is straightforward to see that Theorem 7.3 applies to this case. First we verify that conditions (7.2) and (7.1) in Definition 7.2 are satisfied with L = ∞. This is very similar to Lemma 5.3.

243


To specialize from Section 7, let 0 < R1 < R2 < 1 and 0 R , R1 < |λ| < R2 Q ⊂ λ ∈ D : λ ∈ ᏹ 0 is relatively closed in D since ᏮR be a fixed closed cube. As observed in [41], ᏹ is a normal family. With this choice of Q, let ' = {1, . . . , l}N be equipped with |ω∧τ | and uniform product measure µ. Finally, let ((λ, ω) = the metric d(ω, τ ) = R2 ∞ n n=0 sωn λ . 1+β

Lemma 8.1. Let R2 > R1 . Then Q is a region of transversality of order 2β for ( in the sense of Definition 7.2. Proof. There exists δ = δ(Q) ∈ (0, 1) such that for any g ∈ ᏮR and λ ∈ Q, |g(λ)| < δ

implies

|g (λ)| > δ.

This follows since ᏮR is a normal family. Clearly, 3λ (ω, τ ) = (λR2−1 )|ω∧τ | g(λ) with some g = gω,τ ∈ ᏮR . Fix ω, τ ∈ ' and let k = |ω ∧ τ |. Therefore, " ! η ∂ 3λ ≤ Cη k |η| |λ|k−|η| R −k g ∞ + |λ|k R −k sup ∂ σ g ∞ 2

−|η|

≤ Cη k |η| R1

−β|η|k

≤ Cη,β R2

2

|σ |=|η|

(1 − R2 )−1 + (1 − R2 )−|η|−1

,

since R2 < 1 and β > 0. Thus condition (7.2) in Definition 7.2 holds with L = ∞. βk To check (7.1), assume |3λ | ≤ δbβ R2 , where the constant bβ ∈ (0, 1) is determined below. Then

k

k βk R1 R2−1 |g(λ)| ≤ |λ|R2−1 |g(λ)| ≤ δbβ R2 implies |g(λ)| ≤ δbβ (R1−1 R2

1+β k ) .

Hence, (here 3λ denotes the complex derivative)

3 ≥ R1 R −1 k g (λ) − R −1 k|g(λ)| λ 1 2

βk 1+β k ≥ R2 δ − δbβ R1−1 k R1−1 R2 βk

δR2 βk ≥ ≥ δbβ R2 2 if bβ ≤ 21 [1 + supk≥0 (k/R1 )(R1−1 R2 )k ]−1 . Since the Cauchy-Riemann equations imply that det[D3λ ] = |3λ |2 , condition (7.1) in Definition 7.2 holds with 2β instead of β and Cβ = (δbβ )2 . 1+β

244

PERES AND SCHLAG

Theorem 8.2. Let 0 < r < R < 1. Then 0 R : r < |λ| < R, Ᏼ2 C S = 0 ≤ 4 − log # , dim λ ∈ ᏹR \ ᏹ λ − log r log # 0 R : r < |λ| < R, dim C S < log # . ≤ dim λ ∈ ᏹR \ ᏹ λ − log R − log |λ| Proof. Fix any β ∈ (0, 1). There exist countably many cubes Qj ⊂ C so that % 0 R : r < |λ| < R = λ ∈ D\ᏹ Qj j 1+β

and Qj ⊂ {rj < |λ| < Rj } with rj > Rj . By the previous lemma, each Qj is a region of transversality of order 2β for (. It is easy to check that with the choice of |ω∧τ | , we have Ᏹα (µ) < ∞ if Rjα l > 1. Since supp(νλ ) = CλS , metric dj (ω, τ ) = Rj Theorem 8.2 now follows by applying Theorem 7.3 and letting β → 0+. 8.2. Intersections of sets with spheres. Let S0 ⊂ Rd , d ≥ 2, be a closed, strictly convex C ∞ -hypersurface surrounding the origin (the differentiability assumption will be relaxed later). Thus S0 = {ρ(u)u : u ∈ S d−1 } for some function ρ : S d−1 → (0, ∞) in C ∞ . Define f ∈ C ∞ (Rd \ {0}) to be f (x) = |x|/ρ(x/|x|). In particular, f is homogeneous of degree one and {x ∈ Rd : f (x) = r} = rS0 . Moreover, by the assumption of strict convexity, there exists κ > 0 so that for any x = 0 and any tangent vector v of S0 , (8.2)

2 4 2 5 D f (x)v, v ≥ κ |v| , |x|

whereas D 2 f (x)x = 0 by homogeneity. For any Borel set E ⊂ Rd and σ < 1, let Bσ = Bσ (E) = x ∈ Rd : dim f (−x + E) < σ . For σ = 1, we set

B1 = B1 (E) = x ∈ Rd : Ᏼ1 f (−x + E) = 0 .

Our main result in this section concerns the dimension of Bσ . Theorem 8.3. Let S0 be a strictly convex, C ∞ -hypersurface surrounding the origin and define f and Bσ as above. If E ⊂ Rd is a Borel set and σ ∈ [0, 1 ∧ dim E], then (8.3)

dim Bσ ≤ d + σ − max(1, dim E).

Furthermore, for any hyperplane H ⊂ Rd , (8.4)

dim(Bσ ∩ H ) ≤ d − 1 + σ − max(1, dim E).


245

Proof. By classical theorems of Marstrand and Mattila, (8.3) follows from (8.4) (see Theorem 10.10 in [31]). For the sake of illustration, however, we begin by showing directly that (8.3) follows from Theorem 7.3. In fact, we first consider the case S0 = S d−1 because it is easy to check the conditions in Definition 7.2 for Euclidean spheres. In this special case, it is more convenient to work with the square of the Euclidean norm, that is, f (x) = |x|2 . Define ( : Rd ×E → R to be ((λ, x) = |x −λ|2 , and let 3λ (x, y) = |x −y|−1 [|x −λ|2 −|y −λ|2 ] = /x −y/|x −y|, x +y −2λ0. Therefore, |∂ η 3λ (x, y)| ≤ 2 for any nonzero multi-index η, and condition (7.2) holds with β = 0. Moreover, |D3λ (x, y)| = 2|x −y/|x −y|| = 2, and (7.1) is also satisfied with β = 0. Hence, Definition 7.2 holds with Q = Rd , ' ⊂ E an arbitrary compact set, L = ∞, and β = 0. For any > 0, we can choose a Frostman measure µ on ' with exponent > dim E − . Since f (−λ + E) ⊃ supp(νλ ), (8.3) with S0 = S d−1 follows from Theorem 7.3 as → 0+. For general S0 as described above, fix a small > 0. Dilating, if necessary, we have dim(E \Q0 ) > dim E − for any cube Q0 of side length 2. Now fix a cube Q ⊂ Rd of side length 1. To bound dim(Bσ ∩Q), we may therefore assume that E ⊂ 2rQ\(rQ) for some r ≥ 2. Thus, if |v| = 1, x ∈ E, and λ ∈ Q, (8.2) implies that 4 5 ∇f (x − λ), v = 0 ,⇒ D 2 f (x − λ)v ≥ ρ (8.5) for some ρ > 0 depending on r and κ. Now define (λ : E → R to be (λ (x) = f (x −λ), and let 3λ (x, y) = |x −y|−1 [f (x −λ)−f (y −λ)]. It is clear that condition (7.2) holds with β = 0. To check (7.1), notice that # $ 4 5 3λ (x, y) = |x − y|−1 f (x − λ) − f (y − λ) = ∇f (x − λ), v + O(|x − y|), where v = x − y/|x − y|. Moreover, (8.6)

D3λ (x, y) = −D 2 f (x − λ)v + O(|x − y|),

where D denotes differentiation with respect to λ. In view of (8.5), there exists some small constant C0 such that 3λ (x, y) < C0 ρ ,⇒ |D3λ (x, y)| > ρ , (8.7) 2 provided λ ∈ Q and x, y ∈ E satisfy |x − y| < C0 ρ. Now let ' = E ∩ B(x0 , C0 ρ/2) be equipped with the usual Euclidean metric where x0 is arbitrary. We have shown that Definition 7.2 is satisfied with β = 0 with this choice of ', Q, and (λ . Therefore, selecting x0 so that dim(E ∩B(x0 , C0 ρ)) > dim E − and letting µ be an appropriate Frostman measure, we conclude that (8.3) follows from (7.4) and (7.6) with L = ∞, β = 0, n = d, m = 1, and α = dim E − as → 0+. The proof of (8.4) proceeds by induction in the dimension d. First we need to verify that transversality holds in all dimensions. Fix a hyperplane H ⊂ Rd . Without loss of generality, 0 ∈ H . Let e1 , . . . , ed−1 be a basis in H , and define (λ (x) =

246

PERES AND SCHLAG

−1 d−1 , and f (x − d−1 i=1 λi ei ), 3λ (x, y) = |x − y| [(λ (x) − (λ (y)] for any λ ∈ R d distinct x, y ∈ R . Clearly, with p = x − λ1 e1 − · · · − λd−1 ed−1 , 4 5 (8.8) 3λ (x, y) = ∇f (p), v + O(|x − y|), 5 4 5 4 (8.9) D3λ (x, y) = − D 2 f (p)v, e1 , . . . , D 2 f (p)v, ed−1 + O(|x − y|), where v = x − y/|x − y| and D denotes differentiation with respect to λ. We claim that for any v ∈ S d−1 and any z ∈ H , (8.10)

d−1 4 2 5 5 D f (z)v, ei 2 > 0. ∇f (z), v = 0 ,⇒

4

i=1

In fact, we show that (8.10) can only fail if z ∈ H . More precisely, suppose the sum in (8.10) vanishes for some z ∈ Rd , z = 0, and some v ∈ S d−1 . Without loss of generality, f (z) = 1. Then the functionals u ( → /D 2 f (z)u, ei 0, i = 1, . . . , d − 1, on the tangent space to S at z are linearly dependent. Thus d−1

(8.11)

ξi D 2 f (z)ei ∇f (z)

i=1

for some (ξ1 , . . . , ξd−1 ) = (0, . . . , 0). Let e = d−1 i=1 ξi ei . Clearly, e = 0. It is easy to verify that (8.11) implies that /D 2 f (z)e, e0 = 0. First notice that by strict convexity of S, there is a parameterization z+te = r(t)γ (t) for small t where γ is a smooth curve in S with γ (0) = z and r is a smooth positive function with r(0) = 1. Let ∂e f = /∇f, e0 be the directional derivative. Since ∂e f is homogeneous of degree zero, 7 6 5 4 2 d2 d d = ∂e f (γ (t)) = D 2 f (z)e, γ (0) = 0, D f (z)e, e = 2 f (z + te) t=0 t=0 dt dt dt by (8.11). In view of (8.2) and D 2 f (z)z = 0, we conclude that e z and thus z ∈ H , as claimed. To check transversality, we need the following quantitative version of (8.10) (cf. (8.5)). For any ρ0 > 0 there exists ρ1 > 0, depending on κ from (8.2), so that for any z ∈ Rd with f (z) = 1 and dist(z, H ) > ρ0 , we have (8.12)

d−1 4 4 2 5 5 ∇f (z), v < ρ1 ,⇒ D f (z)v, ei 2 > ρ1 . i=1

Indeed, by (8.10) and compactness, (8.13)

min

min

z∈S0 :dist(z,H )>ρ0 v∈Tz S0 :|v|=1

d−1 4 2 5 D f (z)v, ei 2 > ρ2 > 0 i=1

(Tz S0 denotes the tangent space to S0 at z). If |/∇f (z), v0| < ρ1 , then there exists v˜ ∈ Tz S0 , |v| ˜ = 1, so that |v − v| ˜ ≤ Cρ1 /|∇f (z)| ≤ Cρ1 and (8.12) follows from (8.13), provided ρ1 is small compared to ρ2 .


247

We start by proving (8.4) for d = 2. Fix some e ∈ S 1 , and let # ⊂ R2 be the line through the origin in direction e. If dim(E ∩ #) = dim E, then dim r > 0 : (x + rS) ∩ E = ∅ = dim E for all x ∈ # and (8.4) holds with H = #. Otherwise, for any > 0, there exists ρ0 > 0 and R > 0 so that dim(E ∩ B(0, R) \ #ρ0 ) > dim E − , where #ρ0 denotes a ρ0 -neighborhood of #. Now fix such > 0 and ρ0 > 0, and let J ⊂ # be an interval of length one. In view of (8.8), (8.9), and (8.12), there exist ρ > 0 and C0 (depending on J , κ, ρ0 , R) so that 3λ (x, y) < ρ ,⇒ D3λ (x, y)2 > ρ for any x, y ∈ B(0, R)\#ρ0 satisfying |x −y| < C0 ρ. Let ' = E ∩B(x0 , C0 ρ/2)\#ρ0 , where x0 ∈ B(0, R) is selected so that dim ' > dim E − . We conclude that with this choice of ' and (, J satisfies Definition 2.7 with L = ∞ and β = 0. Since f (− d−1 i=0 λi ei + E) ⊃ supp(νλ ), (8.4) therefore follows from Theorem 2.8 with α = dim E − as → 0+. Now suppose that (8.4) holds up to d − 1 for some d ≥ 3, and fix a hyperplane H ⊂ Rd passing through the origin. If dim(E ∩H ) = dim E, then (8.4) follows from (8.3) in dimension d −1. (Recall that the latter estimate follows from (8.4) in dimension d − 1.) Otherwise, for any > 0, there exists ρ0 > 0 so that dim(E \ H ρ0 ) > dim E − , where H ρ0 denotes a ρ0 -neighborhood of H . By the same argument we gave for d = 2, we conclude from (8.8), (8.9), and (8.12) that Definition 7.2 holds for any bounded cube Q ⊂ H and a suitable choice of '. Hence, (8.4) follows from Theorem 7.3 with L = ∞, β = 0, n = d − 1, m = 1, and α = dim E − as → 0+. Suppose E ⊂ # ⊂ Rd for some line # is a set of dimension 1 but with Ᏼ1 (E) = 0. Then it is easy to see that for all x ∈ Rd , (x +rS d−1 )∩E = ∅ for a.e. r. This implies that there are no nontrivial bounds for σ = 1 and dim(E) ≤ 1 in Theorem 8.3. In [10] Falconer showed that for any Borel set A ⊂ Rd with dim A > (d + 1)/2, the distance set D(A) = {|x − y| : x, y ∈ A} has positive measure. In [32] Mattila and Sjölin showed that under the same assumption, D(A) has nonempty interior. On the other hand, for d = 2 Bourgain [4] proved that D(A) has positive measure if dim A > 13 9 and Wolff [46] improved this to dim A > 43 . (8.3) implies the following sharpened version of Falconer’s theorem concerning “pinned” distance sets Dx (A) = {|x − y| : y ∈ A}. Corollary 8.4. Suppose A ⊂ Rd with dim A > (d + 1)/2. Then (d + 1) dim x ∈ Rd : Dx (A) has Lebesgue measure zero ≤ d + 1 − dim A < . 2 Proof. Apply (8.3) to f (x) = |x| and E = A.

248

PERES AND SCHLAG

Similarly, we obtain a sufficient condition for pinned distance sets to contain an interval. Using (7.4) with σ > 2 in the proof of Theorem 8.3 yields the following statement: Let A ⊂ Rd with d ≥ 3 satisfy dim A > (d + 2)/2. Then (d + 2) dim x ∈ Rd : Dx (A) has empty interior ≤ d + 2 − dim A < . 2 We now discuss the case where S0 ∈ C k,δ for some positive integer k and δ ∈ [0, 1). Theorem 8.5. Let S0 be a strictly convex C k,δ -hypersurface surrounding the origin with k + δ > 2, and define f and Bσ as above. If E ⊂ Rd is a Borel set and σ ∈ [0, 1 ∧ dim E], then

(8.14) dim Bσ ≤ d + σ − max 1, min(dim E, k + δ − 1) . Furthermore, for any hyperplane H ⊂ Rd , (8.15)

dim(Bσ ∩ H ) ≤ d − 1 + σ − max 1, min(dim E, k + δ − 1) .

In particular, if k + δ ≥ dim E + 1, then (8.3) and (8.4) remain valid. Proof. Since f ∈ C L,δ by assumption, (8.6), (8.8), and (8.9) show that 3λ ∈ in the sense of Definition 7.1. The proof of Theorem 8.3 therefore yields that 3λ as defined above satisfies Definition 7.2 with L = k −1 and β = 0, and as before, Theorem 7.3 implies (8.14) and (8.15). C k−1,δ

Finally, we give some examples to show that Theorem 8.3 can fail under weaker assumptions. Suppose S0 = ∂[−1, 1]d is the boundary of the unit cube. Let K ⊂ [0, 1] be a Cantor set and let E = K × [−1, 1]d−1 . It is easy to see that Bσ (E) ⊃ [3, ∞) × [−1, 1]d−1 for any dim K < σ ≤ 1. The bound in (8.3) would therefore require that d ≤ 1 + σ − dim K, which is clearly false. A similar example shows that (8.3) fails if S0 contains an arbitrarily small piece of a hyperplane. In view of (8.4), one might ask whether (8.16)

dim(B1 ∩ π) ≤ 1 + dim π − dim E

for any plane π ⊂ Rd . It turns out that this fails if dim π ≤ d − 2, even if S0 = S d−1 . For simplicity, let d = 3 and let (8.17) E = (0, r cos θ, r sin θ) : θ ∈ [0, 2π], r ∈ K , where K ⊂ [0, 1] is a Cantor set. Clearly, for any x ∈ R, dim{r > 0 : (x, 0, 0)+rS 2 ∈ E} = dim K, whereas (8.16) would require that dim(B1 ∩ {(x, 0, 0) : x ∈ R}) ≤ 2 − dim E = 1 − dim K. 9. Questions and comments 9.1. Bernoulli convolutions. (i) For which intervals J ⊂ ( 21 , 1) is ∞?

J

νλ 42 dλ
2k has nonempty interior. Can 2k be replaced by a smaller threshold if k > 1? (ii) Suppose that µ is a finite planar measure satisfying the Frostman condition µ(B(x, r)) ≤ r α for all x ∈ R2 and r > 0. Let νθ be the projection of µ onto a line in direction θ . For p ∈ [1, 2/(2 − α)), the Sobolev embedding theorem implies that the projected measures νθ have a density in Lp for a.e. θ ∈ S 1 . Can the threshold 2/(2 − α) be increased? (iii) Is Theorem 8.3 optimal? Heuristically speaking, estimates for distance sets are related to the Erd˝os problem of bounding the minimum number gd (n) of different distances determined by n points in R d . It is known that C −1 n4/5− < g2 (n) < 8 Cn/ log n; see [5] and the discussion in [1, Chapter 12]. The analogy between cardinality and dimension suggests that dim(E) > 5/4 ⇒ Ᏼ1 (D(E)) > 0. Generally speaking, however, it is presently unclear how to make such a deduction. Nevertheless, T. Wolff [44], [45] has successfully applied sophisticated methods from combinatorial geometry to obtain Hausdorff dimension estimates. In particular, he showed that a Borel set in the plane that contains a circle of every radius must have Hausdorff dimension 2. For pinned distance sets as in Corollary 8.4, the relevant combinatorial problem seems to be estimating the minimum number of distinct distances between n points and one “typical” point. More precisely, it is known (see [1, Corollary 12.11]) that for any n points p1 , p2 , . . . , pn ∈ R2 , one has card{|pi − pj | : j = 1, 2, . . . , n} ≥ Cn3/4 for at least n/2 choices of i. (This appears to be the best-known bound. In particular, the authors of [5] point out that their method does not show that there exists a point from which there are n4/5− different distances.) The analogy between cardinality and dimension alluded to above then suggests that dim(E) > 43 ⇒ Ᏼ1 (Dx (E)) > 0 for most points x ∈ E. Note that Corollary 8.4 requires dim(E) > 23 for the same conclusion in R2 . In the recent preprint [46], Wolff shows that dim(E) > 43 implies that Ᏼ1 (D(E)) > 0. His methods are in the spirit of [4] and involve Bochner-Riesz-type arguments. (iv) Does Theorem 8.3 hold if S0 ∈ C k,δ with k +δ < dim E +1 (cf. Theorem 8.5)? In particular, is Theorem 8.3 valid if S0 is only assumed to be a continuous, strictly convex hypersurface surrounding the origin? References [1] [2] [3]

P. Agarwal and J. Pach, Combinatorial Geometry, Wiley-Intersci. Ser. Discrete Math. Optim., Wiley, New York, 1995. J. C. Alexander and J. A. Yorke, Fat Baker’s transformations, Ergodic Theory Dynam. Systems 4 (1984), 1–23. J. Bourgain, Besicovitch type maximal operators and applications to Fourier analysis, Geom. Funct. Anal. 1 (1991), 147–187.

250 [4] [5] [6] [7] [8] [9] [10] [11] [12] [13]

[14] [15] [16] [17] [18]

[19] [20] [21] [22]

[23] [24] [25] [26] [27]

[28] [29]

PERES AND SCHLAG , Hausdorff dimension and distance sets, Israel J. Math. 87 (1994), 193–201. F. Chung, E. Szemerédi, and W. Trotter, The number of different distances determined by a set of points in the Euclidean plane, Discrete Comput. Geom. 7 (1992), 1–11. ˝ On a family of symmetric Bernoulli convolutions, Amer. J. Math. 61 (1939), 974–976. P. Erdos, , On the smoothness properties of a family of Bernoulli convolutions, Amer. J. Math. 62 (1940), 180–186. K. J. Falconer, Continuity properties of k-plane integrals and Besicovitch sets, Math. Proc. Cambridge Philos. Soc. 87 (1980), 221–226. , Hausdorff dimension and the exceptional set of projections, Mathematika 29 (1982), 109–115. , On the Hausdorff dimension of distance sets, Mathematika 32 (1985), 206–212. , The Geometry of Fractal Sets, Cambridge Tracts in Math. 85, Cambridge University Press, Cambridge, 1986. , The Hausdorff dimension of self-affine fractals, Math. Proc. Camb. Philos. Soc. 103 (1988), 339–350. M. Frazier, B. Jawerth, and G. Weiss, Littlewood-Paley Theory and the Study of Function Spaces, CBMS Regional Conf. Ser. in Math. 79, Conf. Board Math. Sci., Washington, Amer. Math. Soc., Providence, 1991. D. Funaro, Polynomial Approximation of Differential Equations, Lecture Notes in Phys. New Ser. m Monogr. 8, Springer-Verlag, Berlin, 1992. A. M. Garsia, Arithmetic properties of Bernoulli convolutions, Trans. Amer. Math. Soc. 102 (1962), 409–432. X. Hu and S. J. Taylor, Fractal properties of products and projections of measures in Rd , Math. Proc. Cambridge Philos. Soc. 115 (1994), 527–544. B. Hunt and V. Y. Kaloshin, How projections affect the dimension spectrum of fractal measures, Nonlinearity 10 (1997), 1031–1046. B. Hunt, T. D. Sauer, and J. A. Yorke, Prevalence: A translation-invariant “almost every” for infinite-dimensional spaces, Bull. Amer. Math. Soc. (N.S.) 27 (1992), 217–238; Addendum, Bull. Amer. Math. Soc. (N.S.) 28 (1993), 306–307. J. E. Hutchinson, Fractals and self-similarity, Indiana Univ. Math. J. 30 (1981), 713–747. M. Jarvenpää, On the upper Minkowski dimension, the packing dimension, and orthogonal projections, Ann. Acad. Sci. Fenn. Ser. A I Math. Dissertationes, 1994. B. Jessen and A. Wintner, Distribution functions and the Riemann zeta function, Trans. Amer. Math. Soc. 38 (1935), 48–88. J. P. Kahane, “Sur la distribution de certaines séries aléatoires” in Colloque de Théorie des Nombres (Univ. Bordeaux, Bordeaux, 1969), Bull. Soc. Math. France 25 (1971), 119– 122. R. Kaufman, On Hausdorff dimension of projections, Mathematika 15 (1968), 153–155. , An exceptional set for Hausdorff dimension, Mathematika 16 (1969), 57–58. R. Kaufman and P. Mattila, Hausdorff dimension and exceptional sets of linear transformations, Ann. Acad. Sci. Fenn. Ser. A I Math. 1 (1975), 387–392. M. Keane, M. Smorodinsky, and B. Solomyak, On the morphology of γ -expansions with deleted digits, Trans. Amer. Math. Soc. 347 (1995), 955–966. F. Ledrappier, “On the dimension of some graphs” in Symbolic Dynamics and Its Applications (New Haven, Conn., 1991), Contemp. Math. 135, Amer. Math. Soc., Providence, 1992, 285–293. J. Marstrand, Some fundamental geometrical properties of plane sets of fractional dimensions, Proc. London Math. Soc. (3) 4 (1954), 257–302. P. Mattila, Hausdorff dimension, orthogonal projections and intersections with planes, Ann. Acad. Sci. Fenn. Ser. A I Math. 1 (1975), 227–244.

SMOOTHNESS OF PROJECTIONS [30] [31] [32] [33]

[34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] [46]

251

, Orthogonal projections, Riesz capacities and Minkowski content. Indiana Univ. Math. J. 39 (1990), 185–198. , Geometry of Sets and Measures in Euclidean Spaces, Cambridge Stud. Adv. Math. 44, Cambridge University Press, Cambridge, 1995. P. Mattila and P. Sjölin, Regularity of distance measures and sets, to appear. Y. Peres, W. Schlag, and B. Solomyak, “Sixty years of Bernoulli convolutions” in Fractals and Stochastics, II, Proceedings of the Greifswald 1998 Conference, ed. Bandt, Graf, and Zähle, to appear. Y. Peres and B. Solomyak, Absolute continuity of Bernoulli convolutions, a simple proof, Math. Res. Lett. 3 (1996), 231–239. , Self-similar measures and intersections of Cantor sets, Trans. Amer. Math. Soc. 350 (1998), 4065–4087. M. Pollicott and K. Simon, The Hausdorff dimension of λ-expansions with deleted digits, Trans. Amer. Math. Soc. 347 (1995), 967–983. F. Przytycki and M. Urbansky, On Hausdorff dimension of some fractal sets, Studia Math. 93 (1989), 155–186. T. D. Sauer and J. A. Yorke, Are the dimensions of a set and its image equal under typical smooth functions? Ergodic Theory Dynam. Systems 17 (1997), 941–956. B. Solomyak, On the random series ±λn (an Erd˝os problem), Ann. of Math. (2) 142 (1995), 611–625. , On the measure of arithmetic sums of Cantor sets, Indag. Math. (N.S.) 8 (1997), 133–141. , Measure and dimension for some fractal families, Math. Proc. Cambridge Philos. Soc. 124 (1998), 531–546. E. Stein, Singular integrals and differentiability properties of functions, Princeton Math. Ser. 30, Princeton University Press, Princeton, 1970. , Harmonic Analysis, Princeton University Press, Princeton, 1993. T. Wolff, A Kakeya type problem for circles, Amer. J. Math. 119 (1997), 985–1026. , “Recent work connected with the Kakeya problem” in Prospects in Mathematics (Princeton, NJ, 1996), Amer. Math. Soc., Providence, 1999. , Decay of circular means of Fourier transforms of measures. Preprint (1999). To appear in Internat. Math. Res. Notices 1999, 547–567.

Peres: Department of Mathematics, Hebrew University, Jerusalem, Israel and Department of Statistics, University of California, Berkeley, California 94720-3860, USA; peres @math.huji.ac.il Schlag: Mathematical Sciences Research Institute, 1000 Centennial Drive, Berkeley, CA 94720-5070, USA; Current: Department of Mathematics, Princeton University, Fine Hall, Princeton New Jersey 08544, USA; [email protected]

Vol. 102, No. 2


© 2000

HIGHER INTEGRABILITY FOR PARABOLIC SYSTEMS OF p-LAPLACIAN TYPE JUHA KINNUNEN and JOHN L. LEWIS

1. Introduction. In this work, we study regularity of solutions to second-order parabolic systems: (1.1)

∂ui = div Ai (x, t, ∇u) + Bi (x, t, ∇u), ∂t

i = 1, . . . , N.

In particular, we are interested in systems of p-Laplacian type. We present more precise structural assumptions later, but the principal prototype that we have in mind is the p-parabolic system ∂ui = div |∇u|p−2 ∇ui , ∂t

i = 1, . . . , N,

with 1 < p < ∞. As usual, solutions to (1.1) are taken in a weak sense, and they are assumed to belong to a parabolic Sobolev space. A good source for the regularity theory is [D]. In the elliptic case when the system is (1.2)

div Ai (x, t, ∇u) + Bi (x, t, ∇u) = 0,

i = 1, . . . , N,

it is known that solutions locally belong to a slightly higher Sobolev space than assumed a priori. This self-improving property was first observed by Elcrat and Meyers in [ME] (see also [Gi] and [Str]). Their argument is based on reverse Hölder inequalities and a modification of Gehring’s lemma [Ge], which originally was developed to study the higher integrability of the Jacobian of a quasiconformal mapping. In the elliptic case, higher integrabilty results play a decisive role in studying the regularity of solutions (see [GM] and [Gi]). The purpose of this work is to obtain higher integrablity results in the p-parabolic setting. We prove that the gradient of a weak solution to (1.1) satisfies a reverse Hölder inequality for p > 2n/(n+2). The critical exponent 2n/(n+2) occurs also in parabolic regularity theory (see [D]). We note that reverse Hölder inequalities and the local higher integrability for weak solutions were already proved for p = 2 in [GS] (see also [C]). Our result appears to be new even in the scalar case if p = 2. Received 12 November 1998. Revision received 3 June 1999. 1991 Mathematics Subject Classification. Primary 35K55. Lewis supported by National Science Foundation grant number DMS-9876881. 253

254

KINNUNEN AND LEWIS

One of the difficulties in proving our main result is that a solution does not remain a solution under multiplication by a constant that is neither 0 nor 1. Since reverse Hölder inequalities are invariant under multiplication by a constant, we have to choose a class of cylinders whose side lengths depend on the size of the function in order to obtain a reverse Hölder inequality as in [Ge] and then higher integrability. It seems to us that our results can be used to extend partial regularity results in [GM] for nonlinear elliptic systems to cover some parabolic systems. For p = 2, this was done in [GS], but our method also applies when p = 2. 2. Preliminaries. In order to be more precise about the structure and solutions of the system (1.1), we need some notation. Let Ω ⊂ Rn be an open set and let W 1,p (Ω) denote the Sobolev space of real-valued functions g such that g ∈ Lp (Ω) and the distributional first partial derivatives ∂g/∂xi , i = 1, 2, . . . , n, exist in Ω and belong to Lp (Ω). The space W 1,p (Ω) is equipped with the norm n ∂g

g 1,p,Ω = g p,Ω + ∂x . i p,Ω i=1

Given O

⊂ Rn

open, N a positive integer, and −∞ ≤ S < T ≤ ∞, let u = (u1 , . . . , uN ) : O × (S, T ) → RN

and suppose that whenever −∞ ≤ S < S1 < T1 < T ≤ ∞ and Ω ⊂ O, we have u ∈ L2 Ω × [S1 , T1 ] ∩ Lp [S1 , T1 ]; W 1,p (Ω) . (2.1) Here the notation Lp [S1 , T1 ]; W 1,p (Ω) means that for almost every t, S1 < t < T1 , with respect to one-dimensional Lebesgue measure, the function x → u(x, t) is in W 1,p () componentwise, and T1 N p p ui (·, t)p (2.2) dt < ∞. |||u|||p,Ω = u p,Ω×(S1 ,T1 ) + 1,p,Ω S1 i=1

Let ∇u denote the distributional gradient of u (taken componentwise) in the x variable only. We suppose that A = (A1 , . . . , AN ) where Ai = Ai (x, t, ∇u) : O × (S, T ) × RnN −→ Rn , and B = (B1 , . . . , BN ) where Bi = Bi (x, t, ∇u) : O × (S, T ) × RnN −→ R are Lebesgue (n + 1)-measurable functions on O × (S, T ). This is the case, for example, if Ai and Bi , i = 1, 2, . . . , N , satisfy the well-known Carathéodory-type conditions. We assume that there exist positive constants ci , i = 1, 2, 3, such that (2.3)

|Ai | ≤ c1 |∇u|p−1 + h1 ,

HIGHER INTEGRABILITY FOR PARABOLIC SYSTEMS

255

|Bi | ≤ c2 |∇u|p−1 + h2 ,

(2.4) and

N

(2.5)

Ai , ∇ui ≥ c3 |∇u|p − h3 ,

i=1

for i = 1, 2, . . . , N and almost every (x, t) ∈ O × (S, T ). Here ·, · denotes the standard inner product in Rn , and hi , i = 1, 2, 3, are measurable functions in O × (S, T ) so that (|h1 | + |h2 |)p/(p−1) + |h3 | = c4 < ∞, (2.6) q ,O×(S,T ) where q > 1. Finally, u satisfying (2.1) is said to be a weak solution in O ×(S, T ) to the nonlinear parabolic system ∂ui = div Ai (x, t, ∇u) + Bi (x, t, ∇u), ∂t

i = 1, . . . , N,

if the structural conditions (2.3)–(2.6) hold and (2.7)

T S

N O i=1

− ui

∂φi + Ai , ∇φi − Bi φi dx dt = 0 ∂t

for every test function φ = (φ1 , . . . , φN ) ∈ C0∞ (O × (S, T )). The following theorem is our main result. Theorem 2.8. Let p > 2n/(n + 2) and suppose that u is a weak solution to (1.1). Then there exists ε > 0 such that u ∈ L2 Ω × [S1 , T1 ] ∩ Lp+ε [S1 , T1 ]; W 1,p+ε (Ω) , where ε > 0 depends only on n, p, q , and ci , for i = 1, 2, 3, while |||u|||p+ε,Ω depends on these quantities as well as N, Ω, S1 , T1 , and c4 . The proof of our main result follows from two propositions in Section 4. 3. Fundamental estimates. In this section, we state and outline the proofs of some Sobolev- and Caccioppoli-type lemmas that are used in the proof of the main result. To do this, we need some notation. Given r, s > 0, (x, t) ∈ Rn+1 , let Dr (x) = {y ∈ Rn : |y − x| < r} denote the open ball in Rn and let Qr,s (x, t) = Dr (x) × (t − s, t + s)

256

KINNUNEN AND LEWIS

denote a cylinder in Rn+1 . Let |E| denote the Lebesgue (n + 1)-measure of the measurable set E, and if f is integrable on E with 0 < |E| < ∞, then the integral average of f over E is 1 f dx dt = f dx dt. |E| E E If Qρ,s (z, τ ) ⊂ O × (S, T ), then −1 Iρ (t) = Iρ (t, u, z, τ ) = m Dρ (z)

Dρ (z)

u(x, t) dx,

whenever τ −s < t < τ +s. Here m denotes Lebesgue measure in Rn , and the integral is taken componentwise. Lemma 3.1. Suppose that u is a weak solution to (1.1). If Q4ρ,s (z, τ ) ⊂ O × (S, T ), then there exists ρ , ρ < ρ < 2ρ, and a constant c depending on p, n, c1 , and c2 , such that Iρ(t2 ) − Iρ(t1 ) ≤ csρ −1 |∇u|p−1 + |h1 | + |h2 | dx dt Q2ρ,s (z,τ )

for almost all ti , τ − s < ti < τ + s, i = 1, 2. Proof. To prove this lemma, let δ, η > 0 be small, ρ < ρ < 2ρ, t1 < t2 , ψ1 ∈ C0∞ (t1 − η, t2 + η) with ψ1 = 1 on the interval (t1 , t2 ), and let ψ2 ∈ C0∞ (Dρ+δ (z)) be a radial function with ψ2 = 1 on Dρ(z). For fixed j = 1, 2, . . . , N , we put φj = ψ1 ψ2 and φi = 0 otherwise. Using (2.7) and letting first η → 0 and then δ → 0, we get from well-known Sobolev-type arguments that for almost every t1 , t2 , and ρ , as above, m Dρ(z) Iρ(t2 , uj ) − Iρ(t1 , uj )

−1 x − z, Aj (x, t) |x − z| dσ (x) dt + = ∂Dρ(z)×(t1 ,t2 )

Dρ(z)×(t1 ,t2 )

Bj dx dt.

Here σ denotes (n − 1)-dimensional surface area on ∂Dρ(z). Choose ρ , ρ < ρ < 2ρ, so that |∇u|p−1 + |h1 | + |h2 | dσ dt ∂Dρ(z)×(t1 ,t2 )

≤ 100ρ −1

D2ρ (z)×(t1 ,t2 )

|∇u|p−1 + |h1 | + |h2 | dx dt.

Using this choice, (2.3) and (2.4) in the above inequality, and summing over j = 1, 2, . . . , N , we deduce the claim.

257


The following lemma is a Caccioppoli-type estimate for parabolic systems of pLaplacian type. For short, we write p/(p−1) hp = |h1 | + |h2 | + |h3 |. Lemma 3.2. Let u be a weak solution to (1.1) and a = (a1 , . . . , aN ) ∈ RN . Then there exists a constant c depending on n, N , p, ci for i = 1, 2, 3, 4, such that if Qρ1 ,s1 (z, τ ) ⊂ Qρ2 ,s2 (z, τ ) ⊂ O × (S, T ) with 0 < ρ2 , s2 < 1, then we have |∇u|p dx dt + ess sup |u − a|2 dx Qρ1 ,s1 (z,τ )

≤ c(s2 − s1 )−1

t∈(τ −s1 ,τ +s1 ) Dρ1 (z)

Qρ2 ,s2 (z,τ )

+ c(ρ2 − ρ1 )−p

|u − a|2 dx dt

Qρ2 ,s2 (z,τ )

|u − a|p dx dt + c

Qρ2 ,s2 (z,τ )

hp dx dt.

Proof. Lemma 3.2 follows from a standard Caccioppoli-type estimate obtained from (2.7) by formally choosing test functions of the form p

φi = (u − a)i ψi , i = 1, 2, . . . , N, where ψi ∈ C0∞ Qρ2 ,s2 (z, τ ) is a cutoff function with ψi = 1 on Qρ1 ,s1 (z, τ ), 0 ≤ ψi ≤ 1, and −1 −1 ∂ψi < 1000. |∇ψi | ∞ + (s2 − s1 ) (ρ2 − ρ1 ) ∂t ∞ There is a difficulty with the test functions φi since the solution usually has a very modest degree of regularity with respect to the time variable. We refer the reader to [D, pp. 24–27] for an argument to overcome this difficulty. Next we prove a Sobolev-type inequality. Lemma 3.3. Let 1 ≤ ν < ∞ and suppose that u ∈ Lν ((τ − 2s, τ + 2s); 2ρ (z))). Then there is a constant c depending on n and ν such that

W 1,ν (D

Qρ,s (z,τ )

u(x, t) − Iρ (t) ν(1+2/n) dx dt

≤c

Q2ρ,2s (z,τ )

|∇u(x, t)|ν dx dt ·

ess sup

t∈(τ −2s,τ +2s) D2ρ (z)

u(x, t) − Iρ (t) 2 dx

ν/n

.

Proof. Let t, τ −2s < t < τ +2s be such that x → u(x, t) belongs to W 1,ν (D2ρ (z)), and denote ρ ∗ = 2ρ. Let ψ ∈ C0∞ (Q2ρ,2s (z, τ )) be a cutoff function such that ψ = 1 on Qρ,s (z, τ ) and |∇ψ| ≤ 10/ρ. Let v(x, t) = u(x, t) − Iρ (t) ψ(x, t).

258

KINNUNEN AND LEWIS

Hölder’s inequality implies that J= v(x, t)ν(1+2/n) dx Dρ ∗ (z)

≤

1/n v(x, t) dx 2

Dρ ∗ (z)

Dρ ∗ (z)

v(x, t)

(ν+(2/n)(ν−1))n/(n−1)

dx

(n−1)/n

.

We use Sobolev’s theorem for functions in W 1,1 (Dρ ∗ (z)) to deduce that there is constant c = c(n) such that (n−1)/n v(x, t)(ν+(2/n)(ν−1))n/(n−1) dx Dρ ∗ (z)

≤c

Dρ ∗ (z)

v(x, t)(ν−1)(1+2/n) ∇v(x, t) dx

≤c

Dρ ∗ (z)

Thus J ≤ cJ

(ν−1)/ν

v(x, t)

Dρ ∗ (z)

Clearly,

ν

Dρ ∗ (z)

ν(1+2/n)

dx

(ν−1)/ν Dρ ∗ (z)

∇v(x, t) ν dx

1/ν

|∇v(x, t)| dx

≤ cρ

∗ −1

1/ν Dρ ∗ (z)

Dρ ∗ (z)

Dρ ∗ (z)

1/ν .

1/n v(x, t) dx

.

2

+

∇v(x, t) ν dx

u(x, t) − Iρ (t) ν dx

∇u(x, t) ν dx

1/ν

1/ν .

Poincaré’s inequality in W 1,ν (Dρ ∗ (z)) implies that 1/ν u(x, t) − Iρ (t) ν dx Dρ ∗ (z)

≤

Dρ ∗ (z)

u(x, t) − Iρ ∗ (t) ν dx

≤c and hence

Dρ ∗ (z)

Dρ ∗ (z)

1/ν

u(x, t) − Iρ ∗ (t) ν dx

∇v(x, t) ν dx

1/ν + Iρ ∗ (t) − Iρ (t) m Dρ ∗ (z)

1/ν

1/ν

≤ cρ

∗

≤c

Dρ ∗ (z)

Dρ ∗ (z)

∇u(x, t) ν dx

∇u(x, t) ν dx

1/ν .

1/ν ,

259


The same argument as above gives 1/n v(x, t)2 dx ≤c Dρ ∗ (z)

≤c

Dρ ∗ (z)

Dρ ∗ (z)

u(x, t) − Iρ (t) 2 dx

1/n

u(x, t) − Iρ ∗ (t) 2 dx

1/n .

Collecting the obtained estimates, we arrive at ν/n ∇u(x, t) ν dx u(x, t) − Iρ ∗ (t) 2 dx J ≤c . Dρ ∗ (z)

Dρ ∗ (z)

The claim follows by integrating this inequality with respect to t over the interval (τ − 2s, τ + 2s). Observe that the proof applies to the case n = 1 as well. The following two lemmas are essential tools in proving our main result. We divide the discussion into two parts depending on whether p ≥ 2 or 2n/(n + 2) 0, s = λ2−p ρ 2 , and Q40ρ,402 s (z, τ ) ⊂ O × (S, T ). Denote Q = Qρ,s (z, τ ), Q = Q4ρ,42 s (z, τ ), and Q = Q20ρ,202 s (z, τ ). If there is c5 ≥ 1 such that −1 p p p c5 λ ≤ |∇u| + h dx dt ≤ c5 |∇u|p + hp dx dt ≤ c52 λp , Q

Q

then there is c ≥ 1 such that p |∇u| dx dt ≤ c Q

q

Q

|∇u| dx dt

p/q

+c

Q

hp dx dt,

where q = max{p − 1, pn/(n + 2)}. The constant c has the same dependence as the constant in Lemma 3.2, except that it also depends on c5 . Proof. First suppose that p ≥ 2. From Lemma 3.2 with ρ1 = ρ, s1 = s, ρ2 = 2ρ, and s2 = 2s, we have |∇u|p dx dt ≤ cs −1 |u − a|2 dx dt Qρ,s (z,τ )

(3.5)

Q2ρ,2s (z,τ )

+ cρ −p

Q2ρ,2s (z,τ )

|u − a|p dx dt + c

= T 1 + T2 + c

Q2ρ,2s (z,τ )

Q2ρ,2s (z,τ )

hp dx dt

hp dx dt.

Since p ≥ 2, we may estimate T1 in terms of T2 using Hölder’s and Young’s inequalities and the assumption that s = λ2−p ρ 2 as p−2 −2 (3.6) ρ |u − a|2 dx dt ≤ λp /(4c5 ) + cT2 , T1 ≤ cλ Q2ρ,2s (z,τ )

260

KINNUNEN AND LEWIS

where c ≥ 1 has the same dependence as c in Lemma 3.2, except that it also depends on c5 . Hence it is enough to estimate T2 . By Lemma 3.1, we choose ρ , 2ρ < ρ < 4ρ, so that Iρ(t) − Iρ(ξ ) ≤ csρ −1 (3.7) |∇u|p−1 + |h1 | + |h2 | dx dt, Q

= Qρ,2s (z, τ ), and in (3.5) take for almost every ξ , τ − 2s < ξ < τ + 2s. Let Q a = a Q = a1 Q , . . . , aN Q , where ai Q = ui dx dt, Q

for i = 1, 2, . . . , N . Then we have −p u − Iρ(t) p dx dt + cρ −p (3.8) T2 ≤ cρ Q

ess sup

t∈(τ −2s,τ +2s)

p Iρ(t) − a Q .

We begin with estimating the second term on the right side of (3.8). Using (3.7), we have τ +2s Iρ(t) − a Q Iρ(t) − Iρ(ξ ) dξ ≤ (4s)−1 (3.9) τ −2s ≤ csρ −1 |∇u|p−1 + |h1 | + |h2 | dx dt, Q

and hence using the definition of λ, we obtain cρ −p

ess sup

t∈(τ −2s,τ +2s)

p Iρ(t) − a Q ≤ cλp(2−p) ≤c

(3.10)

≤c

|∇u|p−1 + |h1 | + |h2 | dx dt

Q

|∇u|p−1 + |h1 | + |h2 | dx dt

Q

Q

|∇u|

p−1

p/(p−1) dx dt +c

p

p/(p−1)

Q

hp dx dt.

Observe that the assumption p ≥ 2 is used in the second inequality above. Next we estimate the first term on the right side of (3.8). Lemma 3.3 implies that (3.11) u − Iρ(t) p dx dt ≤ c |∇u|q dx dt Q

Q

ess sup

t∈(τ −4s,τ +4s) Dρ(z)

= Q2 where q = pn/(n + 2), Q = 2 ρ. ρ ,4s (z, τ ), and ρ

u − Iρ(t) 2 dx

q/n

,

261


We estimate the essential supremum on the right side of (3.11). Let Q∗ = Q10ρ,10s (z, τ ). Clearly, u − Iρ(t) 2 dx ≤ c u − a Q∗ 2 dx + cm Dρ(z) a Q∗ − Iρ(t) 2 Dρ(z)

≤c

Dρ(z)

Dρ(z)

u − a Q∗ 2 dx.

Hence, using Lemma 3.2 with a = a(Q∗ ), we have u − Iρ(t) 2 dx ess sup t∈(τ −4s,τ +4s) Dρ(z)

≤c

(3.12)

ess sup

t∈(τ −4s,τ +4s) Dρ(z)

≤ cs −1 where s (3.13)

and

(3.14)

−1

Q∗

Q∗

u − a Q∗ 2 dx

u − a Q∗ 2 dx dt + cρ −p

u − a Q∗ 2 dx dt ≤ cs −1 + cs

+ cρ

Q∗

ess sup

t∈(τ −10s,τ +10s)

Q∗ −p ∗

Q

ess sup

t∈(τ −10s,τ +10s)

τ −10s

≤ cs −1 ρ 2

(3.15)

I10ρ (t) − a Q∗ 2

u − I10ρ (t) p dx dt

By Poincaré’s inequality in W 1,2 (D10ρ (z)), we have τ +10s u − I10ρ (t) 2 dx dt = s −1 s −1 Q∗

Q∗

u − a Q∗ p dx dt,

u − I10ρ (t) 2 dx dt

−1

p ρ −p u − a Q∗ dx dt ≤ cρ −p Q∗

Q∗

≤ cs −1 ρ 2

D10ρ (z)

Q∗

I10ρ (t) − a Q∗ p .

u − I10ρ (t) 2 dx dt

|∇u|2 dx dt

Q∗

|∇u|p dx dt

2/p

|Q∗ |

≤ cρ n+2 λ2 . Here we used the assumption that p ≥ 2 again. Exactly the same argument gives u − I10ρ (t) p dx dt ≤ c ρ −p (3.16) |∇u|p dx dt ≤ cρ n+2 λ2 . Q∗

Q∗

262

KINNUNEN AND LEWIS

Using Lemma 3.1, we choose ρ , 10ρ < ρ < 20ρ, such that Iρ(t) − Iρ(ξ ) ≤ csρ −1 |∇u|p−1 + |h1 | + |h2 | dx dt, Q

when τ − 10s < ξ < τ + 10s. This implies that I10ρ (t) − a Q∗ 2 ess sup s −1 Q∗ t∈(τ −10s,τ +10s)

(3.17)

2 n−2

≤ cs ρ

Q

p

p

2(p−1)/p

|∇u| + h dx dt

≤ cρ n+2 λ2 . A similar argument (see (3.10)) also gives I10ρ (t) − a Q∗ p ess sup ρ −p Q∗ t∈(τ −10s,τ +10s) n

(3.18)

≤ cρ sλ

p(2−p)

Q

|∇u|

p−1

+ |h1 | + |h2 | dx dt

p

≤ cρ n+2 λ2 . Using (3.12)–(3.18), we conclude that u − Iρ(t) 2 dx ≤ cρ n+2 λ2 . ess sup t∈(τ −4s,τ +4s) Dρ(z)

By (3.11) and Young’s inequality, we see that the first term on the right side of (3.8) can be estimated as p cρ −p u − Iρ(t) dx dt ≤ cλ2q/n |∇u|q dx dt Q

Q

≤c

q

Q

|∇u| dx dt

p/q

+ λp /(4c5 ).

Finally, using (3.5), (3.6), (3.8), and (3.10), we have (3.19)

p

Q

p

|∇u| dx dt ≤ λ /(2c5 ) + c +c

Q

q

Q

|∇u| dx dt

|∇u|p−1 dx dt

p/q

p/(p−1)

+c

Q

hp dx dt.

The claim follows from this estimate by absorbing the term containing λp into the left side.

263


Next we prove an analogue of Lemma 3.4 for 2n/(n + 2) 0, s = λ2−p ρ 2 , and Q40ρ,402 s (z, τ ) ⊂ O × (S, T ). Denote Q = Qρ,s (z, τ ), Q = Q4ρ,42 s (z, τ ), and Q = Q20ρ,202 s (z, τ ). If there is c6 ≥ 1 such that 2 −1 p c6 λ ≤ |∇u|p + s −1 u − a(Q) + hp dx dt Q

2 |∇u|p + s −1 u − a Q + hp dx dt ≤ c62 λp , ≤ c6 Q

then there is c ≥ 1 such that |∇u|p dx dt ≤ c Q

Q

|∇u|q dx dt

p/q

+c

Q

hp dx dt,

where q = 2n/(n + 2). The constant c has the same dependence as the constant in Lemma 3.2, except that it also depends on c6 . Proof. We use the same notation as in the proof of Lemma 3.4. Clearly, (3.5) also holds this case. Since p < 2, we use Hölder’s inequality to estimate T2 in (3.5) in terms of T1 and obtain (3.21)

T2 ≤ c ρ −2

Q2ρ,2s (z,τ )

|u − a|2 dx dt

p/2

p/2

≤ cλ(1−p/2)p T1

≤ λp /(4c6 ) + cT1 .

To estimate T1 by Lemma 3.1, we choose ρ , 2ρ < ρ < 4ρ, so that (3.7) holds. Let Q = Qρ,2s (z, τ ). Using the same argument that led to (3.8), we see that 2 −1 u − Iρ(t) 2 dx dt + cs −1 ess sup Iρ(t) − a Q . T1 ≤ cs (3.22) Q

t∈(τ −2s,τ +2s)

Using (3.9), the definition of λ, and Young’s inequality, we obtain s −1

ess sup

t∈(τ −2s,τ +2s)

≤ cλ2−p (3.23)

2 Iρ(t) − a Q

|∇u|p−1 + |h1 | + |h2 | dx dt

Q

≤ λp /(4c6 ) + c ≤ λp /(4c6 ) + c

2

|∇u|p−1 + |h1 | + |h2 | dx dt

Q

Q

|∇u|p−1 dx dt

p/(p−1)

p/(p−1)

+c

Q

hp dx dt.

264

KINNUNEN AND LEWIS

To estimate the first term on the right side of (3.22), we use Lemma 3.3 and argue first as in (3.11) to get u − Iρ(t) 2 dx dt (3.24)

Q

≤c

q

Q

|∇u| dx dt

u − Iρ(t) 2 dx

ess sup

t∈(τ −4s,τ +4s) Dρ(z)

q/n

,

= Q2 = 2 ρ . The essential supremum on where q = 2n/(n + 2), Q ρ ,4s (z, τ ), and ρ the right side of (3.24) is then estimated as in (3.12), and we obtain u − Iρ(t) 2 dx ess sup (3.25)

t∈(τ −4s,τ +4s) Dρ(z)

≤ cs

−1

Q∗

u − a Q∗ 2 dx dt + cρ −p

Q∗

u − a Q∗ p dx dt,

where Q∗ = Q10ρ,10s (z, τ ) as before. Using the assumption of the lemma and remembering that s = λ2−p ρ 2 , we have −1 u − a Q∗ 2 dx dt ≤ cλp Q∗ ≤ cρ n+2 λ2 . s Q∗

The second term on the right side of (3.25) can be estimated exactly the same way as in the case p ≥ 2; see (3.14), (3.16), and (3.18). We conclude that u − Iρ(t) 2 dx ≤ cρ n+2 λ2 . ess sup t∈(τ −4s,τ +4s) Dρ(z)

By (3.24) and Young’s inequality, we arrive at 2 −1 p−q cs u − Iρ(t) dx dt ≤ cλ |∇u|q dx dt (3.26)

Q

Q

≤c

q

Q

|∇u| dx dt

p/q

+ λp /(4c6 ).

The claim now follows from using (3.22), (3.23), and (3.26) as before by absorbing the term containing λp into the left side. This completes the proof. Remark 3.27. We record for future reference that the constant c in Lemmas 3.4 and 3.20 remain bounded above if p is in a compact subset of (2n/(n + 2), ∞). This is easily seen by analyzing the constants in Lemmas 3.1, 3.2, and 3.3. 4. Reverse Hölder inequalities. In this section, we show that gradients of weak solutions of (1.1) satisfy a reverse Hölder inequality provided p > 2n/(n + 2). As in the previous section, slightly different arguments are needed to handle the cases p ≥ 2 and 2n/(n + 2) 0 and c ≥ 1 having the same dependence as the corresponding constants in Theorem 2.8 with QR,R p (z,τ )

|∇u|

p+ε

1/(p+ε) σp−1 dx dt ≤ cR + cR

−1

p

Q2R,(2R)p (z,τ )

|∇u| dx dt

+c

Q2R,(2R)p (z,τ )

h

p+ε

σ 1/(p+ε)

dx dt

,

where σ = (2 + ε)/(2(p + ε)). Proof. To prove Proposition 4.1, we assume, as we may, that R = 1 and (z, τ ) = (0, 0), since otherwise we consider v(x, t) = u(z + Rx, τ + R p t) for (x, t) ∈ Q4,4p (0, 0). It is easily seen that v is a weak solution to a partial differential equation similar to and with the same structure as (1.1). Proving Proposition 4.1 for v with R = 1 relative to (0,0) and then transforming back, we get Proposition 4.1 for u. = Q2,2p (0, 0). To begin the proof of Proposition 4.1, we For short, we denote Q into Whitney-type cylinders divide Q Qi = Qri ,r 2 (zi , τi ), i

i = 1, 2, . . . ,

Let us so that ri is comparable to the parabolic distance of Qi to the boundary of Q. n+1 recall that the parabolic distance of sets E, F ⊂ R is inf |x − y| + |t − s|1/2 : (x, t) ∈ E and (y, s) ∈ F . Moreover, the cylinders Qi , i = 1, 2, . . . , are of bounded overlap and Q5ri ,(5ri )2 (zi , τi ) ⊂ Q. we define Next for (x, t) ∈ Q, g(x, t) = |∇u| + h (x, t) and

f (x, t) = c −1 min |Qi |1/2 : (x, t) ∈ Qi g(x, t),

where c ≥ 1 is chosen later. Let

λ20 =

Q

|g|p dx dt

266

KINNUNEN AND LEWIS

and λ > max{λ0 , 1} = λ0 . Suppose that (x, t) ∈ Qi with |f (x, t)| > λ. We set α = |Qi |−1

and

γ = α 1−p/2 λ2−p .

c large enough, we have If ri /20 ≤ r ≤ ri , then for p −1 |g| dx dt ≤ cγ α (4.2) |g|p dx dt ≤ c p λp α p/2 . Q

Qr,γ r 2 (x,t)

By Lebesgue’s differentiation theorem, we have for almost every such (x, t), that |g|p dx dt > c p λp α p/2 . (4.3) lim r→0

Qr,γ r 2 (x,t)

From (4.2), (4.3), and continuity of the integral, we see that there exists ρ, 0 < ρ < ri /20, such that (4.4) |g|p dx dt = c p λp α p/2 Qρ,γρ 2 (x,t)

and (4.5)

Qr,γ r 2 (x,t)

|g|p dx dt ≤ c p λp α p/2

for ρ ≤ r ≤ ri . Let s = γρ 2 and denote Q = Qρ,s (x, t),

Q = Q4ρ,42 s (x, t),

and

Q = Q20ρ,202 s (x, t).

Since λ, α > 1 and p ≥ 2, we have γ ≤ 1. This implies that Q ⊂ Q. Now (4.4) and (4.5) imply that there is a constant c ≥ 1 such that −1 p p/2 p c λ α (4.6) ≤ |g| dx dt ≤ c |g|p dx dt ≤ c2 λp α p/2 . Q

Observe that

Q

2−p 2 ρ . s = γρ 2 = λα 1/2

Thus we can apply Lemma 3.4 with λ replaced by λα 1/2 . Note that c5 in this case depends only on n and p. From Lemma 3.4, we conclude for pn q = max p − 1, n+2

267


and c ≥ 1 that (4.7)

p

|∇u| dx dt ≤ c

Q

q

Q

|∇u| dx dt

p/q

+c

Q

hp dx dt.

Using (4.6) and (4.7), we have c−1 λp ≤ |f |p dx dt Q

≤c

(4.8)

≤ c2

Q

Q

|f |q dx dt

p/q

+c

Q

k dx dt

|f |p dx dt ≤ c3 λp ,

where k(x, t) = min |Qi |p/2 : (x, t) ∈ Qi hp (x, t). : |f (x, t)| > λ and η > 0. Then by (4.8), we obtain Let G(λ) = (x, t) ∈ Q Q

|f |q dx dt

p/q

−1 ≤ cηp λp + Q ≤ cη

(4.9)

p

q

|f | dx dt

Q

p−q

+ cλ

Q ∩G(ηλ)

Q

−1

p/q

|f |q dx dt

+ cη

Q ∩G(ηλ)

p

p/q

Q

k dx dt

|f |q dx dt.

A similar argument gives (4.10)

Q

k dx dt ≤ cηp

+ cηp

Q

Q

|f |q dx dt

p/q

−1 k dx dt + c Q

Q ∩G(ηλ)

k dx dt.

Choosing η > 0 small enough in (4.9), (4.10), and absorbing terms, we arrive at p/q q |f | dx dt + k dx dt Q Q (4.11) −1 p−q −1 q |f | dx dt + c Q k dx dt. Q ≤ cλ Q ∩G(ηλ)

Q ∩G(ηλ)

An examination of the proof of the well-known Vitali-type covering lemma shows

268

KINNUNEN AND LEWIS

that we can choose pairwise disjoint cylinders Qi = Q4ρi ,γ (4ρi )2 (xi , ti ),

i = 1, 2, . . . ,

such that almost everywhere G(λ) ⊂

∞ i=1

where

Qi ⊂ Q,

Qi = Q20ρi ,γ (20ρi )2 (xi , ti ),

i = 1, 2, . . . .

From (4.8) and (4.11), we deduce that for some small η > 0, we have c−1 λp ≤ |f |p dx dt Qi

p−q

≤ cλ

(4.12)

q

Qi

|f | dx dt + c

≤ c2 λp−q |Qi |−1

Qi ∩G(ηλ)

≤ c 3 λp .

Qi

k dx dt

|f |q dx dt + c2 |Qi |−1

Qi ∩G(ηλ)

k dx dt

Multiplying (4.12) by |Q | and summing over i, we get from (4.12) and disjointness of the cylinders Qi , i = 1, 2, . . . , that |f |p dx dt ≤ |f |p dx dt G(λ)

Qi

i

≤ cλp−q

(4.13)

≤ cλ

p−q

i

Qi ∩G(ηλ)

|f |q dx dt +

q

G(ηλ)

i

|f | dx dt + c

G(ηλ)

Qi ∩G(ηλ)

k dx dt

k dx dt.

We can now apply a standard argument to complete the proof of Proposition 4.1. For completeness, we sketch it. Using Fubini’s theorem and (4.13), we have |f |p+ε dx dt G(λ0 ) ∞ ε−1 p ε λ |f | dx dt dλ + λ0 |f |p dx dt =ε λ0 G(λ) G(λ0 ) ∞ λε−1+p−q |f |q dx dt dλ ≤ cε λ0

+ cε

G(ηλ)

∞ λ0

λ

ε−1+p−q

G(ηλ)

k dx dt

ε dλ + λ0

G(λ0 )

|f |p dx dt

269


≤

cε ε+p−q

G(

λ0

)

|f |p+ε dx dt + c

G(

λ0

)

|f |ε k dx dt + λ0

ε

G(λ0 )

|f |p dx dt.

By Young’s inequality, we obtain |f |ε k dx dt ≤ ε |f |ε+p dx dt + c k 1+ε/p dx dt. G(λ0 ) G(λ0 ) G(λ0 ) Choosing ε > 0 small enough, we may absorb the integrals involving |f |p+ε into the left side and we obtain p+ε ε p dx dt ≤ cλ0 |f | |f | dx dt + c k 1+ε/p dx dt. G(λ0 ) G(λ0 ) G(λ0 ) Observe that there is a difficulty in moving terms to the left side since they may be infinite. This technical problem can be treated, for example, by truncating the function f . To be more precise, let > > λ0 and denote f> = min{|f |, >}. If |f | is replaced by f> in the definition of G(λ), we see that (4.13) holds with |f | replaced by f> , and we can go through the above argument. Now all absorbed terms are finite, and we obtain the claim passing to the limit as > → ∞. Thus p+ε ε p |f | dx dt ≤ λ0 |f | dx dt + |f |p+ε dx dt (λ ) Q Q\G G λ ( ) 0 0 ε |f |p dx dt + c k 1+ε/p dx dt. ≤ cλ0 Q

Q

Proposition 4.1 follows easily from this inequality. Then we prove a counterpart of Proposition 4.1 when 2n/(n + 2) 0 and c ≥ 1 having the same dependence as the corresponding constants in Theorem 2.8 with 1/(p+ε) |∇u|p+ε dx dt QR,R p (z,τ )

≤ cR −1 + cR

−1

Q2R,(2R)p (z,τ )

u − a Q2R,(2R)p (z, τ ) 2 dx dt

+c

p+ε

Q2R,(2R)p (z,τ )

h

1/(p+ε) dx dt

where ν = (2ε + (n + 2)p − 2n)/((p + ε)((n + 2)p − 2n)).

,

ν

270

KINNUNEN AND LEWIS

= Q2,2p (0, 0) into Whitney-type cylinders Qi , i = Proof. We again divide Q 1, 2, . . . , exactly in the same way as in the proof of Proposition 4.1. Next for (x, t) ∈ put Q, g(x, t) = |∇u| + h (x, t). Let ((n+2)p−2n)/2 λ0

(4.15)

=

Q

2 u − a Q dx dt

we define and λ > max{λ0 , 1} = λ0 . For (x, t) ∈ Q, f (x, t) = c −1 min |Qi |σ : (x, t) ∈ Qi g(x, t), where σ=

2n + 8 (n + 2) (n + 2)p − 2n

and c ≥ 1 are chosen later. Suppose (x, t) ∈ Qi with |f (x, t)| > λ. Put α = |Qi |−σ and let γ = (λα)2−p . Again by Lebesgue’s differentiation theorem, we have for almost every such (x, t) that lim |g|p dx dt > c p λp α p . r→0

Qr,γ r 2 (x,t)

and for Also, if r = (λα)p/2−1 ri with ri /20 ≤ ri ≤ ri , then Qr,γ r 2 (x, t) ⊂ Q c large enough, we have from Lemma 3.2 and Hölder’s inequality, 2 |g|p + γ −1 r −2 u − a Qr,γ r 2 (x, t) dx dt Qr,γ r 2 (x,t)

−(n+4) (λα)(1−p/2)n ≤ c(n)ri

Q

2 dx dt < hp + 1 + u − a Q c p λp α p ,

since p > 2n/(n + 2). We again use continuity of the integral to find ρ, 0 < ρ < (λα)p/2−1 ri /20 such that 2 (4.16) |g|p + γ −1 ρ −2 u − a Qρ,γρ 2 (x, t) dx dt = c p λp α p Qρ,γρ 2 (x,t)

and (4.17)

Qr,γ r 2 (x,t)

2 |g|p + γ −1 r −2 u − a Qr,γ r 2 (x, t) dx dt ≤ c p λp α p

for ρ ≤ r ≤ (λα)p/2−1 ri . From (4.16) and (4.17), we see that Lemma 3.20 can be applied with Q = Qρ,γρ (x, t) and λ replaced by λα. We can now repeat the proof of Proposition 4.1 essentially verbatim from (4.8) on in order to get Proposition 4.14. We omit the details.


271

The proof of Theorem 2.8 follows easily from Propositions 4.1, 4.14, and Lemma 3.3. References [C] [D] [Ge] [Gi] [GM] [GS] [ME] [Str]

H. J. Choe, On the regularity of parabolic equations and obstacle problems with quadratic growth nonlinearities, J. Differential Equations 102 (1993), 101–118. E. DiBenedetto, Degenerate Parabolic Equations, Universitext, Springer-Verlag, New York, 1993. F. W. Gehring, The Lp -integrability of the partial derivatives of a quasiconformal mapping, Acta Math. 130 (1973), 265–277. M. Giaquinta, Multiple Integrals in the Calculus of Variations and Nonlinear Elliptic Systems, Ann. of Math. Stud. 105, Princeton University Press, Princeton, 1983. M. Giaquinta and G. Modica, Regularity results for some classes of higher order nonlinear elliptic systems, J. Reine Angew. Math. 311/312 (1979), 145–169. M. Giaquinta and M. Struwe, On the partial regularity of weak solutions of nonlinear parabolic systems, Math. Z. 179 (1982), 437–451. N. G. Meyers and A. Elcrat, Some results on regularity for solutions of non-linear elliptic systems and quasi-regular functions, Duke Math. J. 42 (1975), 121–136. E. W. Stredulinsky, Higher integrability from reverse Hölder inequalities, Indiana Univ. Math. J. 29 (1980), 407–413.

Kinnunen: Department of Mathematics, P.O. Box 4, FIN-00014 University of Helsinki, Finland; [email protected] Lewis: Department of Mathematics, University of Kentucky, Lexington, Kentucky 40506-0027, U.S.A.; [email protected]

Vol. 102, No. 2


© 2000

INTEGRALITY AND SYMMETRY OF QUANTUM LINK INVARIANTS THANG T. Q. LE

0. Introduction. Quantum invariants of framed links whose components are colored by modules of a simple Lie algebra g are Laurent polynomials in v 1/D (with integer coefficients), where v is the quantum parameter and D an integer depending on g. We show that quantum invariants, with a suitable normalization, are Laurent polynomials in v 2 . We also establish two symmetry properties of quantum link invariants at roots of unity. The first asserts that quantum link invariants, at rth roots of unity, are invariant under the action of the affine Weyl group Wr , which acts on the weight lattice. A fundamental domain of Wr is the fundamental alcove C¯ r , a simplex. Let G be the center of the corresponding simply connected complex Lie group. There is a natural action of G on C¯ r . The second symmetry property, in its simplest form, asserts that quantum link invariants are invariant under the action of G if the link has zero linking matrix. The second symmetry property generalizes symmetry principles of Kirby and Melvin (the sl2 case) and Kohno and Takata (the sln case) to arbitrary simple Lie algebra. 0.1. Quantum invariants. Suppose L is a framed link with m ordered components and M1 , . . . , Mm are modules of a simple complex Lie algebra g. Then the quantum invariant JL (M1 , . . . , Mm ) is a rational function in the variable v 1/D , where v is the quantum parameter and D is a number depending on g. (See [RT1], [Tu]; we recall the definition of quantum invariants in §1.) The Jones polynomial (see [Jo]) is the simplest in the family of quantum link invariants: When g = sl2 and the modules equal the fundamental representation, JL is the Jones polynomial, with a suitable change of variable. The reader should be able to relate v to any other variable if it is known that the quantum integer [n] is given by [n] =

v n − v −n . v − v −1

0.2. Integrality. A priori JL is a rational function in v 1/D . Lusztig’s result on the integrality of the R-matrix implies that JL is a Laurent polynomial in v 1/D with integer coefficients (see a detailed proof in §1.4.2 below). We study the integrality of the exponents of v. One of our main results shows that JL is essentially a Laurent Received 13 April 1999. 1991 Mathematics Subject Classification. Primary 57M25; Secondary 17B37. Author’s work partially supported by National Science Foundation grant number DMS-9626404. 273

274

THANG T. Q. LE

polynomial in v 2 . More precisely, suppose all the modules M1 , . . . , Mm are irreducible; then JL belongs to v p Z[v ±2 ], where p is a rational number determined by the linking matrix of L (see the strong integrality theorem in §2). Thus one can get rid of fractional and odd powers of v by using a suitable normalization. For example, suppose that the normalization of the quantum invariant is chosen so that the value of the unknot is 1; then the value of any unframed knot is in Z[v ±2 ]. The strong integrality theorem does not follow directly from the integrality result of Lusztig. To prove it, we have to use a geometric lemma about special presentation of links and a result of Andersen on quantum groups at roots of unity. 0.3. Symmetry I. To formulate the symmetry properties, it is more convenient to use another normalization of quantum invariants, QL (M1 , . . . , Mm ) := JL (M1 , . . . , Mm )JU (m) (M1 , . . . , Mm ), where U (m) is the trivial link with m components and each has zero framing. This normalization is the one used in the definition of quantum 3-manifold invariants. Since irreducible g-modules are parametrized by the set X+ of dominant weights, both JL and QL can be considered as functions from (X+ )m to Z[v ±1/D ]. The set X+ is the part of the weight lattice X which lies in a Euclidean space h∗R . For each positive integer r there is defined the fundamental alcove C¯ r , which is a simplex in h∗R (see §2). The reflections along the facets of C¯ r generate the affine Weyl group Wr , for which C¯ r is a fundamental domain. The affine Weyl group plays an important role in the theory of affine Lie algebras (see [Kac]). We show that when v 2 is a primitive rth root of unity, quantum invariants have very nice symmetry properties expressed in the first and the second symmetry principles. The first asserts that QL is componentwise invariant under the action of the affine Weyl group. More precisely, when v 2 is an rth root of unity, ¯ wm ·µm = QL ¯ µm , ¯ w1 ·µ1 , . . . , ¯ µ1 , . . . , QL ¯ µ is the simple module of highest weight µ. Here the where w1 , . . . , wm ∈ Wr and dot means the dot action, and all µ1 , . . . , µm , w1 · µ1 , . . . , wm · µm are in X+ . For a stronger statement that describes the maximal group of symmetry, see §2. Thus when considering quantum invariants at roots of unity, one could restrict the colors—that is, the modules assigned to components of links—to C¯ r , a fundamental domain of the affine Weyl group. The simplex C¯ r contains only a finite number of elements in X+ . For example, the sum over all weights in X+ could be replaced by the sum over all weights in C¯ r . This happens in the theory of quantum 3-manifold invariants. 0.4. Symmetry II. Let G be the (necessarily finite abelian) center of the simply connected complex Lie group associated with g. The group G is also known as the fundamental group; it is isomorphic to the quotient of the weight lattice by the root

QUANTUM LINK INVARIANTS

275

lattice. There is a natural action of G on C¯ r (see §2). Suppose v 2 is a primitive rth root of unity. In its simplest form, the second symmetry principle says that QL is invariant under the action of G if the linking matrix of L is zero. In general, under the action of G, QL is multiplied by a twisting factor determined by the linking matrix of L. More precisely, suppose µ1 , . . . , µm ∈ C¯ r , g1 , . . . , gm ∈ G; then ¯ g1 ·µ1 , . . . , ¯ µ1 , . . . , ¯ gm ·µm = v rt QL ¯ µm , QL where the dot action is, as usual, the one shifted by the half-sum of positive roots and where t = (r − h) lij gi | gj + 2 lij gi | µj , 1≤i, j ≤m

1≤i, j ≤m

with h being the Coxeter number (see Table 1). Here (lij ) is the linking matrix, and (· | ·)’s are scalar products naturally defined using the standard scalar product on g. For a stronger statement, see §2. The action of G is induced from that of the extended affine Weyl group. The second symmetry principle, in fact, describes how quantum invariants behave under actions of the extended affine Weyl group. For g = sl2 , the second symmetry principle was discovered by Kirby and Melvin [KM] and for g = sln by Kohno and Takata [KT1]. Our contribution in these cases is the explicit relation between the twisting factor and the scalar product of g. This relation makes the second symmetry principle more understandable and easier to deal with. We also consider all primitive rth roots of unity, not only e2πi/r . Our proof is different from those of [KM] and [KT1], though it borrows some ideas from [KM]. In order to handle all simple Lie algebras, we have to use deep results of Lusztig and Andersen on quantum groups. 0.5. In [KM] and [KT2], the second symmetry principle was used to define a finer version—the projective version—of quantum 3-manifold invariants. The values of the projective version, so far defined only for g = sln , were proved to be algebraic integers (see [Mu], [MR], [TY]). Then Ohtsuki showed that, for g = sl2 , the projective version has a perturbative expansion, which is a power series invariant of rational homology 3-spheres; see [Oh1] (see also [Le2] for the sln case). This result led Ohtsuki to the definition of finite-type invariants of 3-manifolds (see [Oh2]). In a forthcoming paper [Le3], we will generalize these results to arbitrary simple Lie algebras. 0.6. Various properties of quantum link invariants were proved by first establishing the properties for fundamental modules and then using cablings (see, e.g., [MW], [Yo]). This approach has been widely used for classical Lie algebras (series ABCD), since the invariants corresponding to fundamental representations are essentially the Homflypt and the Kauffman polynomials, which have simple skein relations. The case of exceptional Lie algebras has not been well studied. We do not use that approach in this paper. To uniformly handle all simple Lie algebras we extensively utilize results in quantum group theory.

276

THANG T. Q. LE

Table 1 A"

B" " odd

B" " even

C"

D" " odd

D" " even

E6

E7

E8

F4

G2

d

1

2

2

2

1

1

1

1

1

2

3

D

"+1

2

1

1

4

2

3

2

1

1

1

G

Z"+1

Z2

Z2

Z2

Z4

Z2 × Z 2

Z3

Z2

1

1

1

h

"+1

2"

2"

2"

2" − 2

2" − 2

12

18

30

12

6

h∨

"+1

2" − 1

2" − 1

" + 1 2" − 2

2" − 2

12

18

30

9

4

0.7. The paper is organized as follows. In §1 we recall necessary facts about quantum groups and the definition of quantum link invariants. The integrality theorem, the two symmetry principles, their refinements, and their corollaries are presented in §2. Finally, §3 contains proofs of main theorems. Acknowledgements. The author would like to thank M. Finkelberg, C. Kassel, G. Masbaum, T. Ohtsuki, T. Takata, and H. Wenzl for helpful and stimulating discussions. He is grateful to H. Andersen for explaining many results in quantum group theory. He also thanks W. Menasco, who provided a proof of Proposition 3.6, and the referee, for valuable corrections and comments. 1. Quantum groups and quantum link invariants 1.1. Quantum groups. We recall here some facts from the theory of quantum groups, following [Lu2] (see also [Ka]). We do not use the h-adic version, so the R-matrix does not lie in the quantum group. 1.1.1. Cartan matrix and roots. Let (aij )1≤i,j ≤" be the Cartan matrix of a simple complex Lie algebra g. There are relatively prime integers d1 , . . . , d" in {1, 2, 3} such that the matrix (di aij ) is symmetric. Let d be the maximal of (di ). The reader uncomfortable with Lie algebra theory might want to consider only the case d = 1, that is, the simply laced case (series ADE), for which many formulas become much simpler. The values of d and other data for various Lie algebras are listed in Table 1. We fix a Cartan subalgebra h of g and basis roots α1 , . . . , α" in the dual space h∗ . Let h∗R be the R-vector space spanned by α1 , . . . , α" . The root lattice Y is the Z-lattice generated by αi , i = 1, . . . , ". Define the scalar product on h∗R so that (αi | αj ) = di aij . Then (α | α) = 2 for every short root α. Let Z+ be the set of all nonnegative integers. The weight lattice X (resp., the set of dominant weights X+ ) is the set of all λ ∈ h∗R such that λ, αi := (2(λ | αi ))/(αi | αi ) ∈ Z (resp., λ, αi ∈ Z+ ) for i = 1, . . . , ". Let λ1 , . . . , λ" be the fundamental weights; that is, the λi ∈ h∗R are defined by λi , αj = δij or (λi | αj ) = di δij . Then X is the

277


Z-lattice generated by λ1 , . . . , λ" . The root lattice Y is a subgroup of the weight lattice X, and the quotient G = X/Y is called the fundamental group. If µ ∈ X and α ∈ Y , then (µ | α) is always an integer. Let ρ be the half-sum of all positive roots. Then ρ = λ1 + · · · + λ" ∈ X+ . Finitedimensional simple g-modules are parametrized by X+ : for every λ ∈ X+ , there ¯ λ. corresponds a unique simple g-module 1.1.2. The Hopf algebra ᐁ and its integral form. Consider the algebra Ꮽ = Z[v, v −1 ] and its fractional field Q(v), where v is an indeterminate. The Hopf algebra ᐁ, known as the quantum group associated with g, is defined over Q(v) and is generated by Ei , Fi , Kα , with i = 1, . . . , " and α ∈ Y , subject to some relations. We refer the reader to [Lu2] for the set of relations and the definitions of the coproduct . and the antipode S; the precise formulas are not used in the sequel. Note that the coproduct in [Lu2] is the same as the one in [Tu], but opposite to the one in [Ka] and [KM]; correspondingly, our antipode is the inverse of that in [Ka] and [KM]. In [Lu2], the two lattices X, Y are in different spaces, dual to each other. Here we consider both X and Y as subsets of the same space h∗R (using the scalar product). One of the relations says that Kα+β = Kα Kβ = Kβ Kα and K0 = 1. Hence K−α = Kα−1 . Lusztig introduced an integral version Ꮽ ᐁ of ᐁ, similar to the Kostant Z-form of classical Lie algebras. For each positive integer p, let (p)

Ei

p

=

Ei , [p]i !

(p)

Fi

p

=

Fi , [p]i !

where [p]i ! =

p v di n − v −di n n=1

(p)

v di − v −di

.

(p)

Then Ꮽ ᐁ is the Ꮽ-subalgebra of ᐁ generated by Ei , Fi , Kα , with i = 1, . . . , n, p ∈ Z+ , and α ∈ Y . It is known that Ꮽ ᐁ inherits the Hopf algebra structure of ᐁ. 1.2. Category of ᐁ-modules 1.2.1. Finite-dimensional ᐁ-modules of type 1. Suppose M is a ᐁ-module. For every ν ∈ X, let M ν = x ∈ M | Kα (x) = v ν,α x for every root α . The subspace M ν is called the subspace of weight ν; its elements are vectors of weight ν. Let Ꮿ be the category of finite-dimensional (over Q(v)) ᐁ-modules M such that Mν. M= ν∈X (p)

(p)

It is known that on every M ∈ Ꮿ, both Ei and Fi equal zero for sufficiently large p. A morphism in Ꮿ is just a ᐁ-linear homomorphism. If M, N are in Ꮿ, then M ⊗N

278

THANG T. Q. LE

is also in Ꮿ. Thus Ꮿ is a tensor category (also known as a monoidal category; see, e.g., [Ka], [Tu]). The category Ꮿ is semisimple, and its simple objects are parametrized by the set of dominant weights X+ . For every λ ∈ X+ , there is a unique simple ᐁ-module λ ∈ Ꮿ, (p) with a vector x of weight λ such that Ei (x) = 0 for every i = 1, . . . , " and p ≥ 1. The module λ , called the module of highest weight λ, can be considered as the ¯ λ of the Lie algebra g. Every ᐁ-module deformation of the corresponding module in Ꮿ is the direct sum of simple modules of the form λ . The decomposition of the tensor product of two simple ᐁ-modules in Ꮿ is exactly the same as that of the tensor product of corresponding g-modules. Hence, the tensor category Ꮿ is tensorly equivalent to the tensor category of finite-dimensional g-modules. If ν is a weight of λ , then λ−ν is a sum of positive roots. In particular, λ−ν ∈ Y . 1.2.2. Dual modules. As usual, using the antipode S, for every M ∈ Ꮿ one can define the dual ᐁ-module M ∗ ∈ Ꮿ. By definition, M ∗ = HomQ(v) (M, Q(v)), and for every a ∈ ᐁ, f ∈ M ∗ , x ∈ M, one has (af )(x) = f (S(a)x). The dual of µ is ν , where ν = −w0 (µ) and w0 is the longest element of the Weyl group. 1.2.3. The element K˜ ±2ρ . For β ∈ Y , β = "i=1 ki αi , let K˜ β = ni=1 Kki di αi . Replacing K by K˜ has the following effect: If x is a vector of weight ν, then K˜ β (x) = v (ν|β) (x) (replacing the bracket λ, β by the scalar product (λ | β)). Note that 2ρ, as the sum of all positive roots, is always in the root lattice Y . Hence, (2ρ | µ) ∈ Z for every µ ∈ X. The elements K˜ ±2ρ play an important role. 1.2.4. The evaluation and coevaluation maps. The ground field Q(v) is the ᐁmodule λ , with λ = 0. The algebra ᐁ acts on Q(v) via the co-unit. The module Q(v) is the unit of the tensor product in Ꮿ. The left evaluation map evl : M ∗ ⊗ M → Q(v), defined by evl (f ⊗ x) = f (x), is ᐁ-linear. But the map M ⊗ M ∗ → Q(v) defined by (x ⊗ f ) → f (x) is not ᐁ-linear. However, the one twisted by K˜ −2ρ is: The map evr : M ⊗ M ∗ → Q(v), defined by evr (x ⊗ f ) = f (K˜ −2ρ x), is ᐁ-linear. Similarly, the coevaluation maps xs ⊗ xs∗ , coevl : Q(v) −→ M ⊗ M ∗ , defined by coevl (1) = s

∗

coevr : Q(v) −→ M ⊗ M,

defined by coevr (1) =

s

are ᐁ-linear. Here {xs } is a basis of M and

{xs∗ }

xs∗ ⊗ K˜ 2ρ (xs ),

is the dual basis in M ∗ .

1.2.5. Canonical basis. Lusztig and Kashiwara introduced a canonical basis Bλ for the ᐁ-module λ . The set Bλ is a Q(v)-basis of the Q(v)-vector space λ . Let Ꮽ λ be the Ꮽ-lattice in λ generated by Bλ . Then Ꮽ ᐁ leaves the lattice Ꮽ λ invariant, Ꮽ ᐁ(Ꮽ λ ) ⊂ Ꮽ λ . The set Bλ consists of weight vectors; that is, the intersection Bλ ∩ (λ )ν is a Q(v)-basis of the vector space (λ )ν for every weight ν.


279

1.3. The braiding and the twist 1.3.1. The quasi-R-matrix. Let 7 be the quasi-R-matrix of [Lu2]; it is an infinite sum 7= as ⊗ b s , (1.1) s

where as , bs are in ᐁ. So 7 belongs to an appropriate completion of ᐁ ⊗ ᐁ. On every M ∈ Ꮿ, all the as , bs , except for a finite number of them, act as zero. Hence it makes sense to consider the operator 7 : M ⊗ N → M ⊗ N for every M, N in Ꮿ. In particular, there is 7 : λ ⊗µ → λ ⊗µ . Lusztig proved that 7 is invertible and that both 7, 7−1 leave Ꮽ λ ⊗Ꮽ Ꮽ µ invariant. Let us take the set x ⊗ y, with x ∈ Bλ and y ∈ Bµ as a basis of λ ⊗ µ , and call it the tensor product basis. Then, in this basis, the matrices of 7, 7−1 have entries in Ꮽ = Z[v, v −1 ]. Moreover, Lusztig proved that 7, 7−1 can be defined over Ꮽ. This implies that the as , bs in the formula (1.1) of 7 can be chosen in Ꮽ ᐁ. 1.3.2. Braiding. Let D be the least positive integer such that D(µ | ν) ∈ Z for every µ, ν ∈ X. Equivalently, D is the least positive integer such that DX ⊂ Y . The number D (see Table 1) is always a divisor of the determinant of the Cartan matrix. Let v 1/D be a new variable such that (v 1/D )D = v. To define the braiding, we extend the ground field to Q(v 1/D ) by taking tensor products of ᐁ and every module in Ꮿ with Q(v 1/D ). By abuse of notation, we still use ᐁ, Ꮿ to denote the corresponding objects (after taking tensor products). For two ᐁ-modules M, N in Ꮿ, let : : M ⊗ N → M ⊗ N be defined by :(x ⊗ y) = v (ν|µ) x ⊗ y, if x ∈ M ν and y ∈ N µ . Note that (ν | µ) is always in (1/D)Z. Let σ : M ⊗N → N ⊗M be the flip: σ (x ⊗y) = y ⊗x. Then the braiding operator c = c(M, N) : M ⊗ N → N ⊗ M is defined by c(M, N) = σ :7−1 : M ⊗ N −→ N ⊗ M. Then c commutes with the action of ᐁ. Actually, c is equal to the inverse of the commutativity isomorphism in [Lu2, Chapter 32]. The map : is called the diagonal part of the braiding. It is because of the diagonal part that we need to extend the ground field to Q(v 1/D ). It is clear now that if we take the tensor product bases as the bases of λ ⊗ µ and µ ⊗ λ , then the matrix of the braiding c has entries in Z[v ±1/D ]. A slightly stronger result is given below. Suppose M = µ and N = ν . If µ , ν are weights of M, N, respectively, then both µ−µ and ν −ν are in the root lattice Y . Since the scalar product of an element in Y and an element in X is always an integer, we see that (µ | ν ) ≡ (µ | ν) (mod Z).

280

THANG T. Q. LE

Hence, in any basis that is the tensor product of bases consisting of weight vectors, the matrix of : has entries in v (µ|ν) Ꮽ. Thus we have the following. Proposition 1.1. In the product bases, the matrix of the braiding c : µ ⊗ν → ν ⊗ µ has entries in v (µ|ν) Ꮽ = v (µ|ν) Z[v ±1 ], and its inverse has entries in v −(µ|ν) Ꮽ. 1.3.3. The twist. Recall that 7 = as ⊗bs , where the sum is infinite and as , bs ∈ ᐁ. Let (see [Lu2, Chapter 6]) S(as )bs . == s

This sum should be considered as an element of some completion of ᐁ. Since on M ∈ Ꮿ only a finite number of terms in the sum survive, one can define = : M → M. For every M ∈ Ꮿ, let θ = θ (M) : M → M be defined by θ (x) = v (ν+2ρ|ν) =(x) if x ∈ M ν . Then θ is invertible, commutes with ᐁ-actions, and is known as a quantum Casimir element (see [Lu2, Chapter 6]). We call it the twist. Moreover, for any M, N ∈ Ꮿ, one has that

−1 θ (M ⊗ N) θ (M) ⊗ θ(N) = c(N, M)c(M, N), (1.2) whose proof is similar to that of [Ka, Proposition VIII.4.5]. The twist θ can also be described as follows. First, note that θ(M N) = θ(M) θ (N). Every M in Ꮿ can be uniquely expressed in the form M(λ) , M= λ∈X+

where M(λ) is the direct sum of a finite number of copies of the simple module λ . The twist θ acts on M(λ) as the scalar v (λ+2ρ|λ) times the identity. 1.3.4. Ꮿ is a ribbon category. It is known that Ꮿ, together with the braiding c and the twist θ , is a ribbon category (see [Ka], [Tu]). Actually, Ꮿ is the same as the category Uh (g)-Modf r in [Ka, Chapter XVII] or the category ᐂq g in [Tu, Chapter XI]. Both [Ka] and [Tu] use the h-adic version of quantum group, which is not suitable for studying the roots of unity case. 1.3.5. The variable q. In knot theory, another variable q = v 2 is usually used. The reader should not confuse this q with the quantum parameter used in the definition of quantum groups by several authors. For example, our q is equal to q 2 in [Ka] and [Tu]. In the expression “quantum invariant at an rth root of unity,” the rth root of unity is q (but not v). 1.4. Quantum link invariants. It is known that any ribbon category gives rise to operator invariants of framed links, whose components are colored by objects of the

281


D0

D1

T ◦ T =

D2

T

D4

D3

T ⊗ T = T T

T

Figure 1. Elementary tangle diagrams, composition, and tensor product

category. We first review the definition, following [KM] and [Tu]. 1.4.1. Framed tangles. A tangle T is an oriented 1-manifold properly embedded (up to isotopy) in R2 ×[0, 1], with ∂T ⊂ 0 ×R×∂[0, 1]. Define ∂− T = T ∩(R2 ×0) and ∂+ T = T ∩ (R2 × 1), and call T a (k, l)-tangle if |∂− T | = k and |∂+ T | = l. Thus a link is a (0, 0)-tangle. A framed tangle is a tangle T equipped with a normal vector field that is standard (1, 0, 0) on ∂T . As usual, we consider framed tangles up to isotopy relative to the boundary. In R3 , there is a natural way to identify framings of a component with integers. A diagram of a tangle is its regular projection on 0×R2 , together with the information on over- or undercrossings. A diagram defines a blackboard framing in which the normal vector is always (1, 0, 0). A diagram of T is good if the blackboard framing is coincident with the framing of T . It is well known that every tangle diagram can be factored into the elementary diagrams D0 –D4 depicted in Figure 1 using the composition ◦ (when defined) and the tensor product ⊗ of diagrams. 1.4.2. Operator invariants of colored framed tangles. A coloring of a tangle T is an assignment of an object in the category Ꮿ to each component of T . This induces a coloring of ∂T as follows: If C is an arc of color M, then assign M to each endpoint of C where C is oriented down and assign the dual object M ∗ to each endpoint where C is oriented up. Tensoring from left to right, this gives the boundary objects T± assigned to ∂± T . By convention, the empty product is the unit in Ꮿ (the ground ring Q(v 1/D )). There exists a unique ᐁ-linear operator JT : T− → T+ , assigned to each colored framed tangle T , that satisfies JT ◦T = JT ◦ JT , JT ⊗T = JT ⊗ JT , and, for the tangles given by the elementary diagrams with blackboard framing, JD0 = id, JD2 = coevl ,

JD1 = evl ,

JD2 = coevr ,

JD1 = evr , JD3 = c,

JD4 = c−1 .

282

THANG T. Q. LE

Here we assume that D1 , D2 have orientation pointing from left to right, while D1 , D2 are the same tangles D1 , D2 but with reverse orientation. In particular, if L is a framed link with m ordered components, then JL (M1 , . . . , Mm ), for M1 , . . . , Mm ∈ Ꮿ, is in Q(v 1/D ). We see that only the braiding c and K˜ ±2ρ take part in the construction of JT . Many problems are thus reduced to questions about c and K˜ ±2ρ . Since in the product bases the matrices of c and K˜ ±2ρ have entries in Z[v ±1/D ], we see that JL (µ1 , . . . , µm ) is always a Laurent polynomial in v ±1/D , that is, JL ∈ Z[v ±1/D ]. Masbaum and Wenzl in [MW] proved this fact for g = sln using idempotent decompositions. Later we see that every link can be decomposed into pure braids and tangle diagrams without crossing points. We then prove that JT , when T is a pure braid, can be expressed through the twist θ only (no need to use the braiding). Thus one needs to use only the twist and K˜ ±2ρ . Both are simple, since their actions on highest-weight modules are easily described. 1.4.3. Relation to the Kontsevich integral. Quantum invariants of links can also be defined through the Kontsevich integral; see [LM] and [Ka, Chapter XX]. Roughly speaking, one first takes the (framed version) Kontsevich integral of a link L, then ¯ µ1 , . . . , ¯ µm of the Lie algebra plugs in the weight system coming from the modules g. The result, after a change of variable, is JL (µ1 , . . . , µm ). This approach avoids the theory of quantum groups (although the Kontsevich integral has its origin in quantum group theory). Some properties of quantum link invariants can be easily seen from this point of view. Some other properties, such as the ones proved in this paper, are easier to prove using quantum group theory. Actually, we do not know how to prove the results of this paper by using the Kontsevich integral theory. 1.4.4. The trivial knot. Suppose U is the trivial knot. Then JU (M) is called the quantum dimension of M; its value is well known: v (µ+ρ|α) − v −(µ+ρ|α) JU (µ ) = v (ρ|α) − v −(ρ|α) positive roots α (1.3) v 2(µ+ρ|α) − 1 −(µ|2ρ) =v v 2(ρ|α) − 1 positive roots α

2(µ+ρ|w(ρ)) w∈W sn(w)v = , 2(ρ|w(ρ)) w∈W sn(w)v

(1.4)

where W is the Weyl group and sn(w) is the sign of the linear transformation w. Noting that (µ | 2ρ) is always an integer, we get the following. Corollary 1.2. One has that JU (µ ) is either in Z[v 2 , v −2 ] or in vZ[v 2 , v −2 ]. More precisely, JU (µ ) ∈ v (µ|2ρ) Z[v 2 , v −2 ].


283

1.4.5. Sum, tensor product, and framing. The following facts are well known; see [Ka], [Tu]. One has the sum formula JL (M ⊕ M , . . . ) = JL (M, . . . ) + JL (M , . . . ),

(1.5)

where the dots denote the colors of the components other than the first. Let T (2) be the link obtained from T by replacing the first component by two of its parallel push-offs (using the framing). Then one has the tensor product formula JT (M ⊗ N, . . . ) = JT (2) (M, N, . . . ).

(1.6)

Suppose L is obtained from L by increasing the framing of the first component by 1. Then one has the framing formula JL (µ , . . . ) = v (µ+2ρ|µ) JL (µ , . . . ).

(1.7)

1.4.6. (1, 1)-tangles. Suppose that T is a (1, 1)-tangle and that the open component of T is the first component whose color is M. Then JT is an operator from M to M, commuting with the action of ᐁ. When M is a simple ᐁ-module, JT is a scalar operator, and thus there is a scalar invariant J˜T (M, . . . ) ∈ Z[v ±1/D ] such that JT (M, . . . ) = J˜T (M, . . . ) × id . If we close the (1, 1)-tangle T to get a framed link L, then JL (M, . . . ) = J˜T (M, . . . ) × JU (M).

(1.8)

2. Integrality and symmetry 2.1. Integrality. By integrality we mean the integrality of the coefficients and the exponents of v in JL . Recall that D is the least natural number such that (µ | µ ) ∈ (1/D)Z for every µ, µ in the weight lattice X. 2.1.1. Weak integrality. We have seen that JL (M1 , . . . , Mm ) is always in Z[v ±1/D ]. A little stronger statement is the following. Proposition 2.1 (Weak integrality). The quantum invariant JL (µ1 , . . . , µm ) fractional) number delies in v f Z[v ±1 ] = q f/2 Z[q ±1/2 ], where f is a (generally termined by the linking matrix (lij )1≤i,j ≤m of L: f = 1≤i, j ≤m lij (µi | µj ). Proof. If x ∈ M ν , then K˜ ±2ρ (x) = v (±2ρ|ν) x. Note that (2ρ | ν) is an integer, since 2ρ, as the sum of all positive roots, is always in the root lattice Y . It follows that in any basis consisting of weight vectors, K˜ ±2ρ has entries in Z[v ±1 ]. The braiding c : µ ⊗ ν → ν ⊗ µ has entries in v (µ|ν) Ꮽ (see Proposition 1.1) in the product bases; its inverse has entries in v −(µ|ν) Ꮽ. Counting the positive and negative crossing points of a good diagram of L gives the desired result.

284

THANG T. Q. LE

2.1.2. Strong integrality. The weak integrality says that the quantum invariant is essentially a Laurent polynomial in v. The fractional power can be eliminated by a suitable normalization. We have here a stronger statement, which says that the quantum invariant is essentially a Laurent polynomial in v 2 . Theorem 2.2 (Strong integrality). The invariant JL (µ1 , . . . , µm) is in v p Z[v ±2 ] = q p/2 Z[q ±1 ], where p is a (generally fractional) number determined by the linking matrix lij of L: 1 p= lij µi | µj + (lii + 1)(2ρ | µi ) ∈ Z. D 1≤i, j ≤m

1≤i≤m

The proof, which is presented in §3.6, is much more difficult than that of the weak integrality. We have to use a geometric lemma about special presentation of links together with a result of Andersen on quantum groups at roots of unity. The use of quantum groups at roots of unity seems unnatural, since the strong integrality does not have anything to do with roots of unity (see also the remark in §3.6.7). Andersen constructed an algebra homomorphism from the quantum group at v = −ε to the quantum group at v = ε, where ε is some root of unity. Heuristically, this implies some kind of symmetry between −v and v, which leads to the fact that JL depends essentially only on v 2 . Remarks. (a) The factor q p/2 could be understood as the contribution of the diagonal part of the R-matrix. (b) If the link L is replaced by a tangle T , then one cannot get such nice results about the exponents as in the strong integrality theorem. (c) If we use the normalization JˆL µ1 , . . . , µm := v −p JL µ1 , . . . , µm , then JˆL is a link invariant with values in Z[v 2 , v −2 ]. Note that we can define JˆL only for simple modules in Ꮿ. The normalization JˆL does not behave well under the action of the Weyl group (see below), and we do not use it in the sequel. Corollary 2.3. Consider the knot case. Let JL (µ ) be the nonframed version of the quantum invariant of knots, normalized so that the unknot takes value 1, that is, JL (µ ) :=

JL0 (µ ) , JU (µ )

where L0 is the framed knot with framing zero and of knot type L. Then JL (µ ) ∈ Z[v ±2 ]. Remark. When the link L has more than one component, then in general, the quotient JL /JU (m) is not a Laurent polynomial, but rather a rational function in v 1/D . The following corollary is useful in the theory of quantum invariants of 3-manifolds.


285

(See [Le3]; for the case g = sln , the corollary was proved in [Le2], using cabling.) Corollary 2.4. If all the µj ’s are in the root lattice, then JL (µ1 , . . . , µm ) is in Z[v ±2 ] = Z[q ±1 ]. Proof. The second term in the expression of the exponent p is in 2Z, since (ρ | µj ) is in Z. The first term is lij µi | µj = lii (µi | µi ) + 2 lij µi | µj . ij

i

i>j

Since (α | α) is even for every α in the root lattice, we see that the first term is in 2Z, too. Hence, the exponent p is an even number. 2.2. The first symmetry principle. Recall that q = v 2 . We show that if q is an rth root of unity, then JL has nice symmetry. 2.2.1. The Weyl group and the affine Weyl group. Let C be the fundamental chamber: C = x ∈ h∗R | 0 ≤ (x | αi ), i = 1, . . . , " . Then X+ = X ∩ C. The Weyl group W , by definition, is generated by reflections along the facets of C. It is a finite subgroup of the orthogonal group of h∗R , and C is a fundamental domain of it. Let Wr be the group of affine transformation of h∗R generated by W and the translation group rY . Since the root lattice is invariant under the action of W , one has Wr = W rY. Let α0 be the highest short root. When d = 1, that is, when all the roots have the same length, α0 is simply the highest root. The fundamental alcove is defined by Cr = x ∈ C | (x | α0 ) < r = x ∈ h∗R | 0 ≤ x, α < r for every positive root α . Its topological closure C¯ r is an "-simplex and is a fundamental domain of the affine Weyl group Wr . (See, e.g., [Kac, Chapter 6]; one has to apply the theory in [Kac] to the dual root lattice.) Moreover, Wr is generated by the reflections along the facets of C¯ r . 2.2.2. JL as a function on the weight lattice. Let us define, for µ1 , . . . , µm ∈ ρ + X+ ,

JL (µ1 , . . . , µm ) := JL µ1 −ρ , . . . , µm −ρ ∈ Z v ±1/D . The shift by ρ is more convenient for us. The formula is good only when all the µ1 , . . . , µm are in ρ + X+ = X ∩ C ◦ , where C ◦ is the interior of the fundamental chamber C. We extend the definition to every point in X as follows. If one of the µj is on the boundary of the chamber C, then let JL (µ1 , . . . , µm ) = 0. For every µ ∈ X, there exists w ∈ W such that w(µ) ∈ C; moreover, if w(µ) is in the interior of C, then such a w is unique. For arbitrary µ1 , . . . , µm ∈ X, choose

286

THANG T. Q. LE

w1 , . . . , wm ∈ W such that wj (µj ) ∈ X+ . Then define (recall that sn(w) is the sign of w) JL (µ1 , . . . , µm ) = sn(w1 ) · · · sn(wm )JL w1 (µ1 ), . . . , wm (µm ) . Using formulas (1.3), (1.4) for the unknot, we see that the formula

JU (µ) = v −(µ−ρ|ρ)

positive roots α

v 2(µ|α) − 1 v −2(ρ|α) − 1

(2.1)

is valid for every µ, not only in ρ + X+ , but also in X. 2.2.3. Another normalization. Let U (m) be the zero-framing trivial link of m components. Recall that QL (µ1 , . . . , µm ) := JL (µ1 , . . . , µm ) × JU (m) (µ1 , . . . , µm ). This normalization is more suitable for the study of quantum 3-manifold invariants and helps us to get rid of the ± sign in many formulas. Then QL is componentwise invariant under the action of the Weyl group: For every w1 , . . . , wm ∈ W , QL w1 (µ1 ), . . . , wm (µm ) = QL (µ1 , . . . , µm ). 2.2.4. First symmetry principle. Recall that q = v 2 . Suppose f, g belong to the same q a Z[q ±1 ], where a ∈ (1/2D)Z. We say that f = g at primitive rth roots of unity and write (r)

f =g if, for every primitive rth root of unity ξ , one has q −a f |q=ξ = q −a g|q=ξ . (r)

There is no need to fix a 2Dth root of ξ . When writing f = g, we always assume that f and g belong to the same q a Z[q ±1 ]. Theorem 2.5 (First symmetry principle). At primitive rth roots of unity, the quantum invariant QL is componentwise invariant under the action of the affine Weyl group Wr . This means, for every w1 , . . . , wm ∈ Wr , (r) QL w1 (µ1 ), . . . , wm (µm ) = QL (µ1 , . . . , µm ). (2.2) (r) If one of the µ1 , . . . , µm is on the boundary of C¯ r , then JL (µ1 , . . . , µm ) = 0.

Note that by the strong integrality, the left-hand side and the right-hand side of (2.2) belong to the same q a Z[q ±1 ]. We also show that JL is componentwise skew-invariant under the affine Weyl group: For every w1 , . . . , wm ∈ Wr , (r) JL w1 (µ1 ), . . . , wm (µm ) = sn(w1 ) · · · sn(wm )JL (µ1 , . . . , µm ).

287


0

λ1 (a)

α1

0

rλ1 /2

rλ1

(b) Figure 2. The A1 case

Remark. One can drop the “primitive” in the statements of the theorem. 2.3. The second symmetry principle. Recall that C¯ r is a fundamental domain of the action of W on h∗R . Because of the first symmetry principle, at primitive rth roots of unity, it is enough to consider JL (µ1 , . . . , µm ) with µj in C¯ r ∩ X, a finite set. It turns out that we can do better. There is a finite group G acting on C¯ r , and although JL is not really invariant under this action, it behaves quite nicely. 2.3.1. The extended affine Weyl group and the center group G. Recall that Wr = W rY . Note that X is invariant under the action of the Weyl group. Let Wˆ r be the group generated by W and translation by rX. Then Wˆ r = W rX. If λ ∈ X and w ∈ W , then w(λ)−λ is in Y . This implies Wr is a normal subgroup of Wˆ r . We have an exact sequence 1 −→ Wr −→ Wˆ r −→ G −→ 1, where G = X/Y is the fundamental group of the root data. It is known that G is isomorphic to the center of the simply connected complex Lie group associated with g and that |G| = det(aij ). The group G for various Lie algebras is listed in Table 1. Taking the action of Wˆ r modulo the action of Wr , we get an action of G on C¯ r that can be described explicitly as follows. Suppose g˜ ∈ X is a lift of g ∈ G = X/Y . There is a unique w ∈ Wr such that w(C¯ r + r g) ˜ = C¯ r . Then, for µ ∈ C¯ r , g(µ) = w µ + r g˜ ∈ C¯ r . For g ∈ D and µ ∈ X, we define a scalar product (g | µ) := (g˜ | µ), which is well defined as an element in (1/D)Z/Z. Similarly, for g1 , g2 ∈ G, let (g1 | g2 ) = (g˜ 1 | g˜ 2 ) ∈ (1/D)Z/Z. 2.3.2. Examples of G and its action. Here we give examples of the cases A1 , A2 , and B2 . When g = sl2 (the A1 case), the space h∗R is one-dimensional. The basis root and the weight are depicted in Figure 2(a). The simplex C¯ r is the interval [0, rλ1 ]. The nontrivial element of G = Z2 acts as the reflection about the midpoint rλ1 /2; see Figure 2(b). When g = sl3 (the A2 case), the space h∗R is two-dimensional and d = 1. The basis roots and weights are depicted in Figure 3(a). The simplex C¯ r is the equilateral

288

THANG T. Q. LE

rλ2

α2

rλ1

α0

λ2

λ1 α1

0

(a)

(b) Figure 3. The A2 case

rλ1 /2

α1

α0 = λ 1

rλ2

β0 λ2 α2

(a)

0

Figure 4. The B2 case

(b)

triangle with vertices at 0, rλ1 , and rλ2 . The group G = Z3 acts as rotations by 2π ik/3 about the center point; see Figure 3(b). When g is of B2 type, the space h∗R is again two-dimensional, with d1 = 2, d2 = 1. The basis roots and weights are depicted in Figure 4(a). The simplex C¯ r is the triangle with vertices at 0, rλ1 /2, and rλ2 . The nontrivial element of G = Z2 acts as the reflection about the dashed line that separates C¯ r into two equal halves; see Figure 4(b). 2.3.3. Second symmetry principle Theorem 2.6. Suppose µ1 , . . . , µm ∈ C¯ r and g1 , . . . , gm ∈ G. Then QL g1 (µ1 ), . . . , gm (µm ) = v rt QL (µ1 , . . . , µm ),

(2.3)

at primitive rth roots of unity. Here t depends only on the linking matrix (lij ) of L:


t = (r − h)

lij gi | gj + 2

1≤i, j ≤m

289

lij gi | µj − ρ ,

1≤i, j ≤m

with h being the Coxeter number of the Lie algebra g (see Table 1). Remark. The factor v rt = q rt/2 makes both sides of (2.3) belong to the same q a Z[q ±1 ]. For the special cases g = sl2 and g = sln , the theorem was proved by Kirby and Melvin [KM] and Kohno and Takata [KT1]. In [KM] and [KT1], the twisting factor v rt is derived by direct computations. Here we express the twisting factor through the scalar product in h∗R . We also have the result for every primitive rth root of unity. Since Wˆ r = W rX and since QL is componentwise invariant under W , (2.3) is equivalent to the statement that for x1 , . . . , xm ∈ X, QL (µ1 + rx1 , . . . , µm + rxm )

(r) r[(r−h) lij (xi |xj )+2 lij (xi |µj −ρ)]

=v

QL (µ1 , . . . , µm ).

(2.4)

Corollary 2.7. Suppose L has zero linking matrix. Then QL (µ1 , . . . , µm ), at primitive rth roots of unity, is invariant under componentwise action of Wˆ r . Corollary 2.8. If µj − ρ is in the root lattice and µj ∈ C¯ r , then (r) QL g1 (µ1 ), . . . , gm (µm ) = v r[ lij (gi |gj )] QL (µ1 , . . . , µm ). The corollary follows from Theorem 2.6, since the second term in the expression of t in the theorem is in 2Z. These corollaries have application in the study of quantum 3-manifold invariants. In a subsequent paper, we will use Corollary 2.8 to define the projective version of quantum invariants and its perturbative expansion. 2.4. Refined versions of symmetry principles. When (r, d) = 1, we can strengthen both symmetry principles. Actually, we formulate the result so that it includes the (d, r) = 1 case. In the refined versions, the symmetry groups are larger, actually, the largest possible. For example, if one wants to construct quantum invariants of 3manifolds, one has to use the refined versions because of the nondegeneracy property in Proposition 2.10 below. 2.4.1. Refined versions of Cr , Wr , Wˆ r . If (r, d) = 1, let X = X and Y = Y . If (r, d) = 1, then (r, d) = d > 1. In this case, let X (resp., Y ) be the Z-lattice generated by λi /di (resp., αi /di ), i = 1, . . . , ". In other words, if (r, d) = 1, then X is the lattice dual to Y with respect to our scalar product, that is, X = {x ∈ h∗R | (x | y) ∈ Z, for every y ∈ Y }, and similarly, Y is the lattice dual to X. If (d, r) = 1, then X , Y is a realization of the root lattice and the weight lattice of the dual root system whose Cartan matrix is the transpose of the original one.

290

THANG T. Q. LE

In any case, X ⊂ X , Y ⊂ Y . Note that X and Y are invariant under W , and if x ∈ X , then x − w(x) ∈ Y for every w ∈ W . Let us define Wr = W rY ,

Wˆ r = W rX .

Then Wr is a normal subgroup of Wˆ r , and we have an exact sequence 1 −→ Wr −→ Wˆ r −→ G −→ 1,

(2.5)

where G = X /Y . Note that G is always isomorphic to G. The lattices X , Y thus depend on the root data (of the Lie algebra g) and whether (r, d) = 1 or not. There is a unifying definition good for every r, as described in the following lemma, which is easy to prove. Lemma 2.9. One has that rX = x ∈ X | (x | y) ∈ rZ for every y ∈ Y , rY = y ∈ Y | (x | y) ∈ rZ for every x ∈ X . 2.4.2. The fundamental domain of Wr . If (r, d) = 1, let Cr = Cr . Otherwise, let Cr = x ∈ C | (x | β0 ) < r , where β0 is the long highest root. The closure C¯ r is a simplex and a fundamental domain of Wr . In any case Cr ⊂ Cr , and if (d, r) = 1, then Cr is strictly less than Cr . In fact, if (r, d) = 1, then it can be shown that the volume of Cr is "i=1 (d/di ) times that of Cr . One important property of Cr is the following nondegeneracy property, proved in [AP]. Proposition 2.10 (see [AP]). Suppose µ ∈ C¯ r and ε 2 is a primitive rth root of unity. Then the quantum dimension JU (µ)|v=ε = 0 if and only if µ is on the boundary of C¯ r . Proof. This follows from the explicit formula of the quantum dimension (2.1). This proposition shows that the set Cr (but not Cr in general) is exactly what should be used in the construction of the topological quantum field theory. 2.4.3. Actions of G and the quadratic form on G . Again, the exact sequence (2.5) leads to an action of G on C¯ r . Explicitly it can be described as follows. Suppose g˜ ∈ X ˜ = C¯ r . is a lift of g ∈ G = X /Y . There is a unique w ∈ Wr such that w(C¯ r + r g) Then g(µ) : = w(µ + r g) ˜ for every µ ∈ C¯ r . For g ∈ G and µ ∈ X (not X here), let us define the product (g | µ) : = (g˜ | µ), which is well defined as an element in Q modulo Z. If ζ is a 2Drth root of unity, then ζ 2Dr(g|µ) is well defined as a complex number.

291


rλ1 /2

α1

α0 = λ 1

rλ1 /2

rλ2 /2

β0 λ2

(a)

rλ2

α2

0

0 (b): Cr

(c): Cr

Figure 5. The B2 case

Similarly, for g1 , g2 ∈ G , let (g1 | g2 ) = (g˜ 1 | g˜ 2 ), defined as an element in Q modulo (1/(d, r))Z. If ζ is a 2Drth root of unity and k an integer divisible by (d, r), then ζ 2Drk(g1 |g2 ) is well defined as a complex number. A little bit more difficult to see is that ζ Dkr(g|g) is well defined as a complex number for every g ∈ G . But this follows from the evenness of the quadratic form (· | ·) on the root lattice. When (r, d) = 1, we could drop all the primes. Note that all the “prime” versions, such as X , Y , Cr , Wr , . . . , depend on the root data and whether (r, d) = 1 or not. The group G is isomorphic to G; however, the scalar product on G depends on whether (r, d) = 1 or not. Let us consider an example when (r, d) = 1. Then d = 2 or 3. If we want G to not be trivial, then we are left with only two cases: the B" or C" series of Lie algebras. Let us consider the case of B2 (see Figure 5). In this case Cr is a half of Cr such that Cr is the triangle with vertices at 0, rλ1 /2, rλ2 /2. The nontrivial element of G = Z2 acts as the reflection about the dashed line in Figure 5(c). 2.4.4. Refined symmetry principles. Both symmetry principles remain valid if we replace Wr , Cr by Wr , Cr . However, one has to take care of the Coxeter numbers. Theorem 2.11 (Refined first symmetry principle). At primitive rth roots of unity, QL is componentwise invariant under actions of Wr : For µ1 , . . . , µm ∈ X, w1 , . . . , wm ∈ Wr , (r) QL w1 (µ1 ), . . . , wm (µm ) = QL (µ1 , . . . , µm ). (2.6) (r)

If one of the µj is on the boundary of C¯ r , then JL (µ1 , . . . , µm ) = 0. Since Wr ⊂ Wr , the refined version implies the nonrefined version. We also prove that JL is componentwise skew-invariant under Wr : For w1 , . . . , wm ∈ Wr , (r) (2.7) JL w1 (µ1 ), . . . , wm (µm ) = sn(w1 ) · · · sn(wm )JL (µ1 , . . . , µm ). Certainly this identity implies (2.6). It also implies the second statement of the

292

THANG T. Q. LE

theorem: If one of µ1 , . . . , µm , say, µ1 , is on the boundary of C¯ r , then there is reflection along a facet of C¯ r that fixes µ1 . The sign of any reflection is −1. Hence, (r)

JL (µ1 , . . . , µm ) = 0. Another way to prove the second statement is the following. By (1.8), JL = J˜T JU (µ1 ). If µ1 is on the boundary of C¯ r , then by Lemma 2.10, (r)

(r)

JU (µ1 ) = 0. Hence JL = 0. Again, because of the first symmetry, it is enough to restrict the colors to C¯ r when considering quantum invariants. Let h∨ be the dual Coxeter number of g (see Table 1). Theorem 2.12 (Refined second symmetry principle). Suppose µ1 , . . . , µm ∈ C¯ r and g1 , . . . , gm ∈ G . Then, at primitive rth roots of unity, QL g1 (µ1 ), . . . , gm (µm ) = v rt QL (µ1 , . . . , µm ), (2.8) where t is determined by the linking matrix of L: lij (gi | gj ) + 2 lij (gi | µj − ρ), t = (r − h ) i,j

i,j

with

h =

h dh∨

if (d, r) = 1, if (d, r) = 1.

Note that v rt in (2.8) is well defined as a complex number; see §2.4.3. Again, the factor v rt = q rt /2 makes both sides of (2.8) belong to the same q a Z[q ±1 ]. Since the action of G is obtained from the action of the extended affine Weyl group Wˆ r , let us describe how QL behaves under the action of Wˆ r . Recall that Wˆ r = W rX and that QL is componentwise invariant under W . We need only to describe how QL behaves under the translation group rX . Suppose x1 , . . . , xm ∈ X ; then, QL (µ1 + rx1 , . . . , µm + rxm ) (r)

= QL (µ1 , . . . , µm )v r[(r−h )

lij (xi |xj )+2

lij (xi |µj −ρ)]

.

(2.9)

The theorem certainly follows from this statement. 3. Proofs 3.1. Quantum groups at roots of unity. We recall the theory of quantum groups at roots of unity, following [An], [AP], and [Lu2] and then prove some auxiliary facts. 3.1.1. Quantum group at roots of unity and its category of modules. Suppose ε ∈ C is a number such that ε2 is an rth primitive root of unity. Then ε is either a primitive 2rth root of unity or a primitive rth root of unity. The latter can happen only when r is odd. Fix a number ζ such that ζ D = ε. If a ∈ (1/D)Z, then by ε a we mean ζ Da .


293

Let ε ᐁ be the algebra Ꮽ ᐁ ⊗Ꮽ C, where C is considered as an Ꮽ-algebra by mapping v to ε. Then ε ᐁ is a Hopf C-algebra, called a quantum group at a root of unity (Lusztig’s version). The Cartan subalgebra of Ꮽ ᐁ is not generated (over Ꮽ) by the Kα alone; one needs the elements d −di t 1−s v Kαi i − v 1−s Kαi Kαi = ∈ Ꮽ ᐁ, t v sdi − v −sdi

s=1

where i = 1, . . . , " and t = 1, 2, 3 . . . . For a ε ᐁ-module M and a weight ν ∈ X, let ν, αi Kαi ν ν,α (x) = M = x ∈ M | Kα (x) = ε x and x , t t i where for x ∈ Z, t ∈ Z, t > 0, t ε di (x−s+1) − ε −di (x−s+1) x := . t i ε sdi − ε −sdi s=1

Let ε Ꮿ be the category of finite-dimensional ε ᐁ-modules M such that M = ⊕ν∈X M ν (p)

(p)

and that Ei (x) = Fi (x) = 0 on M for sufficiently large p. Using the same formulas as in the case over Q(v), we define dual modules and the evaluation and coevaluation maps. We define the twist ε θ and the braiding ε c using the same formulas of θ and c, replacing v 1/D by ζ . Then (ε Ꮿ, ε θ, ε c) is a ribbon category. In particular,

−1

ε θ (M, N) ε θ (M) ⊗ ε θ(N)

= ε c(N, M) × ε c(M, N).

(3.1)

3.1.2. Simple modules. In general, ε Ꮿ is not semisimple: there are modules in ε Ꮿ that are indecomposable, but not simple. Since Ꮽ λ is invariant under Ꮽ ᐁ, there is defined ε λ =Ꮽ λ ⊗C, which is a ε ᐁmodule in ε Ꮿ. Here λ is in X+ . Since ε is a root of unity, ε λ may not be simple. But ε λ always has a unique quotient Lλ that is a simple ε ᐁ-module. This ε ᐁ-module Lλ ∈ ε Ꮿ is also of highest weight λ. If λ = µ, then Lλ is not isomorphic to Lν . If λ ∈ Cr , then ε λ is a simple ε ᐁ-module, that is, Lλ = ε λ . 3.1.3. Composition factors and the twist ε θ. In general, if M ∈ ε Ꮿ, then M may not be a direct sum of simple modules. However, there is a decreasing sequence of submodules M = M0 ⊃ M1 ⊃ · · · ⊃ Mn = 0 such that Mi /Mi+1 is simple. The quotient Mi /Mi+1 is called a composition factor of M.

294

THANG T. Q. LE

In [AP], as a corollary of the linkage principle, it was proved that for every M ∈ ε Ꮿ, M= M[µ] . (3.2) µ∈Cr

Here M[µ] is the maximal submodule of M such that each composition factor of it is isomorphic to Lν with ν in the Wr -orbit of µ under the dot action, that is, ν = w(µ + ρ) − ρ for some w ∈ Wr . Lemma 3.1. If ν is in the Wr -orbit of µ (under the dot action), then ε(µ+2ρ|µ) = ε (ν+2ρ|λ) . Proof. Suppose ν = w(µ + ρ) − ρ. Using the fact that ε 1/D is a 2rDth root of unity, it is easy to check the statement for the case when w is in W and the case when w is a translation by a vector in rY . Note that the twist ε θ acts as the scalar ε(µ+2ρ|µ) on ε µ (see §1.3.3); hence, it acts as the same scalar on any composition factor of ε µ . Thus we get the following. Proposition 3.2. In the above notation, the twist ε θ acts as scalar ε(µ+2ρ|µ) on M[µ] . 3.1.4. Quantum link invariants. Recall that ε Ꮿ is a ribbon category. Thus if L is a framed link, then there is defined the invariant ε J L (M1 , . . . , Mm ) ∈ C, where M1 , . . . , Mm are in ε Ꮿ. Although we use the notation with ε, it is understood that ε J T depends on the choice of ζ , a Dth root of ε, since the twist and the braiding do. However, this is not essential, since one can always get rid of fractional powers of v by a suitable normalization. Obviously when Mj = ε µj , then ε J L ε µ1 , . . . , ε µm = JL µ1 , . . . , µm v 1/D =ζ , where the right-hand side means the value of JL (µ1 , . . . , µm ) at v 1/D = ζ . Many modules are not direct sums of ε λ . We can also define invariants of framed links colored by these modules. The presence of these modules helps us to relate values of quantum invariants at various µ. The simplest argument goes as follows. Suppose T is a framed (1, 1)-tangle whose open component is colored by λ . Then JT (λ ) is a scalar operator from λ to λ ,

JT (λ ) = J˜T (λ ) id, with J˜T (λ ) ∈ Z v ±1/D . Hence, when specialized at v 1/D = ζ , the map ε J T (ε λ ) : ε λ → ε λ is also a scalar operator (although ε λ may not be irreducible): ε J T ε λ = ε J˜T ε λ id .


295

Here ε J˜T (ε λ ) is a complex number obtained from J˜L (λ ) by putting v 1/D = ζ . It follows that if M is a composition factor of ε λ , then ε J T (M) : M → M is also a scalar operator with the same scalar ε J˜T (ε λ ). In particular, (3.3) ε J˜T Lλ = ε J˜T ε λ . Proposition 3.3. If ε λ and ε µ have a common composition factor and if T is a (1, 1)-tangle, then ε J˜T ε λ = ε J˜T ε µ . 3.2. Lemmas on quantum dimensions and signs Lemma 3.4. Recall that h = h if (d, r) = 1 and h = dh∨ if (d, h) = 1. (a) For every x1 , x2 ∈ X , the number h (x1 | x2 ) is an integer. (b) For every x ∈ X , one has (2ρ | x) ≡ h (x | x) (mod 2)

and

(2ρ | x) ≡ dh∨ (x | x) (mod 2).

(c) For every x1 , x2 ∈ X (not X ), one has (h − h)(x1 | x2 ) ∈ Z,

and

(h − h)(x1 | x1 ) ∈ 2Z.

Proof. (a) Suppose (d, r) = 1. Then X = X, and (x1 | x2 ) ∈ (1/D)Z. The values of h = h and D in Table 1 show that h is divisible by D. Hence h (x1 | x2 ) ∈ Z. Now suppose (d, r) = 1; then (d, r) = d > 1. This means g is of type B, C, F4 , or G2 . Note that X is the Z-lattice generated by λi /di , and (λi /di | λj /dj ) = (1/di )(A−1 )ij , where A−1 is the inverse of the Cartan matrix. Explicit calculation shows that (λi /di | λj /dj ) ∈ (1/D )Z, where D = 2 for B" , F2 , G2 , and C" with " even, and D = 4 for C" with " odd. In any case, D divides dh∨ , and hence dh∨ /D ∈ Z. (b) Note that if the statement is true for x = x1 and x = x2 , then it is true for x = x1 +x2 . Hence, it is enough to restrict oneself to the case when x is in a basis set. If (d, r) = 1, a basis set is {λi , i = 1, . . . , "}; if (d, r) = 1, a basis set is {λi /di , i = 1, . . . , "}. One can easily check the statement for each simple Lie algebra. (c) If (r, d) = 1, then h = h and both statements are trivial. Suppose (r, d) = 1; then h = dh∨ . Again one needs only to verify the statements when x1 , x2 is in a basis set of X, say, x1 = λi , x2 = λj . Recalling that (λi | λj ) = (A−1 )ij dj , one can easily check both statements. Lemma 3.5. Recall that U is the unknot. Let µ ∈ X. At primitive rth roots of unity, one has JU (µ + ry) = JU (µ)

and

JU (µ + rx) = (−1)(2ρ|x) JU (µ) JU (w(µ)) = sn(w)JU (µ)

QU (µ + ry) = QU (µ) for every y ∈ Y , and and

QU (µ + rx) = QU (µ) for every x ∈ X , QU (w(µ)) = QU (µ) for every w ∈ Wr .

296

THANG T. Q. LE

For the QL version, the statements are much simpler, since there is no sign. Proof. The first two identities for JU follow from the formula (2.1), if we remember that r(x | α) ∈ rZ for every x ∈ X , α ∈ Y (see Lemma 2.9). The third identity for JU follows from the first one and the fact that JU is skew-invariant under the action of the Weyl group W . All the identities for QU follow from the corresponding ones for JU . 3.3. Proof of refined first symmetry principle. The proof utilizes results from [AP] in the theory of quantum groups. We focus only on the first component of L. Suppose the color of this component is µ. Cut the link at a point on the component to get a (1, 1)-tangle T . Then by formula (1.8) we have JL (µ) = J˜T (µ)JU (µ). Since at primitive rth roots of unity we have (see Lemma 3.5) JU w(µ) = sn(w)JU (µ), it is enough to show that J˜T (w(µ)) = J˜T (µ) at primitive rth roots of unity. Here w ∈ Wr . From [AP, Section 3] and [Ja, Chapter II] we know that if λ = w(µ), then there is a sequence of µ1 , . . . , µs such that Lλ−ρ and Lµ−ρ are composition factors of ε µ1 and ε µs , respectively, and two consecutive ε µj , ε µj +1 have a common composition factor. It follows from Proposition 3.3 that J˜(λ) = J˜(µ) at primitive rth roots of unity. This proves the refined first symmetry principle. 3.4. Proof of refined second symmetry principle. As argued in §2.4.4, we need to prove (2.9). We use a result of Lusztig, which we first recall. 3.4.1. A tensor product theorem. Recall that ε2 is a primitive rth root of unity. Let r , i = 1, . . . , " . Xr = x ∈ X+ | x, αi < (r, di ) One can check that (Cr ∩ X+ ) ⊂ Xr . It is easy to check that if ξ ∈ X+ , then there exists unique λ ∈ Xr and ν ∈ rX ∩X+ such that x = λ + ν. We denote ξ (0) = λ ∈ Xr and ξ (1) = ν/r ∈ X . Lusztig [Lu1] proved that, as ε ᐁ-modules, Lξ ∼ = Lλ ⊗ Lν . This is quite nontrivial and very different from the classical case. It is similar to Steinberg’s tensor product theorem for algebraic groups over fields of positive characteristic. In [Lu1] the proof is given only for the case (r, d) = 1. However, the proof can be generalized to the case (r, d) = 1 (see the arguments of [AP, Theorem 3.12]).


297

3.4.2. The square of the braiding on Lλ ⊗ Lν . Assume that, as in §3.4.1, λ ∈ Xr and ν ∈ (rX ∩ X+ ). The square of the braiding ε c(Lν , Lλ ) ε c(Lλ , Lν ) is an operator acting on Lλ ⊗Lν , commuting with the action of ε ᐁ. But Lλ ⊗Lν = Lλ+ν is a simple module. Hence, the square of the braiding is a scalar operator, ε c(Lν , Lλ ) ε c(Lλ , Lν ) = bλ,ν id,

where bλ,ν ∈ C is a constant. For the tangle diagrams D3 , D4 of Figure 1 corresponding to c, c−1 , we have D3 = D32 D4−1 . It follows that ε J D3 (Lλ , Lν ) = bλ,ν ε J D4 (Lλ , Lν ).

(3.4)

This means the operator of a positive crossing and the one of a negative crossing are proportional. The proportional factor bλ,ν can be calculated as follows. By (3.1),

−1 (ε c)2 = ε θ(Lλ ⊗ Lν ) ε θ(Lλ ) ⊗ ε θ(Lν ) . Using Lλ ⊗ Lν = Lλ+ν and the fact that ε θ acts on Lµ as the scalar ε (µ+2ρ|µ) (see §1.3.3), we see that bλ,ν = ε 2(λ|ν) .

(3.5)

3.4.3. The square of the braiding on Lν ⊗ Lν . We continue to assume that, as in the previous subsection, ν ∈ rX ∩ X+ . We show that (ε c)2 acts as a scalar operator on Lν ⊗ Lν . It is enough to show that ε θ(Lν ⊗ Lν ) is a scalar operator, since

−1 c2 = θ(Lν ⊗ Lν ) θ(Lν ) ⊗ θ(Lν ) . The structure of the module Lν and its tensor powers can be understood by classical Lie theory, via the quantum Frobenius map (see [Lu2, Chapter 35]). Every weight of Lν must be of the form ν − α, where α ∈ rY . The tensor product Lν ⊗ Lν is completely reducible: L ν ⊗ L ν = ⊕ τ Lτ , where τ ∈ 2ν − rY . The twist θ acts on Lτ as a scalar operator, with the scalar ε(τ +2ρ|τ ) (see Proposition 3.2). When τ ∈ 2ν −rY , it is easy to see that the scalar is always equal to ε (2ν+2ρ|2ν) . This means θ (Lν ⊗ Lν ) is a scalar operator with the scalar ε (2ν+2ρ|2ν) . So we have c2 = bν id, where the value of bν can be calculated (bν = ε2(ν|ν) ; we do not need this value). It follows that JD3 (Lν , Lν ) = bν JD4 (Lν , Lν ). For example, suppose T is a (1, 1)-tangle with framing zero. Let us calculate ˜ J ε T (Lν ). We switch over- or undercrossing at some points in a good diagram of T to get the trivial (1, 1)-tangle. With each switching we have to multiply the quantum invariant by bν or (bν )−1 . Since the framing is zero, we conclude that ε J˜T (Lν ) = 1.

298

THANG T. Q. LE

3.4.4. Reduction to the framing-zero case. We show here that if (2.9) is true for a link L, then it holds true for any link obtained from L by altering the framing of the components. It is sufficient to consider the case when we increase the framing of the first component by 1. Then the left-hand side of (2.9) is multiplied by a LHS = ε (µ1 +rx1 +ρ|µ1 +rx1 −ρ) and the right-hand side by

a RHS = ε (µ1 +ρ|µ1 −ρ) ε r[(r−h )(x1 |x1 )+2(x1 |µ1 −ρ)] . Hence, to show that a LHS = a RHS it is enough to prove that

1 = ε r[h (x1 |x1 )+2(x1 |ρ)] , which follows from Lemma 3.4. (The term in the square bracket of the exponent is divisible by 2 by Lemma 3.4.) 3.5. A special case. By virtue of the result of the previous subsection, we assume from now on that 0 = l11 = l22 = · · · . Suppose ξ ∈ X+ . Then ξ = ξ (0) + rξ (1) (see the notation in §3.4.1). In this subsection we assume that µ2 , . . . , µm ∈ X+ . We show that (r) QL ξ , µ2 , . . . , µm = v 2rκ QL ξ (0) , µ2 , . . . , µm , (3.6) where κ=

l1j ξ (1) | µj .

j

By Lemma 2.9, r(ξ (1) | α) ∈ rZ for every α ∈ Y . Since w(µj ) − µj is in Y (1) (1) for every w ∈ Wr , we have that ε2r(ξ |µj ) = ε2r(ξ |w(µj )) . It follows that ε 2rκ is invariant under the action of Wr . Thus using the refined first symmetry principle, we see that to prove (3.6) we can assume that µ2 , . . . , µm are in Cr . In this case ε µj = Lµj , j = 2, . . . , m. Hence, to prove (3.6) one just needs to show that 2rκ (3.7) ε QL ε ξ , Lµ2 , . . . , Lµm = ε ε QL ε ξ (0) , Lµ2 , . . . , Lµm . Cut L at a point on the first component to get a (1, 1)-tangle T . From Lemma 3.5 we know that ε QU (ε ξ ) = ε QU (ε ξ (0) ). Hence, by formula (1.8), identity (3.7) is equivalent to 2rκ ˜ (3.8) ε J˜T ε ξ , Lµ2 , . . . , Lµm = ε ε J T ε ξ (0) , Lµ2 , . . . , Lµm . Using (3.3) we can replace ε ξ and ε ξ (0) by, respectively, Lξ and Lξ (0) in (3.8). The Lusztig theorem says Lξ = Lν ⊗ Lξ (0) , where ν = rξ (1) . By the tensor product formula (1.6), we have ε J˜T Lξ , Lµ2 , . . . , Lµm = ε J˜T (2) Lν , Lξ (0) , Lµ2 , . . . , Lµm .

299


There are two parallel push-offs of the first component of T ; let us denote the one colored with Lν by K. If we remove K, then from T (2) we get T . In the tangle diagram T (2) , consider a crossing point of K with the j th component whose color is Lµj . By formula (3.4), switching over- or undercrossing results in a factor bµj ,ν or its inverse. Switching over- or undercrossing to unlink the component K from other components, from T (2) we get T . Then we have ε J˜T (2)

" l Lν , Lξ (0) , Lµ2 , . . . , Lµm = bµj ,ν 1j × ε J˜T Lν , Lξ (0) , Lµ2 , . . . , Lµm

j =1

=

"

bµj ,ν

l1j

ε J˜K (Lν )ε J˜L

j =1

Lξ (0) , Lµ2 , . . . , Lµm . (3.9)

In §3.4.3 we showed that ε J˜K (Lν ) = 1. Using the values of bµ,ν in (3.5), from (3.9) we get (3.8). 3.5.1. The case when x2 = · · · = xm = 0. We prove (2.9) by assuming that x2 = · · · = xm = 0. Recall that the framings of L are zero. In this case, (2.9) reads (r) QL µ1 + rx1 , µ2 , . . . , µm = v 2r [ j l1j (x1 |µj −ρ)] QL µ1 , µ2 , . . . , µm .

(3.10)

By the refined first symmetry principle, QL is invariant under the translation by rY . Hence, we can further assume that µ1 + rx1 and all µ1 , . . . , µm are in the interior of the fundamental chamber C, that is, they are in ρ + X+ . Replacing µj by µj − ρ, we see that (3.10) is equivalent to (r) QL µ1 +rx1 , µ2 , . . . , µm = v 2r[ j l1j (x1 |µj )] QL µ1 , µ2 , . . . , µm . (3.11)

Note that (µ1 +rx1 )(0) = (µ1 )(0) and (µ1 +rx1 )(1) = (µ1 )(1) +x1 . Applying formula (3.6) for ξ = µ1 +rx1 and for ξ = µ1 and then comparing the right-hand sides of the resulting identities, we get (3.11). 3.5.2. End of proof of second symmetry principle. We continue to assume that the framings are 0, l11 = · · · = lmm = 0. The result of the previous subsection certainly holds true if we replace the first component by any component. Successively adding rx1 to µ1 , rx2 to µ2 , and so on, we get (r) QL µ1 + rx1 , . . . , µm + rxm = v 2rτ QL µ1 , . . . , µm , where τ=

i,j

lij xi | µj − ρ + lij xi | rxj . i>j

(3.12)

300

THANG T. Q. LE

...

...

β

β

Figure 6. Braid closure

By Lemma 3.4, h (xi | xj ) ∈ Z, and hence 2r xi | xj ≡ 2 r − h xi | xj (mod 2), and eventually 2r

xi | xj ≡ (r − h ) lij xi | xj (mod 2).

i>j

i,j

This means that when v 2 isan rth root of unity, the second term in the formula of τ can be replaced by (r −h ) i,j lij (xi | xj ), and (3.12) becomes (2.9). This completes the proof of the refined second symmetry principle. Consider the nonrefined version. Dividing the right-hand side of (2.9) by the right −h) l (x |x ) ij i j . By Lemma 3.4(c), if all the x ’s hand side of (2.4), the quotient is v r(h i are in X and v 2r = 1, then v r(h −h) lij (xi |xj ) = 1. Hence (2.9) implies (2.4), which, in turn, implies the nonrefined version of the second symmetry principle. 3.6. Proof of the strong integrality 3.6.1. Presentation of links as plat closures of pure braids Proposition 3.6. Every nonframed link has a diagram of the form Tu ◦ T ◦ Tl , where Tu and Tl do not have any crossing and T is a pure braid. Proof (W. Menasco). First consider the case when L has only one component, that is, L is a knot. Then L is the braid closure of a braid β (see Figure 6). The natural projection from the braid group to the symmetric group maps β to an element with only one cycle, since L is a knot. Any two such elements are conjugate in the symmetric group. Since braids of the same conjugacy class have the same closure, we may assume that the projection of β onto the symmetric group is the permutation (12 · · · n). This means, after some over- or undercrossing switchings, from β we get a braid isotopic to β described in Figure 7(a). The isotopy can be assumed to be horizontal. The closure of β is presented in Figure 7(b); it is a trivial knot. It could be horizontally isotoped into the diagram in Figure 7(c) and, eventually, into the one in

301


···

···

(a)

(b)

(c)

(d)

Figure 7. The trivial knot

a pure braid

(a)

a pure braid

(b)

(c)

Figure 8. The plat closure

Figure 7(d). Now from this picture we go back to L by horizontal isotopy inside the parallel strip, as indicated in Figure 8(a), and undo the over- or undercrossing switchings using finger moves (see Figure 8(b)). We get the desired presentation (see Figure 8(a)). The proof for the case when L has many components is quite similar. The result, for a link of two components, is described in Figure 8(c). 3.6.2. Quantum invariants of pure braid. Suppose T is a pure braid whose components are colored by M1 , . . . , Mn ∈ Ꮿ. Then JT (M1 , . . . , Mn ) is an operator acting on the vector space M1 ⊗ · · · ⊗ Mn . We show here that JT can be expressed though the twist θ alone. If T is the square of D1 , that is, T is a full twist (see Figure 9(a)), then (see §1.3.3)

−1 JT (M1 , M2 ) = θ(M1 ) ⊗ θ(M2 ) θ(M1 ⊗ M2 ). Hence, in this case JT can be expressed through θ alone. Similarly, if T = D22 , then JT can be expressed through θ. If T is the tangle in Figure 9(b), which is obtained from the one in Figure 9(a) by taking parallels, then JT can also be expressed through θ alone, by the tensor product formula. Here the band stands for a bunch of parallel lines. Now we claim that every pure braid can be obtained from those in Figure 9(b) and their mirror images by using composition and tensor product. In fact, the pure

302

THANG T. Q. LE

(a)

(b)

(c)

(d)

Figure 9. The full twist and its parallels

braids depicted in Figure 9(c) and their mirror images generate every pure braid (using composition and tensor product). But these pure braids can be expressed through the ones of Figure 9(b), as shown in Figure 9(d) for a simple case. Hence JT , when T is a pure braid, can be expressed through θ. Using Lemma 3.6 we see that for a link L, up to a framing factor, JL can be expressed through θ and K±2ρ . 3.6.3. The map ϕ. Let ϕ : Z[v ±1/D ] → Z[v ±1/D ] be the algebra homomorphism defined by ϕ(v 1/D ) = eπi/D v 1/D . Then ϕ(v) = −v and ϕ 2D = id. Hence, the space Z[v ±1/D ] decomposes into eigenspaces of ϕ, whose eigenvalues are 2Dth roots of unity, eaπi , with a = 0, 1/D, . . . , (2D − 1)/D. The eigenspaces of ϕ are v a Z[v ±2 ]. Also x ∈ Z[v ±1/D ] is in the eigenspace v a Z[v ±2 ] if and only if ϕ(x) = eaπi x. Thus to prove the strong integrality theorem, one needs to show that ϕ JL µ1 , . . . , µm = eaπi JL µ1 , . . . , µm , (3.13) with a=

i,j

1 lij µi | µj + (lii + 1)(2ρ | µi ) ∈ Z. D i

Since both sides of (3.13) are Laurent polynomials in v 1/D , it is enough to prove (3.13) when v 1/D = e2πi/2Dr for every sufficiently large odd r. 3.6.4. The algebra homomorphism ϕ. ¯ Let us fix an odd integer r. Let ε be a primitive 2rth root of unity. Then −ε is a primitive rth root of unity. Andersen in [An] showed that there is an algebra homomorphism ϕ¯ : −ε ᐁ −→ ε ᐁ with the following properties. If M is a ε ᐁ-module, then pulling back via ϕ, ¯ we get a −ε ᐁ-module ϕ¯ ∗ (M). If M is the highest-weight module of highest weight µ, that is, M = ε µ , then ϕ¯ ∗ (M) is also the highest-weight −ε ᐁ-module of highest weight µ, that is, ϕ¯ ∗ (ε µ ) = −ε µ . Note that both M and ϕ¯ ∗ (M) have the same underlying vector space over C.


303

In order to consider quantum invariants of links, we need to fix a Dth root of ε and a Dth root of −ε. Fix an arbitrary Dth root ζ of ε, and choose ζ = eπi/D ζ = ϕ(v)|v 1/D =ζ as the Dth root of −ε. Now we can define ε θ, ε c, −ε θ, −ε c. In general, ϕ¯ does not commute with the coproduct. If M and N are two ε ᐁmodules, there may be two different −ε ᐁ-module structures on M ⊗C N, one via the coproduct of −ε ᐁ (the usual one) and one via ϕ¯ and the coproduct of ε ᐁ. However, we have the following. Lemma 3.7. Let Mj = ε µj , j = 1, . . . , n. The space M1 ⊗ · · · ⊗ Mn has two −ε ᐁ-module structures as described above. Then the twist −ε θ acts the same way in the two different module structures. Similarly, every Kβ , β ∈ Y acts the same way in the two different module structures. Proof. The statement for Kβ follows from the fact that for Kβ , the map ϕ¯ commutes with ., ϕ(.(K ¯ ¯ β )), which, in turns, follows from the definition β )) = .(ϕ(K l+1 of ϕ¯ : ϕ(K ¯ β ) = Kβ (see [An]). The statement for θ follows from the fact that the action of θ is totally determined by the highest weight (see Proposition 3.2). One needs to decompose M1 ⊗ · · · ⊗ Mn using (3.2) and applying Proposition 3.2. 3.6.5. The action of the twist. Again let Mj = ε µj , j = 1, . . . , n. There are two actions of −ε ᐁ on M1 ⊗ · · · ⊗ Mn . By the result of the previous subsection, the twist −ε θ acts the same way in the two structures. On this same vector space, M1 ⊗· · ·⊗Mn acts the twist ε θ of ε ᐁ. εθ

Proposition 3.8. Let Mj = ε µj . On M1 ⊗ · · · ⊗ Mn , the two operators −ε θ and are proportional: −ε θ

= eπi(µ1 +···+µn +2ρ|µ1 +···+µn ) ε θ.

Proof. As ε ᐁ-modules, one has (see (3.2)) M1 ⊗ · · · ⊗ Mn =

M[ν] .

ν∈Cr

On M[ν] , ε θ acts as the scalar ζ D(ν+2ρ|ν) (see Proposition 3.2). On that same subspace M[ν] , −ε θ acts (through ϕ) ¯ as the scalar (ζ )D(ν+2ρ|ν) = eπi(ν+2ρ|ν) ζ D(ν+2ρ|ν) . Hence on M[ν] , −ε θ

= eπi(ν+2ρ|ν) ε θ.

(3.14)

Note that µ1 + · · · + µn − ν is in the root lattice. Using the fact that the scalar product of a vector in the root lattice and a vector in the weight lattice is always in Z, one can easily show that (ν + 2ρ | ν) ≡ µ1 + · · · + µn + 2ρ | µ1 + · · · + µn (mod 2Z).

304

THANG T. Q. LE

It follows that the proportional factor eπi(ν+2ρ|ν) in (3.14) does not depend on ν and is always equal to eπi(µ1 +···+µn +2ρ|µ1 +···+µn ) . This proves the proposition. 3.6.6. Pure braid. Suppose that a framed tangle T has a good diagram that is a pure braid on n strands, and suppose that the strands are colored by Mj , which are ε ᐁ-modules. Then ε JT (M1 , . . . , Mn ) is an operator acting on M1 ⊗ · · · ⊗ Mn . Using ϕ, ¯ one can consider T to be colored by −ε ᐁ-modules ϕ¯ ∗ (Mj ). Hence, there is defined the operator −ε JT acting on the same space M1 ⊗ · · · ⊗ Mn . Proposition 3.9. Suppose tij is the linking number of the ith and the j th components of the pure braid T . Suppose also that Mj = ε µj . Then on M1 ⊗ · · · ⊗ Mn the two operators −ε JT and ε JT are proportional:

where b =

1≤i<j ≤n 2tij

−ε JT

= ebπi ε JT ,

µi | µj ).

Proof. Since the pure braids in Figure 9(b) generate every pure braid, we assume T is as in Figure 9(b). We suppose that the band has n−1 parallel lines whose colors are M1 , . . . , Mn−1 , and the remaining line has color Mn . Then by the tensor product formula (1.6), ±ε JT (M1 , . . . , Mn−1 , Mn )

=

−1

±ε θ (M1 ⊗ · · · ⊗ Mn−1 ) ⊗ ±ε θ(Mn )

±ε θ(M1 ⊗ · · · ⊗ Mn−1 ⊗ Mn ).

Using the relation between ε θ and −ε θ in Proposition 3.8, we get the desired result. 3.6.7. End of proof of the strong integrality theorem. As noted in §3.6.3, we need to prove (3.13). Using the framing formula (1.7), it is easy to check that if (3.13) holds true for a framed link L, then it does for every framed link obtained from L by altering the framing. Hence, we may assume L has any framing we wish. By Lemma 3.6, we can assume that L, as a nonframed link, has a diagram D = Tu ◦ T ◦ Tl , where T is a pure braid diagram, and Tu and Tl do not have any crossing points. Alter the framing of L so that D is a good diagram of it. Then the framing ljj of the j th component is always even, since T is a pure braid. We know that the operators −ε JT and ε JT are proportional. Now we prove that ε JTu and ε JTl are proportional to −ε JTu and −ε JTl , respectively. Let us consider a diagram corresponding to a maximal or a minimal point. The corresponding operator involves only K˜ ±2ρ . Suppose the component is colored by ε λ . Then ε K˜ ±2ρ (x) = ε ±(2ρ|ν) x if x ∈ (ε λ )ν . Similarly, −ε K˜ ±2ρ (x) = (−ε)±(2ρ|ν) x. Note that if ν is a weight, then µ − ν ∈ Y . Using the fact that (2ρ | α) ∈ 2Z for every α ∈ Y , we see that ˜ ±2ρ is proportional to ε K˜ ±2ρ on ε λ , with the proportional factor (−1)(2ρ|µ) . −ε K The proportional factor does not depend on the sign of ±2ρ. Thus ε JTu and ε JTl are proportional to −ε JTu and −ε JTl , respectively.

305


Now it is clear that κ := scalar factors:

−ε JL /ε JL

−ε JL ε JL

=

−ε JT ε JT

can be presented as the product of three ×

−ε JTu ε JTu

×

−ε JTl ε JTl

.

(3.15)

The first factor can be calculated using Proposition 3.9. Let us calculate the product of the second and third factors. If we replace T by the trivial pure braid T , then we get a trivial link L of m components. The value of JL is known, and one has −ε JL ε JL

= (−1)(2ρ|µ1 +···+µm ) = eπi(2ρ|µ1 +···+µm ) .

Here µ1 , . . . , µm are colors of the link. Applying (3.15), with T replaced by T , we see that the product of the second and the third factor is eπi(2ρ|µ1 +···+µm ) . Using Proposition 3.9 to calculate the first factor, we see that

κ = eπi[

1≤i, j ≤m lij (µi |µj )+(2ρ|µ1 +···+µm )]

.

Remember that lii is even, and (2ρ | µi ) is always an integer. We can alter the second term in the square bracket to get the value κ = eπi[

1≤i, j ≤m lij (µi |µj )+(2ρ|(l11 +1)µ1 +···+(lmm +1)µm )]

= eaπi ,

where eπia is the one in (3.13). Thus we have −ε JL

= eπia ε JL .

(3.16)

If we replace v 1/D by ζ in (3.13), then the right-hand side becomes eπia ε JL . The left-hand side, remembering that ϕ is an algebra homomorphism, is JL |v 1/D =ζ . The latter is −ε JL . Hence (3.16) implies that (3.13) holds true if v 1/D = ζ . Since ζ can take any 2Drth root of unity with r odd, (3.13) must hold true for every v 1/D . This completes the proof of the strong integrality theorem. Remark. As noted earlier, the use of roots of unity in the proof of the strong integrality seems very artificial. One could avoid roots of unity if the following question has an affirmative answer. Question. Is it true that in the product of the canonical bases of µ1 ⊗ · · · ⊗ µm , the twist θ has entries in v (µ1 +···+µm +2ρ|µ1 +···+µm ) Z[v ±2 ]? References [An] [AP] [Ja]

H. Andersen, Quantum groups at roots of ±1, Comm. Algebra 24 (1996), 3269–3282. H. Andersen and J. Paradowski, Fusion categories arising from semisimple Lie algebras, Comm. Math. Phys. 169 (1995), 563–588. J. Jantzen, Representations of Algebraic Groups, Pure Appl. Math. 131, Academic Press, Boston, 1987.

306 [Jo] [Kac] [Ka] [KM] [KT1] [KT2]

[Ko] [Le1] [Le2] [Le3] [LM] [Lu1]

[Lu2] [MR] [MW] [Mu] [Oh1] [Oh2]

[RT1] [RT2] [TY] [Tu] [Yo]

THANG T. Q. LE V. Jones, Hecke algebra representations of braid groups and link polynomials, Ann. of Math. (2) 126 (1987), 335–388. V. Kac, Infinite-Dimensional Lie Algebras, 3d ed., Cambridge Univ. Press, Cambridge, 1990. C. Kassel, Quantum Groups, Grad. Texts in Math. 155, Springer-Verlag, New York, 1995. R. Kirby and P. Melvin, The 3-manifold invariants of Witten and Reshetikhin-Turaev for sl(2, C), Invent. Math. 105 (1991), 473–545. T. Kohno and T. Takata, Symmetry of Witten’s 3-manifold invariants for sl(n, C), J. Knot Theory Ramifications 2 (1993), 149–169. , “Level-rank duality of Witten’s 3-manifold invariants” in Progress in Algebraic Combinatorics (Fukuoka, 1993), Adv. Stud. Pure Math. 24, Math. Soc. Japan, Tokyo, 1996, 243–264. M. Kontsevich, “Vassiliev’s knot invariants” in I. M. Gel’fand Seminar, Adv. Soviet Math. 16, Part 2, Amer. Math. Soc., Providence, 1993, 137–150. T. T. Q. Le, On denominators of the Kontsevich integral and the universal perturbative invariant of 3-manifolds, Invent. Math. 135 (1999), 689–722. , On perturbative PSU(n) invariants of rational homology 3-spheres, to appear in Topology. , Relation between quantum and finite type invariants, in preparation. T. T. Q. Le and J. Murakami, The universal Vassiliev-Kontsevich invariant for framed oriented links, Compositio Math. 102 (1996), 41–64. G. Lusztig, “Modular representations and quantum groups” in Classical Groups and Related Topics (Beijing, 1987), Contemp. Math. 82, Amer. Math. Soc., Providence, 1989, 59–77. , Introduction to quantum groups, Progr. Math. 110, Birkhäuser, Boston, 1993. G. Masbaum and J. Roberts, A simple proof of integrality of quantum invariants at prime roots of unity, Math. Proc. Cambridge Philos. Soc. 121 (1997), 443–454. G. Masbaum and H. Wenzl, Integral modular categories and integrality of quantum invariants at roots of unity of prime order, preprint, 1997. H. Murakami, Quantum SO(3)-invariants dominate the SU(2)-invariant of Casson and Walker, Math. Proc. Cambridge Philos. Soc. 117 (1995), 237–249. T. Ohtsuki, A polynomial invariant of rational homology 3-spheres, Invent. Math. 123 (1996), 241–257. , “A filtration of the set of integral homology 3-spheres” in Proceedings of the International Congress of Mathematicians (Berlin, 1998), Vol. 2, Documenta Mathematica, Bielefeld, 1998, 473–482; available from http://www.mathematik.unibielefeld.de/documenta/. N. Yu. Reshetikhin and V. Turaev, Ribbon graphs and their invariants derived from quantum groups, Comm. Math. Phys. 127 (1990), 1–26. , Invariants of 3-manifolds via link polynomials and quantum groups, Invent. Math. 103 (1991), 547–597. T. Takata and Y. Yokota, The PSU(n) invariants of 3-manifolds are polynomials, preprint, Kyushu University, 1996. V. G. Turaev, Quantum Invariants of Knots and 3-Manifolds, de Gruyter Stud. Math. 18, de Gruyter, Berlin, 1994. Y. Yokota, Skeins and quantum SU(N) invariants of 3-manifolds, Math. Ann. 307 (1997), 109–138.

Department of Mathematics, State University of New York, Buffalo, New York 14214, USA; [email protected]

Vol. 102, No. 2


© 2000

LOCAL VANISHING OF CHARACTERISTIC COHOMOLOGY MICHELE GRASSI Introduction. To a smooth manifold M one can associate in a natural way a new smooth manifold, the manifold of k-jets of n-dimensional submanifolds of M, (k) indicated by Gn (M), which parametrizes in a smooth way the k-jets of immersed (k) submanifolds of M. On Gn (M) one can build in a canonical way a differential ideal, (k) denoted Ᏽ(k) . The cohomology associated to the complex Gn (M)/Ᏽ(k) is called characteristic cohomology. These ideas, which in part go back to [C], are explained here. A more detailed introduction to them can be read, for example, in the introduction to [BG1] or in [BGH1] (see also [BG2], [BGH2], [BGH3]). Characteristic cohomology appears in this picture as a cohomological tool to study n-dimensional submanifolds of M. In this context one should think of submanifolds as solutions to systems of PDEs (partial differential equations). For example, they could be integral manifolds of an integrable distribution or of a differential ideal. Characteristic cohomology (or a variation of it) can then be used to provide invariants for the system of PDEs. The notation for the qth characteristic cohomology group over a smooth manifold M is (k) H q ∗ G(k) n (M) /Ᏽ , d . A first step in the study of characteristic cohomology is to see if something such as a Poincaré lemma holds for it; that is, if the characteristic cohomology vanishes (k) on contractible open subsets of Gn (M). More precisely, it has been conjectured by (k) Griffiths [Gri] that for any contractible open set ᐁ ⊂ Gn (M) one has H q ∗ (ᐁ)/Ᏽ(k) (ᐁ), d = (0) when 0 ≤ q < n. For dim(M) − n = k = 1 the result is classical. Griffiths and his collaborators [Gri] proved it when n = 1 and k, dim(M) is arbitrary, when k = 1 and n, dim(M) is arbitrary, and finally when dim(M) = 3, k = 2, and n = 2. In Section 5, we prove precisely what was conjectured for all k, n, and dim(M). For index q = n, the result is no longer true. This is because on a smooth manifold, one can have many independent functionals on n-dimensional submanifolds, even locally. The extremely rich structure of the nth characteristic cohomology group is probably the next object to research. Some intriguing conjectures on this topic, formulated by Griffiths, suggest possible approaches. Following this line of ideas, Received 13 March 1998. Revision received 23 June 1999. 1991 Mathematics Subject Classification. Primary 35A27, 13C99; Secondary 18G40, 35A15, 58D10. 307

308

MICHELE GRASSI

Griffiths has obtained some promising partial results, which are unfortunately still short of a general answer. In Section 6, we include some simple facts about the nth characteristic cohomology group. In our proof, there is a strong interplay among rather distant areas of mathematics. As we have seen, the motivation for studying characteristic cohomology comes from differential geometry and PDE theory. However, when one localizes to a small open set, things change abruptly. By the right choice of a weight defined on the bundle of differential forms on the open set, the characteristic weight, one obtains a filtration on the complex computing characteristic cohomology, and the associated graded complex is a completely algebraic object (see Lemma 4.7). The algebraic problem at this point is not solved easily. In Lemma 4.13, we translate the problem into one in which we must compute the Koszul homology group of a family of modules over a polynomial ring. To define these modules, we choose to introduce some basic notions from Hopf algebra theory. From the perspective of commutative algebra, the modules in question are of an unusual nature. Therefore, to compute their Koszul homology, one has to develop some ad hoc homological tools— here, Theorem 2.9. Using it, we may prove the main vanishing theorem for Koszul homology, Theorem 3.9, which gives the desired local vanishing for characteristic cohomology, Theorem 5.1. Finally, a word of caution: There are some other results (see, for example, [BG1, p. 555]) that resemble Theorem 5.1. However, the fundamental difference between the result in [BG1] and Theorem 5.1 is that in [BG1] the authors consider the infinite prolongation case (see also Section 5). Equivalently, one can allow the order of jets to increase by a (fixed and often easily computable) amount. Still equivalently, one could ask that the forms in question come from lower order jets. This approach not only simplifies the transition to an algebraic problem but more importantly leads to an algebraic problem of a different nature. In the language of this work, such an algebraic problem would amount to proving the vanishing of Koszul homology for a free module, which is a known algebraic fact. In [BG1], however, the authors consider characteristic cohomology in a different context (see, for example, [BG1, Theorem 2, p. 562]) to which the methods of this work would not apply, at least directly. Acknowledgements. A special thank you goes to Professor M. L. Green for his support, help, and patience in 1996–1997 during the preparation of my Ph.D. thesis at the University of California Los Angeles (UCLA), from which this paper has been derived. I would also like to thank Professor P. A. Griffiths for a very productive week at the Institute of Advanced Study that I spent with him in the spring of 1997, discussing some of these topics. Also, my thanks go to Professor V. S. Varadarajan for his many courses and seminars at UCLA. Notation. The notation that we use is fairly standard. For concepts on differential geometry, see, for example, [BCGGG]. On commutative algebra, see [G] or [BH]. On homological algebra, see, for example, [HS]. Basic notions of Hopf algebra theory can

LOCAL VANISHING OF CHARACTERISTIC COHOMOLOGY

309

be found in any reference text, for example, [Mo] or [Sw]. We used the terms lemma and theorem for our results (unless otherwise noted), while we used proposition and remark for results that are already known (for which sometimes we couldn’t find a reference) or that are straightforward applications of well-known facts. As there are many module and algebra structures involved, to avoid confusion we indicate the category of the operation. This is especially true of the tensor product. Often we use the notation M ⊗A N or M ⊗f N when the tensor product is done with respect to the ring A or with respect to the B-module structure induced by the ring morphism f : B → A, if M and N are A-modules. We also use this notation for the multiplication by ideals, as in I ·A M or I ·f M. We use a, b, cA to denote the submodule generated by the elements a, b, and c over the ring A. We explain the less standard notation and terminology in the first section. 1. Definitions and basic properties. The definition of k-jet of an immersed manifold f : N → M at a point p ∈ M can be found in [BCGGG, pp. 18–26]. We also define it here. Definition 1.1. (1) Two germs of immersed manifolds f, g : (Rn , 0) → (M, p) are said to have the same k-jet at 0 if, for some (and therefore any) choice of coordinates on M around p, the corresponding f˜, g˜ : (Rn , 0) → (Rdim(M) , 0) have equal k-jets (in the ordinary sense) at the origin. In this case, f ≡(k) g. (2) The space of k-jets of n-dimensional parametrized immersed submanifolds of k (R n , M), is the set of equivalence classes of immersed germs M at p, indicated by J0,p n f : (R , 0) → (M, p) with respect to ≡(k) . (3) Consider the group Aut(Rn , 0) of germs of diffeomorphisms of Rn leaving the origin fixed. There is a natural action of the group Aut(Rn , 0) on the vector space k (R n , M). This allows one to consider the orbit space of J k (R n , M) with respect J0,p 0,p k (R n , M)/ Aut(R n , 0). to this action, indicated by J0,p (k) (4) The space of k-jets of n-dimensional submanifolds of M, indicated by Gn (M), as a set is n k G(k) J0,p R , M / Aut Rn , 0 . n (M) = p∈M

Here we want to stress that we use nonparametrized k-jets. See [BCGGG, pp. 18–26] for the related concept of a jet bundle J k (N, M) between (k) two smooth manifolds of dimensions n and m, respectively. On Gn (M) there is a natural smooth structure. The following elementary proposition is included to identify (k) a choice of coordinate neighborhoods on Gn (M). Proposition 1.2. Let M = M n+s be an (n + s)-dimensional manifold. Then admits a smooth structure.

(k) Gn (M)

310

MICHELE GRASSI

Proof. Let p ∈ M be a point. Let f : Rn → M be a smooth map f (0) = p with injective differential at 0, and let f˜ : (Rn , 0) → (M, p) be its germ. Let (x i , zα ) = (x 1 , . . . , x n , z1 , . . . , zs ) be local coordinates in an open set of M containing p, such that f ∗ (dx 1 ∧ · · · ∧ dx n ) = 0 at p. Then the image of f˜ can be expressed as a graph of the form zα = zα (x 1 , . . . , x n ), α = 1, . . . , s. We now associate to the germ f˜ the set of numbers ∂zα ∂ k zα i α (p) . x (p), z (p), i (p), . . . , i ∂x ∂x 1 · · · ∂x ik α=1,...,s; i,i1 ,...,ik =1,...,s To complete the proof, one needs to show that with a different choice of coordinates near p, the numbers above vary smoothly in an open set. We omit this simple verification. (k)

Locally, we can take the following coordinates on Gn (M):

x i , zα , piα , . . . , piα1 ,...,ik

α=1,...,s; i,i1 ,...,ik =1,...,s

,

where the p α are symmetric in the lower indices and stand for the partial derivatives described in the previous proof. Definition 1.3. Let f : N → M be an n-dimensional immersed submanifold (i.e., N has dimension n and df is everywhere injective). Then there is a canonical lifting (k) f (k) : N → Gn (M), defined by associating to each point q ∈ N the k-jet of the germ of f (N) at p = f (q). (1)

(k)

Remark 1.4. Gn (M) ∼ = G(n, T(M)), where Gn (M) is the Grassmann bundle over the tangent bundle. Definition 1.5. The natural maps from the space of i-jets to the space of j -jets, (j ) (i) for i ≥ j , are indicated with πi,j : Gn (M) → Gn (M). A differential ideal on a manifold M is simply a homogeneous (two-sided) ideal inside the algebra of global differential forms, which is closed under exterior differentiation (see, for example, [BCGGG, p. 16]). The notion of an integral manifold is associated to a differential ideal Ᏽ. This is an immersion f : N → M such that (k) f ∗ (Ᏽ) = (0) (see, for example, [BCGGG, p. 16]). On Gn (M) there is a canonical (k) differential ideal, which we denote by Ᏽ . Because a differential ideal is a module over the smooth functions, its restrictions over open sets generate a (fine) sheaf. Therefore the ideal is determined by its local sections, and it is enough to give it over coordinate neighborhoods. We do this for Ᏽ(k) , using the neighborhoods identified (k) above for Gn (M). Definition 1.6. The contact ideal, indicated by Ᏽ(k) , is the differential ideal on generated locally by the forms (using Einstein’s summation convention)

(k) Gn (M)


311

 α α α i  θ = dz − pi dx ···   α θi1 ,...,ik−1 = dpiα1 ,...,ik−1 − piα1 ,...,ik−1 ,l dx l . We call I (k) the (algebraic) ideal generated by the forms above. Proposition 1.7. The contact ideal has all the liftings of n-dimensional immersed submanifolds of M as integral manifolds. Proof. It is enough to check this locally. Definition 1.8. We define the local characteristic cohomology of degree k over ᐁ as H ∗ ( ∗(ᐁ)/Ᏽ(k) , d). Definition 1.9. We call p = pᐁ : ∗ (ᐁ) → ∗ (ᐁ)/Ᏽ(k) the natural projection map. There are many connections of differential ideals with PDEs and with Lagrangians. Consult [BGH1], [BCGGG], and [BG1]. 2. A theorem on Tor. In Sections 2 and 3, we consider some (apparently) unrelated algebraic constructions and results. As usual, k[x] is the algebra of polynomials over the ring k in the variables x = x1 , . . . , xn . One can put a Hopf algebra structure on the k-algebra k[x] in the following way (see [Sw] for the definition of Hopf algebra). Definition 2.1. The comultiplication % : k[x] → k[x] ⊗k k[x] is defined as the k-algebra homomorphism that sends xi to xi ⊗ 1 + 1 ⊗ xi for all i = 1, . . . , n. The antipode S : k[x] → k[x] is defined as the k-algebra homomorphism that sends xi to −xi for all i = 1, . . . , n. All the other elements of the Hopf algebra structure are canonical. Following the standard practice, we indicate by M[a] the graded module M with the degrees shifted by the integer a (i.e., M[a]i := Mi+a ). Remark 2.2. We have (k[x][a]) ⊗k (k[x][b]) ∼ = k[x] ⊗k k[x][a + b] as graded k[x] ⊗k k[x]-modules. Lemma 2.3. (k[x]⊗k k[x])⊗k[x] k inherits a k-algebra structure from k[x]⊗k k[x]. Proof. By definition, k[x] ⊗k k[x] ⊗k[x] k = k[x] ⊗k k[x] /m ·% k[x] ⊗k k[x] = k[x] ⊗k k[x] /%(m) k[x] ⊗k k[x] , where m is the maximal (irrelevant) ideal of k[x]. This identification shows that (k[x] ⊗k k[x]) ⊗k[x] k is a quotient of k[x] ⊗k k[x] by the ideal generated by {%(xi ) |

312

MICHELE GRASSI

i = 1, . . . , n}. From this description, it is clear that (k[x] ⊗k k[x]) ⊗k[x] k inherits a k-algebra structure from k[x] ⊗k k[x]. Lemma 2.4. The maps ψl : k[x] −→ k[x] ⊗k k[x] ⊗k[x] k, ψr : k[x] −→ k[x] ⊗k k[x] ⊗k[x] k,

φl : k[x] ⊗k k[x] ⊗k[x] k −→ k[x], φr : k[x] ⊗k k[x] ⊗k[x] k −→ k[x],

acting as ψl (p(x)) = (p(x)⊗1)⊗1, φl ((p(x)⊗q(x))⊗1)=p(x)S(q(x)), ψr (p(x)) = (1 ⊗ p(x)) ⊗ 1, φr ((p(x) ⊗ q(x)) ⊗ 1) = S(p(x))q(x), induce isomorphisms of k-algebras, and ψl φl = φl ψl = ψr φr = φr ψr = Id. Proof. We prove this for ψl and φl ; the case of ψr and φr is just the same. First, let us show that φl is well defined. Indeed, if φ˜ l is the map Id ⊗S from k[x] ⊗k k[x] to k[x] (which is clearly well defined), we have that φ˜ l is a (surjective) morphism of k-algebras. Moreover, φ˜ l (%(xi )) = 0 ∀i, so it induces a map on (k[x]⊗k k[x])⊗k[x] k, which is φl . This is enough to show that φl is well defined and surjective. Also, ψl is well defined, as k[x] is a free k-algebra. It is also clear that φl ψl = Id. To show that ψl φl = Id, it is then enough to prove that ψl is surjective. As the maps are clearly homogeneous morphisms of graded k-algebras, to prove surjectivity it is enough to consider “monomials” of the form (m1 ⊗ m2 ) ⊗ 1, where m1 and m2 are monomials of k[x], and show that these are in the image of ψl . We do this by induction on d = deg(m2 ). If d = 0, the statement is clear because the tensor products are over k. Suppose now d > 0, and the statement is known for d − 1. Then suppose, without loss of generality, that x1 divides m2 , m2 = x1 n. We have (m1 ⊗m2 )⊗1 = (%(x1 )m1 ⊗n)⊗1−(x1 m1 ⊗n)⊗1 = −(x1 m1 ⊗n)⊗1. By induction, this last element is in the image of ψl . Definition 2.5. Given a module M over the k-algebra k[x], we define a new k[x] = M ⊗S k[x], with the k[x]module structure on the k-module M as follows: M module structure induced from the multiplication on the right factor. Remark 2.6. is an exact homogeneous covariant additive functor. Moreover, = Id ∼ ∼ and k[x][a] =k[x] k[x][a]. If i is an ideal in k[x], stable under S, then k[x]/i =k[x] τ k[x]/i. In particular, this holds for i = m . Lemma 2.7. Let M and N be finitely generated graded k[x]-modules. Then % induces a k[x]-module structure on the k[x] ⊗k k[x]-module M ⊗k N. With this structure, (M ⊗k N) ⊗k[x] k ∼ =k M ⊗k[x] N. Moreover, (M ⊗k N) ⊗k[x] k is naturally a (k[x] ⊗k k[x]) ⊗k[x] k-module. Using ψl and ψr , it obtains two k[x]-module structures.

With the With the structure induced by ψl , (M ⊗k N) ⊗k[x] k ∼ =k[x] M ⊗k[x] N.


313

⊗k[x] N. structure induced by ψr , (M ⊗k N) ⊗k[x] k ∼ =k[x] M Proof. We prove only that with the structure induced by ψl ,

(M ⊗k N) ⊗k[x] k ∼ =k[x] M ⊗k[x] N. All the other statements follow easily from this. We define a k-linear map , from

by extending the natural quotient map , ˜ from M ⊗k (M ⊗k N) ⊗k[x] k to M ⊗k[x] N, ∼

N =k M ⊗k N to M ⊗k[x] N: / M ⊗k[x] N

M ⊗k N

(M ⊗k N) ⊗k[x] k _ _/ M ⊗k[x] N.

We have to check that the map is well defined, bijective, and a morphism of k[x]modules. We omit the simple verifications that the map is well defined and injective. To prove that it is surjective, we proceed exactly as before to define ,−1 : M ⊗k[x]

→ (M ⊗k N) ⊗k[x] k by inducing a map from the natural quotient map ,˜−1 from N

to (M ⊗k N) ⊗k[x] k: M ⊗k N ∼ =k M ⊗k N M ⊗k N

/ (M ⊗k N) ⊗k[x] k

_ _/ (M ⊗k N) ⊗k[x] k. M ⊗k[x] N

We have by inspection that ,,−1 = ,−1 , = Id, and therefore , is also surjective. ˜ clearly is, It remains to be checked that , is a map of k[x]-modules. However, , with respect to multiplication on the left factor on M ⊗k N. Through the quotient map to (M ⊗k N) ⊗k[x] k, this induces the structure induced by ψl on (M ⊗k N) ⊗k[x] k. Therefore , is also a map of k[x]-modules, as desired. Remark 2.8. We have k[x] ⊗k k[x] ∼ =k[x]

∞

k[x][−i]

n+i−1 i

i=0

where k[x][i] is the k[x]-module k[x] with degrees shifted by i. In particular, k[x]⊗k k[x] is a free k[x]-module. The following theorem is fundamental for the proof of the local vanishing theorem. We have not encountered any similar result in the literature. Theorem 2.9. Let M and N be finitely generated graded k[x]-modules. Then % induces a structure of k[x]-module on the k[x] ⊗k k[x]-module M ⊗k N. With this k[x]

j. structure, and for all i, j , Tor ik[x] (M ⊗k N, k)j ∼ =k Tor i (M, N)

314

MICHELE GRASSI

Proof. Let (C∗ , d1 ) be a minimal resolution of M and (D∗ , d2 ) of N. We then consider the double complex C∗ ⊗k D∗ : ↓ →

C i ⊗ k Dj

··· → ···

↓ →

C i ⊗ k D0

↓

···

↓

···

···

···

↓

···

↓

→ C 0 ⊗ k Dj

→ ···

→ C0 ⊗k D0 .

A standard spectral sequence argument shows that the total complex of this double complex is a free-graded resolution (not minimal) of M ⊗k N over k[x] ⊗k k[x]. Indeed, the E1 term of one of the two spectral sequences looks like ··· ···

(0)

··· ···

··· ···

M ⊗ k Dj

···

··· ···

··· →

···

(0) ···

→

M ⊗ k D0 ,

and the E2 term has all zeroes. Note that the tensor products are over k, not over k[x]. From Remark 2.8, the resolution is free-graded as it is over k[x]. It follows that the total complex associated to the double complex (C∗ ⊗k D∗ )⊗k[x] k computes the Tor group in the left-hand side of the equality that we want to show. Using the first standard filtration of this double complex and computing the spectral sequence p,q associated to it, we get the expression for the E1 term: E1 = Hq ((Cp ⊗k D∗ ) ⊗k[x] k, (1⊗d2 )⊗1). By using the map ψr to induce k[x]-structures, the complex ((Cp ⊗k D∗ ) ⊗k[x] k, (1 ⊗ d2 ) ⊗ 1) is easily seen by Lemma 2.7 to be isomorphic over k[x] to p ⊗k[x] D∗ , 1 ⊗ d2 ). Hence, the E0 term may be taken to look like (C ↓ p ⊗k[x] Dq → C

→

··· → ···

↓ →

p ⊗k[x] D0 C

↓

···

↓

···

···

···

↓

···

↓

0 ⊗k[x] Dq C

→ ···

0 ⊗k[x] D0 . → C p,q

p is still free over k[x] from Remark 2.6, that E As a consequence, we have, as C 1

=


315

p,0 p ⊗k[x] N. Now, from Lemma 2.7 C p ⊗k[x] N ∼

0, if q > 0, and E1 ∼ =C =k Cp ⊗k[x] N. p,0 ∼

using ψl . By Moreover, still using Lemma 2.7, we see that E1 =k[x] Cp ⊗k[x] N, inspection, the differential of E1 is d1 ⊗ 1, under the above isomorphism. From this it follows that p,q

E2

∼ =k 0

if q > 0,

p,0

E2

∼ =k Tor pk[x] M, Nˆ .

Therefore the spectral sequence degenerates at the second term, and we obtain the desired result. 3. A vanishing theorem for Koszul homology. If M is a k[x]-module, the Hopf algebra structure introduced before puts a k[x]-module structure on M ⊗k M, as we have seen, by imposing p(x)(m ⊗ n) = %(p(x))(m ⊗ n). We want to extend this method to build some other module structures. Definition 3.1. % induces a map of k-algebras %p : k[x] → p k (k[x]) by the rule %q+1 (xi ) = xi ⊗ 1 ⊗ · · · ⊗ 1 + 1 ⊗ %q (xi ) (using the isomorphism p k (k[x]) = k[x] ⊗k p−1 k (k[x])). If M is a k[x]-module, there is a k[x]-module structure on p−1 p k (M) = M ⊗k k (M) induced by %p through the rule p(x)(m1 ⊗ · · · ⊗ mp ) = %p (p(x))(m1 ⊗ · · · ⊗ mp ). p Remark 3.2. We have xi (m1 ⊗ · · · ⊗ mp ) = j =1 m1 ⊗ · · · (xi mj ) · · · ⊗ mp . Now, note that there is an action of the symmetric group ᏿p over p k (M), given as follows: σ (v1 ⊗ · · · ⊗ vp ) = 2(σ )vσ (1) ⊗ · · · ⊗ vσ (p) , where 2(σ ) is the sign of σ . Relative to this action, there is an operator π∧ = (1/p!) σ ∈᏿p σ . Here we are using the fact that p! is invertible in k. p Lemma 3.3. π∧ is a k[x]-endomorphism of k (M). Proof. We see that xi π∧ (v1 ⊗ · · · ⊗ vp ) 1 2(σ )vσ (1) ⊗ · · · ⊗ vσ (p) = xi p! = =

1 p! 1 p!

σ ∈᏿ p

σ ∈᏿ j =1 p σ ∈᏿ j =1

2(σ )vσ (1) ⊗ · · · ⊗ xi vσ (j ) ⊗ · · · ⊗ vσ (p) j j j j 2(σ ) 1 − δσ (1) + δσ (1) xi vσ (1) ⊗ · · · ⊗ 1 − δσ (p) + δσ (p) xi vσ (p)

p 1 j j j j = 2(σ ) 1 − δσ (1) + δσ (1) xi vσ (1) ⊗ · · · ⊗ 1 − δσ (p) + δσ (p) xi vσ (p) p! j =1

σ ∈᏿

316

MICHELE GRASSI

=

p

π∧ v1 ⊗ · · · ⊗ xi vj ⊗ · · · ⊗ vp = π∧ xi v1 ⊗ · · · ⊗ vp .

j =1

p As usual, this operatorsatisfies π∧ ◦ π∧ = π∧ , and its image inside k (M) is isomorphic (over k) to p k (M). Moreover, the inclusion p k (M) → p k (M) provides a splitting of the exact sequence of k[x]-modules 0 −→ Ker(π∧ ) −→

p

p π∧ k (M) −−→ k (M) −→ 0.

We have therefore proved the following lemma. p p Lemma 3.4. k (M) is a split-graded sub-k[x]-module of k (M). p Remark 3.5. The module structure on k (M) is made so that

xi m1 ∧ · · · ∧ mp =

p

m1 ∧ · · · x i mj · · · ∧ m p ,

j =1

where m1 ∧ · · · ∧ mp = π∧ (m1 ⊗ · · · ⊗ mp ). Proof. This is an immediate consequence of Remark 3.2 and Lemma 3.3. Definition 3.6. If M is a k[x]-module, with Kt (x, M)j , we indicate the j th graded part of the homology of the complex obtained tensoring the Koszul complex over x = (x1 , . . . , xn ) with M over k[x]. This is a standard definition. These objects are called Koszul homology groups of M (over the polynomial ring k[x]). Note that the Koszul complex has all the maps of degree 1, so it is not, properly speaking, in the category of graded k[x]-modules. k[x] Proposition 3.7. If M is a k[x]-module, then Kt (x, M)j ∼ =k[x] Tor t (M, k)j +t .

Proof. This is just an application of the fact that the Koszul complex is a free resolution of k over k[x]. The above proposition is standard. For a general introduction to Koszul homology, see [BH, p. 39] or [G]. Corollary 3.8. Kt (x, −)j is an additive functor from the category of k[x]modules to the category of k-modules. The main result of this section is the following theorem. Theorem 3.9. Let W be a vector space of dimension s over k, and let m be the irrelevant maximal ideal in k[x]. Then Kt (x, p k (W ⊗k k[x]/mτ ))j = 0 whenever t ≥ p and j < p(τ − 1). The proof is at the end of this section.


317

Remark 3.10. The previous result can be easily improved as follows: In the notation of the theorem above, Kt (x, p k (W ⊗k k[x]/mτ ))j = 0 whenever t ≥ p and j = p(τ − 1), or t < p and j = t (τ − 1). Lemma 3.11. For a k[x]-module M, let us call mt (M) the minimum value j for which Kt (x, M)j = 0. Then, for t ≥ 1, mt M ⊗k k[x]/mτ ≥ mt−1 (M) + (τ − 1). Proof. From Theorem 2.9, Remark 2.6, and Proposition 3.7, we have that Kt (x, k[x] τ )j +t . Given a minimal resolution (C∗ , d) of M ⊗k k[x]/mτ )j ∼ =k Tor t (M, k[x]/m M, clearly Ct is generated in a degree greater than or equal to mt (M) + t. For a homogeneous element x in Ct to be sent into mρ Ct−1 by d, it must be that deg(x) ≥ mt−1 (M) + t − 1 + ρ or d(x) = 0. Therefore deg(x) < mt−1 (M) + t − 1 + ρ and d(x) ∈ mρ Ct−1 imply that d(x) = 0, and hence x = d(x ) for some x ∈ C t+1 . So, to get a nonzero homology element in the middle term of the complex Ct+1 ⊗k[x] k[x]/mτ −→ Ct ⊗k[x] k[x]/mτ −→ Ct−1 ⊗k[x] k[x]/mτ , we must go to degree at least mt−1 (M) + τ + t − 1. This, together with the above identification of the Koszul homology group as a Tor group, proves that mt (M ⊗k k[x]/mτ ) + t ≥ mt−1 (M) + τ + t − 1 as desired. Lemma 3.12. Kt (x, p k (W ⊗k[x]/mτ ))j = 0 whenever t ≥ p and j < p(τ −1). Proof. We prove this lemma by induction on p. If p = 1, we have to prove that Kt (x, k[x]/mτ )j = 0 whenever t ≥ 1 and j < τ − 1. This is clear from Lemma 3.11 applied to M = k. Suppose now that we know the statement for p − 1 ≥ 1. We have   p p−1 τ ∼ τ τ k W ⊗ k[x]/m =k[x]  k W ⊗ k[x]/m  ⊗k W ⊗k k[x]/m   p−1 τ τ ∼ =k[x]  k W ⊗ k[x]/m ⊗k k[x]/m  ⊗k W. From Lemma 3.11, we have that     p−1 p−1 τ τ τ mt  k W ⊗ k[x]/m ⊗k k[x]/m  ≥ mt−1  k W ⊗ k[x]/m  + (τ − 1). By induction we conclude therefore that   p−1 τ τ mt  k W ⊗ k[x]/m ⊗k k[x]/m  ≥ p(τ − 1) for t ≥ p.

318

MICHELE GRASSI

This concludes the proof, because clearly  mt 

p−1 k



τ



W ⊗ k[x]/m ⊗k k[x]/mτ  ⊗k W    p−1 τ τ = mt  k W ⊗ k[x]/m ⊗k k[x]/m  .

Proof of Theorem 3.9. The proof of Theorem 3.9 follows immediately if we observe that, from Corollary 3.8 and Lemma 3.4 applied to the case M = W ⊗k[x]/mτ ,  Ktx,

   p−1 τ W ⊗ k[x]/mτ  ∼ =k[x] Ktx, Ker(π8 ) ⊕k[x] k W ⊗ k[x]/m 

p−1 k

j

j



∼ = Kt x, Ker(π8 ) j ⊕k[x] Ktx,

p−1 k

 W ⊗ k[x]/mτ  .

j

We apply Lemma 3.4 to the case M = W ⊗ k[x]/mτ . 4. The algebraic reduction of the vanishing problem. We now reduce the problem of local vanishing of characteristic cohomology to an algebraic problem. This is done in two steps, obtaining a first algebraic reduction and then a second algebraic reduction. We use the notation of Proposition 1.6. In particular, we use Einstein’s summation convention for repeated indices. Proposition 4.1. We have dθiα1 ,...,it = −θiα1 ,...,it ,l ∧ dx l for all 0 ≤ t < k − 1 (if t = 0, then by dθiα1 ,...,it we mean dθ α ) and dθiα1 ,...,ik−1 = −dpiα1 ,...,ik−1 ,l ∧ dx l . Proof. If k < 1, then there is nothing to prove. Suppose therefore k ≥ 1. If t < k − 1, we have dθiα1 ,...,it = −dpiα1 ,...,it ,l ∧ dx l = −(θiα1 ,...,it ,l + piα1 ,...,it ,l,h dx h ) ∧ dx l = −θiα1 ,...,it ,l ∧ dx l − piα1 ,...,it ,l,h dx h ∧ dx l . However, as piα1 ,...,it ,l,h is symmetric in l, h and dx h ∧ dx l is antisymmetric in l, h, the second summand is zero, while the first one is as desired. For the case t = k − 1, we just have to observe that d 2 = 0 and that d is a derivation of the exterior algebra. Given an element in q (ᐁ), we can always write it in pa unique way as a sum of p0

α

k−1

k−1 “monomials” of the form m = θ α0 ∧ · · · ∧ θ α0 ∧ · · · ∧ θ k−1 1

v ∈ C ∞ (ᐁ) < dx i , dpiα1 ,...,ik > ∩ r (ᐁ).

i1

k−1 ,...,ik−1

∧ v with nonzero


319

Definition 4.2. For any element v ∈ q (ᐁ), we define ∂(v) ∈ q+1 (ᐁ) by   ∂ is a derivation of ∗ (ᐁ),    α  ∂ θ α for all i1 , . . . , it , t ≤ k − 1, i1 ,...,it = d θi1 ,...,it ∞  ∂ C (ᐁ) = (0),      α ∂ dpi1 ,...,ik = 0 for all i1 , . . . , ik . For any monomial m as before, we define also the following. Definition 4.3. If m = 0, pj (m) = :{different θiα1 ,...,ij in m}. We define p(m) = (p0 (m), . . . , pk−1 (m)) and |p(m)| = p0 (m) + · · · + pk−1 (m). Note that we do not put pk in p. Clearly, we have r = q − |p(m)|. Definition 4.4 (Characteristic weight). (1) Given p = (p0 , . . . , pk−1 ), we define w(p) = kp0 + (k − 1)p1 + · · · + pk−1 . (2) For any monomial m as before, different from 0, we define w(m) = w(p(m)). (3) If v ∈ ∗ (ᐁ), we define w(v) = min(w(m) | m appears in the expression of v). We say that v is w-homogeneous if all the monomials in its expression have the same w. We call w the characteristic weight. Lemma 4.5. For any v that is w-homogeneous, we have w(∂(v)) = w(v) − 1 if ∂(v) = 0, and w((d − ∂)(v)) > w(v) − 1 if (d − ∂)(v) = 0. Proof. For a typical element m as above, we have pk−1 p0 1 αk−1 d θ α0 ∧ · · · ∧ θ α0 ∧ · · · ∧ θ k−1 ∧ v k−1 =

i1

,...,ik−1

α

pk−1

k−1 (−1)i−1 θ α0 ∧ · · · ∧ d(θi ) ∧ · · · ∧ θ k−1 1

i1

i≤|p| |p| α01

+ (−1) θ

∧···∧θ

p

α0 0

k−1 ,...,ik−1

∧v

p

k−1 αk−1

∧ · · · ∧ θ k−1 i1

k−1 ,...,ik−1

∧ d(v).

We have indicated formally with θi the ith element in the wedge product above. It is clear that the first summation is ∂(m), while the second one is (d −∂)(m). In the first summation all summands have w decreased by 1, while in the last summand w stays the same or increases. This proves the statement. Definition 4.6. We have F w,q = {v ∈ q (ᐁ) | w(v) ≥ w}. Lemma 4.7. The complex F w+1,q−1 /F w+2,q−1 → F w,q /F w+1,q → F w−1,q+1 / is isomorphic to a split subcomplex of the following complex:

F w,q+1

q−1

U ⊗R C ∞ (ᐁ) −→

q

U ⊗R C ∞ (ᐁ) −→

q+1

U ⊗R C ∞ (ᐁ),

320

MICHELE GRASSI

where U = ωi , ηiα1 ,...,ik , ηiα1 ,...,ik−1 , . . . , ηα R and the differential is the derivation of the exterior algebra of U acting as ∂(ηiα1 ,...,iτ ) = −ηiα1 ,...,iτ ,l ∧ ωl , if τ ≤ k − 1, and as ∂(ηiα1 ,...,ik ) = 0 and ∂(ωi ) = 0. Proof. Let us define Ij to be the vector space span of the ηiα1 ,...,ij , and let J be the

span of the ωi , ηiα1 ,...,ik . The identification we are looking into is F w,q ∼ = F w+1,q

p0

q−|p|

pk−1

I 0 ⊗R · · · ⊗ R

Ik−1 ⊗R

J ⊗R C ∞ (ᐁ).

p:w(p)=w

The map providing this isomorphism is   −→ ηiα1 ,...,ij θα   i1 ,...,ij dpiα1 ,...,ik −→ ηiα1 ,...,ik    i dx −→ ωi . Then we extend this by linearity to all the exterior algebras. Observe now that in the quotient complex F w+1,q−1 F w,q F w−1,q+1 −→ w+1,q −→ , w+2,q−1 F F F w,q+1 the induced differential comes only from ∂, as d − ∂ is mapped to zero. Moreover, under the above isomorphism, the ∂ of the quotient complex is sent to the ∂ of the “algebraic” complex q−1

∞

U ⊗R C (ᐁ) −→

q

∞

U ⊗R C (ᐁ) −→

q+1

U ⊗R C ∞ (ᐁ).

This just means that we have built a map of complexes. In the algebraic complex, we can define a weight w (see also Definition 4.8), and the image is just obtained by fixing w + q. From this it follows that the image of the above map is actually a split subcomplex. In view of the result of Lemma 4.7, it is interesting to isolate the algebraic part of the complex considered there. Definition 4.8 (First algebraic reduction). The first algebraic reduction is the complex · · · → q−1 U → q U → q+1 U → · · · , where U is a k-vector space U = ωi , ηiα1 ,...,ik , ηiα1 ,...,ik−1 , . . . , ηα , = ωi , and the differential is the derivation of the exterior algebra of U acting as ∂(ηiα1 ,...,iτ ) = −ηiα1 ,...,iτ ,l ∧ ωl if τ ≤ k − 1, and as ∂(ηiα1 ,...,ik ) = 0 and ∂(ωi ) = 0. Let us define Ij to be the vector space span of the ηiα1 ,...,ij , J be the span of the ωi , ηiα1 ,...,ik , and the span of just the ωi . We then


321

define a weight w that on m ∈ p0 I0 ⊗R · · · ⊗R pk−1 Ik−1 ⊗R q−|p| J takes the value w(m) = kp0 + · · · + pk−1 . We indicate the grade-w piece of the complex as · · · → ( q−1 U )w+1 → ( q U )w → ( q+1 U )w−1 → · · · . Theorem 4.9. H q (Ᏽ∗ (ᐁ), d) = 0 for 0 ≤ q ≤ n for any open set ᐁ small enough and contractible, provided that the complex     q q−1 q+1 · · · −→  U U U −→ −→  −→ · · · w+1

w

w−1

(the grade-w piece of the first algebraic reduction) is exact at the indicated place when w > 0 and q ≤ n. Proof. We may as well assume that ᐁ is included in a standard chart for Gkn (M n+s ) and that it is smooth and topologically trivial. Suppose that v ∈ Ᏽq (ᐁ) = Ᏽ(ᐁ) ∩

q (ᐁ), with dv = 0, v = 0, and q ≤ n. Then modulo d Ᏽ, v is in the ideal generated by the θ ’s. Indeed, the only problem can arise from multiples of the dθ but not of the θ . However, any multiple of a dθ is equivalent modulo d Ᏽ to a multiple of θ, as can be verified easily. For our purposes (proving that v is a boundary), we can replace v by this new element, obtained by adding a boundary to it. We may assume, therefore, without loss of generality, that for some w with w > 0, v ∈ F w,q . Let w(v) be the smallest such v ∈ F w(v),q \F w(v)+1,q . From the exactness of the first algebraic reduction, we see that there is a v ∈ F w+1,q−1 with v −d(v ) =∈F w+1,q . Proceeding by induction, we get that there is a v1 such that v − d(v1 ) =∈ w F w,q = (0). This proves that v can be reduced to zero via boundaries, as desired. In the following, we give a further algebraic translation of the first algebraic rep duction. Consider first the k[x]-module W ⊗k k[x]. We grade ∧k (W ⊗k k[x]/mk+1 ) using the total degree, as usual. Referring to Definition 4.8, we give the following. Definition 4.10. We have( q U )tw = {v ∈ ( q U )w | t elements from appear}. Lemma 4.11. The two following complexes are isomorphic: q t U )w → ( q+1 U )t+1 (1) · · · → ( q−1 U )t−1 w+1 → ( w−1 → · · · ; (2) the graded piece of a Koszul complex:   n−(t−1)

q−(t−1) · · · −→  W ⊗k k[x]/mk+1

 ⊗k ∧k

(q−t)k−(w+1)

n−t

q−t −→

⊗k ∧k W ⊗k k[x]/mk+1 

n−(t+1)

−→ 



(q−t)k−w

q−(t+1)

 ⊗k ∧k W ⊗k k[x]/mk+1

(q−t)k−(w−1)

−→ · · · .

322

MICHELE GRASSI

Proof. Pick a basis {eα | α = 1, . . . , s} for W . Remember that we have already an ordered basis {ωi | i = 1, . . . , n} for . Pick an element of the form ωJ ⊗ ((eα1 ⊗ q−t k+1 m1 ) ∧ · · · ∧ (eα1 ⊗ mq−t )) ∈ n−t ⊗k (∧k (W ⊗k k[x]/m )), where the mi ’s are monomials of degree |mi | smaller than or equal to k, with i |mi | = pk −w, and J is an ordered multiindex of length n−t. Write mi = xEi , where the Ei are multiindices of length |mi |. This element is sent to B(ωJ ⊗ ((eα1 ⊗ m1 ) ∧ · · · ∧ (eα1 ⊗ mq−t ))) = α α1 ∧· · ·∧ηEq−t in the corresponding space of the first complex, where (−1)n−t ∗ωJ ∧ηE q−t 1 ∗ is the usual Hodge ∗ operator on the space ∗ . To define ∗, we use the given ordered basis for (as in the classical case one uses an orthonormal ordered basis). Then B is extended by k-linearity, and in this way B becomes an isomorphism over k. A direct computation shows that an element of degree j is sent to an element with w = (q −t)k −j , and therefore the degrees in the two complexes correspond correctly. We have only to check that B commutes with the differentials in the two complexes. By k-linearity, it is enough to prove this for elements of the kind used above. In the following, we indicate with C the contraction operator, and with (∂/∂ωi ) the dual to ωi with respect to the given bilinear form on . We call d the differential in the second complex. We have ∂ B ωJ ⊗ eα1 ⊗ m1 ∧ · · · ∧ eα1 ⊗ mq−t

α1 αq−t ∧ · · · ∧ η = ∂ (−1)n−t ∗ ωJ ∧ ηE Eq−t 1 = (−1)

n−t

q−t αh α1 α (−1)h−1 ∗ ωJ ∧ ηE ∧ · · · ∧ ∂ ηE ∧ · · · ∧ ηEq−t q−t h 1 h=1

= (−1)n−t

q−t n αh J α1 α (−1)h ∧ ωi ∧ · · · ∧ ηEq−t ∗ ω ∧ ηE1 ∧ · · · ∧ ηE q−t h ∪{i} h=1

=

i=1

q−t n h=1 i=1

α αh α1 (−1)n−t ∗ ωJ ∧ ωi ∧ ηE ∧ · · · ∧ ηE ∧ · · · ∧ ηEq−t q−t h ∪{i} 1

q−t n α J αq−t αh n−t i 1 = (−1) ηE1 ∧ · · · ∧ ηEh ∪{i} ∧ · · · ∧ ηEq−t ∗ω ∧ω ∧ i=1

=

n i=1

h=1

q−t α J αq−t αh i 1 ω ∧ ∗ω ∧ ηE1 ∧ · · · ∧ ηEh ∪{i} ∧ · · · ∧ ηEq−t h=1

(see Proposition 4.12) q−t n ∂ αq−t αh α1 n−t+1 J (−1) ∗ Cω ∧ ηE1 ∧ · · · ∧ ηEh ∪{i} ∧ · · · ∧ ηEq−t = ∂ωi i=1

h=1


323

q−t ∂ J =B C ω ⊗ eα1 ⊗ xE1 ∧ · · · ∧ eαh ⊗ xEh ∪{i} ∧· · ·∧ eαq−t ⊗xEq−t i ∂ω h=1 q−t ∂ J E1 Eq−t =B C ω ⊗ xi eα1 ⊗ x ∧ · · · ∧ eαq−t ⊗ x ∂ωi h=1

= B d ωJ ⊗ eα1 ⊗ xE1 ∧ · · · ∧ eαq−t ⊗ xEq−t as desired. The following simple fact is standard from Hodge theory.

Proposition 4.12. If ωJ has degree n − t (with the notation as in the previous proof) and dim( ) = n, then ωi ∧ (∗ωJ ) = (−1)n−t+1 ∗ ((∂/∂ωi ) C ωJ ). Theorem 4.13 (Second algebraic reduction). We have H q (Ᏽ∗ (ᐁ), d) = 0 for 0 ≤ q≤ n for any open set ᐁ small enough and contractible, provided that Kn−t (x, q−t (W ⊗k k[x]/mk+1 ))j = 0 whenever n − t ≥ q − t and j < (q − t)k. Proof. From Theorem 4.9, it is enough to show that the complex     q q−1 q+1 −→ −→  −→ · · · U U U · · · −→  w+1

w

w−1

(the grade w piece of the first algebraic reduction as described in Definition 4.8) is exact at the indicated place when w > 0 and q ≤ n, provided that Kn−t (x, q−t (W ⊗k k[x]/mk+1 ))j = 0 whenever n − t ≥ q − t and j < (q − t)k. From Lemma 4.11 this is true, because the above complex and the Koszul complex computing the Koszul group are isomorphic in a way that correctly matches degrees. 5. The local vanishing theorem Theorem 5.1 (Local vanishing of characteristic cohomology). For any open set q (k) ᐁ ⊆ Gn (M), we have that p∗ : HDR (ᐁ) → H q ( ∗ /Ᏽ∗ (ᐁ), d) is an isomorphism for 0 ≤ q < n and is injective for q = n. (k)

Proof. First of all, we note that, for ᐂ ⊆ Gn (M) small enough and contractible, = 0 for 0 ≤ q ≤ n. Indeed, from Theorem 4.13, this is equivalent to showing that Kn−t (x, q−t (W ⊗k k[x]/mk+1 ))j = 0 whenever n − t ≥ q − t and j < (q −t)k. This is the content of Theorem 3.9. Let us now consider the complex of sheaves 0 → ( Ᏽ(k) )0 → · · · → ( Ᏽ(k) )n → · · · . The previous vanishing result reveals that this complex of sheaves is exact up to level n. Moreover, all the sheaves involved are fine. Therefore the first spectral sequence of hypercohomology abuts to zero in a degree smaller than or equal to n, while the second one abuts to the cohomology of the global sections of the previous complex over ᐁ. This shows that H q (Ᏽ∗ (ᐁ), d) = 0 H q (Ᏽ∗ (ᐂ), d)

324

MICHELE GRASSI

for 0 ≤ q ≤ n. We conclude using the long exact sequence in cohomology associated to the exact sequence of sheaves 0 −→ Ᏽ∗ , d −→ ∗ (ᐁ), d −→ ∗ (ᐁ)/Ᏽ∗ , d −→ 0. In the remainder of this section we consider the infinite prolongation case, men(k) tioned in the introduction. We have seen that there are natural maps πk,h : Gn (M) → (h) Gn (M), k > h. These maps are smooth and form an inverse system in the category ∗ : ∗ (G(h) (M)) → ∗ (G(k) (M)), k > h, of smooth manifolds. We have also that πk,h n n is a well-defined map of differential graded algebras, which sends Ᏽ(h) into Ᏽ(k) . We therefore have an array of induced maps of complexes that in the end provide a natural map between characteristic cohomologies (h) (k) ∗ q πk,h : H q ∗ G(h)

∗ G(k) n (M) /Ᏽ , d −→ H n (M) /Ᏽ , d . ∗ form a direct system in the category of differential graded algeThe maps πk,h (∞) (k) bras. We want to consider the object Gn (M) = lim ← Gn (M). This limit does k not exist in the category of smooth finite-dimensional manifolds. (For example, (k) (k+1) (M) there is always a dimensional increase.) However, from Gn (M) to Gn it exists as an infinite-dimensional smooth manifold. We are really interested in (k) lim → H q ( ∗ (Gn (M))/Ᏽ(k) , d), which is well defined in the category of vector k

(∞)

spaces. This is the same if, instead of all of Gn (M), we consider an open set −1 ᐁ inside it, eventually stable under πk+1,k , so that we can define for any q and any such ᐁ the vector space lim → H q ( ∗ (ᐁ)/Ᏽ(k) , d). Note also the simple fact that k

lim → H q ( ∗ (ᐁ)/Ᏽ(k) , d) = H q (lim → ( ∗ (ᐁ)/Ᏽ(k) , d)). We have natural maps k

∗ π∞,k :H

q

k

∗ (ᐁ) /Ᏽ(k) , d −→ lim H q ∗ (ᐁ) /Ᏽ(h) , d , →

h

∗ : ∗ (ᐁ)/Ᏽ(k) → lim→ ∗ (ᐁ)/Ᏽ(h) . With this induced by the natural maps π∞,k h notation, we can now state and prove the following theorem.

Theorem 5.2 (Vinogradov). For any open set ᐁ of the form above, which from some level k onward is contractible, lim → H q ( ∗ (ᐁ)/Ᏽ(k) , d) = (0) for 0 < q < n. k

You can find a proof of this theorem in [BG1, pp. 555–556]. Here we sketch how one can use the machinery developed in this paper to get a fairly simple proof. Proof. Let 0 < q < n, and let φ ∈ lim → H q ( ∗ (ᐁ)/Ᏽ(k) , d), d(φ) = 0. We want k to prove that there exists a ψ with φ − d(ψ) ∈ Ᏽ∞ . Take a k ' 0 so that there are (k) a contractible open set ᐁk ⊂ Gn and a form φk ∈ q (ᐁk )/Ᏽ(k) that in the limit reduce to ᐁ and φ, respectively. These two objects exist from the definition of a limit ∗ and that of ᐁ. Moreover, from the injectivity of πk+1,k for all k, d(φk ) ∈ Ᏽ(k) . From


325

Theorem 5.1, there exists a ψk ∈ q−1 (ᐁk ) such that φk −d(ψk ) ∈ Ᏽk . However, this equation can be “passed to the limit,” obtaining exactly what we wanted. 6. The nth cohomology group. Here we discuss the cohomology group H n q q (k) ( ∗ /Ᏽ, d). Because we proved that, ∀q < n, H q ( ∗ /Ᏽ, d) ∼ = HDR (Gn ) ∼ = HDR (G (n, T(M))), this is the next interesting case to study. It turns out that this case is qualitatively different from the previous ones, because, for example, we don’t have local vanishing. The results in this section have been obtained by different classical methods—basically, integrating by parts. We will stick to the algebraic notation. (k)

Definition 6.1. A point transformation of Gn (M) is a smooth automorphism that sends the ideal Ᏽ(k) to itself. Remark 6.2. A point transformation is easily seen to send I (k) to itself. If a transformation sends I (k) to itself, it sends the differential ideal that I (k) generates, Ᏽ(k) , to itself. Therefore, point transformations could also be defined as the automorphisms that leave I (k) fixed. (k)

Definition 6.3. A Lagrangian density on Gn (M) is defined as a (local) section (k) (k) over Gn (M) of the bundle D (k) ⊂ n (Gn (M))/Ᏽ(k) , inside the pullback of n (M). Locally, any such section has a lifting of the form L(x i , zα , . . . , piα1 ,...,ik )d x, ¯ where by 1 n d x¯ we mean dx ∧ · · · ∧ dx . Lemma 6.4. For any Lagrangian density L(x i , zα , . . . , piα1 ,...,ik )d x¯ as before, we ¯ ∈ (Ᏽk ). have d(L(x i , zα , . . . , piα1 ,...,ik )d x) Proof. In the expression for d(L(x i , zα , . . . , piα1 ,...,ik )d x), ¯ we get a summation of terms, one for each partial derivative of L with respect to its arguments. There are basically three different cases. (1) The partial derivatives with respect to the x i ’s. As d x¯ involves all the x i ’s, we do not get any contribution from this. (2) The partial derivatives with respect to the zα or the pIα ’s for |I | < k: For these, remember the formulas  α α α i  θ = dz − pi dx , ···   α θi1 ,...,ik−1 = dpiα1 ,...,ik−1 − piα1 ,...,ik−1 ,l dx l . From these, it follows that  α α ¯  θ ∧ d x¯ = dz ∧ d x, ···   α θi1 ,...,ik−1 ∧ d x¯ = dpiα1 ,...,ik−1 ∧ d x, ¯

326

MICHELE GRASSI

and therefore all the summands coming from these partials are actually in I (k) . (3) The partial derivatives with respect to the pIα ’s for |I | = k. From the formulas above, we know that dθiα1 ,...,ik−1 = −dpiα1 ,...,ik−1 ,l ∧ dx l . Therefore, indicating with ˆ i ∧· · ·∧dx n and for I = I ∪{i0 }, we get |I |+1 = |I | = k x¯[i] = (−1)i−1 dx 1 ∧· · ·∧ dx and dpIα ∧ d x¯ = ±dθIα ∧ d x¯[i0 ] ∈ Ᏽ(k) .

With reference to the nth characteristic cohomology group, some of the most interesting problems arise from studying the following. Conjecture/Question [Gri]. Is it true that any class in H n+1 (Ᏽ, d), which comes from H n ( ∗ /Ᏽ, d) under the natural coboundary map, has a natural point-equivariant representative? At this time this conjecture remains open. Theorem 6.5. There is a unique map of presheaves n+1 PC : H n+1 (Ᏽ, d) −→ Z I (1) ∧ nM + I ∧ I , which, once composed with the natural map n+1 Z I (1) ∧ nM + I ∧ I −→ H n+1

Ᏽ ,d , Ᏽ2

induces the natural map H n+1 (Ᏽ, d) → H n+1 (Ᏽ/Ᏽ2 , d) at the level of sheaves. At the level of global sections, PC : H n+1 (Ᏽ, d) → (I /I 2 ) is Aut(M)-equivariant. Lemma 6.6. Given a local Lagrangian density Ld x, ¯ there exists 8 ∈ (I )n such (1) n+1 that d(Ld x¯ − 8) ∈ (I ∧ d x¯ + I ∧ I ) . Proof. In terms of the filtration F ω,n+1 , we can modify Ld x¯ by an element 8 ∈ (I )n so that d(Ld x) ¯ lands inside F k,n+1 . For this, it is clearly enough to show that the graded piece · · · −→

n

t−1 U

n+1 t n+2 t+1 U U −→ −→ −→ · · ·

ω+1

ω

ω−1

of the first algebraic approximation is acyclic at the indicated place whenever ω < k. From Lemma 4.13, this is equivalent to showing that for all t n−t+1

k+1 Kn−t x, =0 W ⊗k k[x]/m j

whenever j > k. This is in Remark 3.10. Lemma 6.7. We have I (1) ∧ d x¯ + I ∧ I = I (1) ∧ nM + I ∧ I .


327

Proof. The inclusion I (1) ∧ d x¯ + I ∧ I ⊂ I (1) ∧ nM + I ∧ I is clear. For the other direction, note that dx i , dzα C ∞ (G(k) (M)) = dx i , θ α C ∞ (G(k) (M)) . n

n

Lemma 6.8. If 81 and 82 satisfy the properties of Lemma 6.6, then d(81 −82 ) ∈ I ∧I. Proof. This is clear because d(I ) ∩ (I (1) ∧ d x¯ + I ∧ I ) ⊂ I ∧ I . Proof of Theorem 6.5. From Lemmas 6.6, 6.7, and 6.8, we see that there is a well-defined map PC : H n+1 (Ᏽ, d) → Z(I (1) ∧ nM + I ∧ I )n+1 . Moreover, from its definition in Lemma 6.6, we see that, once composed with the natural map (Z(I (1) ∧

nM + I ∧ I ))n+1 → H n+1 (Ᏽ/Ᏽ2 , d), PC induces the natural map H n+1 (Ᏽ, d) → H n+1 (Ᏽ/Ᏽ2 , d) at the level of sheaves. The equivariance with respect to Aut(M) is an immediate consequence of the local uniqueness, which gives PC canonically on every small open set. References [BT] [BH] [BCGGG]

[BG1] [BG2] [BGH1]

[BGH2] [BGH3] [BGH4] [C] [Gr] [G] [Gri] [HS] [Mo]

R. Bott and L. W. Tu, Differential Forms in Algebraic Topology, Grad. Texts in Math. 82, Springer-Verlag, New York, 1982. W. Bruns and J. Herzog, Cohen-Macaulay Rings, Cambridge Stud. Adv. Math. 39, Cambridge Univ. Press, Cambridge, 1993. R. L. Bryant, S. S. Chern, R. B. Gardner, H. L. Goldschmidt, and P. A. Griffiths, Exterior Differential Systems, Math. Sci. Res. Inst. Publ. 18, Springer-Verlag, New York, 1991. R. L. Bryant and P. A. Griffiths, Characteristic cohomology of differential systems, I: General theory, J. Amer. Math. Soc. 8 (1995), 507–596. , Characteristic cohomology of differential systems, II: Conservation laws for a class of parabolic equations, Duke Math. J. 78 (1995), 531–676. R. L. Bryant, P. A. Griffiths, and L. Hsu, “Toward a geometry of differential equations” in Geometry, Topology, and Physics for Raoul Bott, Conf. Proc. Lecture Notes Geom. Topology 4, International Press, Cambridge, 1995, 1–76. , Hyperbolic exterior differential systems and their conservation laws, I, Selecta Math. (N.S.) 1 (1995), 21–112. , Hyperbolic exterior differential systems and their conservation laws, II, Selecta Math. (N.S.) 1 (1995), 265–323. , Poincaré-Cartan forms and the geometry of first order functionals, I, Institute for Advanced Study preprint, 1997. E. Cartan, Les systèmes différentiels extérieurs et leur applications géométriques, Actualiatés Sci. Indust. 994, Hermann, Paris, 1945. M. Grassi, Characteristic cohomology of smooth manifolds, Ph.D. thesis, UCLA, 1997. M. L. Green, “Koszul cohomology and geometry” in Lectures on Riemann Surfaces (Trieste, 1987), World Scientific Press, Teaneck, N.J., 1989, 177–200. P. A. Griffiths, personal communication, January 23, 1997. P. J. Hilton and U. Stammbach, A Course in Homological Algebra, Grad. Texts in Math. 4, Springer-Verlag, New York, 1971. S. Montgomery, Hopf Algebras and Their Actions on Rings, CBMS Reg. Conf. Ser. Math. 82, Amer. Math. Soc., Providence, 1993.

328 [Sw]

MICHELE GRASSI M. E. Sweedler, Hopf Algebras, Math. Lecture Note Ser., Benjamin, New York, 1969.

Scuola Normale Superiore, P.zza dei Cavalieri 7, 56100 Pisa, Italy; [email protected]

Vol. 102, No. 2


© 2000

SYMPLECTIC MODULAR SYMBOLS PAUL E. GUNNELLS

1. Introduction 1.1. Let G be a semisimple algebraic group defined over Q of Q-rank , and let X be the associated symmetric space. Let ⊂ G(Q) be a torsion-free arithmetic subgroup. Then H ∗ (; Z) = H ∗ (\X; Z), and this cohomology vanishes for ∗ > N, where N = dim(X) − , the cohomological dimension of . The theory of modular symbols as formulated by Ash [2] constructs an explicit spanning set for H N (; Z) as follows. Let Ꮾ be the Tits building associated to G [17]. By the Solomon-Tits theorem, Ꮾ has the homotopy type of a wedge of ( − 1)spheres, and thus H˜ ∗ (Ꮾ; Z) is nonzero only in dimension −1. Using the Borel-Serre compactification of the locally symmetric space \X, we may construct a map (1)

: H−1 (Ꮾ; Z) −→ H N (; Z)

that is surjective (cf. §2). Because the left-hand side of (1) is generated by fundamental classes of apartments of Ꮾ, this provides a geometric spanning set for H N (). These cohomology classes (or rather, their duals in homology) are called modular symbols. 1.2. The modular symbols provide a spanning set for H N (; Z), but they do not provide a finite spanning set, a distinction that is important for applications. However, suppose K/Q is a number field with euclidean ring of integers ᏻ, and let G(Q) = SLn (K) and ⊂ SLn (ᏻ). Then in [6], Ash and Rudolph determine an explicit finite spanning set—the unimodular symbols—and present an algorithm to write a modular symbol as a sum of unimodular symbols (cf. §2.9). This algorithm, in conjunction with certain explicit cell complexes, can be used to compute the action of the Hecke operators on H N (). In turn, through work of Ash, Pinch, and Taylor [5], Ash and McConnell [4], and van Geemen and Top [18], corroborative evidence has been produced for certain aspects of the “Langlands philosophy.” In particular, in the case of ⊂ SL3 (Z), many examples of representations of the absolute Galois group ¯ Gal(Q/Q) have been found that appear to be associated to cohomology classes of . 1.3. In this paper we solve the finiteness problem for the symplectic group: G(Q) = Sp2n (K) and of finite index in Sp2n (ᏻ), where ᏻ is euclidean. We characterize a finite spanning set of H N (; Z) and present an algorithm (Theorem 4.11) Received 7 May 1998. Revision received 28 July 1999. 1991 Mathematics Subject Classification. Primary 11F75. Author’s work partially supported by National Science Foundation grant number DMS-9627870. 329

330

PAUL E. GUNNELLS

that generalizes the modular symbol algorithm of [6]. To do this, we prove a relation in the homology of the symplectic building (Theorem 3.16; see Examples 3.9 and 3.10). 1.4. We conclude this introduction by discussing the arithmetic nature of H N () in the symplectic case and by indicating possible applications of Theorem 4.11. First, we consider the complex cohomology. It is known that for n > 1, the groups H N (; C) do not contain cuspidal classes (that is, cohomology classes corresponding to cuspidal automorphic forms). Moreover, work of Schwermer [13], [14], [15] shows the following: • For n > 1, there are Eisenstein cohomology classes in the top degree. These classes are constructed using Eisenstein series and characters attached to the maximal split torus of the minimal parabolic subgroup of G. For n > 3, these are all of the Eisenstein classes appearing in the top degree. • Furthermore, for Sp4 , there are additional classes constructed using Eisenstein series and cuspidal classes for SL2 . Hence, for complex coefficients these classes are either easy to understand arithmetically or are better studied on a lower rank group using the classical theory of modular symbols. Nevertheless, there is one possibility that might be of interest. In the case of Sp6 , one cannot exclude the possibility that there is an Eisenstein cohomology class in the top degree associated to a cuspidal class for Sp4 whose infinity type is not a discrete series representation. It might be interesting to show the existence of this class and to study its arithmetic nature using the results of this paper. (I am grateful to the referee for this suggestion.) 1.5. Next, let p be a prime and let Fp be the corresponding finite field. Consider the mod p cohomology H N (; Fp ) or, more generally, H N (; M), where M is an Fp -module with -action. In this setting, the situation is much less clear. For example, there may be classes that lift to torsion classes in integral cohomology and thus are not associated to automorphic forms in any obvious manner. For discussion and examples of this, see [1] for ⊂ GL3 (Z) and [3], [7] for GLn . One would like to know if such classes arise in the symplectic case and, if so, if the corresponding Hecke eigenclasses are attached to Galois representations. The algorithm in Theorem 4.11, in conjunction with the cell complex described in [12], provides the means to explore this question for H 4 (), where ⊂ Sp4 (Z). 1.6. Finally, in another paper [10], we describe an algorithm to compute the Hecke action on H 5 (), where is a subgroup of SL4 (Z). This cohomology group, whose degree is one less than the cohomological dimension, is within the cuspidal range. One would like to have a symplectic version of this algorithm that could compute the Hecke action on cuspidal classes in H 3 of subgroups of Sp4 (Z) (i.e., on Siegel modular forms of weight 3). The algorithm described in this paper is an essential first step towards solving this problem.

SYMPLECTIC MODULAR SYMBOLS

331

1.7. Acknowledgments and related work. This paper relies heavily on the results of [2] and [6]. We thank Avner Ash for much advice and encouragement. In [12] the authors describe a deformation retract W of the symmetric space Sp4 (R)/ U(2), which may be used to compute H ∗ (; Z), where is a finite index subgroup of Sp4 (Z). The combinatorial data (from [11]) that is used to describe the cell decomposition of W inspired the results in Example 3.9 and led to Theorem 3.16. We thank Bob MacPherson and Mark McConnell for many conversations. Finally, we thank Mark McConnell, Richard Scott, and the referee, who made helpful comments that improved this article. 2. Minimal modular symbols. In this section we review results on minimal modular symbols. These are due to Ash and Rudolph [6] for G = SLn and to Ash [2] for any semisimple Q-group G. Our exposition closely follows these sources. For general results about buildings, the reader may consult Tits [17]. 2.1. Let G, X, and be as specified in the introduction, and let e ∈ X be the distinguished basepoint. Let T be a maximal Q-split torus of G stable under the Cartan involution corresponding to e, and let A = T (R)0 . Since is the Q-rank of G, we have that A ∼ = (R>0 ) . Let X¯ be the partial compactification of X constructed by Borel and Serre [8]. Then the closure Z of Ae in X¯ is homeomorphic to a closed ball of dimension , and ∂Z ¯ Let [I ] ∈ H−1 (∂ X) ¯ be the fundamental class of ∂Z; if m ∈ G(Q), let lies in ∂ X. [m] be the fundamental class of ∂(mZ). Let Ꮾ be the Tits building associated to G. According to [8, §8.4.3], there is a homotopy equivalence h : Ꮾ → ∂ X¯ that takes a distinguished apartment A0 homeomorphically onto ∂Z. This map is compatible with the natural G(Q)-action on Ꮾ ¯ In particular, if m ∈ G(Q), then h(mA0 ) = ∂(mZ). Since G(Q) acts transiand ∂ X. tively on apartments, and H−1 (Ꮾ; Z) is generated by the fundamental classes of the apartments, we have shown the following lemma. ¯ Z). 2.2. Lemma [2]. The classes [m] generate H−1 (∂ X; 2.3. Now let π : X¯ → \X¯ be the projection. Using π and the long exact se¯ ∂ X), ¯ we obtain quence of the pair (X,

(2)

∼ π∗ ¯ ∂ X¯ −− ¯ ∂ \X¯ → H \X, H−1 ∂ X¯ −→ H X, ∼ D −→ H N \X¯ −→ H N (\X).

¯ the last isomorphism Here, the first isomorphism follows from the contractibility of X, ¯ and the map D is Lefis induced by the canonical homotopy equivalence X → X, schetz duality. Since is assumed torsion-free, D is an isomorphism with integral coefficients, and H N (\X; Z) can be identified with H N (; Z). Let [m]∗ ∈ H N (; Z) be the image of [m] under the sequence (2). One of the main results of [2] is that π∗

332

PAUL E. GUNNELLS

is surjective, which with Lemma 2.2 implies the following theorem. 2.4. Theorem [2]. As m varies over G(Q), the classes [m]∗ generate H N (; Z). 2.5. Let K/Q be a number field with ring of integers ᏻ, and let G be the algebraic Q-group such that G(Q) = SLn (K). Here = n−1. We want to describe the classes [m] using the combinatorics of Ꮾ. Let V = K n . We assume that the basis {e1 , . . . , en } ⊂ V are eigenvectors for the torus T from §2.1. Then Ꮾ is a simplicial complex with a vertex for every proper nonzero subspace F of V . A set {F1 , . . . , Fk } of vertices of Ꮾ spans a (k −1)-simplex in Ꮾ if and only if the corresponding subspaces can be arranged in a proper flag: 0 ⊂ F 1 ⊂ · · · ⊂ Fk ⊂ V . The action of SLn (K) on V induces an action on Ꮾ and on H∗ (Ꮾ; Z). To construct the classes [m], we use an auxiliary simplicial complex. Let [[n]] be the set {1, . . . , n}. Let ∂!n−1 be the barycentric subdivision of the boundary complex of the (n − 1)-simplex. In other words, ∂!n−1 is a simplicial complex with vertices corresponding to the proper nonempty subsets I of [[n]], and a collection of subsets I1 , . . . , Ik corresponds to a (k − 1)-simplex in ∂!n−1 if and only if they can be arranged in a proper flag: I1 ⊂ · · · ⊂ Ik . We may orient ∂!n−1 using the standard ordering on [[n]]. Given n points in V {0}, we may construct a class in Hn−2 (Ꮾ; Z) (following [6]). Given v1 , . . . , vn ∈ V {0}, we define a simplicial map φ : ∂!n−1 −→ Ꮾ by sending the vertex I ⊂ [[n]] to the vertex of Ꮾ corresponding to the subspace spanned by {vi | i ∈ I }. If ξ is the fundamental class of the geometric realization of ∂!n−1 , then φ∗ (ξ ) is a class in Hn−2 (Ꮾ; Z). Thus, under the composition φ h ¯ ∂!n−1 −→ Ꮾ −→ ∂ X,

¯ Z). we have constructed a class in Hn−2 (∂ X; 2.6. Definition [6]. The modular symbol associated to v1 , . . . , vn ∈ V {0} is the ¯ Z) constructed above and is denoted [v1 , . . . , vn ]. If m ∈ Mn (K) class in Hn−2 (∂ X; (n×n matrices over K), then by [m] we mean the modular symbol constructed using the columns of m. 2.7. Proposition. If m ∈ SLn (K), then the construction of [m] given in Definition 2.6 coincides with that given in §2.1. In particular, the classes [m] span ¯ Z). Hn−2 (∂ X; Proof. Recall that we have a homotopy equivalence h : Ꮾ → ∂ X¯ taking a distinguished apartment A0 homeomorphically onto ∂Z (§2.1). This apartment is in

333


fact φ(∂!n−2 ), where ∂!n−2 is constructed using the basis e1 , . . . , en ∈ V . These ¯ so the identifications are compatible with the action of SLn (K) on V , Ꮾ, and ∂ X, result follows. Modular symbols have the following properties. ¯ Z) satisfies the following. 2.8. Proposition [6]. The map [ ] : Mn (K) →Hn−2 (∂ X; (1) [v1 , . . . , vn ] = (−1)|τ | [τ (v1 ), . . . , τ (vn )], where τ ∈ Sn is a permutation on n letters and |τ | is the length of τ . (2) If q ∈ K, then [qv1 , v2 , . . . , vn ] = [v1 , . . . , vn ]. (3) If the vi are linearly dependent, then [v1 , . . . , vn ] = 0. (4) If v0 , . . . , vn ∈ V {0}, then (−1)i v0 , . . . , vî , . . . , vn = 0. i

Furthermore, [ ] is surjective. 2.9. Now assume that ᏻ is a euclidean ring with respect to a multiplicative norm : ᏻ → Z≥0 . Using multiplicitivity, extend the norm to a map : K → Q≥0 . We recall how to identify a -finite spanning set of modular symbols. By a primitive vector, we mean a vector v ∈ ᏻn such that the greatest common divisor of the entries of v is a unit. Let L be an ᏻ-submodule of ᏻn of rank k ≤ n. Since ᏻ is a principal ideal domain, L has a free ᏻ-basis B = {v1 , . . . , vk }. Choose W = {wk+1 , . . . , wn } ⊂ ᏻn such that B W is a K-basis of K n and W may be extended to an ᏻ-basis of ᏻn . We define the index of L by i(L) := det(v1 , . . . , vk , wk+1 , . . . , wn ). It is easy to see that i(L) is independent of the choices above and that i(L) = 1

if and only if (L ⊗ᏻ K) ∩ ᏻn = L.

Furthermore, if L has rank n and is the usual norm NK/Q : K → Q, then i(L) = [ᏻn : L]. We write i(v1 , . . . , vk ) for i(L) if we are given a specific set of linearly independent vectors generating L. If the vi are linearly dependent, we define i(v1 , . . . , vk ) = 0. 2.10. Definition. Let v1 , . . . , vk ∈ ᏻn be linearly independent primitive vectors. Then a candidate for the vi is a primitive x ∈ ᏻn {0}, satisfying 0 ≤ i(x, v1 , . . . , vî , . . . , vk ) < i(v1 , . . . , vk ),

1 ≤ i ≤ k.

The following proposition is a fundamental result. 2.11. Proposition [6]. Let v1 , . . . , vk be a linearly independent set of primitive vectors. If i(v1 , . . . , vk ) > 1, then a candidate x for the vi exists.

334

PAUL E. GUNNELLS

Proof. We give the proof since our statement differs slightly from that found in [6]. First, assume k = n. Let L be the lattice spanned by the vi , and let w be a primitive vector in ᏻn that is not in L. Note that w exists since i(L) > 1. Let A ∈ Mn (ᏻ) be the matrix with columns v1 , . . . , vn , and let Ai [w] be the matrix obtained by replacing the ith column of A with w. Since ᏻ is euclidean, there exist αi , βi ∈ ᏻ such that det Ai [w] = αi det A + βi , where 0 ≤ βi < det A. Now let x = w − i αi vi . By our choice of w, the vector x is nonzero. It is easy to check that det Ai [x] = βi . Since 0 ≤ βi < det A and det Ai [x] = i(x, v1 , . . . , vî , . . . , vn ), the result follows. Now assume k < n. Since ᏻ is euclidean, L is a free module. Hence, we can choose an isomorphism L ⊗ K → K k that carries (L ⊗ K) ∩ ᏻn onto ᏻk . This brings us back to the full rank setting, and we may argue as above. Notice that x can be written as qi vi , with qi ∈ K satisfying 0 ≤ qi < 1. Furthermore, since the vi are linearly independent, at least one qi = 0. ¯ 2.12. Theorem [6]. As m ranges over SLn (ᏻ), the classes [m] generate Hn−2 (∂ X; Z). Hence, if ⊂ SLn (ᏻ) is torsion-free and of finite index, then the classes [m]∗ provide a finite spanning set of H N (; Z). Proof. Repeatedly applying Propositions 2.11 and 2.8(4), we may write any class [m] as a sum [mi ], where the determinant of each mi is a unit. Then applying Proposition 2.8(2), we may divide the first column of mi by the determinant of det mi to get mi ∈ SLn (ᏻ), satisfying [mi ] = [mi ]. 2.13. Definition. The classes {[m] | m ∈ SLn (ᏻ)} are called unimodular symbols. 3. Symplectic modular symbols. In this section we generalize the results of §2.5 to the symplectic case. In §§3.1 and 3.2 we recall well-known facts about symplectic geometry and the building associated to the symplectic group. In §3.4 we translate results of §2.5 to the symplectic setting, and in §§3.11–3.15 we describe a relation in the homology of the building that is crucial to our finiteness result. Throughout this section we do not assume that the ground field K has a euclidean ring of integers. 3.1. First, we recall some elementary facts about symplectic geometry to fix notation. Fix a field K, and let V be the vector space K 2n . If k ∈ Z, we use the notation k¯ for 2n + 1 − k. We fix a nondegenerate alternating bilinear form , : V → K. A basis v1 , . . . , vn , vn¯ , . . . , v1¯ of V is said to be symplectic if   if j = ı¯ and i < j , 1 vi , vj = −1 if j = ı¯ and i > j ,   0 otherwise.


335

We let G = Sp2n (K) be the subgroup of GL2n (K) preserving the form. Thus, Sp2n (K) = {g ∈ GL2n (K) | gv, gw = v, w}. The group G has Q-rank = n. Given any x ∈ V , we define x ⊥ to be the set {y ∈ V | x, y = 0}. If x is nonzero, then x ⊥ is a hyperplane containing x. A subspace F ⊂ V is called isotropic if v, w = 0 for all v, w ∈ F . Any one-dimensional subspace is isotropic, and the largest dimension an isotropic subspace may have is n. An n-dimensional isotropic subspace is called a Lagrangian subspace. 3.2. From now on we exclusively use Ꮾ to denote the building associated to Sp2n (K). We have the following description of Ꮾ as a simplicial complex, analogous to that found in §2.5: vertices of Ꮾ correspond to nonzero isotropic subspaces of V , and simplices of Ꮾ correspond to flags of nonzero isotropic subspaces. To describe the geometry of Ꮾ we must use cross-polytopes instead of simplices, and so we recall their definition. Let e1 , . . . , en be the standard basis of Rn . Then a cross-polytope on 2n vertices is a polytope isomorphic to the convex hull of the points ±e1 , . . . , ±en . Examples are the square (n = 2) and the octahedron (n = 3). Since the proper faces of cross-polytopes are simplicial, we may regard their boundary complexes as simplicial complexes. Let ∂βn be the first barycentric subdivision of the boundary complex of the cross-polytope on 2n vertices. To describe the struc¯ We order this set by ture of ∂βn , we use the notation [[n]]± := {1, . . . , n, n, ¯ . . . , 1}. ¯ 1 < · · · < n < n¯ < · · · < 1. 3.3. Definition (cf. [9]). A nonempty subset I ⊂ [[n]]± is called isotropic if for all i, j ∈ I , we have i = ¯. Note that #I ≤ n for any isotropic I . As in the SLn case, vertices of ∂βn correspond to isotropic subsets of [[n]]± . If we take β to be the convex hull of the points ±e1 , . . . , ±en , then the set I corresponds to the linear span of {ei | i ∈ I }, where eı¯ := −ei . Similarly, simplices of ∂βn correspond to proper flags of isotropic subsets. 3.4.

Now suppose that we are given 2n points v1 , . . . , vn , vn¯ , . . . , v1¯ ∈ V {0}.

3.5. Definition. The set v1 , . . . , vn , vn¯ , . . . , v1¯ is said to satisfy the isotropy condition if, for every isotropic subset I ⊂ [[n]]± , the subspace spanned by {vi | i ∈ I } is isotropic. Notice that this condition is strictly weaker than the requirement that the vi form a symplectic basis. In particular, the columns of any m ∈ Sp2n (K) satisfy this condition. Following §2.5, we define a simplicial map φ : ∂βn −→ Ꮾ by sending the vertex corresponding to an isotropic I ⊂ [[n]]± to the vertex of Ꮾ corresponding to the linear span of {vi | i ∈ I }. Via the composition φ ◦h, we have an induced map on homology taking the fundamental class ξ of the geometric realization ¯ Z). of ∂βn to a class in Hn−1 (∂ X;

336

PAUL E. GUNNELLS

3.6. Definition. Given v1 , . . . , vn , vn¯ , . . . , v1¯ ∈ V {0} satisfying the isotropy con¯ Z) denote the class constructed above. dition, let [v1 , . . . , vn ; vn¯ , . . . , v1¯ ] ∈ Hn−1 (∂ X; This class is called a symplectic modular symbol. If the columns of m ∈ M2n (K) satisfy the isotropy condition, we denote the symplectic modular symbol corresponding to its columns by [m]. Symplectic modular symbols satisfy properties similar to those satisfied by the special linear symbols. For instance, minor modification of the proof of Proposition 2.7 shows that the construction of [m] in Definition 3.6 agrees with that of §2.1, and thus ¯ Z). Furthermore, we have the following analog the modular symbols span Hn−1 (∂ X; of the first parts of Proposition 2.8, whose proof is easily checked. 3.7. Proposition. Symplectic modular symbols enjoy the following properties. (1a) Let τ ∈ Sn be a permutation on n letters. Then

v1 , . . . , vn ; vn¯ , . . . , v1¯ = τ (v1 ), . . . , τ (vn ); τ (vn¯ ), . . . , τ v1¯ ,

where τ (vk ) := vτ (k) and τ (vk¯ ) := v τ (k) for k ∈ [[n]]. (1b) We also have [v1 , v2 , . . . , vn ; vn¯ , . . . , v2¯ , v1¯ ] = −[v1¯ , v2 , . . . , vn ; vn¯ , . . . , v2¯ , v1 ]. (2) If q ∈ K, then [qv1 , v2 , . . . , vn ; vn¯ , . . . , v1¯ ] = [v1 , . . . , vn ; vn¯ , . . . , v1¯ ]. (3) If the vi are linearly dependent, then [v1 , . . . , vn ; vn¯ , . . . , v1¯ ] = 0. 3.8. The symplectic analogue of Proposition 2.8(4) is more complicated and forms one of the main results of this article. To motivate the result, we illustrate it for Sp4 (K) and Sp6 (K). In light of the homotopy equivalence Ꮾ → ∂ X¯ (§2.1), we work ¯ in H∗ (Ꮾ) rather than H∗ (∂ X). 3.9. Example. Let G = Sp4 (K). Then the simplicial complex Ꮾ is a graph with two types of vertices, corresponding to the one- and two-dimensional isotropic subspaces of V . Let [m] = [v1 , v2 ; v2¯ , v1¯ ] be a symplectic modular symbol for Sp4 (K). We may picture the subspace configuration in V = K 4 determined by {vi } by passing to P3 (K). Figure 1 shows the projectivized configuration on the left, along with the apartment in Ꮾ corresponding to [m] on the right. In Ꮾ we represent the one-dimensional (respectively, two-dimensional) isotropic subspaces of V by solid (respectively, hollow) vertices. By abuse of notation we use the same symbol for a point in K 4 , the point it determines in P3 (K), and the vertex it determines in Ꮾ. The lines on the left of Figure 1 are the projectivizations of the Lagrangian planes determined by the vi ; we have not drawn the images of the two non-Lagrangian planes. Now choose x ∈ V {0}. We use x to construct a class in H1 (Ꮾ; Z) homologous to [m]. Recall that x ⊥ is the set of all y ∈ V satisfying x, y = 0. In P3 (K), x ⊥ becomes a hyperplane that, for generic x, determines four new points by intersection with the original Lagrangian lines (see Figure 2). These new points and lines determine four

337


v1 v1 v2¯

v2¯

v1¯

v2

v2 v1¯ Figure 1. A configuration in P3 (K) and its corresponding apartment

v1 x v2¯

v1¯ v2

Figure 2. Constructing new points

new apartments in Ꮾ, and provide a relation [m] = 4i=1 [mi ] in homology, as in Figure 3. Now suppose that x is not generic. Then, for some subset I ⊂ [[2]]± , we have x, vi = 0 for i ∈ I . Up to symmetry there are three nontrivial possibilities, which we depict in Figure 4. The corresponding apartments appear in Figure 5. Note that in these cases there are fewer apartments in the images of the relations. 3.10. Example. Consider the case G = Sp6 (K). Now, Ꮾ has three kinds of vertices, corresponding to isotropic points, lines, and planes in P5 (K). The configuration in P5 (K) corresponding to a symplectic modular symbol m is combinatorially an octahedron with isotropic faces. We choose a generic point x and construct twelve new points by intersecting x ⊥ with the isotropic lines of the octahedron. In Figure 6 we show the configuration corresponding to [m] along with these constructed points,

338

PAUL E. GUNNELLS

v1

a v1

a

x v2¯ c

v2¯

v1¯

b

b v2

x c

v2 d

d

v1¯ Figure 3. A homology relation

v1 v2¯

v1 v2¯

v1¯

x

v1 v1¯

x

v2

x v1¯

v2¯

v2

v2

¯ {1, 2}, and {1} Figure 4. I = {1, 1},

v1

v2¯

v1

x v1¯

v2

v1 x

v2¯ v1¯

v2

x

v2¯

v2

v1¯

Figure 5

which are shown as hollow dots. (Although the configuration properly lives in P5 (K), we show it in three dimensions for clarity.) Notice that the constructed points satisfy nontrivial linear dependencies: three constructed points are collinear if they lie on an isotropic plane corresponding to a facet of the original octahedron. (These dependen cies are only true in P5 (K).) These dependencies ensure the relation [m] = [mi ] in homology.


339

Figure 6. A configuration in P5 (K)

3.11. Returning to the general case, let [m] = [v1 , . . . , vn ; vn¯ , . . . , v1¯ ] be a symplectic modular symbol. Let x ∈ V {0}, and let x ⊥ be as above. Let Dx ⊂ [[n]]± be the set of indices such that x, vi = 0 if and only if i ∈ Dx . Given distinct i, j ∈ [[n]]± with i = ¯ and not both i, j ∈ Dx , define xij by

(3) xij = x, vi vj − x, vj vi . If both i and j lie in Dx , then xij is not defined. The point xij lies on the intersection of x ⊥ with the isotropic plane spanned by vi and vj . Now, we define new matrices built from m, x, and the xij . 3.12. Definition. Let [m] be a symplectic modular symbol. Choose x ∈ V {0}, and construct the xij as in (3). If i ∈ Dx , define the matrix mi to be the matrix obtained by altering m according to the following rules: (1) replace vı¯ by x; (2) for j ∈ [[n]]± {i, ı¯}, replace vj by xij . A priori [mi ] may not be a symplectic modular symbol, because its columns might not satisfy the isotropy condition. Hence, we state the following proposition. 3.13. Proposition. For each i ∈ [[n]]± Dx , each [mi ] is a symplectic modular symbol. Proof. It is easy to check that xij , xik = 0 and vi , xij = 0 in all the necessary cases. Since x, xij = 0 by construction, the result follows. As indicated in Example 3.10, the xij satisfy linear dependencies, which we record in the following lemma. 3.14. Lemma. Suppose I = {i, j, k} is an isotropic subset, and #(I ∩ Dx ) ≤ 1. Then the points xij , xj k , and xik are linearly dependent.

340

PAUL E. GUNNELLS

Proof. The points satisfy the identity

x, vk xij = x, vj xik − x, vi xj k . 3.15.

We come now to the main result of this section.

3.16. Theorem. Let [m] = [v1 , . . . , vn ; vn¯ , . . . , v1¯ ] be a symplectic modular symbol for Sp2n (K), and choose x ∈ V {0}. Define Dx as above, and let [mi ] be the ¯ Z), symplectic modular symbols constructed in Definition 3.12. Then in Hn−1 (∂ X; we have [m] = (4) [mi ]. i∈[[n]]± Dx

Proof. As in Examples 3.9 and 3.10, we prove the relation in Hn−1 (Ꮾ; Z). Let A (respectively, Ai ) be the apartment corresponding to the symplectic modular symbol [m] (respectively, [mi ]). We think of these apartments as being explicit simplicial cycles in Ꮾ, and we will show that [m] = i [mi ] by examining these cycles. We begin by fixing some notation. Let (a1 , . . . , ak ) be an ordered tuple of linearly independent points of V lying in a Lagrangian subspace, and let F (a1 , . . . , ak ) ⊂ V be their linear span. Let σ (a1 , . . . , ak ) ⊂ Ꮾ be the simplex corresponding to the flag 0 ⊂ F (a1 ) ⊂ F (a1 , a2 ) ⊂ · · · ⊂ F (a1 , . . . , ak ) ⊂ V . Recall that the (closed) star of a simplex σ in a simplicial complex is the set of all simplices σ meeting σ , as well as the faces of all such σ . Also recall that maximal simplices in Ꮾ are called chambers. First, assume that the point x is generic with respect to the vi , so that Dx = ∅. Consider the column vector vi from [m]. The chambers in A appearing in the star of vi are the simplices of the form (5) σ vi , vk1 , . . . , vkn−1 ⊂ A, where {i, k1 , . . . , kn−1 } ⊂ [[n]]± is isotropic. These chambers correspond to the flags (6) 0 ⊂ F (vi ) ⊂ F vi , vk1 ⊂ · · · ⊂ F vi , vk1 , . . . , vkn−1 ⊂ V . On the right-hand side of (4), in Ai we have the chambers (7) σ vi , xi,k1 , . . . , xi,kn−1 ⊂ Ai . These chambers correspond to the flags 0 ⊂ F (vi ) ⊂ F vi , xi,k1 ⊂ · · · ⊂ F vi , xi,k1 , . . . , xi,kn−1 ⊂ V . (8) The sets of flags in (6) and (8) coincide by the definition of the xij , and these chambers appear with the same orientations on both sides of (4). Taking all permutations of


341

{k1 , . . . , kn−1 } in (5)–(8), we obtain all chambers in the star of vi . Hence, any chamber in A appears once in a unique Ai with the same orientation. Now, we claim that each of the remaining chambers in the Ai appears exactly twice with opposite orientations. Any such chamber must appear in the star of x or xij , and we first consider the star of x. Choose an apartment Ai and a point xij , and let I ⊂ [[n]]± be a maximal isotropic subset of the form {i, j, k1 , . . . , kn−2 }. Let FI be the Lagrangian subspace, corresponding to I . Consider the chambers with x and xij as vertices, and with all vertices other than x corresponding to subspaces lying in FI . These chambers have the form σ x, xij , xi,k1 , . . . , xi,kn−2 ⊂ Ai (9) or (10)

σ x, xij , xj,k1 , . . . , xj,kn−2 ⊂ Aj .

In (9) and (10), we allow all permutations of {k1 , . . . , kn−2 }. By Lemma 3.14, these chambers correspond to the same isotropic flag. Furthermore, it is not difficult to see that the chambers in (9) and (10) appear in Ai and Aj with opposite orientations. We may apply this argument to any pair {i, j } and any FI with {i, j } ⊂ I , and so all the chambers in the star of x in the Ai cancel each other in (4). To complete the proof for generic x, we investigate any remaining chambers on the right-hand side of (4). These must appear in the stars of the xij . Fix xij , and let I = {i, j, k1 , . . . , kn−2 } and FI be as above. The chambers meeting the star of xij and with all vertices corresponding to subspaces lying in FI are of two types: first, σ xij , xi,k1 , . . . , xi,kn−2 , vi ⊂ Ai (11) and (12)

σ xij , xj,k1 , . . . , xj,kn−2 , vj ⊂ Aj ,

and second, (13)

σ xij , xi,k1 , . . . , xi,kn−2 , x ⊂ Ai

and (14)

σ xij , xj,k1 , . . . , xj,kn−2 , x ⊂ Aj .

In (11)–(14), we allow all permutations of the right n−1 vertices. By Lemma 3.14, the isotropic flags corresponding to (11) and (12) (respectively, (13) and (14)) coincide, and checking orientations shows that these cancel in pairs. This accounts for all the chambers on both sides of (4), and hence the result follows for generic x. Now assume that Dx = ∅. Write Dx = I J , where I = I¯ and J ∩ J¯ = ∅. We claim it is sufficient to assume I = ∅. Indeed, let S = [[n]]± I , and let V ⊂ V be

342

PAUL E. GUNNELLS

the span of {vi | i ∈ S}. Then x ∈ V , and V is a symplectic space whose form , is the restriction of , . Furthermore, the vectors {vi | i ∈ S} define an apartment in the building Ꮾ associated to (V , , ) and thus determine a symplectic modular symbol = (1/2)#S. Writing [m] = [m ] ∈ Hn −1 (Ꮾ ; Z), where n [mi ] in Hn−1 (Ꮾ; Z) is equivalent to writing [m ] = [mi ] in Hn −1 (Ꮾ; Z). Hence, by induction we may assume that Dx contains no subset I with I¯ = I . This is the same as Dx being isotropic, and so up to symmetry the only invariant of Dx is its cardinality. As before, we proceed by investigating the stars of vertices. We merely indicate which chambers cancel which and omit the details. We first consider the case that #Dx ≤ n − 1. Any chamber in the star of vi has the form σ vi , vk1 , . . . , vkn−1 ⊂ A, (15) where {i, k1 , . . . , kn−1 } ⊂ [[n]]± is isotropic. This is matched on the right side of (4) by the chamber σ vi , vk1 , . . . , vkj , xkj ,kj +1 , . . . , xkj ,kn−1 ⊂ Akj . (16) In (16), kj is the first subscript reading from the left that does not appear in Dx . (The possibility that i = kj is included.) This chamber appears with the same orientation as that of (15), and chambers of this form account for all chambers on the left side of (4). In the star of x, the chamber σ x, xij , xi,k1 , . . . , xi,kn−2 ⊂ Ai (17) is canceled by the chamber σ x, xij , xj,k1 , . . . , xj,kn−2 ⊂ Aj , (18) exactly as in the generic case. If #Dx < n − 1, then every chamber in the star of x is of this form up to symmetry. If #Dx = n − 1, the remaining chambers in the star of x have the form σ x, vi1 , . . . , vin−1 , (19) where Dx = {i1 , . . . , in−1 }. This chamber appears in Ak and Ak¯ with opposite orientations, where k and k¯ are the unique elements that extend Dx to an isotropic subset. The remaining chambers on the right side of (4) appear in the stars of the xij where {i, j } ⊂ Dx . These cancel exactly as in (11)–(14). Finally we consider the case #Dx = n. This case is slightly different, since x is in the span of the {vi | i ∈ Dx }. Again we begin with the star of the vi and consider the chamber σ vi , vk1 , . . . , vkn−1 .


343

If Dx = {i, k1 , . . . , kn−2 }, then this chamber is matched by the chamber (20) σ vi , vk1 , . . . , vkj , xkj ,kj +1 , . . . , xkj ,kn−1 ⊂ Akj , exactly as in (16). Otherwise, if Dx = {i, k1 , . . . , kn−2 }, then this chamber is matched by σ vi , vk1 , . . . , vkn−2 , x ⊂ Akn−1 . Now consider the star of x. If {i, j }∩Dx = ∅, then we have cancellation as in (17) and (18). Otherwise, write Dx = {i1 , . . . , in }. Then in the remaining chambers, σ x, vi1 , . . . , vin−1 ⊂ Ain cancels σ x, vi1 , . . . , vin−2 , vin ⊂ Ain−1 . Finally, the chambers in the star of the xij with {i, j } ⊂ Dx cancel exactly as in (11)–(14). This completes the proof. 3.17. Remark. Theorem 3.16 can be proven in more generality than stated here. For example, the proof applies to buildings associated to the odd orthogonal groups SO2n+1 and can be modified to work for buildings associated to the even orthogonal groups SO2n . Using the notion of a “perspectivity” (see [16]), we may prove a similar result for buildings of type G2 . We would like to have a “building-theoretic” proof of Theorem 3.16. 4. Finiteness. Throughout this section we assume that ᏻ is a euclidean ring with respect to the norm : ᏻ → Z≥0 . We also assume, using Proposition 3.7(2), that all modular symbols have integral columns. ¯ Z) a unimodular symbol. 4.1. If m ∈ Sp2n (ᏻ), we call the class [m] ∈ Hn−1 (∂ X; ¯ Z). The goal of this section is to prove that the unimodular symbols span Hn−1 (∂ X; We begin with a notion that lets us measure how far a symplectic modular symbol is from being unimodular. In contrast to the special linear case (§2.9), we use the symplectic pairing rather than the determinant as a measure of non-unimodularity. 4.2. Definition. Let [m] be a symplectic modular symbol with primitive columns. The depth of [m] = [v1 , . . . , vn ; vn¯ , . . . , v1¯ ] is the number d(m) := Max vi , vı¯ . i∈[[n]]

Notice that d(m) = 1 implies that each vi , vı¯ ∈ ᏻ× . Hence, if d(m) = 1, we may divide the columns of m by appropriate units to obtain m ∈ Sp2n (ᏻ) satisfying ¯ Z), [m] = [m ]. Thus, in order to show that the unimodular symbols span Hn−1 (∂ X;

344

PAUL E. GUNNELLS

it is sufficient to show that through a homology we may replace a modular symbol with depth greater than 1 with a cycle of modular symbols, each of which has smaller depth. 4.3. Lemma. Let [m] = [v1 , . . . , vn ; vn¯ , . . . , v1¯ ] be a symplectic modular symbol with primitive columns. If d(m) > 1, then i(v1 , . . . , vn ; vn¯ , . . . , v1¯ ) > 1, and there exists a candidate for the vi . Proof. Assume that i(v1 , . . . , vn , vn¯ , . . . , v1¯ ) = 1. Then the lattice generated by the v1 is ᏻn , and thus m ∈ GL2n (ᏻ). Therefore vi , vı¯ ∈ ᏻ× , det(m) = i∈[[n]]

and d(m) = 1, a contradiction. This implies i(v1 , . . . , vn , vn¯ , . . . , v1¯ ) > 1, and a candidate exists by Proposition 2.11. 4.4. Suppose that [m] = [v1 , . . . , vn , vn¯ , . . . , v1¯ ] is a symplectic modular symbol with d(m) > 1, and let x be a candidate for m. As in §2.9, write qi v i x= i∈[[n]]±

with qi ∈ K satisfying 0 ≤ qi < 1. Define [mi ] as in Definition 3.12, so that (21) [mi ]. [m] = i∈[[n]]± Dx

Notice that for i ∈ [[n]]± Dx , we have x, vi = qı¯ vı¯ , vi < vı¯ , vi ≤ d(m), so that the norm of at least one of the symplectic pairings in each mi has decreased. However, it is not true that d(mi ) < d(m) in general, and so more than just the construction of x is required to show that the unimodular symbols span. The following example illustrates our strategy. 4.5. Example. Consider the case of G = Sp6 (K). As in Example 3.10, [m] corresponds to an octahedron in P5 (K). Suppose that x is a candidate for [m] and that Dx = ∅. Construct the modular symbols [mi ]. Then the configuration in P5 (K) corresponding to [m1 ] is the octahedron with vertices v1 , x, and the four constructed points lying on the lines in the original configuration in the link of v1 . (See Figure 7.) The point x has been chosen so that x, v1 < d(m), but in general x12 , x12¯ and x13 , x13¯ will be larger than d(m). However, we claim that i(x12 , x13 , x13¯ , x12¯ ) > 1 and that we may identify the quadruple (x12 , x13 , x13¯ , x12¯ ) with a modular symbol [m ] for Sp4 (K). This means we may argue inductively and use the reduction of this “link” modular symbol to reduce [m1 ]. (See Figure 8.)

345


v1 x13¯ v3¯

x12¯

x12

v2

v2¯ x13 x

v3

Figure 7. The configuration corresponding to [m1 ]

Figure 8. Reducing the link modular symbol

4.6. To study the [mi ], we first prove a version of the Hermite normal form for our matrix representatives. 4.7. Lemma. Let m ∈ M2n (ᏻ) be a matrix with nonzero, primitive columns, and assume that m satisfies the isotropy condition. Then there is a matrix γ ∈ Sp2n (ᏻ) such that γ m is upper-triangular and γ m satisfies the isotropy condition. Proof. The left action of Sp2n (ᏻ) corresponds to row operations on m, so we begin by listing the elementary symplectic row operations. Let {ri | i ∈ [[n]]± } denote the rows of m, and use the notation a ← b to mean that the row vector a is to be replaced by the expression b. Then we find that we may effect the following: (T1)

ri ←− ri + r ı¯ .

346

PAUL E. GUNNELLS

For 1 ≤ i < k ≤ n, ri ←− ri + rk ,

(T2)

r k¯ ←− r k¯ − r ı¯ ,

and ri ←− ri + r k¯ ,

(T3)

rk ←− rk + r ı¯ .

Also ri ←− r ı¯ ,

(P1)

r ı¯ ←− −ri .

If τ ∈ Sn , then ri ←− rτ (i) .

(P2)

Here T stands for transvection and P stands for permutation. In (P2) we mean that τ permutes the first n rows among themselves, with the action on the last n rows determined by τ (¯ı ) := τ (i). Furthermore, the inverses and transpositions of these operations are also elementary row operations. Now, using operations of type (T1) and (P1), we may carry m into the form 

m11  ..  .  mn,1   0   .  .. 0

m12 .. .

···

m12 ¯

···

mn,2 mn,2 ¯ .. .

··· ···

 m11¯ ..  .   mn,1¯  .  mn, ¯ 1¯  ..  .  m1¯ 1¯

In the first column, the entry in row i is the greatest common divisor of the original entries in row i and row ı¯. Next, using operations of type (T2) and (P2), we can take m into the form   m11 m12 · · · m11¯  0 m22 · · · m21¯     .. ..  . ..  . .  . 0

m12 ¯

···

m1¯ 1¯


347

Because left multiplication by γ ∈ Sp2n (ᏻ) preserves the isotropy condition, it follows that m actually has the form   m11 m12 · · · m11¯  0 m22 · · · m21¯     .. .. ..   . . .  .    ..  . m22 · · · m2¯ 1¯  ¯ 0 ··· 0 m1¯ 1¯ Now we may apply the induction hypothesis to the middle (2n − 2) × (2n − 2) block to complete the proof of the lemma. 4.8. Let [m] be a symplectic modular symbol with m upper-triangular, and let x be a candidate for m. Without loss of generality, we may assume v1 = e1 by Proposition 3.7(3) and that 1 ∈ Dx . We want to study the modular symbol [m1 ] from (21) in greater detail. 4.9. Lemma. Suppose that m is upper-triangular with v1 = e1 . Let the points x1,j j ∈ [[n]]± 1, 1¯ and the matrix m1 be defined as in Definition 3.12. Let X be the 2n×(2n−2) matrix with columns the vectors x12 , . . . , x1,n , x1,n¯ , . . . , x12¯ . That is, X is the matrix obtained by deleting the first and last columns from m1 . ¯ by Define points wj for j ∈ [[n]]± {1, 1}

wj = ej , x e1 − e1 , xej , and let W be the 2n × (2n − 2) matrix with columns the vectors w2 , . . . , wn , wn¯ , . . . , w2¯ . Let m be the central (2n − 2) × (2n − 2) block of m1 . Then X = W m . Proof. Given a column vector v, let v k be the entry in the kth row. Suppose first that j ≤ n. Then, by computation, using Definition 3.12, we find ¯

1 = x 2 m2,j + · · · + x ¯ mjj x1,j

and

¯

k x1,j = −x 1 mkj

for 2 ≤ k ≤ j .

348

PAUL E. GUNNELLS

On the other hand, W has the form  ¯ ¯ x3 x2  1¯ −x 0   ¯  0 −x 1  . ..  . .  . 0 ···

··· ··· .. . ..

.

0

x2 0 .. .



    ,   0  ¯ −x 1

and so x1,j is this matrix times the j th column of m . The computation is similar if j > n, so the result follows. 4.10.

We come now to the main result of this article.

¯ Z). 4.11. Theorem. As m ranges over Sp2n (ᏻ), the classes [m] span Hn−1 (∂ X; Proof. By the paragraph following Definition 4.2, it is sufficient to show that if [m] satisfies d(m) > 1, then [m] = [mα ], where d(mα ) < d(m). We proceed by induction. Since SL2 (ᏻ) = Sp2 (ᏻ), the statement is true by the usual modular symbol algorithm. So we assume the statement is true for Sp2k (ᏻ) with k < n. Assume that d(m) > 1, and let x be a candidate for m. Write (22) [mi ]. [m] = i∈[[n]]± Dx

We permute the columns of each [mi ] so that vi is the first column. Choose an [mi ] from the right-hand side of (22). Permuting labels if necessary, we can assume i = 1. Because candidate selection and the relation (4) are Sp2n (ᏻ)-equivariant, using Lemma 4.7 we may assume m1 is upper-triangular. As in Lemma 4.9, we construct the matrix m and the vectors wj . Since m is a (2n − 2) × (2n − 2) matrix satisfying the isotropy condition, we have that [m ] corresponds to a class in Hn−2 (Ꮾ ; Z), where Ꮾ is the building associated to Sp2n−2 (K). By the induction hypothesis, we may write (23) mα , [m ] = α∈A

where mα ∈ Sp2n−2 (ᏻ), and the sum is finite. We may use the [mα ] to write (24) [mα ] [m1 ] = α∈A

as follows. Each mα corresponds to an endomorphism of the ᏻ-module generated by the wj . We may apply mα to W to produce a 2n × (2n − 2) matrix Wα . Then mα is the matrix with first column e1 , last column x, and with middle columns Wα . The


349

induction hypothesis asserts that the columns of Wα form an ᏻ-basis of the ᏻ-module generated by the wj . In particular,

d(mα ) ≤ wj , w¯ = x, v1 2 . We claim that we may reduce each [mα ] further to write [mα ] = [mαβ ], β∈B

where d(mαβ ) ≤ x, v1 . To see this, consider the j and ¯ columns of W : t ¯ wj = x ¯ , 0, . . . , 0, −x 1 , 0, . . . , 0 and t ¯ w¯ = − x j , 0, . . . , 0, −x 1 , 0, . . . , 0 , ¯

where −x 1 appears in the j th and ¯th rows, respectively. Assume first that these ¯ vectors are primitive. Then some multiple of wj added to w¯ will be divisible by x 1 . This means wj and w¯ generate a lattice L satisfying ¯ i(L) ≥ x 1 = x, v1 . Therefore, we may apply the SL2 -modular symbol algorithm to the pair (wj , w¯ ) to reduce L to (L ⊗ K) ∩ ᏻn . If either wj or w¯ is not primitive, then a simple modification of this argument achieves the same end. Applying this to each pair (wj , w¯ ) and adapting the construction that passes from (23) to (24), we find [mα ] = (25) [mαβ ], β∈B

with d(mαβ ) ≤ x, v1 . Together, (24) and (25) imply [m1 ] = [mαβ ]

with d(mαβ ) ≤ x, v1 < d(m).

α∈A β∈B

Because this argument may be applied to any of the [mi ] from (22), this completes the proof of the theorem.

350

PAUL E. GUNNELLS

4.12. Corollary. If ⊂ Sp2n (ᏻ) is torsion-free of finite index, and N is the cohomological dimension of , then the classes [m]∗ provide a finite spanning set of H N (; Z). References [1]

[2] [3] [4] [5] [6] [7]

[8] [9] [10] [11]

[12] [13] [14] [15] [16] [17] [18]

G. Allison, A. Ash, and E. Conrad, Galois representations, Hecke operators and the mod-p cohomology of GL(3, Z) with twisted coefficients, Experiment. Math. 7 (1998), 361– 390. A. Ash, A note on minimal modular symbols, Proc. Amer. Math. Soc. 96 (1986), 394–396. , Galois representations attached to mod p cohomology of GL(n, Z), Duke Math. J. 65 (1992), 235–255. A. Ash and M. McConnell, Experimental indications of three-dimensional Galois representations from the cohomology of SL(3, Z), Experiment. Math. 1 (1992), 209–223. 4 extension of Q attached to a non-selfdual automorA. Ash, R. Pinch, and R. Taylor, An A phic form on GL(3), Math. Ann. 291 (1991), 753–766. A. Ash and L. Rudolph, The modular symbol and continued fractions in higher dimensions, Invent. Math. 55 (1979), 241–250. A. Ash and W. Sinnott, An analogue of Serre’s conjecture for Galois representations and Hecke eigenclasses in the mod-p cohomology of GL(n, Z), preprint, available at http://can.dpmms.cam.ac.uk/Algebraic-Number-Theory/0185/index.html. A. Borel and J.-P. Serre, Corners and arithmetic groups, Comm. Math. Helv. 48 (1973), 436–491. I. M. Gelfand and V. V. Serganova, On the general definition of a matroid and a greedoid, Dokl. Akad. Nauk SSSR 292 (1987), 15–20. P. Gunnells, Computing Hecke eigenvalues below the cohomological dimension, to appear in Experiment. Math. R. MacPherson and M. McConnell, “Classical projective geometry and modular varieties” in Proceedings of the JAMI Inaugural Conference, Johns Hopkins University Press, Baltimore, 1989, 237–290. , Explicit reduction theory for Siegel modular threefolds, Invent. Math. 111 (1993), 575–625. J. Schwermer, On arithmetic quotients of the Siegel upper half space of degree two, Compositio Math. 58 (1986), 233–258. , Eisenstein series and cohomology of arithmetic groups: the generic case, Invent. Math. 116 (1994), 481–511. , On Euler products and residual Eisenstein cohomology classes for Siegel modular varieties, Forum Math. 7 (1995), 1–28. J. Tits, Sur la trialité et certains groupes qui s’en déduisent, Inst. Hautes Études Sci. Publ. Math. (1959), 13–60. , Buildings of Spherical Type and Finite BN-Pairs, Lecture Notes in Math. 386, SpringerVerlag, Berlin, 1974. B. van Geemen and J. Top, A non-selfdual automorphic representation of GL3 and a Galois representation, Invent. Math. 117 (1994), 391–401.

Department of Mathematics, Columbia University, New York, New York 10027, USA; [email protected]

Vol. 102, No. 2


© 2000

EXCEPTIONAL INTEGERS FOR GENERA OF INTEGRAL TERNARY POSITIVE DEFINITE QUADRATIC FORMS RAINER SCHULZE-PILLOT

0. Introduction. In [2] it was shown that in a certain sense most integers represented by some form in the genus of a given integral ternary positive definite quadratic form are represented by all forms in this genus. More precisely, let (L, q) be a Zlattice of rank 3 with integral positive definite quadratic form. Then all sufficiently large integers a that are represented primitively by some lattice in the spinor genus of (L, q) are represented by all lattices in that spinor genus (see corollary to Theorem 3 in [2]). The theorems of that article actually imply a slightly sharper characterization of the set of exceptional integers that are represented by some forms in the genus of L but not by all of them. Since there seems to be some interest to have available a characterization of this set that is as sharp as possible, I give such a description and some examples in this note. I also comment on the question of effectivity of the results and on results for primitive representations. As in [2], a special role is played by the integers t in the square class of a primitive spinor exception, that is, in the square class of an integer represented primitively by some but not by all spinor genera in the genus of (L, q). The Fourier coefficients of the spinor generic theta series at integers tp 2 for primes p in certain arithmetic progressions do not grow for growing p. In view of the positivity of the Fourier coefficients of theta series, this implies that the Shimura lift with respect to such a square class of the difference of the theta series of lattices in the same spinor genus omits these primes in its Fourier expansion. One might therefore be tempted to speculate about a connection to CM-forms. By looking at an example, we see, however, that this is not the case; in general one has to expect that one is looking at the sum of a cusp form and of its quadratic twist. Acknowledgement. I thank Peter Sarnak for stimulating discussions on the questions mentioned above. 1. Exceptional integers. Let (L, q) as above be a quadratic lattice of level N, that is, for the dual lattice L# , we have q(L# )Z = N −1 Z. Let d denote the discriminant of (L, q). Let T denote the (finite) set of primes p for which (L, q) remains anisotropic over the p-adic completion Qp (for p ∈ T , we have p | N) and write q(L) for the set of numbers represented by some lattice in the genus of (L, q) (or equivalently, locally everywhere by (L, q)), qr (L) for the set of t ∈ q(L) that are divisible at Received 5 March 1999. 1991 Mathematics Subject Classification. Primary 11E45; Secondary 11E12, 11E20, 11F27. 351

352

RAINER SCHULZE-PILLOT

most to the rth power by the primes in T , and q ∗ (L) for the set of t ∈ q(L) that are represented primitively by some lattice in the genus of (L, q). For t ∈ qr (L), there is an integer m (which is bounded) consisting only of primes in T such that t/m2 ∈ q ∗ (L); all statements about representation of sufficiently large t ∈ qr (L) are therefore immediately reduced to the corresponding statements for sufficiently large t ∈ q ∗ (L). Recall the following facts about representation of numbers by spinor genera of ternary lattices (see [3], [8], [10], and [11]): • There is a finite set of square classes ti Z2 (the spinor exceptional square classes) such that numbers in q(L) outside these square classes are represented (primitively if they are in q ∗ (L)) by all spinor genera in the genus of L (i.e., in each spinor genus, there is at least one lattice representing the number), and all spinor genera have the same measure of representation (or Darstellungsmaß) for such an integer. An integer t whose square-free part does not divide d is not in any of these exceptional square classes. • For each of the spinor exceptional square classes, the set of spinor genera in the genus of (L, q) is divided into two half-genera containing equally many spinor genera such that all spinor genera in the same half-genus (primitively) represent the same numbers in that square class (and with equal representation measures). The numbers that are (primitively) represented only by one of the half-genera are called the (primitive) spinor exceptions of the genus; if t is a primitive spinor exception and m is an integer prime to the level N , then tm2 is a primitive spinor exception too. The sets of (primitive) spinor exceptions have√been explicitly determined in [10] and [4]. • For each ti from above, let Ei = Q( −dti ). If p is a prime that splits in Ei /Q, then for all t ∈ ti Z2 , the integer tp2 is a (primitive) spinor exception if and only if t is a (primitive) spinor exception and t and tp2 are represented by the same spinor genera in the genus of (L, q). • Let ti , Ei be as above and let p be a prime that is inert in Ei /Q. Let t ∈ ti Z2 be a primitive spinor exception of the genus of (L, q) represented by the halfgenus of spn(L). If p |N, then tp2 is primitively represented by the other halfgenus not containing spn(L) (but not by the half-genus of L). In particular, the tm2 with (m, N ) = 1, for which at least one prime factor of m is inert in Ei /Q, are primitive spinor exceptions but not spinor exceptions. If p | N , then either there is a ν0 depending on N such that tp2ν is not a primitive spinor exception for ν ≥ ν0 or the tp 2ν behave in the the same way as in the case p |N. • Let ti , Ei be as above and let t ∈ ti Z2 , p be a prime. If p is ramified in Ei /Q and p 2ν divides t for ν ∈ N large enough (depending on N), then t is neither a spinor exception nor a primitive spinor exception. We proved in [2] for positive definite (L, q) that all sufficiently large integers that are primitively represented by the spinor genus of (L, q) are represented by all lattices in that spinor genus and gave an asymptotic formula for the number of representations. However, we made no statement about the representation behaviour

EXCEPTIONAL INTEGERS OF TERNARY QUADRATIC FORMS

353

of those integers that are primitive spinor exceptions but not spinor exceptions: If they are sufficiently large, they are represented by all lattices in the spinor genera representing them primitively and by at least one lattice in each of the spinor genera in the other half of the set of spinor genera in the genus. The following theorem shows that most of these integers are also represented by all lattices in the genus with possibly (and usually) infinitely many exceptions. Theorem. There is some constant c depending on N and r such that all t ∈ qr (L) with t ≥ c are represented by all lattices in the genus of (L, q) unless one of the following conditions is satisfied: (i) t is a spinor exceptional integer, in which case it is represented by all lattices in the half-genus representing t and by no lattice in the other half-genus; (ii) t/p 2 is a spinor exceptional integer for some prime p that is inert in the quadratic extension E/Q associated to the square class of t, in which case t is represented by all lattices in the half-genus not representing t/p 2 and by precisely those lattices in the other half-genus that represent t/p 2 (hence by all of them if t/p 2 ≥ c holds). Proof. We assume without loss of generality that t ∈ q ∗ (L). By the results of [2], there is a constant c0 depending on N such that all integers t ≥ c0 in qr (L) that are primitively represented by some lattice in the spinor genus of (L, q) are represented by all lattices in the spinor genus of (L, q). Choosing c ≥ c0 , we therefore have only to deal with t that are represented by the spinor genus of (L, q) but are not represented primitively by that spinor genus. Let E/Q be the quadratic extension associated to the square class of t. Let tˆ ∈ q ∗ (L) ∩ t (Q× )2 be such that all representations of tˆ by the lattices in gen(L) are primitive; we call such a number a primitive element of q ∗ (L) and notice that such numbers are almost square-free in the following sense: There is some constant c1 depending on N such that primitive elements of q ∗ (L) are divisible by m2 only for m ≤ c1 (this is an easy consequence of the well-known representation properties of local lattices [9]). Hence there is a constant c2 depending on N such that tˆ ≤ c2 holds for all primitive elements tˆ of q ∗ (L) that are in one of the spinor exceptional square classes. We can choose tˆ such that it divides our given t, write t = tˆm2 , and decompose m = ma mr mi ms , where ma is the largest divisor of m consisting only of primes p for which Lp is anisotropic, mr is the largest divisor of m consisting only of primes ramified in E/Q and prime to ma , and mi is the largest divisor of m consisting only of primes inert in E/Q and prime to ma . By assumption, the “anisotropic part” ma of m is bounded, and we can restrict ourselves to the case ma = 1; in particular, we can assume t ∈ q ∗ (L). Since, by assumption, t is not represented primitively by the spinor genus of L, it is a primitive spinor exception. The results of [4] quoted above imply then that mr is bounded as well, and we can restrict to mr = 1 (the restrictions made being justified by suitably enlargening c0 ). Let K be a lattice in the genus of L representing tˆ. The integer t1 := tˆm2s is then a spinor exception that is primitively represented by some lattice K in the half-genus

354


of K; it is (primitively) represented by all classes in that genus if t ≥ c0 by the results of [2]. We are left with the case that t1 ≤ c0 and mi = 1 hold. We choose c ≥ c02 . If mi is composite, it has at least one proper divisor mi such that t1 m 2i ≥ c0 holds. Hence t1 m 2i is represented (primitively) by all classes in one half-genus H1 in the genus of L, and for any prime p | (mi /mi ), we see that t1 m 2i p 2 is represented (primitively) by all classes in the other half-genus H2 of the genus of L and represented (but not primitively) by all classes in H1 . Then, of course, t is represented by all classes in the genus of L as well, and our assertion is proved. Remark 1. An anologous result is true for representations with additional congruence conditions. The proof is the same as above. Remark 2. The result we gave in [2] is not effective. It would be no principal problem to make the estimates on the error term in our asymptotic formula effective; in fact, for the estimates of Fourier coefficients of modular forms of half-integral weight greater than or equal to 5/2, this was done in the Diplom thesis (or Diplomarbeit) of M. Bienert [1]. However, as already remarked in [2], the growth of the main term in our asymptotic formula is of the same order of magnitude as the growth of the class number of imaginary quadratic fields. The effective bound of Goldfeld [5], following from the work of Gross and Zagier, for the class number is much too weak for the present purpose. It is, however, well known that under the assumption of the generalized Riemann hypothesis, the class number bound h(d) d 1/2− can be made effective. In fact, we have (from Siegel’s proof of his class number estimate), √ for any 0 < < 1 for which the Dedekind zeta function ζD (s) of Q( −D) satisfies ζD (1 − /2) ≤ 0, the estimate wD 1/2−(1−2) h(D) > 4πe4π √ (where w is the number of units of Q( −D). Under the assumption of the generalized Riemann hypothesis, our asymptotic formula (and the sharpening in the theorem above) therefore becomes effective too. The author is frequently asked whether the results of [2] also hold for primitive representations. We take this occasion to state this fact in a corollary. Corollary. There is some constant c∗ depending on N and r, such that all t ∈ qr (L) with t ≥ c∗ that are represented primitively by some lattice in the spinor genus of (L, q) are represented primitively by all lattices in the spinor genus of (L, q). The same is true for representations with additional congruence conditions. Moreover, the primitive representations of t satisfying these conditions by (L, q) are asymptotically equidistributed in the sense of [2, Theorem 3] on the ellipsoid surface {x ∈ L ⊗ R | q(x) = t}. Proof. Denote by r(L, q, t) = #{x ∈ L | q(x) = t} the number of representations of t by (L, q) and by r ∗ (L, q, t) the number of primitive representations. By the

EXCEPTIONAL INTEGERS OF TERNARY QUADRATIC FORMS

355

Möbius inversion formula, we have r ∗ (L, q, t) =

d 2 |t

As usual, we denote by

t µ(d)r L, q, 2 . d

r spn(L, q), t :=

{K} r(K, q, t)/|O(K)|

{K} 1/|O(K)|

(where the summation goes over a set of representatives of the classes of lattices in the spinor genus of L and where |O(K)| is the order of the group of units or isometries of (K, q)) the weighted mean of the representation numbers of t by the lattices in the spinor genus of L and analogously for the averaged primitive representation number r ∗ (spn(L, q), t). Obviously, we then have t µ(d)r spn(L, q), 2 , r ∗ spn(L, q), t = d 2 d |t

and hence

t t r ∗ (L, q, t) − r ∗ spn(L, q), t = µ(d) r L, q, 2 − r spn(L, q), 2 . d d 2 d |t

The number of terms in the sum on the right-hand side is at most the number of divisors of t, hence O(t ) for all > 0. Each summand µ(d)(r(L, q, t/d 2 ) − r(spn(L, q), t/d 2 )) is estimated as a cusp form coefficient in the same way as in [2], and the main term r ∗ (spn(L, q), t) is at least of the order of magnitude of t 1/2− as in [2]. This proves the first assertion; the remainder of the corollary is proved in the same way as in [2]. 2. Shimura correspondence. As usual, ϑ(K, z) = x∈K exp(2πiq(x)z) is the theta series of the lattice K, and {K} ϑ(K, z)/|O(K)| ϑ spn(L), z := {K} 1/|O(K)| (where the summation goes over a set of representatives of the classes of lattices in the spinor genus of L and where |O(K)| is the order of the group of units or isometries of (K, q)) is the theta series of the spinor genus of L. We proved in [11], [12] that the Shimura lift with respect to any t of ϑ(L, z)−ϑ(spn(L), z) is cuspidal. Let us consider the following example taken from [7]: We look at the lattice K giving the quadratic form 4x 2 + 48y 2 + 49z2 + 48yz + 4xz and the lattice K giving the quadratic form x 2 + 48y 2 + 144z2 . The forms are in the same spinor genus, with another spinor genus in the same genus consisting of the lattice L with quadratic

356


form 9x 2 + 16y 2 + 48z2 and the lattice L with quadratic form 16x 2 + 25y 2 + 25z2 + 14yz + 16xz + 16xy. The exceptional square class√is just the set of integral squares, and the associated quadratic extension is E = Q( −3) by [10]. Write f (z) = ϑ(K , z) − ϑ(K, z) and g(z) = ϑ(L, z) − ϑ(L , z). By results from [12], f, g are “good” cusp forms, that is, forms whose Shimura lifting is cuspidal. The explicit calculation of T (p 2 ) acting on theta series given in [12] also shows that T (p 2 )f is a scalar multiple λp f of f for p ≡ 1 mod 3 and λp g of g for p ≡ −1 mod 3, and vice versa with λp replaced by some µp for g. This follows from the fact that T (p2 )f is in the first case, a cusp form in the one-dimensional space of cusp forms generated by ϑ(K, z), ϑ(K , z), and in the second case, a cusp form in the one-dimensional space of cusp forms generated by ϑ(L, z), ϑ(L , z). In fact, an explicit (computer-assisted) calculation of the theta series and their T (p 2 )-images for the first few primes p shows that we have T (p2 )f = λp f , T (p 2 )g = λp g (respectively, T (p 2 )g = λp f , T (p 2 )f = λp g) with λp = −2, 0, −4, −2, 4 for p = 5, 7, 11, 13, 19. But since for p, p ≡ 1 mod 3 and q, q ≡ −1 mod 3, we have λp µq = λq µp and λq µq = µq λq , by the commutativity of the Hecke algebra, we see that λp = µp holds for all p = 2, 3. The associated eigenforms f + g, f − g and their Shimura lifts are thus finally seen to have Hecke eigenvalues λp , χ (p)λp , where χ(p) = (−3/p) is the quadratic character associated to E/Q, that is, they are quadratic twists of each other. In general, the situation may be slightly more complicated, but the apparent lacunarity of the Fourier coefficients of ϑ(spn(L))−ϑ(L) in the square class of a primitive spinor exception is also caused by adding up character twists of cusp forms for the quadratic character associated to the square class in question. References [1] [2]

[3] [4] [5] [6] [7] [8] [9] [10]

M. Bienert, Fourier-Koeffizienten von Modulformen halbganzen Gewichts vom Nebentyp, Diplomarbeit, Universität zu Köln, 1996. W. Duke and R. Schulze-Pillot, Representation of integers by positive ternary quadratic forms and equidistribution of lattice points on ellipsoids, Invent. Math. 99 (1990), 49–57. A. Earnest, Representation of spinor exceptional integers by ternary quadratic forms, Nagoya Math. J. 93 (1984), 27–38. A. Earnest, J. S. Hsia, and D. Hung, Primitive representations by spinor genera of ternary quadratic forms, J. London Math. Soc. (2) 50 (1994), 222–230. D. Goldfeld, Gauss’s class number problem for imaginary quadratic fields, Bull. Amer. Math. Soc. (N.S.) 13 (1985), 23–37. J. S. Hsia, Representations by spinor genera, Pacific J. Math. 63 (1976), 147–152. B. W. Jones and G. Pall, Regular and semiregular positive definite ternary quadratic forms, Acta Math. 70 (1940), 165–191. M. Kneser, Darstellungsmaße indefiniter quadratischer Formen, Math. Z. 77 (1961), 188–194. O. T. O’Meara, Introduction to Quadratic Forms, 2d ed., Grundlehren Math. Wiss. 117, Springer-Verlag, New York, 1971. R. Schulze-Pillot, Darstellung durch Spinorgeschlechter ternärer quadratischer Formen, J. Number Theory 12 (1980), 529–540.

EXCEPTIONAL INTEGERS OF TERNARY QUADRATIC FORMS [11] [12]

357

, Darstellungsmaße von Spinorgeschlechtern ternärer quadratischer Formen, J. Reine Angew. Math. 352 (1984), 114–132. , Thetareihen positiv definiter quadratischer Formen, Invent. Math. 75 (1984), 283–299.

Fachbereich 9 Mathematik, Bau 27, Universität des Saarlandes, Postfach 15 11 50, D-66041 Saarbrücken, Germany; [email protected]

Vol. 102, No. 2


© 2000

FORMAL CHOW GROUPS, p-DIVISIBLE GROUPS, AND SYNTOMIC COHOMOLOGY TAKAO YAMAZAKI

Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359 2. Syntomic complex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363 3. Calculation of PD-algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368 4. Surjectivity of symbol map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370 5. The Leray spectral sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374 6. Syntomic cohomology and p-divisible groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379 7. K-cohomological functor and p-divisible groups . . . . . . . . . . . . . . . . . . . . . . . . . 384 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386 1. Introduction. Let K be an absolutely unramified complete discrete valuation field of mixed characteristic (0, p) with perfect residue field k, and V a smooth projective variety over K. In this article, we consider the Chow groups CHr (V ) in the infinitesimal method, which was proposed by Spencer Bloch (cf. [5, p. 24]). We review this method in what follows. First of all, recall the modified version of the Bloch-Quillen formula (cf. [16]) 1 1 M ∼ CHr (V ) ⊗ Z , ⊗Z = H r V , ᏷r,V (r − 1)! (r − 1)! M the Zariski sheaf of Milnor K -groups where, for any scheme Z, we denote by ᏷r,Z r on Z, that is, the sheafification of the presheaf that associates the Milnor K-group KrM ( (U, ᏻU )) to an open subscheme U of Z. Here, for any commutative ring R, we denote by KrM (R) the group R ∗⊗r /H, where H is the subgroup of R ∗⊗r generated by elements of the form x1 ⊗ · · · ⊗ xr with xi + xj = 0 or 1 for some i = j . If r is equal to the dimension of V , the above formula is valid without tensoring Z[1/(r − 1)!] (cf. [10]). Now assume that V admits a smooth projective model X over the valuation ring W of K (i.e., W = W (k) is the ring of Witt vectors of k). Let Ꮽ be the category of Artinian local W -algebras that have k as their residue field. For any object A of Ꮽ,

Received 27 April 1999. Revision received 20 September 1999. 1991 Mathematics Subject Classification. Primary 14G20; Secondary 11G25, 14F30, 19E20. Author’s research supported by Japan Society for the Promotion of Science (JSPS) Research Fellowships for Young Scientists. 359

360

TAKAO YAMAZAKI

we write XA = X ⊗W A. Set Y = Xk . For r > 0, define M M M ᏷r,X = ker ᏷r,X −→ ᏷r,Y , A ,Y A where the map is induced by the reduction map A → k. Note that since XA and Y M M as have the same underlying topological space, we can regard both ᏷r,X and ᏷r,Y A sheaves on (XA )Zar = YZar . For r > 0 and q ≥ 0, we define the K-cohomological M ) as functor Hˆ q (᏷r,X M M (A) = H q XA , ᏷r,X . Hˆ q ᏷r,X A ,Y This is a covariant functor from Ꮽ to the category of abelian groups. According to the M ) as the formal completion of CHr (V ). Bloch-Quillen formula, we regard Hˆ r (᏷r,X It seems natural to expect that it reflects an infinitesimal behavior of CHr (V ) at M ), mainly when the origin. The aim of this article is to study the functor Hˆ q (᏷r,X q = dim V . As an application of our theory, we prove the following theorem, which was conjectured by Bloch (cf. [5, p. 24]). Let Ꮽ0 be the full subcategory of Ꮽ consisting of m m all objects A such that A/pA ∼ = k[T1 , . . . , Ts ]/(T1 1 , . . . , Ts s ) for some s ≥ 0 and m1 , . . . , ms ≥ 1. (For more explanation of this category, see (1) below.) Theorem A. Assume p ≥ 3. Let X be a projective smooth scheme over W of relative dimension d. Assume that d < p and that the de Rham cohomology groups j HdR (X/W ) have no torsion for all j . Let 0 < r < p. When r = d, we assume that Y is of Hodge-Witt type (cf. Remark 6.11; more generally, we only need to assume that d+r−1 the de Rham cohomology group HdR (X/W ) is of Hodge-Witt type as a filtered Dieudonné module—cf. Definition 6.10). Let Gc be the p-divisible group over W , which is characterized by the condition in Remark 1.1 below. Then there exists a surjective homomorphism (functorial in A) M (A) −→ Gc (A) Hˆ d ᏷r,X for any object A of Ꮽ0 . d+r−1 Remark 1.1. We regard the de Rham cohomology group HdR (X/W ) as a filtered Dieudonné module (cf. remark after Definition 5.1). Let N be the level [r −1, d+r−1 r]-part of HdR (X/W ) (cf. Proposition 6.12 for the definition). Its translation N[r − 1] is a filtered Dieudonné module of level [0, 1], and so it corresponds to a p-divisible group G via the Dieudonné functor (cf. (15)). Then Gc is the connected component of G. The tangent space Lie Gc of Gc is isomorphic to H d (X, "r−1 X/W ). One should regard the homomorphism in Theorem A for r = d as the formal completion of the Albanese map

A0 (V ) −→ AlbV (K),

FORMAL CHOW GROUPS

361

deg

where A0 (V ) = ker(CHd (V ) −→ Z), and AlbV (K) is the group of K-rational points of the Albanese variety AlbV of V . M ) is to use the syntomic cohomology theory of Our method of studying Hˆ q (᏷r,X Fontaine-Messing. Let Ꮽsyn be the full subcategory of Ꮽ consisting of all objects A such that A/pA is syntomic over k. In Section 2, for any object A of Ꮽsyn we define a group H q ((XA , Y ), ᏿(r)), which we call the relative syntomic cohomology group. This is a slight modification of the syntomic cohomology group of Fontaine-Messing. We need to restrict the category Ꮽ to the smaller one Ꮽsyn , since we cannot define the group H q ((XA , Y ), ᏿(r)) if A does not belong to Ꮽsyn . We further restrict the category Ꮽsyn to the much smaller one Ꮽ0 , for a technical reason (since Ꮽsyn contains rings that the author could not control). We remark that the following rings belong to Ꮽ0 :

• OK /p n OK where n is any positive integer and OK is the integer ring of a totally ramified finite extension of K; (1) • the affine ring of a connected finite flat group scheme over W . In Section 3, we make some preliminary calculation about rings in Ꮽ0 . In the following theorem, which is proven in Section 4, we compare the Kcohomological functor with the relative syntomic cohomology group. Theorem B (Corollary 4.6). Let X be a quasi-projective smooth scheme over W of relative dimension d. Let 0 < r < p. For any object A of Ꮽ0 , there is a canonical surjective homomorphism (functorial in A) M Hˆ d ᏷r,X (A) −→ H d+r (XA , Y ), ᏿(r) . In the appendix, we show that the kernel of this map is essentially zero, in the sense of Stienstra (see Definition A.1, taken from [18, p. 2]). However, we do not know whether this map is in fact an isomorphism or not (cf. Conjecture 4.2, Remark 4.7). M ) for q = d. We also do not know anything about Hˆ q (᏷r,X Our next objective is to analyze the syntomic cohomology groups. In Section 5, we construct a Leray spectral sequence for the syntomic cohomology groups. The E2 -terms are given by the relative syntomic cohomology groups with coefficients (cf. Definition 5.2), which are written as H i ((A, k), ᏿(M, r)), where M is a filtered Dieudonné module (cf. Definition 5.1) and A is an object of Ꮽsyn . The de Rham j cohomology group HdR (X/W ) has a natural structure of a filtered Dieudonné module (cf. remark after Definition 5.1). Theorem C (Corollary 5.5). Let X be a projective smooth scheme over W of relative dimension d. Assume that d < p and that the de Rham cohomology groups j HdR (X/W ) have no torsion for all j . Let 0 < r < p. Then, for any object A of Ꮽsyn there exists a spectral sequence

362

TAKAO YAMAZAKI

j i,j E2 = H i (A, k), ᏿ HdR (X/W ), r ⇒ H i+j (XA , Y ), ᏿(r) . This spectral sequence induces a filtration on H q ((XA , Y ), ᏿(r)). We find an analogous filtration in Stienstra [18, p. 5]. We might wonder whether this filtration corresponds to the “motivic filtration” of Deligne-Beilinson. Roughly speaking, the following theorem says that the top-graded quotient is prorepresentable by a pdivisible group. Theorem D (Corollary 6.8). Assume p ≥ 3. Let M be a filtered Dieudonné module corresponding to a p-divisible group G via the Dieudonné functor (15). Let A be an object of Ꮽ0 . Then there exists a canonical isomorphism ∼ = Gc (A) −−→ H 1 (A, k), ᏿(M, 1) , where Gc is the connected component of G. This theorem is proven in Section 6. Taking care of a switch of the twist, we show that those three theorems imply Theorem A in Section 7. In the appendix, we consider the injectivity of the map in Theorem B. Such a problem might be considered in the equal characteristic situation, which was also proposed by Bloch. He obtained a result in the case of characteristic zero (cf. [2], [3], [4, Chapter 6]). Stienstra got a very general result in the positive characteristic case, by using his theory of the formal de Rham-Witt complex (cf. [18], [17], [4, Chapter 6]). Though we do not use their results, we might see some analogy with those works, as mentioned above. According to Stienstra’s theory [18], it would be interesting to interpret our theory in the context of a generalization of the Dieudonné theory, in which Theorem D plays a basic role. Perhaps further study of the syntomic cohomology groups (like H 2 ((A, k), ᏿(M, 2))) will be a guide for a study of a deeper part of the Chow groups (like the albanese kernel). For this point, see Proposition 7.1. Acknowledgements. This research is a subject of my Ph.D. thesis, which is directed by Kazuya Kato. Without his advice, this paper would not exist. I would like to express my deep gratitude to him. I am also grateful to Kanetomo Sato, Jinya Nakamura, and especially Jan Stienstra for a lot of fruitful discussions. In particular, the context of the appendix arose from discussion with Stienstra. I heartily thank Thomas Geisser, who carefully read the earlier version of this manuscript and gave a lot of comments which improved this paper very much. I acknowledge once again the great influence of profound works of Spencer Bloch and Jan Stienstra. Conventions 1.2. For a ∈ Q, write [a] = inf{n ∈ Z | n ≥ a}. For an abelian group A and an integer n, we write n A = {a ∈ A | na = 0}. Let k denote a fixed perfect field of characteristic p > 0, W = W (k) the ring of Witt vectors of k, and σ the frobenius on W . For a ring or a scheme S, Sn denotes S ⊗Z Z/p n Z and "1S denotes the absolute

363

FORMAL CHOW GROUPS

differential module "1S/Z . If S is a ring, Sˆ denotes the ring lim Sn . If S is a scheme, ← − Sˆ denotes the formal scheme lim Sn . − → A PD-envelope of a W -algebra (or a W -scheme) is the one with respect to the canonical divided power structure on pW . If D is a PD-algebra (or a PD-scheme) with PD-ideal (sheaf) JD , JD[r] denotes the rth divided power ideal (sheaf). For r ≤ 0, we use the convention JD[r] = D (or ᏻD ). All sheaves and cohomology groups are taken in the Zariski site, unless stated otherwise. 2. Syntomic complex. In this section, we define a relative syntomic complex for a W -scheme. This is a slight modification of the theory of Fontaine-Messing and Kato. Our main references are [11] and [12]. Let U be a W -scheme such that p N ᏻU = 0 for some N > 0. We assume that U1 is syntomic over k, and that there exists a W -immersion U +→ Z with Z a smooth W -scheme endowed with a Frobenius φ : Z → Z, that is, a morphism of schemes such that φ ⊗ Z/pZ : Z1 → Z1 is the absolute Frobenius and such that the diagram φ

Z Spec W

σ

/Z / Spec W

commutes (cf. [11, I.1.2]). Let D = DU (Z) be the PD-envelope with PD-ideal sheaf JD , and let Dˆ = lim Dn be its p-adic completion endowed with the induced PD− → structure on JDˆ (cf. Conventions 1.2). Note that the PD-envelope DUn (Zn ) is canonically isomorphic to Dn (= D ⊗ Z/pn Z), whose PD-ideal sheaf JDn is the image of JD (cf. [1, 3.33]). We also remark that the underlying topological spaces of Dˆ and U are the same. ˆ q be the continuous differential module lim "q . For r ≥ 0 and n > 0, we Let " Z ← − Zn denote the complexes of sheaves on UZar (cf. [1, 7.23]) d

d

d

ˆ 1Z −→ J [r−2] ⊗ᏻ " ˆ 2Z −→ · · · J [r] −→ J [r−1] ⊗ᏻZˆ " ˆ ˆ ˆ Zˆ D

D

D

and d

d

d

⊗ᏻZn "1Zn −→ JD[r−2] JD[r]n −→ JD[r−1] ⊗ᏻZn "2Zn −→ · · · n n [r] by J[r] U,Z and by Jn,U,Z , respectively.

Lemma 2.1. (i) For 0 ≤ r 0, we have r [0] φ J[r] n,U,Z ⊂ p Jn,U,Z . (ii) The following sequence is exact for any n, m > 0: pm

pn

[0] [0] [0] J[0] m+n,U,Z −−→ Jm+n,U,Z −−→ Jm+n,U,Z −−→ Jn,U,Z −→ 0. can

364

TAKAO YAMAZAKI

Proof. Since D ∼ = DU1 (Z), the proof of [11, I.1.3] works in this situation as well. Because of the lemma, we can define [0] φr = p −r φ : J[r] n+r,U,Z −→ Jn,U,Z

and [0] φr : J[r] U,Z −→ JU,Z

as the inverse limit. We remark that an analogous sequence of (ii) for J[r] n,U,Z with

[0] r > 0 is not exact, and hence we cannot define φr : J[r] n,U,Z → Jn,U,Z .

Definition 2.2. Let 0 ≤ r < p. We define a complex ᏿(r)U,Z of Zariski sheaves on U to be the mapping fiber of [0] 1 − φr : J[r] U,Z −→ JU,Z .

More precisely, the degree-i part of ᏿(r)U,Z is [r−i] ˆ iZ ⊕ ᏻ ˆ ⊗ᏻ " ˆ i−1 , Jˆ ⊗ᏻZˆ " Z D Zˆ D

and the boundary map is given by (x, y) −→ dx, (1 − φr )(x) − dy ˆ i , y ∈ ᏻ ˆ ⊗" ˆ i−1 ). We denote by H q (U, ᏿(r)) the qth hypercoho(x ∈ J [r−i] ⊗" Z Z D Dˆ mology group of the complex ᏿(r)U,Z . To justify the notation, we remark that ᏿(r)U,Z is independent of the choice of Z in the derived category D(UZar ). This can be shown by the same way as in [11, p. 212] (cf. also [12, p. 412]), by using [1, 7.24]. For the same reason, we write the cohomology sheaf of the complex ᏿(r)U,Z as Ᏼq (᏿(r)U ), instead of Ᏼq (᏿(r)U,Z ). We define the symbol map KrM (U, OU ) −→ H r U, ᏿(r) in completely the same way as in [11, I, Section 3] (also cf. [12, pp. 412–413]). For the sake of completeness, we copy it here. Let r and r be nonnegative integers satisfying r + r < p. We define the product structure ᏿(r)U,Z × ᏿(r )U,Z −→ ᏿(r + r )U,Z

by (x, y)(x , y ) = xx , (−1)i xy − yφ(x ) ,

FORMAL CHOW GROUPS

365

ˆ i )⊕(ᏻ ˆ ⊗ ⊗" where (x, y) and (x , y ) are local sections of the degree-i part (J [r−i] Z D Dˆ ] [r−i ˆ i )⊕(ᏻ ˆ ⊗ " ˆ i −1 ), respectively. This product ˆ i−1 ) and the degree-i part (J ⊗" " Z

structure induces a product

Dˆ

Z

Z

D

H i U, ᏿(r) ⊗ H i U, ᏿(r ) −→ H i+i U, ᏿(r + r ) , which satisfies an associative law. Denote the immersion U → Z by i. Let Ꮿ be the complex ker i −1 ᏻ∗Z −→ ᏻ∗U −→ i −1 ᏻ∗Z , which is canonically quasi-isomorphic to ᏻ∗U [−1]. We define a morphism of complexes Ꮿ → ᏿(1)U,Z as follows: The degree-zero part ker(i −1 ᏻ∗Z → ᏻ∗U ) → JDˆ is defined by x −→ log(x) ¯ =

∞ (−1)i (i − 1)!(x¯ − 1)[i] , i=1

where x¯ is the image of x in ᏻDˆ and [ ] denotes the PD-structure. (Note that x¯ − 1 is ˆ 1 ) ⊕ ᏻ ˆ is defined by in JDˆ .) The degree-one part i −1 ᏻ∗Z → (ᏻDˆ ⊗ " Z D ¯ x¯ p . x −→ 1 ⊗ dx/x, p −1 log φ(x)/ (Note that φ(x)/ ¯ x¯ p − 1 is in p ᏻDˆ and that log(φ(x)/ ¯ x¯ p ) is in p ᏻDˆ .) This morphism of complexes induces a canonical homomorphism U, ᏻ∗U = H 1 (U, Ꮿ) −→ H 1 U, ᏿(1) . Hence, by the product structure we obtain a homomorphism ⊗r U, ᏻ∗U −→ H r U, ᏿(r) for r < p. It is shown in [11, I, 3.2] that this homomorphism factors through KrM ( (U, ᏻ∗U )). By sheafification, we get a morphism of sheaves on UZar ,

M ᏷r,U −→ Ᏼr ᏿(r)U ,

which we call the symbol map. Let X be a quasi-projective smooth W -scheme and A an object of Ꮽsyn . Since we assumed X to be quasi-projective, we can find a smooth W -scheme Z endowed with a Frobenius φ and an immersion XA +→ Z. Hence we can apply the above definitions to the scheme XA . Let Y = Xk . Note that we have not only ᏿(r)XA ,Z but also ᏿(r)Y,Z by considering the composition Y +→ XA +→ Z.

366

TAKAO YAMAZAKI

Definition 2.3. Let 0 ≤ r < p. We define the complex ᏿(r)(XA ,Y ),Z to be the mapping fiber of the natural map ᏿(r)XA ,Z −→ ᏿(r)Y,Z .

We denote by H q ((XA , Y ), ᏿(r)) and Ᏼq (᏿(r)(XA ,Y ) ) the qth hypercohomology group and the qth cohomology sheaf of the complex ᏿(r)(XA ,Y ),Z , respectively. Again, these hypercohomology groups and cohomology sheaves are independent of the choice of Z and φ. We remark that the map ᏿(r)XA ,Z → ᏿(r)Y,Z is injective. Hence, if we write the cokernel of this map by Ꮿ, ᏿(r)(X,Y ),Z is quasi-isomorphic to Ꮿ[−1]. But this mapping fiber definition is useful in Section 5. The following lemma, which can be seen easily, may be helpful in considering this complex. Lemma 2.4. Let Ꮿ1

fC

g1

Ᏸ1

/ Ꮿ2 g2

fD

/ Ᏸ2

be a commutative diagram of complexes. Writing the mapping fiber of a morphism h of complexes by Ᏺ(h), there are two morphisms of complexes: f = (fC , fD ) : Ᏺ(g1 ) −→ Ᏺ(g2 ), g = (g1 , g2 ) : Ᏺ(fC ) −→ Ᏺ(fD ). Then we have a canonical isomorphism Ᏺ(f ) ∼ = Ᏺ(g).

Consider the diagram 0

/ ᏷M r,XA ,Y

0

/ Ᏼr ᏿(r)(X ,Y ) A

α

/ ᏷M r,XA

/ ᏷M r,Y

/ Ᏼr ᏿(r)X A

/ Ᏼr ᏿(r)Y ,

/0

where the middle and the right vertical arrows are the symbol maps. The upper row is exact by definition, and the right square is commutative. Lemma 2.5 shows the lower row is also exact. Hence we can define the symbol map M ᏷r,X −→ Ᏼr ᏿(r)(XA ,Y ) A ,Y by the unique map α that makes the left square commutative.

367

FORMAL CHOW GROUPS

Lemma 2.5. The complex ᏿(r)Y,Z is acyclic outside [r, ∞). Define ᏿n (r)Y,Z by the exact sequence pn

0 −→ ᏿(r)Y,Z −−→ ᏿(r)Y,Z −→ ᏿n (r)Y,Z −→ 0. It is enough to show that ᏿n (r)Y,Z is acyclic outside [r, ∞). We prove a more precise lemma as follows. Lemma 2.6. Let 3 : Yet → YZar be the natural morphism of sites. Then ᏿n (r)Y,Z is quasi-isomorphic to R3∗ Wn "rY,log [−r]. Proof. By a standard induction argument, this is reduced to the case n = 1. In the étale site, "rY,log [−r] is quasi-isomorphic to the complex deg(r)

"rY

deg(r+1) 1−F

(2)

/ "r /d"r−1 , Y Y

where F is the inverse Cartier operator. Since each component of complexes ᏿1 (r)Y,Z and (2) is a coherent sheaf, it suffices to prove that ᏿1 (r)Y,Z is quasi-isomorphic to the complex (2) for the étale site. Then, we can assume that X itself has a Frobenius φ and that Z = X. By definition, ᏿1 (r)Y,X is the total complex associated to the double complex p r ᏻX /p r+1 ᏻX

/ p r−1 "1 /p r "1 X X

−φr

/ ···

/ "r Y

/ ···

/ "r Y

−φr

1−F

/ "1 Y

ᏻY

/ "r+1 Y

/ ···

1

/ "r+1 Y

/ ··· .

If 0 ≤ q < r, we have a commutative diagram q

q

p r−q "X /p r−q+1 "X

−φr

/ "q Y

∼ =

q "Y

−F

/ "q . Y q

q

d

q+1

The lower arrow is injective, and its cokernel is "Y / ker("Y − → "Y ). Using this repeatedly, we see that ᏿1 (r)Y,X is quasi-isomorphic to the total complex associated to 0

/ ···

/0

/ "r Y

/ ···

/0

/ "r /d"r−1 Y Y

1−F

0

From this, the assertion follows.

/ "r+1 Y

/ ···

1

/ "r+1 Y

/ ··· .

368

TAKAO YAMAZAKI

3. Calculation of PD-algebra. In this section, we calculate PD-envelope algebras of Artinian rings. The results are used in Sections 4 and 6. In this section, except in Remark 3.5, we use the following convention:  s ≥ 0; m1 , . . . , ms ≥ 1; m ≥ 2.    

   A = k[T1 , . . . , Ts , T ] T1m1 , . . . , Tsms , T m .    

m1   ms m−1  = k[T , . . . , T , T ] T , . . . , T , T . A 1 s  s 1      B = W [T1 , . . . , Ts , T ].     p φ : B −→ B; Tj −→ Tj , T −→ T p , a −→ σ (a) (a ∈ W ). (3)    m m m 1  D = DB T , . . . , T s , T  ⊃ JD ,  s 1       D = DB T1m1 , . . . , Tsms , T m−1 ⊃ JD ,        C = DW [T1 ,...,Ts ] T1m1 , . . . , Tsms ⊃ JC ,      are the PD-envelope algebras and their PD-ideals. Note that there is a natural map D +→ D induced by the obvious maps B → A → A . It is easily seen that D= CT [i,m] . i≥0

Here we use the following notation: T [i,m] denotes γa (T m )T b , where γ denotes the PD-structure a = [i/m] and b = i −am. The sub-C-module of D generated by T [i,m] is written as CT [i,m] . If i < pm, then T [i,m] = T i /[i/m]!. Noting this, we have for 0 ≤ r < p,  rm−1

 [r−[i/m]] i J [r] = J T ⊕ CT [i,m]  . D

i=0

C

i≥rm

If we replace m by m − 1 in the right-hand side of the above two equations, we get a description of D and JD[r] . We consider groups JD[r] /JD[r] . In the case of r = 0, since CT [i,m−1] /CT [i,m] = 0 for i < p(m − 1), we have D ∼ = D

i≥p(m−1)

CT [i,m−1] . CT [i,m]

(4)

ˆ [i,m−1] in Dˆ . This is a subgroup of Let S be the (p-adic) closure of ⊕i≥(m−1)p CT for any 0 ≤ r < p. The following lemma can be shown by elementary calculation.

J [r] Dˆ

369

FORMAL CHOW GROUPS

Lemma 3.1. Let 0 ≤ r < p. For any element x of S, φr (x) again belongs to S, and j φr (x) = φr ◦ · · · ◦ φr (x) (j times) converges p-adically to zero as j → ∞. Corollary 3.2. The map 1 − φr :

J [r] ˆ

D J [r] Dˆ

−→

Dˆ Dˆ

is a surjection for any 0 ≤ r < p and is an isomorphism if r = 0. Proof. From (4), it is enough to show that, for any x ∈ S, the class of x in Dˆ /Dˆ j is in the image of 1 − φr . But the above lemma shows that y = ∞ j =0 φr (x) is a , and it is clear that (1 − φr )(y) = x. When r = 0, well-defined element of S ⊂ J [r] Dˆ this correspondence defines the inverse map of 1 − φ. [r−[i/(m−1)]]

[r−[i/m]]

Next, we consider the case of r = 1. Since (JC T [i,m−1] )/(JC if 0 ≤ i < m − 1 or if m ≤ i < p(m − 1), we can write   JD ∼ CT m−1  CT [i,m−1]  ⊕ . = JD CT [i,m] JC T m−1

T [i,m] ) = 0

i≥p(m−1)

We remark that the second term is the same as the right-hand side of (4). This shows that the inclusion map JD → D induces a surjection JD /JD → D /D, which has a splitting (inclusion to the second term in the above presentation) ι:

D JD −→ . D JD

Passing to the limits, we have ˆ m−1 Dˆ JDˆ ∼ CT ⊕ , = JDˆ JCˆ T m−1 Dˆ

ι:

J ˆ Dˆ +→ D . JDˆ Dˆ

Now the following lemma is seen easily. Lemma 3.3. There exists an exact sequence 0 −→

ˆ m−1 s JDˆ 1−φ1 Dˆ CT −→ 0, −−→ −−−→ JDˆ JCˆ T m−1 Dˆ

ˆ m−1 , the image of the class of x via s is the class of where, for x ∈ CT (Since φ1 (x) ∈ S, this is a well-defined element of JDˆ by Lemma 3.1.)

∞

j j =0 φ1 (x).

370

TAKAO YAMAZAKI

Remark 3.4. Assume rm ≤ p(m − 1). (If r ≤ p/2, this assumption holds for any m ≥ 2.) Then the above argument works for J [r] /J [r] as well. In fact, under this Dˆ Dˆ assumption, there exists a description   [r−[i/(m−1)]] i rm−1 J T J [r] ˆ Cˆ Dˆ ∼  ⊕ D , = [r−[i/m]] i Dˆ J [r] J T i=0

Dˆ

Cˆ

and an exact sequence 0 −→

rm−1 i=0

[r−[i/(m−1)]] i T C [r−[i/m]] i Jˆ T C

Jˆ

sr

−→

J [r] ˆ

D J [r] Dˆ

1−φr

−−−→

Dˆ −→ 0, Dˆ

where sr is defined by the similar way. (If r = 1, s1 is nothing but s of Lemma 3.3.) The following remark is used in Sections 4 and 6. Remark 3.5. Let A be an object of Ꮽsyn such that pN+1 A = 0. Take a surjective W homomorphism B → A where B is a smooth W -algebra endowed with a Frobenius φ. Let A = A/p N A. Let D = DB (ker(B → A)) be the PD-envelope algebra. The PD-ideal JD of D contains pN+1 . We see that DB (ker(B → A )) ∼ = D by ignoring the PD-structure, and the PD-ideal JD of this PD-envelope algebra is JD + p N D. 4. Surjectivity of symbol map. In this section, let X be a smooth quasi-projective scheme over W . The aim of this section is to prove the following theorem and its Corollary 4.6. Theorem 4.1. Let A be an object of Ꮽ0 and 0 < r < p. (i) For any q > r, we have Ᏼq ᏿(r)(XA ,Y ) = 0. (ii) The symbol map

M ᏷r,X −→ Ᏼr ᏿(r)(XA ,Y ) A ,Y

is a surjection. In the appendix, we consider the kernel of the symbol map in (ii). Here we propose an optimistic conjecture, or, perhaps, a hope, as follows. Conjecture 4.2. Let 0 < r < p. For any object A of Ꮽsyn , (i) Ᏼq (᏿(r)(XA ,Y ) ) = 0 for any q > r; M (ii) the symbol map ᏷r,X → Ᏼr (᏿(r)(XA ,Y ) ) is an isomorphism. A ,Y Instead of proving Theorem 4.1, we prove a slightly more general proposition.

371

FORMAL CHOW GROUPS

Proposition 4.3. Let 0 < r < p. (i) If A is an object of Ꮽsyn , then we have Ᏼq (᏿(r)XA ) = 0 for q ≥ r + 2. (ii) If A is an object of Ꮽ0 , then Ᏼr+1 ᏿(r)XA −→ Ᏼr+1 ᏿(r)Y is an isomorphism, and the sequence M ᏷r,X −→ Ᏼr ᏿(r)XA −→ Ᏼr ᏿(r)Y −→ 0 A ,Y is exact, where the first map is the symbol map. For a smooth quasi-projective scheme X over W , Take a closed immersion X → Z over W such that Z is smooth over W and endowed with a Frobenius φ. Let T = DX (Z) be the PD-envelope with PD-ideal sheaf JT .

(5)

For an object A of Ꮽsyn , Take a surjective W -homomorphism B → A such that B is smooth over W and endowed with a Frobenius φ. Let D = DB (ker(B → A)) be the PD-envelope with PD-ideal JD .

(6)

Then the fiber product XA → Z ⊗W B = ZB is a closed W -immersion and ZB is smooth over W endowed with the Frobenius φ ⊗ φ. It can be seen that DXA (ZB ) ∼ = T ⊗W D with PD-ideal JT ⊗ D + ᏻT ⊗ JD , by using the universality of PD-envelope algebras and the flatness of D and X over W . Hence, to calculate the cohomology sheaves Ᏼq (᏿(r)XA ), we can use [r]

[0]

1−φr

᏿(r)XA ,ZB = mapping fiber of JXA ,ZB −−−→ JXA ,ZB , [0] where J[r] XA ,ZB and JXA ,ZB is the complex

ˆ 1Z −→ J [r−2] ⊗ᏻ " ˆ 2Z −→ · · · −→ JT[r−1] ⊗ᏻZˆ " JT[r]⊗D ˆ ˆ ˆ B B Zˆ ⊗D T ⊗D B

B

and ˆ 1Z −→ (ᏻT ⊗D) ˆ 2Z −→ · · · , ˆ −→ (ᏻT ⊗D) ˆ ˆ ᏻT ⊗D ⊗ᏻZˆ " ⊗ᏻZˆ " B B B

B

respectively. If q ≥ r + 1, q

q

ˆ ˆ ˆ ˆ 1 − φr : (ᏻT ⊗D) ⊗" ZB −→ (ᏻT ⊗D) ⊗ "ZB is an isomorphism, because it has the inverse map implies Proposition 4.3(i).

j j ≥0 φr

(note φr = p q−r φq ). This

372

TAKAO YAMAZAKI

To prove Proposition 4.3(ii), we first work under the assumptions and notation of (3), and we consider Ᏼq (᏿(r)XA ) → Ᏼq (᏿(r)XA ). Define a complex Ꮿ by exactness of 0 −→ ᏿(r)XA ,ZB −→ ᏿(r)XA ,ZB −→ Ꮿ −→ 0.

(7)

This complex Ꮿ is equal to the mapping fiber of J[r] X ,ZB A

J[r] XA ,ZB

1−φr

−−−→

J[0] X ,ZB A

,

J[0] XA ,ZB

[r] [0] [0] where J[r] X ,ZB /JXA ,ZB and JX ,ZB /JXA ,ZB are the complexes A

A

deg(r−2)

/

···

deg(r−1)

ˆ r−2 JT[2]⊗D ⊗" ZB ˆ ˆ r−2 JT[2]⊗D ⊗" ZB ˆ

/

deg(r)

ˆ r−1 JT[1]⊗D ⊗" ZB ˆ ˆ r−1 JT[1]⊗D ⊗" ZB ˆ

/

ˆr ˆ ) ⊗ " (OT ⊗D ZB ˆr ˆ (OT ⊗D) ⊗" ZB

/ ···

and ···

/

ˆ r−2 ˆ ) ⊗ " (OT ⊗D ZB ˆ r−2 ˆ (OT ⊗D) ⊗" ZB

/

ˆ r−1 ˆ ) ⊗ " (OT ⊗D ZB ˆ r−1 ˆ (OT ⊗D) ⊗" ZB

/

ˆr ˆ ) ⊗ " (OT ⊗D ZB ˆr ˆ (OT ⊗D) ⊗" ZB

/ ··· ,

respectively. We denote by αr,q the map [r−q] ˆq JT ⊗D ⊗" ZB ˆ

[r−q] ˆq JT ⊗D ⊗" ZB ˆ

1−φr

−−−→

q

ˆ ˆ ) ⊗ " (OT ⊗D ZB ˆq ˆ (OT ⊗D) ⊗" ZB

.

By using Corollary 3.2, we see that αr,q is a surjection for any q and is an isomorphism if q ≥ r, so that Ᏼq (Ꮿ) = 0 if q ≥ r. By (7), this implies that Ᏼq ᏿(r)XA −→ Ᏼq ᏿(r)XA is an isomorphism if q ≥ r + 1 and is a surjection if q = r. By using Lemma 3.3, one gets isomorphisms

ˆ r−1 ∼ ᏻX ⊗ (C/JC ) ⊗ " ZB =

m−1 ˆ ᏻT ⊗CT

ˆ r−1 ⊗" ZB m−1 ˆ ˆ JT ⊗C + ᏻT ⊗JC T

∼ =

−→ ker(αr,r−1 ),

373

FORMAL CHOW GROUPS

j where the last map “ ∞ j =0 φ1 ” is defined by the similar way as in Lemma 3.3. By (7), we get a surjection ˆ r−1 −→ ker Ᏼr ᏿(r)XA −→ Ᏼr ᏿(r)X . ᏻX ⊗ (C/JC ) ⊗ " (8) ZB A By induction, for A as in (3), the proof of Proposition 4.3(ii) is completed by the following. Lemma 4.4. Let a ∈ ᏻX ⊗(C/JC ) and b1 , . . . , br−1 ∈ (ᏻX ⊗A)∗ be local sections. The image of {1 + aT m−1 , b1 , . . . , br−1 } under the symbol map coincides with the image of a ⊗ d b˜1 /b˜1 ∧ · · · ∧ d b˜r−1 /b˜r−1 under (8), where b˜i is a local lift of bi to OZ∗ B (i = 1, . . . , r − 1). Proof. Once we see the lemma for r = 1, the case r > 1 follows immediately from ˆ of the definition of the symbol and that of (8). Assume r = 1. Take a lift c ∈ ᏻT ⊗C j m−1 ) is well defined in ᏻT ⊗D a. Then a section ξ = j ≥0 φ1 (cT ˆ (cf. Lemma 3.1). The section ˆ1 dξ, (1 − φ1 )(ξ ) ∈ ᏻT ⊗D ˆ ⊗ "ZB ⊕ ᏻT ⊗D ˆ ˆ1 is in (ᏻT ⊗D its cohomology class is the image of aT m−1 under ˆ ⊗ "ZB ) ⊕ ᏻT ⊗D ˆ , and (8). Once we see the fact that n≥0 ξ n /n! is a well-defined section of ᏻT ⊗D ˆ , we complete the proof of the lemma, by taking this as a lift of 1 + aT m−1 . To see this fact, we first note that  n 1 j j  cp φ1 T m−1  n! n≥0

j ≥0

is a well-defined section of ᏻT ⊗D ˆ . (This is a consequence of the fact that the Artin j Hasse exponential n≥0 (1/n!)( j ≥0 X p /p j )n is in Z(p) [[X]].) By the way, it can be seen that j j j cp φ1 T m−1 − φ1 cT m−1 j ≥0

is in p ᏻT ⊗D ˆ . Then the fact follows. Next, we work under the assumptions and notation as in Remark 3.5. Again, we ∼ define a complex Ꮿ by exactness of the sequence (7). We remark that J[0] XA ,ZB = J[0] X ,ZB . Then it turns out that Ꮿ is equal to A

J [r] ˆ T ⊗D JT[r]⊗D ˆ

−→

J [r−1] ˆ T ⊗D

ˆ 1Z ⊗" B JT[r−1] ˆ ⊗D

−→

J [r−2] ˆ T ⊗D JT[r−2] ˆ ⊗D

ˆ 2Z −→ · · · . ⊗" B

374

TAKAO YAMAZAKI

Hence, we get a surjection ˆ r−1 ᏻX ⊗ p N A ⊗ " ZB

J ˆ ∼ T ⊗D ˆ r−1 −→ ker Ᏼr ᏿(r)XA −→ Ᏼr ᏿(r)X . ⊗" = ZB A JT ⊗D ˆ

(9)

Lemma 4.5. Let a ∈ ᏻX ⊗(C/JC ) and b1 , . . . , br−1 ∈ (ᏻX ⊗A)∗ be local sections. The image of {1+p N a, b1 , . . . , br−1 } under the symbol map coincides with the image of pN a ⊗ d b˜1 /b˜1 ∧ · · · ∧ d b˜r−1 /b˜r−1 under (9), where b˜i is a local lift of bi to OZ∗ B (i = 1, . . . , r − 1). The proof is similar to and easier than that of Lemma 4.4. By induction, this lemma completes the proof of Proposition 4.3. Corollary 4.6. Let d be the relative dimension of X over W . Let 0 < r < p. For any object A of Ꮽ0 , there is a canonical surjective homomorphism M Hˆ d ᏷r,X (A) −→ H d+r (XA , Y ), ᏿(r) . Proof. Consider the spectral sequence i,j E2 = H i Y, Ᏼj ᏿(r)(XA ,Y ) ⇒ H i+j (XA , Y ), ᏿(r) . i,j

We have E2 = 0 if j > r, by Theorem 4.1(i)—or if i > d, since the Zariski cohomological dimension of Y is d. Hence we have H d+r ((XA , Y ), ᏿(r)) = E2d,r . Since H d (Y, −) is a right exact functor, the corollary follows from Theorem 4.1(ii). Remark 4.7. The above proof shows that, if Conjecture 4.2 is true, the map in Corollary 4.6 is an isomorphism for any object A of Ꮽsyn . 5. The Leray spectral sequence. In this section, let X be a smooth projective scheme over W . This section is devoted to constructing a spectral sequence of Leray type for the relative syntomic cohomology groups H q ((XA , Y ), ᏿(r)). The following definition is taken from Wintenberger [19]. Definition 5.1. A filtered Dieudonné module over W of finite type is a W -module M endowed with (i) a decreasing filtration (M i )i∈Z such that M i = M(i % 0), M i = 0(i & 0); (ii) a family of σ -linear homomorphism (φi : M i → M)i∈Z such that φi |M i+1 = pφi+1 for all i ∈ Z; satisfying the conditions (iii) M i is of finite type over W for all i ∈ Z; (iv) M i is a direct summand of M as a W -module for all i ∈ Z; (v) M = i∈Z φi (M i ).

375

FORMAL CHOW GROUPS

We denote by MFW,tf the category of filtered Dieudonné modules of finite type over W . This is an abelian category (cf. [19]). The following fact, due to Fontaine [8] and Kato [11], plays an essential role in this section: If X is a smooth projective scheme over W of relative dimension d < p, then q ˆ " ˆ · ) has the structure of the de Rham cohomology group M = HdR (X/W ) ∼ = Hq (X, X an object of MFW,tf , defined as follows. The filtration (M i )i∈Z is the Hodge filtration, ˆ " ˆ " ˆ " ˆ ·≥i ). (Note that the natural map Hq (X, ˆ ·≥i ) → Hq (X, ˆ· ) that is, M i = Hq (X, X X X is injective; cf. [11, 2.5].) Let Z, φ be as in Section 4 (5). The homomorphism φi : M i → M(i < p) is defined as the composition φi ˆ " ˆ J[i] −− ˆ J[0] ∼ ˆ ·≥i ∼ M i = Hq X, → Hq X, = Hq X, X X,Z X,Z = M. Note that M p = 0. Let M be an object of MFW,tf . Assume that M is torsion-free. Let A be an object of Ꮽsyn and B, φ, D as in Section 4 (6). For 0 ≤ r < p, we define the syntomic complex ᏿(M, r)A,B and the relative syntomic complex ᏿(M, r)(A,k),B with coefficients in M. First we define for r ≥ 0 J (M)[r] A,B =

r i=0

ˆ M i ⊗W J [r−i] ⊂ M ⊗W D. ˆ D

We remark that this group is the inverse limit of the rth filtration subgroups of the sections of the crystals associated to M (on the big crystalline site of Spec(W )) at the PD-thicking (Spec(A), Spec(Dn )). We have a connection [r−1] ˆ1 ∇ = 1 ⊗ d : J (M)[r] A,B −→ J (M)A,B ⊗Bˆ "B

and its expansion ∇ [r−q] [r−q−1] ˆ q −− ˆ q+1 ⊗Bˆ " J (M)A,B ⊗Bˆ " B → J (M)A,B B

defined in the usual way. We denote the complex of abelian groups ∇

∇

∇

ˆ 1 −→ J (M)[r−2] ⊗ ˆ " ˆ 2 −→ · · · −→ J (M)[r−1] J (M)[r] A,B − A,B ⊗Bˆ "B − A,B B B − by J(M)[r] A,B . Since M is torsion-free, we can define for 0 ≤ r < p [0] φr = p −r φ : J(M)[r] A,B −→ J(M)A,B ,

whose map on the degree-q component is the unique map that coincides with φi ⊗ [r−q−i] ˆ q for any i. φr−q−i ⊗ φq on M i ⊗ J ˆ ⊗" B D

376

TAKAO YAMAZAKI

Definition 5.2. Let 0 ≤ r < p. We define the complex ᏿(M, r)A,B of abelian groups to be the mapping fiber of the map [0] 1 − φr : J(M)[r] A,B −→ J(M)A,B .

We denote by H q (A, ᏿(M, r)) the qth cohomology group of ᏿(M, r)A,B . We also define the complex ᏿(M, r)(A,k),B as the mapping fiber of ᏿(M, r)A,B −→ ᏿(M, r)k,B ,

where the latter complex is defined by using the composition B → A → k. We denote by H q ((A, k), ᏿(M, r)) the qth cohomology group of ᏿(M, r)(A,k),B . Again, those cohomology groups are independent of the choice of B, which can be seen by the same method as in Section 2 and [1, 7.24]. First we study the nonrelative syntomic cohomology groups. Theorem 5.3. Let X be a smooth projective scheme over W of relative dimension d. Let A be an object of Ꮽsyn . Assume d < p and the de Rham cohomology group q HdR (X/W ) is torsion-free for any q. Then, for 0 ≤ r < p, there exists a spectral sequence j i,j E2 = H i A, ᏿ HdR (X/W ), r ⇒ H i+j XA , ᏿(r) . Let π : (XA )Zar → (Spec A)Zar be the map of sites induced by the structure morphism. Let Ꮿ = F 0 Ꮿ ⊃ F 1 Ꮿ ⊃ F 2 Ꮿ . . . be a finite filtration (by subcomplexes) of a complex Ꮿ of Zariski sheaves on XA , whose components in negative degrees are ·,j trivial. The following fact, due to Deligne, is explained in [13, 3.3]: Let E1 (Ꮿ) be the complex of E1 -terms of the spectral sequence of a finitely filtered complex Ꮿ: i,j

E1 (Ꮿ) = Ri+j π∗ (gr i Ꮿ) ⇒ Ri+j π∗ (Ꮿ), where gr i Ꮿ = F i Ꮿ/F i+1 Ꮿ. We remark that the boundary map of this complex is induced by the connecting homomorphism of 0 −→ gr i+1 Ꮿ −→ F i Ꮿ/F i+2 Ꮿ −→ gr i Ꮿ −→ 0.

(10)

Then there exists (another) spectral sequence ·,j i,j E2 = Ri E1 (Ꮿ) ⇒ Ri+j ◦ R0 π∗ Ꮿ, where = (Spec A, −). Let Z, T , B, D, and φ be as in Section 4, (5) and (6). We define a filtration ·,j j i F ᏿(r)XA ,ZB and show (E1 (᏿(r)XA ,ZB )) = ᏿(HdR (X/W ), r)A,B . We also show

377

FORMAL CHOW GROUPS ·,j

that each component of E1 (᏿(r)XA ,ZB ) is a quasi-coherent sheaf. Then, the above fact implies the theorem. We start with an exact sequence ˆ 1Z −→ "1 ⊗W Bˆ −→ 0. ˆ 1B −→ " 0 −→ ᏻZˆ ⊗W " B Zˆ [r−q] ˆ q as ⊗ᏻZˆ " Using this, we define a filtration F i (i ≥ 0) on JT ⊗D ZB ˆ

B

[r−q] ˆq ⊗ᏻZˆ " F i JT ⊗D ZB ˆ B [r−q] ˆ q−i ⊗ᏻ ˆ iB −→ J [r−q] ⊗ᏻ " ˆq . = Im JT ⊗D ⊗ " " ᏻ ⊗ ᏻ W ˆ ZB ZB Z ˆ ˆ Zˆ Zˆ Zˆ T ⊗D B

B

B

J[r] XA ,ZB

is defined to be the complex The filtration F i (i ≥ 0) on ∇ ∇ ∇ [r−1] [r−2] i 1 i 2 ˆ ˆ " " − − → F ⊗ − − → F ⊗ −→ · · · . J J F i JT[r]⊗D ᏻZˆ ᏻZˆ ZB ZB − ˆ ˆ ˆ T ⊗D T ⊗D B

Some linear algebra shows that deg(i−1)

B

gr i J[r] XA ,ZB

is isomorphic to the complex

deg(i)

· · · −→ 0 −→ JT[r−i] ⊗ᏻZˆ ˆ ⊗D

B

deg(i+1)

[r−i−1] ⊗ᏻZˆ ˆ T ⊗D B

ˆ i −→ J ᏻZˆ ⊗W " B

ˆ 1 ⊗W " ˆ i −→ · · · . " Z B

i [0] Let 0 ≤ r < p. We remark that φr (F i J[r] XA ,ZB ) ⊂ F JXA ,ZB . We now define for i ≥ 0 a filtration F i ᏿(r)XA ,ZB to be the mapping fiber of i−1 [0] F i J[r] JXA ,ZB . XA ,ZB −−−→ F 1−φr

[0] (We use the convention F −1 J[0] XA ,ZB = JXA ,ZB .) The above remark shows that 0 [r] [0] gr i ᏿(r)XA ,ZB ∼ = the mapping fiber of gr i JXA ,ZB −→ gr i−1 JXA ,ZB [r] [0] ∼ = gr i JXA ,ZB ⊕ gr i−1 JXA ,ZB [−1]. ·,j

Hence, the degree-i component of the complex E1 (᏿(r)XA ,ZB ) is [r] [0] Ri+j π∗ gr i ᏿(r)XA ,ZB ∼ = Ri+j π∗ gr i JXA ,ZB ⊕ Ri+j −1 π∗ gr i−1 JXA ,ZB . The following lemma shows that this is a quasi-coherent sheaf and that the global j section of this sheaf is equal to the degree-i component of ᏿(HdR (X/W ), r)A,B . Lemma 5.4. We have an isomorphism of sheaves on Spec(A), j [r−i] ∼ î Ri+j π∗ gr i J[r] XA ,ZB = J HdR (X/W ) A,B ⊗Bˆ "B .

378

TAKAO YAMAZAKI

Proof. Since this lemma is independent of φ, we can assume Z = T = X. We remark that ·−i ∼ i+j π∗ " ˆ ⊗W J [r−·] ⊗ ˆ " î Ri+j π∗ gr i J[r] X XA ,XB = R B B ˆ D

∼ ˆ ·X ⊗W J [r−i−·] ⊗ ˆ " î = Rj π∗ " B B . Dˆ

First we deal with the case r = 0. In this case, we have · · ˆ ˆ X ⊗W Dˆ ⊗ ˆ " ˆ î ∼ j î R j π∗ " B B = R π∗ "X ⊗W D ⊗Bˆ "B , ˆ ˆ î ˆ · ⊗W Dˆ ⊗ ˆ " î because the boundary map in " X B B is a D-linear map and D ⊗ "B is a ˆ free D-module. This proves the lemma in the case r = 0. Furthermore, the spectral sequence · ˆ iB ⇒ Ra+b π∗ " ˆ X ⊗W Dˆ ⊗ ˆ " ˆ aX ⊗ Dˆ ⊗ " î E1a,b = Rb π∗ " (11) B B degenerates at E1 , by [11, II.2.5]. Consider the natural morphism of spectral sequences from · ˆ iB ⇒ Ra+b π∗ " ˆ X ⊗W J [r−i−·] ⊗ ˆ " ˆ aX ⊗ J [r−i−a] ⊗ " î E1a,b = Rb π∗ " (12) B B ˆ ˆ D

D

to (11). We see that the maps on E1 -terms are injective. Using the fact that the spectral sequence (11) degenerates at E1 , we see that (12) also degenerates at E1 , and that the map · · ˆ X ⊗W J [r−i−·] ⊗ ˆ " ˆ X ⊗W Dˆ ⊗ ˆ " ˆ iB −→ Ra+b π∗ " ˆ iB Ra+b π∗ " Dˆ

B

B

is injective. The lemma follows. ·,j

It remains to show that the boundary maps of (E1 (᏿(r)XA ,ZB )) coincide with j those of ᏿(HdR (X/W ), r)A,B . But this follows from the theory of Gauss-Mannin connection for crystals, except a part coming from 1 − φr , and this part can be seen directly from the definition of the filtration and (10). Next, we deduce a Leray spectral sequence for the relative syntomic cohomology groups. Corollary 5.5. Assume X, d, A, and r satisfy the hypothesis of Theorem 5.3. Let Y = Xk . Then, for 0 ≤ r < p, there exists a spectral sequence j i,j E2 = H i (A, k), ᏿ HdR (X/W ), r ⇒ H i+j (XA , Y ), ᏿(r) . Proof. We use the same method as in the proof of the theorem. Let F i ᏿(r)XA ,ZB and F i ᏿(r)Y,ZB be as in the above proof. We define a subcomplex F i ᏿(r)(XA ,Y ),ZB of ᏿(r)(XA ,Y ),ZB as the mapping fiber of the natural map F i ᏿(r)XA ,ZB −→ F i−1 ᏿(r)Y,ZB .

FORMAL CHOW GROUPS

379

(We use the convention F −1 ᏿(r)Y,ZB = ᏿(r)Y,ZB .) Then gr i ᏿(r)(XA ,Y ),ZB ∼ = gr i ᏿(r)XA ,ZB ⊕ gr i−1 ᏿(r)Y,ZB , and we see that j ·,j E1 ᏿(r)(XA ,Y ),ZB = ᏿ HdR (X/W ), r (A,k),B . The corollary follows. 6. Syntomic cohomology and p-divisible groups. In this section, we study the syntomic cohomology groups H q (A, ᏿(M, r)) and the relative syntomic cohomology groups H q ((A, k), ᏿(M, r)). We begin with a definition taken from [11, II.2.8]. Definition 6.1. Let M be an object of MFW,tf , and let I be a closed interval in R of the form [i, j ] or [i, ∞)(i, j ∈ Z). We say that M is of level I if M i = M for i ≤ inf(I ) and if M i+1 = 0 for i ≥ sup(I ). Proposition 6.2. Let M be a torsion-free object of MFW,tf of level [0, ∞) and let 0 ≤ r < p. (i) If q ≥ r + 2, then we have for any object A of Ꮽsyn H q A, ᏿(M, r) = 0. (ii) If q ≥ r + 1, then we have for any object A of Ꮽ0 H q (A, k), ᏿(M, r) = 0. (iii) If q ≥ 2, we have

H q k, ᏿(M, r) = 0.

In particular, if 1 ≤ r r, the degree-q component [0] of the map 1 − φr : J(M)[r] A,B → J(M)A,B is an isomorphism. Hence (i) follows. To prove (ii), let A, A , B, and φ be as in Section 3 (3) or Remark 3.5. By the distinguished triangle ᏿(M, r)(A,k),B −→ ᏿(M, r)A,B −→ ᏿(M, r)k,B −→ (+1),

(13)

it is enough to show that

H q A, ᏿(M, r) −→ H q k, ᏿(M, r)

is an isomorphism if q = r + 1 and is a surjection if q = r. Define a complex Ꮿ by exactness of 0 −→ ᏿(M, r)A,B −→ ᏿(M, r)A ,B −→ Ꮿ −→ 0. [q]

If we put M q in place of JT in the proof of Proposition 4.3, we can prove that Ꮿ is acyclic outside [0, r − 1]. Hence (ii) follows. We also remark that the degree-(r − 1)

380

TAKAO YAMAZAKI

component of Ꮿ is isomorphic to ˆ r−1 . M/M 1 ⊗ ker(A −→ A ) ⊗ " B

(14)

For (iii), we use the obvious surjection W → k with Frobenius φ = σ . Then

ˆ 1 is trivial. ᏿(r)k,W is acyclic outside [0, 1], since " W

In the rest of this section, we study a relation between the rational points of a p-divisible group and the syntomic cohomology groups. Assume p = 2. Then we have the contravariant equivalence functor ILM of [9, 9.11], between the category of p-divisible groups over W and the full subcategory of MFW,tf consisting of objects M, which is torsion-free and is of level [0, 1]. (Strictly speaking, the subject of [9] is not p-divisible groups but finite flat group schemes. However, the proof in [9] also works for p-divisible groups, by using the corresponding result of [7, Chapter IV, 1.6] instead of [6].) But since we need a covariant theory, we define, for a p-divisible group G, the object MG of MFW,tf corresponding to G as MG = Hom ILM(G)[1], W . (15) Here we use the following notation: Hom denotes the Hom-object in MFW,tf (cf. [19, 1.7]). W is the object of MFW,tf defined as W q = W (resp., 0) if q ≤ 0 (resp., q > 0), and φi = p −i σ (i ≤ 0). For an object M of MFW,tf , M = M[q] is the translation of M, that is, an object of MFW,tf defined as M i = M i+q and φ i = φi+q . We use the 1 ∼ Lie G. fact that there is a canonical isomorphism MG /MG = Instead of the symbol map in Proposition 4.3, we have the following theorem. Theorem 6.3. Assume p = 2. Let G be a connected p-divisible group over W and MG the object of MFW,tf corresponding to G. Then we have a canonical homomorphism for any object A of Ꮽsyn G(A) −→ H 1 A, ᏿(MG , 1) . Furthermore, if A belongs to Ꮽ0 , then this is an isomorphism and H q (A, ᏿(MG , 1)) = 0 unless q = 1. Consequently, for any object A of Ꮽ0 we have G(A) if q = 1, H q (A, k), ᏿(MG , 1) ∼ = H q A, ᏿(MG , 1) ∼ = 0 if q = 1. Note G(k) = 0 since G is connected. The definition of this map is as follows. Let x ∈ G(A). For each n > 0, define Fn as the pullback of the extension 0

/

pn G

/G O

0

/

pn G

/ Fn

pn

/G O

/0

/Z

/ 0,

where the right vertical arrow is defined as 1 → x. (This pullback is taken in the

381

FORMAL CHOW GROUPS

category of abelian sheaves on the flat site of Spec(A).) Let En = Fn /p n Fn . Then we have a short exact sequence 0 −→ pn G −→ En −→ Z/pn Z −→ 0, so that En is a finite flat group scheme over A. Furthermore, E = (En )n>0 forms a p-divisible group over A, and there is an exact sequence 0 −→ G −→ E −→ Qp /Zp −→ 0. Take B, D, and φ as in Section 4 (6). By taking the section of the associated crystalline Dieudonné module (and its canonical filtration) of the above sequence at a PD-thicking (Spec(A), Spec(Dn )), we have a commutative diagram with exact rows 0

/

0

/ M 1 ⊗W Dn + M G ⊗W J D n G

M G ⊗W Dn

/ Nn

/ Dn

/ 0

/ N1 n

/ Dn

/ 0,

where Nn ⊃ Nn1 is the one associated to E. By passage to the limit, we have 0

/

0

/ M 1 ⊗W Dˆ + MG ⊗W J ˆ G D

MG ⊗W Dˆ

/ N

/ Dˆ

/ 0

/ N1

/ Dˆ

/ 0.

Take a ∈ N 1 such that its image in Dˆ is 1. Since (1 − φ1 )(1) = 0 and ∇(1) = 0, we have 1 ˆ 1B . ⊗ Dˆ + MG ⊗ JDˆ n ⊕ Dˆ ⊗ " (1 − φ1 )(a), ∇(a) ∈ MG The image of x is defined to be the class of this element in H 1 (A, ᏿(MG , 1)). The proof of Theorem 6.3 is similar to that of Proposition 4.3. Using the notation 1 ∼ Lie G, we have as in the proof of Proposition 6.2, by (14) and the fact MG /MG = H 0 (Ꮿ) ∼ = Lie G ⊗W ker(A −→ A ). But the former group is isomorphic to ker(G(A) → G(A )). By induction, Lemma 6.4 completes the proof. The last assertion is deduced from (13). Lemma 6.4. We have H q (k, ᏿(MG , 1)) = 0 for any q. Proof. To calculate H q (k, ᏿(MG , 1)), we use the obvious surjection W → k with φ = σ . Then the complex ᏿(MG , 1)k,W is 1−φ1

1 + pMG −−−→ MG . MG

382

TAKAO YAMAZAKI

1 + pM → M is an isomorphism. There is a σ −1 We must show that 1 − φ1 : MG G G linear map V : MG → MG such that φ ◦ V = V ◦ φ = p (cf. [9, 9.8]). This map 1 ) (cf. Definition is characterized by V (φ0 (x) + φ1 (y)) = px + y (x ∈ MG , y ∈ MG 1 5.1(v)). This shows V (MG ) = MG + pMG . We also get 1 1 + pMG −→ MG + pMG V ◦ (1 − φ1 ) = V − 1 : MG

(1 − φ1 ) ◦ V = V − 1 : MG −→ MG . Since G is connected, the map V is topologically nilpotent. Hence both of V − 1 are isomorphisms. Thus the lemma follows. Next, we consider an étale p-divisible group. Proposition 6.5. Assume p = 2. Let G be an étale p-divisible group over W , and MG the object of MFW,tf corresponding to G. Then we have for any object A of Ꮽ0  1−φ1 1 −−   −→ MG if q = 0, ker MG = MG   q 1−φ 1 H A, ᏿(MG , 1) = coker MG = M 1 −−−→ MG if q = 1, G     0 if q ≥ 2. In particular, for any q ≥ 0 we have H q (A, k), ᏿(MG , 1) = 0. Proof. Since G is an étale p-divisible group, MG is of level [1, 1]. Then the following two lemmas complete the proof. Lemma 6.6. Let 0 ≤ s ≤ r < p. Let M be a torsion-free object of MFW,tf of level [s, ∞) and A an object of Ꮽsyn . Take B as in Section 4 (6). Then we have ᏿(M, r)A,B = ᏿ M[s], r − s A,B , ᏿(M, r)(A,k),B = ᏿ M[s], r − s (A,k),B . The proof is clear from the definition. Lemma 6.7. Let M be a torsion-free object of MFW,tf object A in Ꮽ0 , we have  1−φ 0   ker M −−−−→ M   1−φ 0 H q A, ᏿(M, 0) = coker M −− −−→ M     0 In particular, for any q ≥ 0 we have

of level [0, ∞). For any

if q = 0, if q = 1, if q ≥ 2.

FORMAL CHOW GROUPS

383

H q (A, k), ᏿(M, 0) = 0. Proof. As in the proof of Proposition 6.2, we see that H q (A, ᏿(M, 0)) = To calculate the latter group, we can use the natural surjection W → k with φ = σ . Then the complex ᏿(M, 0)k,W is H q (k, ᏿(M, 0)).

1−φ 0

M −−−−→ M. Hence the equality follows. The last assertion follows from (13). Corollary 6.8. Assume p = 2. Let G be a p-divisible group over W , and MG the object of MFW,tf corresponding to G. Then we have for any object A of Ꮽ0 H

q

Gc (A) if q = 1, (A, k), ᏿(MG , 1) = 0 if q = 1,

where Gc is the connected component of G. Proof. Let MGc be the object of MFW,tf corresponding to Gc . Then MG /MGc corresponds to the étale part of G. Taking B as in Section 4 (6), we have an exact sequence of complexes 0 −→ ᏿(MGc , 1)(A,k),B −→ ᏿(MG , 1)(A,k),B −→ ᏿(MG /MGc , 1)(A,k),B −→ 0. By the deduced long exact sequence, the corollary follows from Theorem 6.3 and Proposition 6.5. Remark 6.9. The sequence 0 −→ H 1 (A, k), ᏿(MG , 1) −→ H 1 A, ᏿(MG , 1) −→ H 1 k, ᏿(MG , 1) −→ 0 (16) is not exact in general. An example for a failure of the injectivity of the first map is given by A = Wn with n ≥ 2 and by a p-divisible group G that is a nontrivial extension of Qp /Zp by Qp /Zp (1). This is a big difference from the equal characteristic situation, which was considered in [3] and [18]. In fact, if A is an object of Ꮽsyn such that the structure morphism W → A factors through k, it is clear that (16) is a split exact sequence. We also remark that, if G is a trivial extension of an étale p-divisible group by a connected p-divisible group, for any object A of Ꮽ0 , (16) is also a split exact sequence by Theorem 6.3 and Proposition 6.5. (Remember that any p-divisible group over k is a trivial extension of an étale p-divisible group by a connected p-divisible group.) The following definition is taken from [11, II.2.12].

384

TAKAO YAMAZAKI

Definition 6.10. Let M be an object of MFW,tf . We say that M is of Hodge-Witt type if there is a finite sequence of subobjects (Mi )0≤i≤N in MFW,tf such that 0 = M 0 ⊂ M 1 ⊂ · · · ⊂ MN = M and such that, for each i, Mi+1 /Mi is of level [mi , mi + 1] for some integer mi . The following remark is proven in [11, II.2.15]. Remark 6.11. For X a smooth projective scheme over W of relative dimension d < p, the following conditions are equivalent. (i) Y is of Hodge-Witt type, that is, H q (Y, W "sY ) is of finite type over W for all q and s. q (ii) HdR (X/W ) is of Hodge-Witt type for all q. Proposition 6.12. Assume p = 2. Let M be a torsion-free object of MFW,tf of level [0, ∞) and of Hodge-Witt type. By [11, II.2.14], we have the “level-[0, 1] part” of M, that is, a subobject N of M such that N is of level [0, 1] and M/N is of level [1, ∞). Let Gc be the connected component of the p-divisible group corresponding to N. (Though the choice of N may not be unique for the level-[1, 1] part, Gc itself is independent of this choice. We have Lie Gc ∼ = N/N 1 ∼ = M/M 1 .) Then there exists a canonical isomorphism Gc (A) −→ H 1 (A, k), ᏿(M, 1) for any object A of Ꮽ0 . Furthermore, H q ((A, k), ᏿(M, 1)) = 0 unless q = 1. Proof. For q ≥ 0, we have H q ((A, k), ᏿(M/N, 1)) ∼ = H q ((A, k), ᏿(M/N[1], 0)) = 0 by Lemmas 6.6 and 6.7. From an exact sequence 0 −→ ᏿(N, 1)(A,k),B −→ ᏿(M, 1)(A,k),B −→ ᏿(M/N, 1)(A,k),B −→ 0, we have H q (A, k), ᏿(M, 1) ∼ = H q (A, k), ᏿(N, 1) . Then the proposition follows from Corollary 6.8. 7. K-cohomological functor and p-divisible groups. In this section, we prove Theorem A. Let the notation and assumptions be as in Theorem A. By Corollary 5.5, we have a spectral sequence j i,j E2 = H i (A, k), ᏿ HdR (X/W ), r ⇒ H i+j (XA , Y ), ᏿(r) . i,j

We have E2 = 0 if i > r by Proposition 6.2. Since j ˆ " ˆ " ˆ ·X ∼ ˆ ·≥j −d , HdR (X/W ) = Hj X, = Hj X, X

FORMAL CHOW GROUPS

385

j

we see that HdR (W/X) is of the level [j − d, d]. Hence we have by Lemma 6.6 j i,j E2 = H i (A, k), ᏿ HdR (X/W ), r j = H i (A, k), ᏿ HdR (X/W )[j − d], r − j + d . i,j

Thus Proposition 6.2 shows E2 = 0 if i > r − j + d. Moreover, E20,d+r = 0 by 1,d+r−1 and that there exists a Lemma 6.7. These facts imply that E21,d+r−1 = E∞ canonical surjection d+r−1 H d+r (XA , Y ), ᏿(r) −→ H 1 (A, k), ᏿ HdR (X/W ), r . d+r−1 (X/W ) to be of Hodge-Witt type, Proposition 6.12 and Since we assume HdR Lemma 6.6 provide canonical isomorphisms d+r−1 Gc (A) ∼ (X/W )[r − 1], 1 = H 1 (A, k), ᏿ HdR d+r−1 ∼ (X/W ), r . = H 1 (A, k), ᏿ HdR

These two maps and Corollary 4.6 complete the proof. Proposition 7.1. Let the notation and assumptions be as in Theorem A. Assume further d = r = 2. If we assume Conjecture 4.2, we have an analogue of Bloch’s conjecture about the Albanese kernel; that is, admitting Conjecture 4.2, the following conditions are equivalent: (i) H 2 (X, ᏻX ) = 0. M )(A) → G(A)) = 0 for any object A of Ꮽ . (ii) ker(Hˆ 2 (᏷2,X 0 Proof. Let A be an object A of Ꮽ0 . By Remark 4.7, we have M H 2 ᏷2,X (A) ∼ = H 4 (XA , Y ), ᏿(2) . 3 (X/W ), 2)) Using the notation in the above proof, we have E20,3 = H 0 ((A, k), ᏿(HdR i,j = 0 by Proposition 6.12, and E2 = 0 if i ≥ 3 by Proposition 6.2. Hence we have 2,2 E22,2 = E∞ . From Remark 4.7, we have 2 M ker H ᏷2,X (A) −→ G(A) 3 ∼ (X/W ), 2 = ker H 4 (XA , Y ), ᏿(2) −→ H 1 (A, k), ᏿ HdR 2 ∼ (X/W ), 2 . = H 2 (A, k), ᏿ HdR 2 (X/W ) is of level [1, 2], and we have, by Lemma 6.6 and Assume (i). Then HdR Proposition 6.2, 2 2 H 2 (A, k), ᏿ HdR (X/W ), 2 ∼ (X/W )[1], 1 = 0, = H 2 (A, k), ᏿ HdR

that is, (ii) holds. Assume H 2 (X, ᏻX ) = 0. Then the following lemma completes the proof.

386

TAKAO YAMAZAKI

Lemma 7.2. Let M be a torsion-free object of MFW,tf of level [0, ∞). Assume M 0 = M 1 . We set A = Wn [T ]/(T e ). Then for sufficiently large e and n, we have H 2 (A, k), ᏿(M, 2) = 0. Proof. Set r = rank W (M), r1 = rank W (M 1 ). Let n ≥ 3 and e > ps with s > r1 /(r − r1 ). Since H 2 (k, ᏿(M, 2)) = 0 by Proposition 6.2, it is enough to show that H 2 (A, ᏿(M, 2)) = 0. Let B = W [T ] endowed with the Frobenius φ : T → T p . Let D = DB ((p n , T e )) be the PD-envelope algebra and Dˆ its p-adic completion with the induced PD-structure on JDˆ . Define a complex ᏿1 (M, 2)A,B by the exactness of p

0 −→ ᏿(M, 2)A,B −→ ᏿(M, 2)A,B −→ ᏿1 (M, 2)A,B −→ 0. Since H 3 (A, ᏿(M, 2)) = 0 by Proposition 6.2, it is enough to show that H 2 (A, ˆ 2 = 0, we have ᏿1 (M, 2)) = 0. Noting " B H 2 A,᏿1 (M, 2) (1−φ2 ,∇) ˆ −− −−−−→ M ⊗ Dˆ ⊗ "1B1 . = coker M 1 ⊗ Dˆ + M ⊗ JDˆ ⊗ "1B1 ⊕ (M ⊗ D) i Let F be the image of ⊕si=1 M ⊗ W T p −1 dT under the natural map M ⊗ Dˆ ⊗ "1B → ˆ ∩ F = 0 and (1 − φ2 )(M ⊗ M ⊗ Dˆ ⊗ "1B1 . Then dimk F = rs. We have ∇(M ⊗ D) 1 1 JDˆ ⊗ "B1 ) ∩ F = 0. We also have dimk ((1 − φ2 )(M ⊗ Dˆ ⊗ "1B1 ) ∩ F ) ≤ r1 (s + 1). Since rs > r1 (s + 1) by definition, this completes the proof.

Remark 7.3. Let the notation and assumption be as in Theorem A, but do not d+r−1 assume HdR (X/W ) to be of Hodge-Witt type. For r = d, we have a composition map 2d M −→ H 2d XA , ᏿(d) −→ H 0 A, ᏿ HdR (X/W ), d ∼ H d XA , ᏷d,X = Zp , A 2d (X/W ) = W [−d] (cf. Lemmas 6.6 and 6.7). This map can be viewed because of HdR as the formal completion of the degree map

CHd (X ⊗W K) −→ Z, where K is the field of fractions of W . Appendix In this appendix, we show that the map in Theorem B is an isomorphism modulo essentially zero functors. We need the following definition, which is taken from Stienstra [18, p. 2].

FORMAL CHOW GROUPS

387

Definition A.1. A functor F from Ꮽsyn to the category of abelian groups is called essentially zero if, for any object A of Ꮽsyn , there exists a surjection A → A in Ꮽsyn such that the induced map F (A ) → F (A) is zero. Theorem A.2. Let the notation and assumptions be as in Theorem B. Then the functors M A −→ ker Hˆ d ᏷r,X (A) −→ H d+r (XA , Y ), ᏿(r) , M A −→ coker Hˆ d ᏷r,X (A) −→ H d+r (XA , Y ), ᏿(r) are essentially zero. Remember that a functor F from Ꮽsyn to the category of abelian groups is called formally smooth if, for any surjection A → A in Ꮽsyn , the induced map F (A ) → F (A) is surjective. This theorem suggests that, roughly speaking, at least the “formally M ) and H d+r ((X , Y ), ᏿(r)) are isomorphic. So, the smooth parts” of functors Hˆ d (᏷r,X . author wishes that this theorem play a role when one considers a generalization of the Dieudonné theory, which might be an analogue of Stienstra’s theory [18]. Proof. If A → A¯ is a surjection in Ꮽsyn , and if the theorem is true for A, then it ¯ It therefore suffices to prove the theorem for is also true for A.

A = Wn [T1 , . . . , Ts ] T1m1 , . . . , Tsms with s ≥ 0, n, m1 , . . . , ms ≥ 1. Thus, the theorem about the cokernel is already proven in Theorem B. In the rest of this appendix, we fix such an A and show M ker Hˆ d ᏷r,X (A) −→ H d+r (XA , Y ), ᏿(r) = 0. By the proof of Corollary 4.6, it is enough to show that the symbol map M ᏷r,X −→ Ᏼr ᏿(r)(XA ,Y ) A ,Y is an isomorphism (cf. Remark 4.7). Let R be a p-adically complete local ring that is essentially smooth over W . We use the notation RA = R ⊗W A, Rk = R ⊗W k, and so forth. We can define the syntomic cohomology groups H r ((RA , Rk ), ᏿(r)) and the symbol map, as in Section 2. What we need is to prove the following. Proposition A.3. The symbol map induces an isomorphism ker KrM (RA ) −→ KrM (Rk ) −→ H r (RA , Rk ), ᏿(r) . Since this proposition is proved by following Kurihara’s proof in [14], [15], we only sketch it. We can define the groups H r ((RA , RWn ), ᏿(r)), by replacing k by

388

TAKAO YAMAZAKI

Wn in the definition of the relative syntomic cohomology groups. Noting that the surjection A → Wn (defined by Ti → 0 for any i) has a section Wn → A (induced by the natural map W → A), we have a commutative diagram with exact rows 0

/ K M RA , RW n r

0

r

κ1

/H

RA , RWn , ᏿(r)

/ K M RW , R k n r

/ K M (RA , Rk ) r / H (RA , Rk ), ᏿(r)

r

/0

κ2

/H

r

RWn , Rk , ᏿(r)

/ 0.

(Here we use an ad hoc notation KrM (RA , Rk ) = ker(KrM (RA ) → KrM (Rk )), etc., since it corresponds to our notation of the relative syntomic cohomology groups. This is not the standard notation.) Now the proposition is reduced to showing that κ1 and κ2 are isomorphisms. First we consider κ2 . Using the natural surjection R → Rn , we can see that d ˆ r−1 /p n " ˆ r−2 −→ ˆ r−1 . H r RWn , Rk , ᏿(r) ∼ p" = coker p 2 " R R R Kurihara constructed the exponential homomorphism (cf. [14, 0.1, 3.3], [15, 2.10])

ˆ r−2 −→ KrM (R, Rk )ˆ. ˆ r−1 d p 2 " p" R R (Here “ ˆ ” means the completion with respect to some filtration; cf. loc. cit.) By compositing it with the natural map KrM (R, Rk )ˆ→ KrM (RWn , Rk ), we get the inverse map of κ2 . Next, we consider κ1 . Choose a p-base of R/pR, and take their lift I . One can find a Frobenius φ on R such that φ(X) = X p for any X ∈ I . Let B = R[T1 , . . . , Ts ], and p extend φ to B by φ(Ti ) = Ti for any i. Let (D, J ) be the PD-envelope of B with respect to the kernel of the natural surjection B → RA . Define ˆq , ˆ "qB = ker D ⊗ ˆ "qB −→ " UD ⊗ R ˆq . ˆ "qB = ker J ⊗ ˆ "qB −→ " UJ ⊗ R Then we can see H r RA , RWn , ᏿(r) (1−φr ,d) ∼ ˆ "r−1 ˆ "r−1 ˆ "r−2 −−−−−−→ U D ⊗ ⊕ UD ⊗ . = coker U J ⊗ B B B Following Kurihara’s proof (cf. [14], [15]), we have a homomorphism M ˆ "r−1 E : UD ⊗ B −→ Kr RA , RWn

389

FORMAL CHOW GROUPS

such that

dX1 dXr−1 E a ··· X1 Xr−1

= s(a), X1 , . . . , Xr−1 ,

a ∈ (T1 , . . . , Ts )B,

Xi ∈ I ∪ {T1 , . . . , Ts } ˆ "r−1 = 0, E UJ ⊗ B

for any i,

j where s = exp( ∞ j =0 φ1 ) (cf. [15, 1.1]). By [15, 1.3], we see that E factors through r H ((RA , RWn ), ᏿(r)) and gives the inverse map of κ1 . References [1] [2]

[3] [4] [5] [6] [7] [8]

[9] [10]

[11]

[12] [13] [14] [15] [16] [17]

P. Berthelot and A. Ogus, Notes on Crystalline Cohomology, Princeton Univ. Press, Princeton, 1978. S. Bloch, “Algebraic K-theory and algebraic geometry” in Algebraic K-Theory, I: Higher KTheories (Seattle, Wash., 1972), Lecture Notes in Math. 341, Springer-Verlag, Berlin, 1973, 259–265. , K2 of Artinian Q-algebras, with application to algebraic cycles, Comm. Algebra 3 (1975), 405–428. , Lectures on Algebraic Cycles, Duke Univ. Math. Ser. 4, Duke Univ. Mathematics Department, Durham, N.C., 1980. , “p-adic étale cohomology” in Arithmetic and Geometry, Vol. 1, Progr. Math. 35, Birkhäuser, Boston, 1983, 13–26. J.-M. Fontaine, Groupes finis commutatifs sur les vecteurs de Witt, C. R. Acad. Sci. Paris Sér. A-B 280 (1975), A1423–A1425. , Groupes p-divisibles sur les corps locaux, Astérisque 47–48, Soc. Math. France, Paris, 1977. , “Cohomologie de de Rham, cohomologie cristalline et représentations p-adiques” in Algebraic Geometry (Tokyo/Kyoto, 1982), Lecture Notes in Math. 1016, SpringerVerlag, Berlin, 1983, 86–108. J.-M. Fontaine and G. Laffaille, Construction de représentations p-adiques, Ann. Sci. École Norm. Sup. (4) 15 (1982), 547–608. K. Kato, “Milnor K-theory and the Chow group of zero cycles” in Applications of Algebraic K-Theory to Algebraic Geometry and Number Theory, Contemp. Math. 55, Amer. Math. Soc., Providence, 1986, 241–253. , “On p-adic vanishing cycles (application of ideas of Fontaine-Messing)” in Algebraic Geometry (Sendai, 1985), Adv. Stud. Pure Math. 10, North-Holland, Amsterdam, 1987, 207–251. , The explicit reciprocity law and the cohomology of Fontaine-Messing, Bull. Soc. Math. France 119 (1991), 397–441. N. Katz, Nilpotent connections and the monodromy theorem: Applications of a result of Turrittin, Inst. Hautes Études Sci. Publ. Math. 39 (1970), 175–232. M. Kurihara, The exponential homomorphisms for the Milnor K-groups and an explicit reciprocity law, J. Reine Angew. Math. 498 (1998), 201–221. , The Milnor K-groups of a local ring over a ring of p-adic integers, preprint. C. Soulé, Opérations en K-théorie algébrique, Canad. J. Math. 37 (1985), 488–550. J. Stienstra, “The formal completion of the second Chow group, a K-theoretic approach” in Journées de géométrie algébrique de Rennes (Rennes, 1978), Vol. 2, Astérisque 64, Soc. Math. France, Paris, 1979, 149–168.

390 [18] [19]

TAKAO YAMAZAKI , Cartier-Dieudonné theory for Chow groups, J. Reine Angew. Math. 355 (1985), 1–66; Correction, J. Reine Angew. Math. 362 (1985), 218–220. J.-P. Wintenberger, Un scindage de la filtration de Hodge pour certaines variétés algébriques sur les corps locaux, Ann. of Math. (2) 119 (1984), 511–548.

Institute of Mathematics, University of Tsukuba, Tendoudai 1-1-1, Tsukuba-shi, Ibaraki, 305-8571, Japan; [email protected]

Vol. 102, No. 3


© 2000

VARIATIONAL PROPERTIES OF A NONLINEAR ELLIPTIC EQUATION AND RIGIDITY M. L. BIALY and R. S. MACKAY

1. Introduction and main results. In this paper, we discuss variational properties of classical solutions of the nonlinear equation u = −Vu (u, x1 , . . . , xn ) , which is the Euler-Lagrange equation of the functional 1 I (u) = (∇u)2 − V (u, x1 , . . . , xn ) dx1 · · · dxn . 2

(1.1)

(1.2)

The main question addressed here is the following: Under what conditions on the potential V are all classical solutions of (1.1) globally minimising for the functional (1.2)? By a global minimiser we mean a smooth function on a domain of Rn minimising the integral (1.2) over all bounded subdomains with smooth boundaries with respect to smooth functions with the same boundary values. The motivation for this question comes from variational problems of classical mechanics. In geodesic problems it is well known that all geodesics are globally minimising on manifolds of negative sectional curvature. However, it was shown first by Hopf [12] and Green and Gulliver [9] that the situation is completely different for Riemannian (two) tori or for Riemannian planes that are flat outside a compact set. They proved that for these manifolds, there always exist geodesics with conjugate points, which are therefore nonminimal, unless the metric is flat. We refer the reader to [4] and [6] for higher-dimensional generalisations of Hopf and Green-Gulliver theorems and to [8], [7], [13], and [10] for very important previous developments. It was observed first in [2] that the Hopf phenomenon is not entirely Riemannian. In [3] it was shown that a similar type of rigidity holds for Newton’s equations with periodic or compactly supported potentials. For the equation (1.1) with periodic potentials, it was shown in [15] and [16] that far-reaching generalisations of Aubry-Mather and KAM (Kolmogorov-ArnoldMoser) theories apply. Using these theories, one can construct families of minimal solutions that form laminations or sometimes even foliations of the configuration torus. It is an interesting open question, however, if it happens that, for all slope Received 3 February 1999. 1991 Mathematics Subject Classification. Primary 35J60; Secondary 58F18, 53C99. Authors’ work supported by Engineering and Physical Sciences Research Council grant number GR/M11349. 391

392

BIALY AND MACKAY

vectors, these laminations are genuine foliations. This question is tightly related to the one we are addressing in this paper, since all the leaves of a foliation are globally minimal. We assume throughout this paper that the potential V is compactly supported. The main reason for this is to make the analogy with the aforementioned situations of ≤ 0 everywhere which Hopf rigidity and to exclude, in particular, the case with Vuu is analogous to nonpositive curvature. Nevertheless, several of our results apply in more generality, as we indicate. We show that in case of dimensions greater than 2, there are many potentials such that all solutions of (1.1) are minimising (see Theorem 1). In dimension 2, however, at least for radially symmetric potentials, there always exist nonminimal solutions of (1.1) unless V vanishes identically (see Theorem 2). We state Theorems 1 and 2 here. Theorem 1. For n ≥ 3, let V be a compactly supported potential on Rn+1 . As (u, x) ≤ U (x) for some function U such that either sume that Vuu (A) U (x) ≤ ((n − 2)/2)2 / x − x0 2 for some point x0 ∈ Rn , or (B) U n/2 ≤ (n(n − 2)/4)|Sn | where |Sn | is the volume of the unit n-sphere. Then any solution of (1.1) is globally minimising. Theorem 2. Let V (u, x1 , x2 ) be a radially symmetric compactly supported potential (n = 2). There always exist radial nonminimal solutions of (1.1) unless V vanishes identically. Our approach to the proof of Theorem 2 is based on the reduction to a Newton equation with the potential supported in the semistrip. It turns out that a technique analogous to Hopf’s can be applied for such a shape of support. In Section 3, we consider the case of radially symmetric potentials for n ≥ 3. We illustrate the minimality property of radial solutions of (1.1), organising them in foliations (see Theorem 3). Acknowledgements. This paper was written while the first author was visiting the University of Cambridge. We would like to thank the EPSRC for their support. 2. Proof of Theorem 1. Let u be a solution of (1.1) in a domain , and let υ be any function with the same boundary values. Set ξ = υ − u and write (∇ξ )2 I (υ) − I (u) = − V (u + ξ, x) − V (u, x) d n x. ∇u∇ξ + (2.1) 2 Integrating the first term by parts and using the equation (1.1), we have ∇u∇ξ dx = Vu (u, x)ξ d n x.

By the assumption on V , we have 1 V (u + ξ, x) − V (u, x) − Vu (u, x)ξ ≤ U (x)ξ 2 . 2

VARIATIONAL PROPERTIES AND RIGIDITY

393

Substituting this in (2.1), we obtain I (υ) − I (u) ≥

(∇ξ )2 1 − U (x)ξ 2 d n x. 2 2

(2.2)

We show that under hypothesis (A) or (B), the last integral is always positive unless ξ ≡ 0. Indeed, for case (A), introduce spherical coordinates centred at x0 , r = x − x0 . The required assertion follows from the next lemma. Lemma 1. For any function ξ(r) defined on [r1 , r2 ] (0 ≤ r1 < r2 ≤ ∞) with ξ(r2 ) = 0, it follows that

r2 r1

r

n−1

2 n−2 2 ξ2 ξ − dr ≥ 0 2 r2

with equality for ξ ≡ 0 only. Proof. Introduce the new function ϕ(r) = ξ(r) · r (n/2)−1 and substitute into the integral. We have

r2 r1

r

n−1

2 2 − n −n/2 (n − 2)2 −n 2 r r ϕ dr ϕ + ϕ − r 2 4 r2 r2 2 2 n−2 2 ϕ (r1 ) + r ϕ dr. r · ϕ + (2 − n)ϕϕ dr = = 2 r1 r1

1−(n/2)

Since n ≥ 3, we see that the last expression is always nonnegative and equals zero only for ξ ≡ 0. This completes the proof of Lemma 1 and of the theorem in case (A). Alternatively, under (B), Sobolev’s inequality (e.g., [14, §8.3]) implies that |∇ξ |2 d n x ≥ Sn ξ 22n/(n−2) , where

n(n − 2) n |S |. 4 Then Hölder’s inequality applied to the second term of the integrand (as in [14, §11.3]) yields U ξ 2 d n x ≤ ξ 2 U n/2 . n/(n−2) Sn =

Thus,

1 Sn − U n/2 ξ 22n/(n−2) . 2 This completes the proof in case (B). I (v) − I (u) ≥

394

BIALY AND MACKAY

Remark 1. Lemma 1 and its proof have a precursor in [5, Ch. 6, §5], and the whole of Theorem 1 fits in the general domain of “absence of bound states,” described, for example, in [17, §8.3]. Remark 2. In addition, our proof shows that any solution is a nondegenerate minimum for (1.2). In a different way, one can say this as follows: For any solution u on , the linearised equation ξ + Vuu (u, x)ξ = 0 has only the trivial solution satisfying the zero boundary conditions ξ |∂ = 0. 3. Radial potentials. Let us now consider the case of radial, compactly supported (u, r) ≤ 1/4r 2 . Radial solutions u(r) of (1.1) are potentials V (u, r) satisfying Vuu given by the following: u +

n−1 u + Vu (u, r) = 0. r

It follows from Theorem 1(A) that for n ≥ 3, all radial solutions are without conjugate points. Moreover, Lemma 1 gives that the points r1 and ∞ are not conjugate for any r1 > 0. This fact enables us to organise the solutions into foliations in the following way. Denote by NA the class of all those solutions that can be written as u(r, α) = (α/r n−2 ) + A for some α outside the support of V . (u, r) ≤ Theorem 3. For compactly supported radial potential V satisfying Vuu 2 1/4r , the set NA is totally ordered and the graphs of the solutions define a smooth foliation of Rn+1 − R(u) × {x = 0}.

Proof of Theorem 3. Given A, we have to show that the function u(r, α) is monotone in α. Indeed, the function ξ(r) = ∂u/∂α is a solution of the linearised equation. Note that by definition, ξ(∞) = 0. Then the nonconjugacy property implies that ξ > 0, and then it is easy to complete the proof. Remark 3. In some cases, there is another way of organising the set of all solutions into foliations. Assume we are given a radial potential with V (u, r) ≡ 0 for 0 < r ≤ r0 and r ≥ R0 satisfying the inequality of Theorem 3. Define the set MA of all those solutions that can be written as u(r, α) = (A/r n−2 )+α when r ≤ r0 . Then one shows that for any given A the set MA is ordered and the graphs of the solutions smoothly foliate the space Rn+1 −R(u)×{x = 0}. In addition, M0 foliates the whole of Rn+1 . In order to check the order property, one shows that the linearised equation has no focal points in the following sense: Any solution satisfying ξ˙ (r1 ) = ξ(r2 ) = 0 is trivial, provided r1 < r2 (this is not necessarily true for r1 > r2 ). Then one proceeds exactly as in the proof of Theorem 3. 4. Rigidity for the case n = 2. Let u(r) be a radial solution of the equation (1.1) for compactly supported rotationally symmetric V (u, r), V (u, r) ≡ 0 for |u| ≥ U or


395

r > R. With the substitution r = et , the equation for u as a function of t can be written in the Hamiltonian form u˙ = p, p˙ = −e2t Wu (u, t).

(4.1)

The Hamiltonian function of (4.1) is 1 H = p2 + e2t W (u, t) 2 with the function W (u, t) = V (u, et ). Note that the support of W is contained in the semistrip " = {|u| ≤ U, t ≤ T = ln R}. We prove the following rigidity result, which implies Theorem 2. Theorem 4. There always exist solutions with conjugate points for (4.1), unless W vanishes identically. The strategy of the proof follows the original one of Hopf, but it takes special care about the noncompactness of the situation, which requires careful estimates on ω given in Lemmas 2 and 3. In what follows, we assume that all solutions of (4.1) are without conjugate points. The first step in the proof is the following very well-known construction: If the solution u(t) has no conjugate points, then one can easily construct a nonvanishing solution ξ of the linearised Jacobi equation ξ + e2t Wuu u(t), t ξ = 0. Having such a ξ(t) for every u(t), one defines the function ω(p, u, t) by the formula ω(p, u, t) =

ξ˙ (t) ξ(t)

when p = u(t), ˙ u = u(t).

Then ω satisfies the Riccati equation along the flow of (4.1): ω˙ + ω2 + e2t Wuu u(t), t = 0.

(4.2)

Here · stands for the derivative along the flow. It should be mentioned that by the construction of ξ and ω, the function ω is a priori only measurable (and smooth along the flow). Introduce K by the following:

(u, t). K= sup Wuu (t,u)&"

We need the following two lemmas specifying the behaviour of the function ω at infinity.

396

BIALY AND MACKAY

Lemma 2. The following statements hold true: |ω(p, u, t)| ≤ KeT

for all (p, u, t), 1 for t > T . 0 ≤ ω(p, u, t) < t −T

(4.3) (4.4)

There exists a constant K˜ such that for all (p, u, t) with t < T , ˜ t/4 < ω(p, u, t) < Ket . −Ke

(4.5)

Lemma 3. The function ω satisfies the following inequalities: (I) For t ≤ T , then 0 ≤ ω < Ket−((u−U )/p) ;

if u > U, p > 0, if u < −U, p < 0,

then 0 ≤ ω < Ke

if u > U − p(T − t), p ≤ 0

or

(4.6)

t+((−U −u)/p)

;

(4.7)

u < −U − p(T − t), p ≥ 0,

then ω ≡ 0. (4.8)

(II) For t > T , if u > U + p(t − T ), p > 0,

then 0 ≤ ω < Ket−((u−U )/p) ;

(4.9)

if u U, p ≤ 0,

then ω ≡ 0.

(4.11)

We postpone the proof of Lemmas 2 and 3 and first finish the proof of the theorem. Proof of Theorem 4. In order to achieve decay also in the p direction, introduce the Gibbs density α(p, u, t) = e−H = e−1/2p

2 −e2t W (u,t)

.

We have α˙ = −e−H H˙ = −e−H Ht = −α e2t W t .

(4.12)

Now recall that the Hamiltonian flow of (4.1) preserves the Liouville measure dµ = dp du. Multiply the Riccati equation (4.2) by α, and write it in the form ( ˙

αω − αω ˙ + αω2 + αe2t Wuu = 0.

Substitute equation (4.12) to obtain ( ˙ αω + αω e2t W t + αω2 + αe2t Wuu (u, t) = 0.

397


Owing to the estimates of Lemmas 2 and 3, one can integrate this equation over the whole (p, u) space. In addition, use the invariance of the measure and write 2t d 2 αω dµ + αω e W t dµ + αω dµ + αe2t Wuu dµ = 0. (4.13) dt Using integration by parts, the last term can be replaced by α(e2t Wu )2 dµ. Now integrate the last equation (4.13) for −A ≤ t ≤ A for a large constant A and pass to the limit A → +∞. Note that by the uniform estimate of Lemma 2, the term αω dµ|A −A vanishes in the limit. So we have 2 αω W e2t t dµ dt + αω2 dµ dt + α e2t W dµ dt = 0. (4.14) By the Cauchy-Schwarz inequality, we can estimate the first integral of (4.14) by

αω W e2t t dµ dt ≥ −

αω2 dµ dt

2 α W e2t t dµ dt

1/2 .

With the notation x = ( αω2 dµ dt)1/2 , we have the quadratic inequality 2t 2 2 e Wu α dµ dt ≤ 0. x 2 − x · α W e2t t dµ dt + Then its discriminant must be nonnegative: 2t 2 2 4 α e Wu dµ dt ≤ α e2t W t dµ dt.

(4.15)

The final argument in the proof is the following rescaling trick. It is similar to one invented in [3] for periodic potentials. Consider the family of Hamiltonians for every natural number N: HN =

1 1 1 H (Np, Nu, t) = p2 + 2 e2t W (Nu, t) . 2 N2 N

It can be immediately checked that the property of having all solutions without conjugate points remains valid for all N. Thus the inequality (4.15) implies the following inequalities for all N: 4

e−(1/N

2 1 Wu (Nu, t) du dt N 2 1 2t −(1/N 2 )W (Nu,t)e2t e W (Nu, t) t du dt. ≤ e N2

2 )W (Nu,t)e2t

e2t

(Here the integration with respect to p has been performed on both sides.)

398

BIALY AND MACKAY

Change the variable in both integrals to υ = Nu. We obtain the inequality: 4 N3

e−(1/N

2 )W (υ,t)e2t

1 ≤ 5 N

2 e2t Wu (υ, t) dυ dt

e−(1/N

2 )W (υ,t)e2t

2 e2t W (υ, t) t dυ dt.

Now it is clear that if W is not zero identically, then the left side is of order 1/N 3 while the right side is of order 1/N 5 as N → ∞. This then proves that W ≡ 0 identically. The proof of the theorem is complete. Proof of Lemma 2. By the definition of K, we have ω˙ ≤ K 2 e2t − ω2

for t ≤ T

and ω˙ = −ω2

for t > T .

The proof is based on the following elementary fact. Fact. Any solution of the inequality ω˙ ≤ B 2 − ω2 ,

ω (t0 ) = ω0

blows up if |ω0 | > B. Moreover, the blow-up time t∗ is estimated as follows for ω0 > B,

t 0 + < t ∗ < t0 ;

for ω0 < −B,

t0 < t∗ < t0 + , ω0 − B 1 ln . where = 2B ω0 + B To prove (4.3), we take B = KeT and obtain the required estimate. In the same manner, we obtain (4.4) and the right-hand side of (4.5). Let us prove the rest of (4.5). Pick two moments of time τ0 < τ1 < 0. On the segment τ ∈ [τ0 , τ1 ], we have ω˙ ≤ B 2 − ω2 with B = Keτ1 . If ω (τ0 ) = ω0 is too negative, then the blow-up happens before τ1 , and this is impossible. Thus we obtain 1 ω0 − B = ln > τ 1 − τ0 . 2B ω0 + B This implies 1 + e2Bd . e2Bd − 1 Choose τ1 = τ0 /2; then this inequality can be written in the form: ω0 ≥ −B

ω0 ≥ −eτ0 /4 · f (τ0 ) ,


where

399

τ /2

f (τ0 ) = Keτ0 /4 ·

1 + e−Kτ0 e 0

τ /2

1 + e−Kτ0 e 0

.

An easy calculation shows that for τ0 → −∞, f → 0, and thus f (τ0 ) is bounded ˜ Since τ0 was arbitrary, (4.5) follows. from above by some positive constant K. Proof of Lemma 3. Any point (p, u, t) with |u| > U is situated outside the support " and so moves in the straight line u(t) = u+pt unless it hits ". If it does not touch ", then ω vanishes identically. This is the case in both (4.8) and (4.11). In the cases of (4.6)–(4.7) and (4.9)–(4.10), the line u(t) = u + pt hits " in backward time. Since ω˙ = −ω2 for the free motion, it follows that ω(p, u, t) has to be nonnegative and can be bounded from above by Ket˜, where t˜ is the time of entering ". This completes the proof of Lemma 3. Although by Theorem 4 there are always solutions with conjugate points, one cannot claim that they appear on those radial solutions that are regular at zero. The next example shows that all regular solutions may remain minimal. Example 4.1. Consider two compactly supported functions ,(u) and -(t). Define the function 2 1 ˙ ˙ + -(t) ,(u) . W (u, t) = −e−2t -(t),(u) 2 Then W (u, t) is compactly supported, and one can easily verify that all the solutions of ˙ u˙ = f (u, t) with f (u, t) = ,(u)-(t)

(4.16)

are the solutions of the Newton equation ü = −e2t Wu (u, t). The solutions of (4.16) form a foliation of R(u) × R(t) and then, by Weierstrass’s theorem of calculus of variations (see, e.g., [11, §23]), are globally minimal. It is clear from the construction that all corresponding radial solutions u(r) of (1.1) are regular at zero. 5. Discussion and open problems. The results of this paper can be generalised in several directions. We have formulated our results in the context of compactly supported potentials V on R × Rn , but most of them extend to some noncompactly supported cases. Theorem 1 and its proof apply verbatim to V of noncompact support, in particular to V periodic in u, but periodicity in x is excluded by each of the hypotheses (A) and (B). Theorem 2 can be generalised to V periodic in u and compactly supported and radially symmetric in x. Our work, however, leaves open some very natural questions. (1) To what other radially symmetric cases can Theorem 2 be extended?

400

BIALY AND MACKAY

(2) What results can be obtained for potentials periodic in both u and x? The fundamental papers [15] and [16] by Moser imply, for our equation, the existence of minimal laminations for all irrational slopes which for certain slopes are foliations. In this context, Question 2 can be reformulated as follows: Are there other periodic potentials except those with Vu (u, x) = 0 such that for any slope there is a smooth foliation of Tn+1 by minimal solutions? We refer the reader to the survey article by Bangert [1] for detailed discussions of this and related questions. (3) Does Theorem 2 extend to nonradial potentials V ? The proof of Theorem 2 is based on the reduction to the case of Hamiltonian systems and so cannot be generalised in a straightforward way to nonradial potentials V . However, it is very reasonable to expect that the equation perturbed by a compactly supported W , u = −Vu (u, r) + εWu (u, x1 , x2 ), always has nonminimal solutions for ε sufficiently small. In the case of Hamiltonian systems, it would follow from the theorem on continuous dependence of solutions on the initial values. (4) Regarding Theorem 4, if the support of W (u, t) were compact, then absence of conjugate points would imply that every trajectory on the (u, t)-plane leaves the support of W in the same direction as it entered it. It would be interesting to know if this a priori weaker assumption already implies that W is zero. Related questions in the Riemannian case were considered in [6]. References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13]

V. Bangert, “Minimal foliations and laminations” in Proceedings of the International Congress of Mathematicians, Vol. 1, 2 (Zürich, 1994), Birkhäuser, Basel, 1995, 453–464. M. Bialy, Convex billiards and a theorem by E. Hopf, Math. Z. 214 (1993), 147–154. M. Bialy and L. Polterovich, Hopf-type rigidity for Newton equations, Math. Res. Lett. 2 (1995), 695–700. D. Burago and S. Ivanov, Riemannian tori without conjugate points are flat, Geom. Funct. Anal. 4 (1994), 259–269. R. Courant and D. Hilbert, Methods of Mathematical Physics, Vol. 1, Interscience Publishers, New York, 1953. C. Croke, Rigidity and the distance between boundary points, J. Differential Geom. 33 (1991), 445–464. C. Croke and A. Fathi, An inequality between energy and intersection, Bull. London Math. Soc. 22 (1990), 489–494. C. Croke and B. Kleiner, On tori without conjugate points, Invent. Math. 120 (1995), 241–257. L. Green and R. Gulliver, Planes without conjugate points, J. Differential Geom. 22 (1985), 43–47. J. Heber, On the geodesic flow of tori without conjugate points, Math. Z. 216 (1994), 209–216. D. Hilbert, Mathematical problems, Bull. Amer. Math. Soc. 8 (1902), 437–479. E. Hopf, Closed surfaces without conjugate points, Proc. Nat. Acad. Sci. U.S.A. 34 (1948), 47–51. A. Knauf, Closed orbits and converse KAM theory, Nonlinearity 3 (1990), 961–973.

VARIATIONAL PROPERTIES AND RIGIDITY [14] [15] [16] [17]

401

E. Lieb and M. Loss, Analysis, Grad. Stud. Math. 14, Amer. Math. Soc., Providence, 1997. J. Moser, Minimal solutions of variational problems on a torus, Ann. Inst. H. Poincaré Anal. Non Linéaire 3 (1986), 229–272. , A stability theorem for minimal foliations on a torus, Ergodic Theory Dynam. Systems 8 (1988), Charles Conley Memorial Issue, 251–281. M. Reed and B. Simon, Methods of Modern Mathematical Physics, IV: Analysis of Operators, Academic Press, New York, 1978.

Bialy: School of Mathematical Sciences, Tel-Aviv University, Tel-Aviv 69978, Israel Mackay: Nonlinear Centre, Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Silver Street CB3 9EW, United Kingdom

Vol. 102, No. 3


© 2000

EXAMPLES WITH BOUNDED DIAMETER GROWTH AND INFINITE TOPOLOGICAL TYPE X. MENGUY

0. Introduction. In this paper, we construct an example of an open manifold with positive Ricci curvature, infinite topological type, and bounded diameter growth. Let M n be an open manifold with base point p ∈ M n . For r > 0, denote by C(p, r) the union of the unbounded components of M n \Br (p). U. Abresch and D. Gromoll [AG] defined diameter growth as 3 diam(p, r) = sup diam k , C p, r , (0.1) 4 where the supremum is taken over all components k of ∂C(p, r) and diam(k , C(p, (3/4)r)) is the diameter of k calculated with respect to the intrinsic distance of C(p, (3/4)r). Here, we show the following theorem. Theorem 0.2. There exists a complete 4-dimensional manifold with (0.3)

positive Ricci curvature,

(0.4)

infinite topological type, and

(0.5)

bounded diameter growth.

This result can be generalized to any dimension greater than 5 by taking products with any compact manifold of positive Ricci curvature. If we take a product with a circle, we get a 5-dimensional example satisfying (0.4) and (0.5) but whose Ricci curvature is nonnegative. Theorem 0.2 is the first example of a manifold satisfying (0.3), (0.4), and (0.5). Previously, J. Sha and D. G. Yang [ShYg] constructed manifolds satisfying (0.3) and (0.4); see also the examples of M. T. Anderson [An]. Examples with Ricci-flat Kähler metrics were later given by Anderson, Kronheimer, and LeBrun [AnKL]. All these examples are collapsing, namely, limr→0 r −n Vol(Br (p)) = 0. Among them, the smallest diameter growth is due to [ShYg] and is equivalent to r 2/3 . More recently, G. Perelman [P] constructed 4-dimensional compact manifolds with positive Ricci curvature, large volume, and large Betti numbers. We used these constructions in [M2] to give examples of manifolds with (0.3), (0.4), and Euclidean volume growth. In contrast to the examples of Sha and Yang, the sectional curvature of our example in Theorem 0.2 is not bounded from below. Indeed, according to a result of Abresch Received 11 May 1999. 1991 Mathematics Subject Classification. Primary 53C20; Secondary 53C21. 403

404

X. MENGUY

and Gromoll [AG], an n-dimensional manifold with nonnegative Ricci curvature, sectional curvature bounded from below, and diameter growth smaller than r 1/n has finite topological type. Note also that in dimensions 2 and 3 any complete manifold satisfying (0.3) is diffeomorphic to Euclidean space. This follows from the work of S. Cohn-Vossen [C] and of R. Schoen and S. T. Yau [SY], respectively. The idea of this paper is to start with a cylinder over a lemon (that is, a spherical suspension over a small 2-sphere). We replace some cylindrical sectors by spherical sectors such that locally it looks like a singular 4-sphere. We then remove a ball centered on the singular set and glue in a CP 2 with a point removed via the following metric of Perelman [P]. Let (S n , g) be a rotationally symmetric metric with sectional curvature greater than 2 , where dσ 2 1 on the n-sphere. That is, g = dt 2 + B 2 (t)dσn−1 n−1 is the metric of the unit (n − 1)-sphere. We denote by [0, πr0 ] the range of t and denote by R0 the maximal value of B(t) on [0, πr0 ]. For any given ρ > 0 with (0.6)

(n−1)/n

R0

< ρ < r0 ,

there exists a metric of positive Ricci curvature on S n × [0, 1] with the following properties: • S n × {0} is isometric to a round sphere of radius ρ/λ; • the principal curvatures of S n × {0} are greater than −λ; • S n × {1} is isometric to (S n , g) and its principal curvatures are greater than 1. We refer to a metric of this form as a neck of Perelman. In our construction, we look for removing balls Bri (oi ) and gluing in a rescaled neck of Perelman. That is why we make the following definition. Definition 0.7. Let N n be a compact hypersurface in M n+1 . We denote by II its second fundamental form, SN n its unit tangent bundle, and KN its intrinsic curvature. We say that N n satisfies Perelman’s property if for all x ∈ N and all linearly independent X, Y in Sx N n , KN (X ∧ Y ) > maxn II2 (Z, Z). Z∈SN

It is also possible to construct a metric with positive Ricci curvature and convex boundary on CP n with a point removed (see [P] for details). Therefore, if the boundary of a ball in M n+1 has a rotationally symmetric metric and satisfies Perelman’s property, we can rescale it by the inverse of the maximum of the principal curvatures, use a neck of Perelman if R0 (which appears in the definition of the neck) is sufficiently small, and glue in a CP n . What is important is that the gluing has positive Ricci curvature, thanks to the neck of Perelman. See [M2] for another construction using a neck of Perelman. Note that in higher even dimension, we can modify slightly the construction of Theorem 0.2 and glue in CP n ’s instead of CP 2 ’s. The main point of this paper then is to smooth the metric near the singular set and check Perelman’s property for some balls Bri (oi ). From C. Sormani [So] we

DIAMETER GROWTH AND INFINITE TOPOLOGICAL TYPE

405

know that the annuli Br+1 (p) \ Br (p) are converging to cylinders (see also [ChCo] for almost rigidity results in the case of almost maximal volume). Therefore, our glued-in CP 2 ’s have to be smaller and smaller. If we apply the stability theorem of T. H. Colding [Co1], we know that at least one cross section of the cylinders has to be a singular space, and in our case the singular cross section looks like a lemon. See also [Co2] for background and general results about manifolds with lower-bounded Ricci curvature. The main difference between this construction and the one in [M2] is the way we smooth the metric near the singular set. In [M2] we smoothed the metric slowly and progressively in order to have both positive Ricci curvature and Perelman’s property. Namely, the metric of the main block was written as g#(r) (x), where #(r) = 1/ logβ r and r denotes the distance from a base point (see [M2, Sections 1.2 and 1.3]; compare also with Section 1.2 of this paper). Roughly speaking, # measures the difference between g# (x) and the metric of a lemon. In the case of bounded diameter growth, the curvatures of the spherical sectors are big, and therefore the width of these sectors decreases rapidly. As a consequence the radii of the balls Bri (oi ) and the parameter #(r) decrease rapidly too. If we apply the same smoothing as in [M2], some very negative Ricci curvatures are created along the singular set. The new idea of this paper is to use the neck of Perelman not only to glue in CP 2 ’s but also to smooth the metric. The parameter # now depends on the balls Bri (oi ) that are removed and not on r. The function #(r) is constant except inside Bri (oi ), where it changes a lot. Therefore, the created negative Ricci curvature is localized inside the balls Bri (oi ). However, a slight modification of the metric g# (x) used in [M2] enables us to keep very good control of the principal curvatures of ∂Bri (oi ) (see Lemma 1.26 and Section 1.4; compare with [M2, Section 1.6] and [P, Section 3]). After removing Bri (oi ) and gluing in a CP 2 with a point removed, the Ricci curvature is positive. Finally, we point out that the construction of Theorem 0.2 can be adapted to the noncollapsing case. This then gives a new, shorter, and more elegant construction than the one of [M2]. Acknowledgment. This paper is part of my dissertation [M1]. I would like to thank sincerely Professor T. H. Colding for his advice, remarks, and encouragement. 1. Construction 1.1. Main block. The metric of our manifold is given by (1.1)

ds 2 = dt 2 + u2 (t) dx 2 + f (t, x)2 dσ 2 ,

where dσ 2 is the standard metric of the unit 2-sphere. T , X, 1 , and 2 form an orthonormal basis corresponding to the forms dt 2 , dx 2 , dσ 2 . Let ti ≥ 2 be any increasing sequence going to infinity to be determined. We define inductively the function u(t) and sequences ui , ui , Ki , ψi , ri , oi as follows:

406

X. MENGUY

(1.2)

u1 = u(t1 ) = 1

(1.3)

ui = u(ti )

(1.4) (1.5) (1.6) (1.7) (1.8) (1.9)

and

u1 = ut (t1 ) =

and ui = ut (ti ); 1 − u 2 i Ki = ; u2i

2 Ki ψi = 1 − ui ; sin

1 ; t12

π ψi ri < √ − ; 2 4 Ki oi = (t = ti + ri , x = 0);

1 Ki t − ti + ψi , for ti < t < ti + 2ri ; u(t) = √ sin Ki ti+ 2 ut (t) = ut ti + 2ri , for ti + 2ri < t < ti+1 . t

The function f (x, t) is defined for t > t1 in Section 1.2. The metric is defined near the origin, that is, for t < t1 , in Section 1.6. From now on, we suppose that ri is sufficiently rapidly decreasing so that the diameter growth is bounded. 1.2. Smoothing near the singular set. In this subsection, we construct the C 2 function f (t, x) in order to have both Perelman’s property at the boundary of Bri (oi ) and positive Ricci curvature everywhere outside these balls. First, we define a function f# (x) as follows: Let R and # be two constants such that (1.10)

Let θ be a C 2 function such that (1.11)

1 0 0, there exists #0 > 0 such that if # < #0 , then (1.23) (1.24)

(f# )xx ≥ 1 − η, f# 1 − (f# )2x ≥ 1 − η. f#2 −

Proof. Note first that it suffices to prove (1.23), since (1.24) follows from (1.23) and the Ricatti comparison theorem. From (1.13), we have −(f# )xx /f# = l 2 ≥ 1 for

408

X. MENGUY

0 < x 1 for b < x < #. For # < x < 1, we calculate from (1.15) that (1.25)

−

2 (f# )xx = 1 + δθ# (x) − cot x + δθ# (x) δθ# (x). f#

From (1.12), (1.19), and (1.25), we see that | − (f# )xx /f# − 1| is as small as we want for sufficiently small #. For 1 ≤ x ≤ π/2, −(f# )xx /f# = 1, which proves eventually the lemma. The following lemma is crucial to control the principal curvatures of ∂Bri (oi ). Lemma 1.26. There exists #0 > 0 such that if # < #0 , then for 0 ≤ x ≤ 1, (f# )x (1.27) − cot x < 2#. A(x) = tan(x) f# Proof. From Lemma 1.22 and a standard comparison argument, we get that 1 (f# )x < . f# x

(1.28) For b ≤ x ≤ #, (1.14) gives

1−# (f# )x , = f# x

(1.29) and for x ≤ b, (1.13) gives (1.30)

1−# (f# )x . ≥ f# x

Therefore, from (1.28), (1.29), and (1.30), we get that (1.31)

A(x) ≤

# tan x < 2# x

for x ≤ # and # sufficiently small. For # ≤ x ≤ 1, we get from (1.15) that (1.32) and thus (1.33)

(1.34)

(f# )x = cot x + δθ# (x) 1 + δθ #(x) , f# tan(x) 1 + δθ# (x) − 1 , A(x) = tan x + δθ# (x) A(x) < 2#,

for # sufficiently small. This proves the lemma.


409

Remark 1.35. We see that f# can be smoothed near x = b and x = # to a C 2 function satisfying Lemmas 1.22 and 1.26. We now define the function f (x, t) for t > t1 . Let #i be a decreasing sequence converging to zero to be determined. For ti + 3ri /2< t < ti+1 + ri+1 /2, we set (1.36)

f (t, x) = f#i+1 (x),

and for ti + ri /2 < t < ti + 3ri /2, we set (1.37)

f (t, x) = f#(t) (x),

where #(t) is bounded by #i+1 and #i . For t1 < t < t1 + r1 /2, we set (1.38)

f (t, x) = f#1 (x).

1.3. Calculation of curvatures. In this subsection, we prove that the Ricci curvature of the main block is positive outside the balls Bri (oi ) provided that #i is sufficiently rapidly decreasing. Indeed, we assume that the sequence #i is decreasing so rapidly that ft = 0 outside Bri (oi ). We then compute the curvature outside the ball Bri (oi ) as follows: utt (1.39) K(X, T ) = K 1 , T = K 2 , T = − , u fxx ut 2 (1.40) , K X, 1 = K X, 2 = − 2 − u fu 1 − (fx )2 ut 2 (1.41) − . K 1 , 2 = u u2 f 2 The mixed curvature is zero. From Lemma 1.22, (1.8), (1.9), and the above calculations, the Ricci curvature is then positive, provided that #1 is small enough. 1.4. Establishing Perelman’s property. In this subsection, we prove that if #i is sufficiently rapidly decreasing, the intrinsic curvatures of ∂Bri (oi ) are greater than the square of the principal curvatures. This permits us to glue in a CP 2 with a point removed via a neck of Perelman. As in Section 1.3, we suppose that #i is sufficiently rapidly decreasing such that ft = 0 on ∂Bri (oi ). Note that in this case for (t, x) ∈ ∂Bri (oi ), f (t, x) = f# (x) with # = #i or #i+1 . We denote by N the outward pointing unit vector to T ∂Bri (oi ). Let θ be given by (1.42)

N = X sin θ + T cos θ.

Define Y by (1.43)

Y = X cos θ − T sin θ.

First, we calculate the second fundamental form of T ∂Bri (oi ):

410 (1.44)

X. MENGUY

ut (f# )x cos θ + sin θ. II 1 , 1 = II 2 , 2 = ∇1 N | 1 = u f# u

Using the√fact that the metric of the T -X-plane is locally isometric to the 2-sphere of radius 1/ Ki for ti < t < ti + 2ri (see (1.8)), we get

II(Y, Y ) = Ki cot K i ri . (1.45) II(·,·) is diagonal in the orthonormal basis 1 , 2 , and Y . The intrinsic curvatures are then calculated by (1.44), (1.45), and the Gauss equations so that (1.46) and

(1.47)

2 (f# )x ut K N 1 , 2 = K 1 , 2 + cos θ + sin θ u f# u KN Y, 1 = KN Y, 2 = K X, 1 cos2 θ + K T , 1 sin2 θ

u (f# )x t Ki cos θ + sin θ . + Ki cot u f# u

The mixed curvature is zero. We note √ that if f# (x) = sin x, the metric is locally isometric to the 4-sphere of radius 1/ Ki (see (1.8)), and therefore

ut cot x cos θ + sin θ = Ki cot Ki r i . (1.48) u u Combining (1.44) and (1.48), we get

(1.49)

(f ) sin θ # x K i ri = − cot x II 1 , 1 − Ki cot f# u cot x| sin θ| ≤ A(x) u

u t ≤ A(x) Ki cot Ki ri + | cos θ| u 1

≤ A(x) ; Ki cot K i ri + u

consequently, by Lemma 1.26,

4# (1.50) K i ri ≤ . II 1 , 1 − Ki cot ri Conclusion. From (1.45), (1.46), (1.47), (1.50), and Section 1.3, we conclude that Perelman’s property is satisfied, provided that #i is sufficiently rapidly decreasing.


411

1.5. Gluing in CP 2 ’s. To glue a CP 2 in at ∂Bri (oi ) via a neck of Perelman, we need to check that ∂Bri (oi ) rescaled by (1.51)

Ki cot

4# K i ri + ri

satisfies (0.6) for sufficiently small #. If #i and #i+1 are going to zero, then the metric of Bri (oi ) is converging to (1.52)

ds 2 = dt 2 + u2 (t) dx 2 + R 2 sin2 (x)dσ 2 .

Bri (oi ) is then a ball of a singular 4-sphere with center on the singular set. Therefore, after rescaling by (1.50) with # = 0, we get that

r0 = cos Ki ri , (1.53)

R0 = R cos Ki ri . (1.54) Therefore, (0.6) is satisfied if #i is sufficiently rapidly decreasing and if R is smaller than some fixed constant. Conclusion. Combining the result of this section with the conclusion of Section 1.4, we can glue a CP 2 in at ∂Bri (oi ) via a neck of Perelman, provided that #i is sufficiently rapidly decreasing and R is small enough. 1.6. Smoothing near the origin. The submanifold defined by t = t1 satisfies Perelman’s property if t1 ≥ 2. Indeed, (1.55) (1.56) (1.57)

ut II(X, X) = II 1 , 1 = II 2 , 2 = , u fxx KN X, 1 = KN X, 2 = − 2 , fu 2 1 − (fx ) . KN 1 ∧ 2 = (uf )2

From (1.56), (1.57), and Lemma 1.22, the intrinsic curvatures are greater than 1/(2u2 ) for sufficiently small #1 ; from (1.55) and (1.2), the principal curvatures are smaller than 1/(4u). For t ≤ t1 , we then can continue the manifold via a neck of Perelman and glue the end of the neck in a ball of R4 in order to complete the metric. References [AG] [An]

U. Abresch and D. Gromoll, On complete manifolds with nonnegative Ricci curvature, J. Amer. Math. Soc. 3 (1990), 355–374. M. T. Anderson, Short geodesics and gravitational instantons, J. Differential Geom. 31 (1990), 265–275.

412 [AnKL] [ChCo] [C] [Co1] [Co2]

[M1] [M2] [P]

[SY]

[ShYg] [So]

X. MENGUY M. T. Anderson, P. Kronheimer, and C. LeBrun, Complete Ricci-flat Kähler manifolds of infinite topological type, Comm. Math. Phys. 125 (1989), 637–642. J. Cheeger and T. H. Colding, Lower bounds on Ricci curvature and the almost rigidity of warped products, Ann. of Math. (2) 144 (1996), 189–237. S. Cohn-Vossen, Totalkrümmung und geodätische Linien auf einfach zusammenhängenden offenen vollständigen Fläschenstücken, Recueil Math. Moscou 43 (1936), 139–163. T. H. Colding, Ricci curvature and volume convergence, Ann. of Math. (2) 145 (1997), 477–501. , “Spaces with Ricci curvature bounds” in Proceedings of the International Congress of Mathematicians (Berlin, 1998), Vol. 2, Documenta Mathematica, Bielefeld, 1998, 299–308; available from http://www.mathematik.uni-bielefeld.de/documenta/. X. Menguy, Examples of manifolds and spaces with positive Ricci curvature, Ph.D. thesis, Courant Institute, New York University, 2000. , Noncollapsing examples with positive Ricci curvature and infinite topological type, to appear in Geom. Funct. Anal. G. Perelman, “Construction of manifolds of positive Ricci curvature with big volume and large Betti numbers” in Comparison Geometry (Berkeley, 1993–94), Math. Sci. Res. Inst. Publ. 30, Cambridge Univ. Press, Cambridge, 1997, 157–163. R. Schoen and S. T. Yau, “Complete three-dimensional manifolds with positive Ricci curvature and scalar curvature” in Seminar on Differential Geometry, Ann. of Math. Stud. 102, Princeton Univ. Press, Princeton, 1982, 209–228. J. Sha and D.-G. Yang, Examples of manifolds of positive Ricci curvature, J. Differential Geom. 29 (1989), 95–103. C. Sormani, The almost rigidity of manifolds with lower bounds on Ricci curvature and minimal volume growth, to appear in Comm. Anal. Geom.

Courant Institute of Mathematical Sciences, 251 Mercer Street, New York, New York 10012, USA; [email protected]

Vol. 102, No. 3


© 2000

RIGID LOCAL SYSTEMS, HILBERT MODULAR FORMS, AND FERMAT’S LAST THEOREM HENRI DARMON

Contents 1. Frey representations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415 1.1. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415 1.2. Classification: The rigidity method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416 1.3. Construction: Hypergeometric abelian varieties . . . . . . . . . . . . . . . . . . . . . . 419 2. Modularity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428 2.1. Hilbert modular forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428 2.2. Modularity of hypergeometric abelian varieties . . . . . . . . . . . . . . . . . . . . . . 430 3. Lowering the level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434 3.1. Ribet’s theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434 3.2. Application to x p + y p = zr . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435 4. Torsion points on abelian varieties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446 Introduction. Historically, two approaches have been followed to study the classical Fermat equation x r + y r = zr . The first, based on cyclotomic fields, leads to questions about abelian extensions and class numbers of K = Q(ζr ) and values of the Dedekind zeta-function ζK (s) at s = 0. Many open questions remain, such as Vandiver’s conjecture that r does not divide the class number of Q(ζr )+ . The second approach is based on modular forms and the study of 2-dimensional representations of ¯ Gal(Q/Q). Even though 2-dimensional representations are more subtle than abelian ones, it is by this route that Fermat’s last theorem was finally proved (cf. [Fre], [Se2], [Ri2], [W3], and [TW]; or [DDT] for a general overview). This article examines the equation x p + y q = zr .

(1)

¯ Certain 2-dimensional representations of Gal(K/K), where K is the real subfield of a cyclotomic field, emerge naturally in the study of equation (1), giving rise to a blend of the cyclotomic and modular approaches. The special values ζK (−1), which Received 23 September 1998. Revision received 23 June 1999. 1991 Mathematics Subject Classification. Primary 11G18; Secondary 11D41, 11F80, 11G05. Author’s work partially supported by grants from Natural Sciences and Engineering Research Council of Canada and from Fonds pour la Formation de Chercheurs et l’Aide à la Recherche, and by an Alfred P. Sloan research award. 413

414

HENRI DARMON

in certain cases are related to the class numbers of totally definite quaternion algebras over K, appear as obstructions to proving that (1) has no solutions. The condition that r is a regular prime also plays a key role in the analysis leading to one of our main results about the equation x p + y p = zr (see Theorem 3.22). One is interested in primitive solutions (a, b, c) to equation (1), that is, those satisfying gcd(a, b, c) = 1. (Such a condition is natural in light of the abc conjecture, for example; see also [Da2].) A solution is called nontrivial if abc = 0. It is assumed from now on that the exponents p, q, and r are prime and that p is odd. Let (a, b, c) be a nontrivial primitive solution to equation (1). One wishes to show that it does not exist. The program for obtaining the desired contradiction, following the argument initiated by Frey and brought to a successful conclusion by Wiles in the case of x p + y p = zp , can be divided into four steps. Step 1 (Frey, Serre). Associate to (a, b, c) a mod p Galois representation ¯ ρ : Gal(K/K) −→ GL2 (F) having “very little ramification,” that is, whose ramification can be bounded precisely and a priori independently of the solution (a, b, c). Here K is a number field and F is a finite field. For the Fermat equation x p + y p = zp , one may take K = Q and F = Z/pZ: the representation ρ is then obtained by considering the action of GQ on the p-division points of the Frey elliptic curve y 2 = x(x −a p )(x +b p ). As explained in Section 1, one is essentially forced to take K = Q(ζq , ζr )+ and F, the residue field of K at a prime above p, in studying equation (1). Step 2 (Wiles). Prove that ρ is modular, that is, arises from a Hilbert modular form on GL2 (AK ). In the setting of Fermat’s equation, Wiles proves that all semistable elliptic curves over Q arise from a modular form, which implies the modularity of ρ. Step 3 (Ribet). Assuming step 2, show that ρ comes from a modular form of small level, and deduce (in favorable circumstances) that its image is small, that is, contained in a Borel subgroup or in the normalizer of a Cartan subgroup of GL2 (F). In the setting of Fermat’s equation, Ribet showed that ρ has to be reducible; for reasons that are explained in Section 3, one cannot rule out the case where the image of ρ is contained in the normalizer of a Cartan subgroup when dealing with equation (1). Step 4 (Mazur). Show that the image of ρ is large; for example, that it contains SL2 (F). Historically, this is the step in the proof of Fermat’s last theorem that was carried out first, in the seminal papers [Ma1] and [Ma2], which also introduced many of the tools used in steps 2 and 3. In the classical setting, combining the conclusions of steps 3 and 4 leads to a contradiction and shows that (a, b, c) does not exist, thus proving Fermat’s last theorem. In [Da1] and [DMr], it was observed that the program above can be used to show that x p + y p = zr has no nontrivial primitive solutions when r = 2, 3 and p ≥ 6 − r (the

HILBERT MODULAR FORMS AND FERMAT’S LAST THEOREM

415

result for r = 3 being conditional on the Shimura-Taniyama conjecture, which is still unproved for certain elliptic curves whose conductor is divisible by 27). The purpose of this article is to generalize the analysis to the general case of equation (1). Sections 1, 2, 3, and 4 describe the generalizations of steps 1, 2, 3, and 4, respectively. As a concrete application, the main results of Section 3 relate solutions to x p +y p = zr to questions about p-division points of certain abelian varieties with real multiplications by Q(cos(2π/r)). Alas, our understanding of these questions (and of the arithmetic of Hilbert modular forms over totally real fields) is too poor to yield unconditional statements. For the time being, the methods of this paper should be envisaged as a way of tying equation (1) to questions that are more central, concerning Galois representations, modular forms, and division points of abelian varieties. Acknowledgements. The author is grateful to F. Diamond, J. Ellenberg, A. Kraus, and K. Ribet for their helpful comments, and to N. Katz and J.-F. Mestre for pointing out a key construction used in Section 1. The author greatly benefitted from the support of le Centre interuniversitaire en calcul mathématique algébrique (CICMA) and the hospitality of the Université Paris VI (Jussieu) and the Institut Henri Poincaré, where the work on this paper was started, and of the Eidgenössische Technische Hochschule (ETH) in Zürich, where it was completed. 1. Frey representations ¯ 1.1. Definitions. If K is any field of characteristic zero, write GK := Gal(K/K) for its absolute Galois group. Typically, K is a number field; let K(t) be the function field over K in an indeterminate t. The group GK(t) fits into the exact sequence 1 −→ GK(t) −→ GK(t) −→ GK −→ 1. ¯ Let F be a finite field, embedded in a fixed algebraic closure of its prime field. Definition 1.1. A Frey representation associated to the equation x p +y q = zr over K is a Galois representation = (t) : GK(t) −→ GL2 (F) satisfying the following conditions. (1) The restriction of to GK(t) has trivial determinant and is irreducible. Let ¯ −→ PSL2 (F) ¯ geom : GK(t) ¯ be the projectivization of this representation. (2) The homomorphism ¯ geom is unramified outside {0, 1, ∞}. (3) It maps the inertia groups at 0, 1, and ∞ to subgroups of PSL2 (F) of order p, q, and r, respectively. The characteristic of F is also called the characteristic of the Frey representation. One should think of = (t) as a 1-parameter family of Galois representations of

416

HENRI DARMON

GK indexed by the parameter t. Condition (1) in Definition 1.1 ensures that this family has constant determinant but is otherwise “truly varying” with t. The motivation for the definition of (t) is the following. Lemma 1.2. There exists a finite set of primes S of K depending on in an explicit way, such that, for all primitive solutions (a, b, c) to the generalized Fermat equation x p + y q = zr , the representation ρ := (a p /cr ) has a quadratic twist that is unramified outside S. Sketch of proof. Let ¯ : GK(t) −→ PGL2 (F) be the projective representation deduced from . The field fixed by the kernel of ¯ is a finite extension of K(t), whose Galois group is identified with a subgroup G of PGL2 (F) by ; ¯ in other words, it is the function field of a G-covering of P1 over K. This covering is unramified outside {0, 1, ∞} and its ramification indices are p, q, and r above those three points: it is a G-covering of “signature (p, q, r)” in the sense of [Se3, Sec. 6.4]. The lemma now follows from a variant of the Chevalley-Weil theorem for branched coverings (see, e.g., [Be] or [Da2]). Definition 1.3. Two Frey representations 1 and 2 attached to equation (1) are said to be equivalent if their corresponding projective representations ¯ 1 and ¯ 2 differ ¯ that is, if 1 is conjugate (over F) ¯ to a central by an inner automorphism of PGL2 (F), twist of 2 . To a Frey representation we assign a triple (σ0 , σ1 , σ∞ ) of elements in PSL2 (F) of orders p, q, and r satisfying σ0 σ1 σ∞ = 1 as follows (cf. [Se3, Ch. 6]). The element σj is defined as the image by ¯ geom of a generator of the inertia subgroup of GK(t) at ¯ t = j . The elements σ0 , σ1 , and σ∞ are well defined up to conjugation, once primitive p, q, and rth roots of unity have been chosen. One can choose the decomposition groups in such a way that the relation σ0 σ1 σ∞ = 1 is satisfied (cf. [Se3, Th. 6.3.2]). The triple (σ0 , σ1 , σ∞ ) is then well defined up to conjugation. If Cj is the conjugacy class of σj in PSL2 (F), one says that the Frey representation is of type (C0 , C1 , C∞ ). For the following definition, assume that the exponents p, q, and r are odd, so that σ0 , σ1 , and σ∞ lift to unique elements σ˜ 0 , σ˜ 1 , and σ˜ ∞ of SL2 (F) of orders p, q, and r, respectively. Definition 1.4. The Frey representation attached to x p + y q = zr is said to be odd if σ˜ 0 σ˜ 1 σ˜ ∞ = −1, and is said to be even if σ˜ 0 σ˜ 1 σ˜ ∞ = 1. 1.2. Classification: The rigidity method. If n is an integer, let ζn denote a primitive√nth root of unity. Given an odd prime p, write p ∗ := (−1)(p−1)/2 p, so that Q( p ∗ ) is the quadratic subfield of Q(ζp ). We now turn to the classification of Frey representations, beginning with the classical Fermat equation.


417

The equation x p + y p = zp Theorem 1.5. Let p be an odd prime. There is a unique Frey representation (t) of characteristic p (up to equivalence) associated to the Fermat equation x p + y p = zp . One may take K = Q and F = Fp , and the representation (t) is odd. Remark. This theorem is originally due to Hecke [He], where it is expressed as a characterization of a certain field of modular functions of level p. Proof of Theorem 1.5. Set F = Fp . We begin by classifying conjugacy classes of triples σ0 , σ1 , and σ∞ of elements of order p in PSL2 (F) satisfying σ0 σ1 σ∞ = 1. There are two conjugacy classes of elements of order p in PSL2 (F), denoted pA and pB, respectively. The class pA (resp., pB) is represented by an upper-triangular unipotent matrix whose upper right-hand √ ∗ entry is a square (resp., a nonsquare). These two classes are rational over Q( p ) in the sense of √ [Se3, Sec. 7.1], and they are interchanged by the nontrivial element in Gal(Q( p ∗ )/Q) as well as by the nontrivial outer automorphism of PSL2 (F). Lift σ0 , σ1 , and σ∞ to elements σ˜ 0 , σ˜ 1 , and σ˜ ∞ of order p in SL2 (F). The group SL2 (F) acts on the space V = F2 of column vectors with entries in F. Since σ˜ j is unipotent, there are nonzero vectors v1 and v2 in V which are fixed by σ˜ 0 and σ˜ 1 , respectively. Because σ0 and σ1 do not commute, the vectors for V . Scale v2 so that σ˜ 0 is expressed by the matrix 1 1 v1 and v2 form 1a 0basis in this basis; let be the matrix representing σ˜ 1 . Since σ˜ ∞ has trace 2, the 01 x 1 −1 relation σ˜ 0 σ˜ 1 = σ˜ ∞ forces x = 0, which is impossible since σ1 is of order p. Hence −1 there are no even Frey representations of characteristic p. The relation σ˜ 0 σ˜ 1 = −σ˜ ∞ gives x = −4. Note that the resulting elements σ0 , σ1 , and σ∞ belong to the same conjugacy class in PSL2 (F). It is well known that they generate PSL2 (F). Hence there are exactly two distinct conjugacy classes of surjective homomorphisms geom

¯ A

geom

, ¯ B

: GQ(t) −→ PSL2 (F), ¯

of type (pA, pA, pA) and (pB, pB, pB), respectively, which are interchanged by the outer automorphism of PSL2 (F). By the rigidity theorem of Bely˘ı, Fried, Thompson, geom geom and Matzat (cf. [Se3, Sec. 7]), ¯ A and ¯ B extend uniquely to homomorphisms ¯ A , ¯ B : GQ(t) −→ PGL2 (F) = Aut PSL2 (F) , which are conjugate to each other. Thus there is at most one Frey representation attached to x p + y p = zp , whose corresponding projective representation ¯ is conjugate to ¯ A and ¯ B . To prove the existence of , it is necessary to show that ¯ A (say) lifts to a linear representation GQ(t) → GL2 (F). Choose a set-theoretic lifting s of ¯ A to GL2 (F) satisfying det(s(x)) = χ(x), where χ is the mod p cyclotomic character, and note that such a lifting satisfies s(x)s(y) = ±s(xy). Hence, the obstruction to lifting ¯ A to a homomorphism into GL2 (F) is given by a cohomology class c(x, y) := s(x)s(y)s(xy)−1 in H 2 (Q(t), ±1). We note that (for j = 0, 1, and ∞) the

418

HENRI DARMON

homomorphism ¯ A maps the decomposition group at t = j to the normalizer of σj , which is the image in PGL2 (Fp ) of a Borel subgroup B of upper-triangular matrices. Since the inclusion F× p → B splits, it follows that the restrictions c0 , c1 , and c∞ of the cohomology class c in H 2 (Q((t)), ±1), H 2 (Q((t −1)), ±1), and H 2 (Q((1/t)), ±1) vanish. In particular, c has trivial “residues” at t = 0, 1, ∞ in the sense of [Se4, Ch. II, Annexe, Sec. 2]. Hence, c is “constant,” that is, comes from H 2 (Q, ±1) by inflation (see [Se4, Ch. II, Annexe, Sec. 4]). But note that H 2 (Q, ±1) injects into H 2 (Q((t)), ±1), since a nontrivial conic over Q cannot acquire a rational point over Q((t)). Therefore, the class c vanishes, and the result follows. (For an alternate and, perhaps, less roundabout argument, see [By].) The equation x p + y p = zr . Let us now turn to the equation x p + y p = zr , where r and p are distinct primes. One is faced here with the choice of considering Frey representations either of characteristic p or of characteristic r. From now on, we adopt the convention that the prime p is always used to denote the characteristic of the Frey representation, so that the equations x p + y p = zr and x r + y r = zp require seperate consideration. The following theorem is inspired from the proof given in [Se3, Prop. 7.4.3 and 7.4.4] for the case r = 2 and r = 3; the general case follows from an identical argument (see also [DMs]). Theorem 1.6. Suppose that r and p are distinct primes and that p = 2. There exists a Frey representation of characteristic p over K associated to x p + y p = zr if and only if (1) the field F contains the residue field of Q(ζr )+ at a prime p above p, and (2) the field K contains Q(ζr )+ . When these two conditions are satisfied, there are exactly r − 1 Frey representations up to equivalence. When r = 2, exactly (r − 1)/2 of these are odd and (r − 1)/2 are even. Proof. for condition (3) in Definition 1.1 to be satisfied, it is necessary that PSL2 (F) contain an element of order r. This is the reason for condition (1) in Theorem 1.6. Condition (2) arises from the fact that (for r = 2) the (r − 1)/2 distinct conjugacy classes of elements of order r in PSL2 (F) are rational over Q(ζr )+ (in the sense of [Se3, Sec. 7.1]) and are not rational over any smaller extension. Assume conversely that conditions (1) and (2) are satisfied. Let σ0 , σ1 , and σ∞ be chosen as in the proof of Theorem 1.5, and let σ˜ j be the lift of σj to SL2 (F) of order p when j = 0, 1. Finally, let σ˜ ∞ be a lift of σ∞ to an element of order r if r is odd and to an element of order 4 if r = 2. Let ω¯ ∈ F be the trace of σ˜ ∞ . When r = 2, one has ω¯ = 0, and when r is odd, ω¯ is of the form ϕ(ζr + ζr−1 ) where ϕ is a homomorphism from Z[ζr + ζr−1 ]+ to F. Note that there are exactly (r − 1)/2 such ϕ’s. One now finds, as in the proof of Theorem 1.5, that (σ˜ 0 , σ˜ 1 , σ˜ ∞ ) is conjugate to one of the following two triples:


1 0 1 0

1 1 , −(2 + ω) ¯ 1 1 1 , −(2 − ω) ¯ 1

419

−1 1 0 , , −2 − ω¯ 1 + ω¯ 1 1 −1 0 . , 2 − ω¯ −1 + ω¯ 1

When r = 2, these triples are equal in PSL2 (F). When r is odd, they are distinct. An argument based on rigidity as in the proof of Theorem 1.5 produces (r − 1) inequivalent homomorphisms from GK(t) to GL2 (F), yielding the desired odd and even Frey representations. These Frey representations are constructed explicitly in Section 1.3 (cf. Lemma 1.9 and Theorem 1.10). The equation x r + y r = zp Theorem 1.7. Suppose that r and p are distinct odd primes. There exists a Frey representation of characteristic p over K associated to x r + y r = zp if and only if (1) the field F contains the residue field of Q(ζr )+ at a prime p above p, and (2) the field K contains Q(ζr )+ . When these two conditions are satisfied, there are exactly (r − 1)(r − 2)/2 inequivalent Frey representations: (r − 1)2 /4 odd representations and (r − 1)(r − 3)/4 even representations. Although the conclusion is somewhat different, the proof of Theorem 1.7 follows the same ideas as the proof of Theorem 1.6. Each triple (C0 , C1 , pA), where C0 and C1 each range over the (r − 1)/2 possible conjugacy classes of elements of order r in PSL2 (F), gives rise to a unique odd and even projective representation of GK(t) of type (C0 , C1 , pA), with one caveat: There is no even representation of type (C0 , C1 , pA) when C0 = C1 . The equation x p + y q = zr . We finally come to the general case of equation (1). Assume that the exponents p, q, and r are distinct primes and that p is odd. Theorem 1.8. There exists a Frey representation of characteristic p over K associated to x p + y q = zr if and only if (1) the field F contains the residue fields of Q(ζq )+ and of Q(ζr )+ at a prime p above p, and (2) the field K contains Q(ζq )+ and Q(ζr )+ . When these two conditions are satisfied, there are (r − 1)(q − 1)/2 inequivalent Frey representations over Q(ζq , ζr )+ . If q, r = 2, then (r − 1)(q − 1)/4 of these are odd and (r − 1)(q − 1)/4 are even. The proof is the same as for Theorems 1.5, 1.6, and 1.7. 1.3. Construction: Hypergeometric abelian varieties The equation x p + y p = zp . One can construct the Frey representation (t) of Theorem 1.5 explicitly, by considering the Legendre family of elliptic curves

420

HENRI DARMON

J = J (t) : y 2 = x(x − 1)(x − t). It is an elliptic curve over Q(t) which has multiplicative reduction at t = 0 and 1 and has potentially multiplicative reduction at t = ∞. The module J [p] of its p-division points is a 2-dimensional F-vector space on which GQ(t) acts linearly. The corresponding representation (t) is the Frey representation of characteristic p attached to x p + y p = zp . The equation x p +y p = zr . When r = 2, let C2 (t) be the elliptic curve over Q(t) given by the equation C2 (t) : y 2 = x 3 + 2x 2 + tx.

(2)

Lemma 1.9. The mod p Galois representation associated to C2 is the Frey representation associated to x p + y p = z2 . The proof of this lemma is omitted. It follows the same ideas but is simpler than the proof of Theorem 1.10 for the case of odd r, for which all the details are given. j −j Suppose now that r is an odd prime. Let ωj = ζr + ζr , and write ω for ω1 , so that K = Q(ω) is the real subfield of the cyclotomic field Q(ζr ). Let ᏻK denote its ring of integers, and let d = (r − 1)/2 be the degree of K over Q. Let g(x) = dj =1 (x +ωj ) be the characteristic polynomial of −ω, and let f (x) be an antiderivative of ±rg(x)g(−x); for example, we take f (x) = xg x 2 − 2 = g(−x)2 (x − 2) + 2 = g(x)2 (x + 2) − 2. Following [TTV], consider the following hyperelliptic curves over Q(t) of genus d: Cr− (t) : y 2 = f (x) + 2 − 4t, Cr+ (t) : y 2 = (x + 2) f (x) + 2 − 4t .

(3) (4)

Let Jr− = Jr− (t) and Jr+ = Jr+ (t) be their Jacobians over Q(t). In [TTV], Tautz, Top, and Verberkmoes show that these families of hyperelliptic curves have real multiplications by K, that is, that ± EndQ(t) Jr ᏻK . (5) ¯ Their proof shows that the endomorphisms of Jr± are in fact defined over K, and that the natural action of Gal(K/Q) on EndK(t) (Jr± ) and on ᏻK are compatible with the identification of equation (5), which is canonical (see also [DMs]). Fix a residue field F of K at a prime above p, and let ϕ be a homomorphism of ᏻK to F. The module Jr± [p] ⊗ϕ F is a 2-dimensional F-vector space on which GK acts F-linearly. By choosing an F-basis for this vector space, one obtains Galois representations (depending on the choice of ϕ, although this dependence is supressed from the notation) r± (t) : GK(t) −→ GL2 (F).


421

Theorem 1.10. The representations r− (t) and r+ (t) (as ϕ varies over the (r − 1)/2 possible homomorphisms from ᏻK to F) are the r − 1 distinct Frey representations of characteristic p associated to x p +y p = zr . The representations r− are odd, and the representations r+ are even. Proof. (See also [DMs, Prop. 2.2 and 2.3].) Observe the following. (1) Outside of t = 0, 1, ∞, the curves Cr± (t) have good reduction. Hence r± (t) satisfies condition (2) in Definition 1.1 of a Frey representation. (2) The Cr± (t) are Mumford curves over Spec(K[[t]]) and Spec(K[[t −1]]), that is, the special fiber of Cr± (t) over these bases is a union of projective lines intersecting transversally at ordinary double points. For example, replacing y by 2y +(x +2)g(x) yields the following equation for Cr+ (t) over Spec(K[[t]]), whose special fiber is the union of two projective lines crossing at the d + 1 ordinary double points (x, y) = (−2, 0), (−ωj , 0): y 2 + (x + 2)g(x)y + t (x + 2) = 0.

(6)

Likewise, replacing y by 2y + xg(−x) gives the following equation for Cr+ (t) over Spec(K[[t − 1]]): y 2 + xg(−x)y + g(−x)2 + (x + 2)(t − 1) = 0.

(7)

Its special fiber is a projective line with the d ordinary double points (x, y) = (ωj , 0). A similar analysis can be carried out for Cr− (t). By Mumford’s theory, the Jacobians Jr± (t) have purely toric reduction at t = 0 and t = 1, and hence r± maps the inertia at these points to unipotent elements of SL2 (F). (3) The curve Cr− (t) has a quadratic twist that acquires good reduction over K[[(1/t)1/r ]], while Cr+ (t) acquires good reduction over this base. For example, setting t˜ = (1/t)1/r and replacing x by 1/x and y by (2y + 1)/x (r+1)/2 in equation (4) for Cr+ (t) gives the model y 2 + y = x r + t˜h(x, y, t˜/2),

(8) r− (resp., r+ ) SL2 (F) whose

where h is a polynomial with coefficients in Z. Therefore maps the inertia at t = ∞ to an element of order 2r (resp., r) of image in PSL2 (F) is of order r. It follows from (2) and (3) that r± (t) satisfies condition (3) in Definition 1.1. (4) A strong version of condition (1) in Definition 1.1 now follows from the following group-theoretic lemma. Lemma 1.11. Let σ0 , σ1 , and σ∞ be elements of PSL2 (F) of order p, p, and r satisfying σ0 σ1 σ∞ = 1. Then σ0 , σ1 , and σ∞ generate PSL2 (F) unless (p, r) = (3, 5) and σ˜ 0 σ˜ 1 σ˜ ∞ = −1, in which case they generate an exceptional subgroup isomorphic to A5 ⊂ PSL2 (F9 ). Proof. Let G be the subgroup of PSL2 (F) generated by the images of σ0 , σ1 , and σ∞ . The proper maximal subgroups of PSL2 (F) are conjugate to one of the groups

422

HENRI DARMON

in the following list (cf., e.g., [Hu, Ch. II.8, Th. 8.27]): (1) the Borel subgroup of upper-triangular matrices; (2) the normalizer of a Cartan subgroup; (3) a group isomorphic to PSL2 (F ) or PGL2 (F ) for some F ⊂ F; (4) one of the exceptional subgroups A4 , S4 , or A5 . The fact that G contains two unipotent elements that do not commute rules out the possibility that G is contained in a Borel subgroup or in the normalizer of a Cartan subgroup, and the fact that it contains an element of order r rules out the groups isomorphic to PSL2 (F ) or PGL2 (F ). Obviously, G can be contained in one of the exceptional subgroups only if both p and r are less than or equal to 5, that is, if (r, p) = (2, 3), (2, 5), (3, 5), or (5, 3). In the first three cases, G is isomorphic to PSL2 (Fp ). (Note that PSL2 (F3 ) A4 and that PSL2 (F5 ) A5 .) When (r, p) = (5, 3) and σ˜ 0 σ˜ 1 σ˜ ∞ = −1, one checks directly that G is isomorphic to the exceptional subgroup A5 ⊂ PSL2 (F9 ). The equation x r +y r = zp . Choose a parameter j ∈ {1, 3, 5, . . . , r −2}, and define curves over the function field Q(t) by the equations j +2 − 2r 2 j −2 x − 1 , Xr,r (t) : y = u x x −u j +2 t + r 2 j −2 x − 1 Xr,r (t) : y = u x . , u= x −u t −1 A role is played in our construction by the Legendre family J (t) of elliptic curves, whose equation we write in the more convenient form: x − 1 j +2 J (t) : y 2 = u2 x j −2 . x −u These curves are equipped with the following structures. − and X + defined by (1) A canonical action of µr on Xr,r r,r ζ (x, y) = (x, ζy),

ζ ∈ µr .

− , X + , and J defined by (2) An involution τ on Xr,r r,r

τ (x, y) = (u/x, 1/y). + and has no fixed points on X − and on J . This involution has two fixed points on Xr,r r,r − − + are defined by (3) Maps π : Xr,r → J and πr : Xr,r → Xr,r π(x, y) = (x, y r ); πr (x, y) = x, y 2 .

These maps obey the rules τ ζ = ζ −1 τ,

π ζ = π,

πr ζ = ζ 2 πr ,

τ π = πτ,

τ πr = πr τ.


423

Let ± ± Cr,r = Xr,r /τ,

J = J /τ.

− to J and C + , The maps π and πr commute with τ and hence induce maps from Cr,r r,r respectively, which are denoted by the same letters by abuse of notation. Write π ∗ + , and C − induced by π and and πr∗ for the maps between the Jacobians of J , Cr,r r,r + denote the Jacobian of πr , respectively, by contravariant functoriality. Finally let Jr,r + , and let J − be the quotient of the Jacobian Jac(C − ) of C − defined by Cr,r r,r r,r r,r

− ∗ + − Jr,r := Jac Cr,r π (J ) + πr∗ Jr,r . + (resp., J − ) have dimension equal to Proposition 1.12. The abelian varieties Jr,r r,r (r − 1)/2 when j ∈ {1, 3, 5, . . . , r − 4} (resp., j ∈ {1, 3, 5, . . . , r − 2}). In these cases there is a natural identification

± EndK Jr,r = ᏻK , which is compatible with the action of Gal(K/Q) on each side. ± is a direct calculation based on Proof. The computation of the dimension of Jr,r ± , let the Riemann-Hurwitz formula. To study the endomorphism rings of Jr,r ± ± ± ηζ : Xr,r −→ Cr,r × Cr,r ± to C ± given by η := (pr, pr ◦ζ ), where pr is the be the correspondence from Cr,r ζ r,r ± ± . The resulting endomorphism η of Pic(C ± ) is natural projection of Xr,r to Cr,r ζ r,r defined (on effective divisors) by the equation

ηζ (pr P ) = pr(ζ P ) + pr(ζ −1 P ). The commutation relations between ζ , π, and πr show that π ηζ = 2π,

π r η ζ = η ζ 2 πr .

+ ) of Jac(C − ) are preserved by these correHence, the subvarieties π ∗ (J ) and πr∗ (Jr,r r,r − as well as of J + . The assignment spondences, which induce endomorphisms of Jr,r r,r ± ). It is an isomorphism since J ± has ζ → ηζ yields an inclusion of ᏻK into End(Jr,r r,r multiplicative reduction at t = ∞ and hence is not of complex multiplication (CM) type. The result follows. ± be the Galois repreChoose as before a homomorphism ϕ : ᏻK → F, and let r,r ± [p] ⊗ F. Note that sentations obtained from the action of GK(t) on the modules Jr,r ϕ ± depend on the choice of the parameter j as well as on the the representations r,r choice of ϕ.

424

HENRI DARMON

− , as j ranges over {1, 3, . . . , r −2} and Theorem 1.13. (1) The representations r,r ϕ over the different homomorphisms ᏻK → F, are the (r − 1)2 /4 distinct odd Frey representations attached to x r + y r = zp . + , as j ranges over {1, 3, . . . , r − 4} and ϕ over the (2) The representations r,r different homomorphisms ᏻK → F, are the (r − 1)(r − 3)/4 distinct even Frey representations attached to x r + y r = zp .

Proof. See, for example, [Ka, Th. 5.4.4] or [CW]. ± , as functions of the variable Remarks. (1) The periods of the abelian varieties Jr,r t, are values of certain classical hypergeometric functions. These functions arise as solutions of a second-order differential equation having only regular singularities at t = 0, 1, and ∞ and monodromies of order r at 0 and 1 and quasi-unipotent monodromy (with eigenvalue −1 for the odd Frey representation and 1 for the even Frey representation) at t = ∞. (2) Katz’s proof, which is based on his analysis of the behaviour of the local monodromy of sheaves under the operation of “convolution on Gm ,” is significantly more general than the rank 2 case used in our application. It also gives a motivic construction of rigid local systems over P1 − {0, 1, ∞} of any rank. Katz’s “hypergeometric motives” suggest the possibility of connecting equation (1) to higher-dimensional Galois representations, for which questions of modularity are less well understood. (3) In computing finer information such as the conductors of the Frey representa± (a r /c p ) at the “bad primes,” it may be desirable to have a direct proof of tions r,r Theorem 1.13 along the lines of the proof of Theorem 1.10. The details, which are omitted, will be given in [DK].

The equation x p + y q = zr . The notion of “hypergeometric abelian variety” explained in [Ka, Th. 5.4.4] and [CW, Sec. 3.3] also yields a construction of the (r − 1)(q − 1)/2 Frey representations of characteristic p over K = Q(ζq , ζr )+ associated to x p + y q = zr , when p, q, r are distinct primes and p is odd. We do not describe the construction here, referring instead to [Ka, Sec. 5.4] for the details. All that is used in the sequel is the following theorem. − and Theorem 1.14. If q, r = 2 (resp., q = 2), there exist abelian varieties Jq,r + (resp., J ) over Q(t) of dimension (r − 1)(q − 1)/2 satisfying Jq,r 2,r

± EndK Jq,r = ᏻK

(resp., End(J2,r ) = ᏻK ),

whose mod p representations give rise to all the Frey representations in characteristic p associated to x p + y q = zr . More precisely, fix a residue field F of K at p, and let ϕ be a homomorphism of Q(ζq )+ Q(ζr )+ to F. There are (r − 1)(q − 1)/4 (resp., ± (resp., (r − 1)/2) such ϕ’s. Extending ϕ to a homomorphism ᏻK −→ F, let q,r ± 2,r ) be the Galois representation obtained from the action of GK(t) on Jq,r [p] ⊗ϕ F ± (resp., ) are the distinct Frey (resp., J2,r [p] ⊗ϕ F). Then the representations q,r 2,r


425

− representations of characteristic p attached to x p +y q = zr . The representations q,r + (resp., q,r ) are odd (resp., even).

Frey abelian varieties. We may now assign to each solution (a, b, c) of equation (1) a Frey abelian variety, obtained as a suitable quadratic twist of the abelian variety ± (a r /c p ) for x r + y r = J (a p /b p ) for x p + y p = zp , Jr± (a p /cr ) for x p + y p = cr , Jr,r p ± p r p q r z , and Jq,r (a /c ) for x + y = z . These twists are chosen in such a way as to make the corresponding mod p representations as “little ramified” as possible, in accord with Lemma 1.2. The equation x p + y p = zp . If (a, b, c) is a solution to the Fermat equation x p + p p 2 p p = zr , the elliptic curve J (a √ /c ) has equation y = x(x − 1)(x − a /c ), which is a quadratic twist (over Q( c)) of the familiar Frey curve yq

J (a, b, c) : y 2 = x(x + a p )(x − b p ). Let ρ be the associated mod p representation of GQ . The equation x p + y p = zr . When r = 2, we associate to a solution (a, b, c) of equation (1) the following twist of C2 (a p /c2 ): C2 (a, b, c) : y 2 = x 3 + 2cx 2 + a p x.

(9)

When r is odd, the Frey hyperelliptic curves Cr− (a, b, c) and Cr+ (a, b, c) are given by the equations Cr− (a, b, c) : y 2 = cr f (x/c) − 2(a p − b p ), Cr+ (a, b, c) : y 2 = (x + 2c) cr f (x/c) − 2(a p − b p ) .

(10) (11)

− c) is a nontrivial quadratic twist of Cr− (a p /cr ) (over the field Note √ that Cr (a, b, + Q( c)), while Cr (a, b, c) is isomorphic to Cr+ (a p /cr ) over Q. Here are the equations of Cr− (a, b, c) for the first few values of r:

r =3:

y 2 = x 3 − 3c2 x − 2(a p − b p ).

r =5:

y 2 = x 5 − 5c2 x 3 + 5c4 x − 2(a p − b p ).

r =7:

y 2 = x 7 − 7c2 x 5 + 14c4 x 3 − 7c6 x − 2(a p − b p ).

Let Jr± (a, b, c) be the Jacobian of Cr± (a, b, c), and let ρr± be the corresponding mod p Galois representations (which depend, as always, on the choice of a homomorphism ϕ from ᏻK to F). The representation ρr± is a quadratic twist of r± (a p /cr ). ± (a, b, c) or C ± (a, b, c), as we have no We do not write down the equations for Cr,r q,r further use for them in this paper. A more careful study of the Frey abelian varieties ± (a, b, c) associated to x r + y r = zp will be carried out in [DK]. Jr,r

426

HENRI DARMON

Conductors. We say that a Galois representation ρ : GK → GL2 (F) is finite at a prime λ if its restriction to a decomposition group at λ comes from the Galois action on the points of a finite flat group scheme over ᏻK,λ . When ᐉ = p, this is equivalent to ρ being unramified. Let N(ρ) denote the conductor of ρ, as defined, for example, in [DDT]. In particular, N(ρ) is divisible precisely by the primes for which ρ is not finite. The equation x p + y p = zp . By interchanging a, b, and c and changing their signs if necessary so that a is even and b ≡ 3 (mod 4), one finds that the conductor of ρ := (a, b, c) is equal to 2 (cf. [Se2]). The presence of the extraneous prime 2 in the conductor (in spite of the fact that all the exponents involved in the Fermat equation are odd) can be explained by the fact that the Frey representation used to construct ρ is odd, so that one of the monodromies of (t) is necessarily of order 2p. In contrast, we will see that the Galois representations obtained from even Frey representations are unramified at 2. The equation x p + y p = zr . Let r = (2 − ω) be the (unique) prime ideal of K above r. Proposition 1.15. (1) The representation ρr− is finite away from r and the primes above 2. (2) The representation ρr+ is finite away from r. Proof. The discriminants 2± of the polynomials used in equations (10) and (11) to define Cr± (a, b, c) are 2− = (−1)(r−1)/2 22(r−1) r r (ab)((r−1)/2)p , 2+ = (−1)(r+1)/2 22(r+1) r r a ((r+3)/2)p b((r−1)/2)p . If 3 is a prime that does not divide 2± , then Cr± (a, b, c) has good reduction at 3; hence ρr± is finite at all primes above 3. So it is enough to consider the primes that divide 2ab. Suppose first that 3 = 2 divides a, and let λ denote any prime of K above 3. Let Kλ be the completion of K at λ and ᏻλ its ring of integers, and denote by ± ρr,λ the restriction of ρr± to an inertia group Iλ ⊂ Gal(K¯ λ /Kλ ) at λ. We observe that ± ± ρr,λ = r± (a p /cr )|Iλ , since 3 does not divide c. To study ρr,λ , we consider the abelian ± variety Jr over Kλ ((t)). Let M be the finite extension of Kλ ((t)) cut out by the Galois representation r± on the p-division points of Jr± . From the proof of Theorem 1.10, one knows that Cr± is a Mumford curve over Kλ [[t]]. Hence, its Jacobian Jr± is equipped with a (t)-adic analytic uniformization 1 −→ Q −→ T −→ Jr± Kλ ((t)) −→ 1, where T (Kλ ((t))× )d is a torus and Q is the sublattice of multiplicative periods. Hence M is contained in L((t 1/p )), where L is a finite extension of Kλ . Because


427

Jr± extends to an abelian scheme over the local ring ᏻλ ((t)), the extension L/Kλ is unramified when 3 = p and comes from a finite flat group scheme over ᏻλ when 3 = p. ± But the extension of Kλ cut out by ρr,λ is contained in L(t 1/p ), where t = a p /cr . Since ord3 (t) ≡ 0 (mod p), this extension is unramified at λ when 3 = p and comes from a finite flat group scheme over ᏻλ when 3 = p. The proof when 3 = 2 divides b proceeds in an identical manner, considering this time Cr± (t) over Kλ ((t −1)) and using the fact that ord3 ((a p /cr ) − 1) = ord3 (−b p /cr ) ≡ 0 (mod p) to conclude. Consider finally the case where 3 = 2. If 2 does not divide ab, then c is even. Making the substitution (x, y) = (1/u, (2v + 1)/u(r+1)/2 ), the equation of Cr+ (a, b, c) becomes

(a p − b p ) r u + (lower-order terms in u). 2 The coefficients involved in this equation are integral at 2, and (a p − b p )/2 is odd; hence, Cr+ (a, b, c) has good reduction at 2, and therefore, ρr+ is unramified at λ. If 2 divides ab, suppose without loss of generality that it divides a, and note that the equation (6) for Cr+ (t) also shows that Cr+ (a, b, c) is a Mumford curve over Kλ . The result follows. v 2 + v = 4c(a p − b p )ur+1 −

Remark. The reader will find in [Ell] a more general criterion for the Galois representations arising from division points of Hilbert-Blumenthal abelian varieties to be unramified, which relies on Mumford’s theory in an analogous way. Proposition 1.15 implies that the conductor of ρr+ is a power of r and that the conductor of ρr− is divisible only by r and by primes above 2. We now study the exponent of r that appears in these conductors. Proposition 1.16. (1) If r divides ab, then the conductor of ρr− and ρr+ at r divides r. (2) If r does not divide ab, then the conductor of ρr− and ρr+ at r divides r3 . Proof. We treat the case of ρr+ , since the calculations for ρr− are similar. By making the change of variable x = (2 − ω)u − 2, y = (2 − ω)d+1 v in equation (6), one finds the new equation for Cr+ : 2 − ωj t v+ u = 0. (12) u− Cr+ : v 2 + u 2−ω (2 − ω)d j

Setting t˜ = t/(2 − ω)d , one sees that Cr+ (t˜) is a Mumford curve over Spec(ᏻr[[t˜]]). (The singular points in the special fiber have coordinates given by (u, v) = (0, 0) and ((2 − ωj )/(2 − ω), 0), which are distinct since (2 − ωj )/(2 − ω) ≡ j 2 (mod r).) One concludes that when ordr(t) > d, the representation r+ (t) is ordinary at r, and its conductor divides r. When r divides a one has ordr(a p /cr ) ≥ pd > d. A similar reasoning works when r divides b, and so part (1) of Proposition 1.16 follows. Part (2) is proved by analyzing Jr± (t) over Spec(ᏻr[t, 1/(1−t), 1/t]). The conductor of Jr± over this base is constant, and one finds that the conductor of ρr± is equal to r3 .

428

HENRI DARMON

By combining the analysis of Propositions 1.15 and 1.16, we have shown the following theorem. Theorem 1.17. (1) The conductor of ρr− is of the form 2u rv , where u = 1 if ab is even. One has v = 1 if r divides ab, and v ≤ 3 otherwise. (2) The conductor of ρr+ divides r if r divides ab, and r3 otherwise. 2. Modularity 2.1. Hilbert modular forms. Let K be a totally real field of degree d > 1, and let ψ1 , . . . , ψd be the distinct real embeddings of K. They determine an embedding of the group ; = SL2 (K) into SL2 (R)d by sending a matrix ac db to the d-tuple ai bi d ci di i=1 , where aj = ψj (a) and likewise for bj , cj , and dj . Through this embedding, the group ; acts on the product Ᏼd of d copies of the complex upper half-plane by Möbius transformations. More precisely, if τ = (τ1 , . . . , τd ) belongs to Ᏼd , then a i τi + b i d . Mτ := ci τi + di i=1 If f is a holomorphic function on Ᏼd and γ ∈ GL2 (K), we define (f |2 γ )(τ ) = det(γ ) (ci τi + di )−2 f (γ τ ). Let ; be a discrete subgroup of GL2 (K). Definition 2.1. A modular form of weight 2 on ; is a holomorphic function on

Ᏼd which satisfies the transformation rule

f |2 γ = f, for all γ in ;. A function that vanishes at the cusps is called a cusp form on ;. The space of modular forms of weight 2 on ; is denoted M2 (;), and the space of cusp forms is denoted S2 (;). Let n be an ideal of K. We now introduce the space S2 (n) of cusp forms of weight 2 and level n, as in [W1, Sec. 1.1]. For this, choose a system c1 , c2 , . . . , ch of representative ideals for the narrow ideal classes of K. Let d denote the different of K, and assume that the ci have been chosen relatively prime to nd. Define

a b −1 ∈ GL+ ;i (n) := M = 2 (K) | a, d ∈ ᏻK , b ∈ (ci d) , c d c ∈ ci dn, ad − bc ∈ ᏻ× K . Definition 2.2. A cusp form of weight 2 and level n is an h-tuple of functions (f1 , . . . , fh ), where fi ∈ S2 (;i (n)).


429

Denote by S2 (n) the space of cusp forms of weight 2 and level n. To the reader acquainted with the case K = Q, the definition of S2 (n) may appear somewhat contrived. It becomes more natural when one considers the adelic interpretation of modular forms of level n as a space of functions on the coset space GL2 (AK )/GL2 (K). As in the case where K = Q, the space S2 (n) is a finitedimensional vector space and is endowed with an action of the commuting self-adjoint Hecke operators Tp for all prime ideals p of K which do not divide n (cf. [W1, Sec. 1.2]). A modular form f ∈ S2 (n) is called an eigenform if it is a simultaneous eigenvector for these operators. In that case one denotes by ap(f ) the eigenvalue of Tp acting on f . Let Kf be the field generated by the coefficients ap(f ). It is a finite totally real extension of Q. If λ is any prime of Kf , let Kf,λ be the completion of Kf at λ and let ᏻf,λ be its ring of integers. Eigenforms are related to Galois representations of GK thanks to the following theorem. Theorem 2.3. Let f be an eigenform in S2 (n). There is a compatible system of λ-adic representations ρf,λ : GK −→ GL2 (ᏻf,λ ) for each prime λ of Kf , satisfying trace ρf,λ frobq = aq(f ) ,

det ρf,λ frobq = Norm(q),

for all primes q of K which do not divide nλ. Sketch of proof. When K is of odd degree or when K is of even degree and there is at least one finite place where f is either special or supercuspidal, this follows from work of Shimura, Jacquet and Langlands, and Carayol (cf. [Ca]). In this case, the representation ρf,λ can be realized on the λ-adic Tate module of an abelian variety over K. (It is a factor of the Jacobian of a Shimura curve associated to a quaternion algebra over K which is split at exactly one infinite place.) In the general case, the theorem is due to Wiles [W2] (for ordinary forms) and to Taylor [Tay] for all f . The constructions of [W2] and [Tay] are more indirect than those of [Ca]: They exploit congruences between modular forms to reduce to the situation that is already dealt with in [Ca], but they do not realize ρf,λ on the division points of an abelian variety (or even on the étale cohomology of an algebraic variety). A different construction, by Blasius and Rogawski [BR], exhibits the Galois representations in the cohomology of Shimura varieties associated to an inner form of U (3). Let A be an abelian variety over K with real multiplications by a field E. More precisely, one requires that E is a finite extension of Q whose degree is equal to the dimension of A, and one also requires that A is equipped with an inclusion: E −→ EndK (A) ⊗ Q.

430

HENRI DARMON

Following the terminology of Ribet, call A an abelian variety of GL2 -type over K. It gives rise to a compatible system ρA,λ of 2-dimensional λ-adic representations of GK for each prime λ of E by considering the action of GK on (T3 (A) ⊗ Q3 ) ⊗E Eλ . The conductor of A is defined to be the Artin conductor of ρA,λ for any prime λ of good reduction for A. (One can show that this does not depend on the choice of λ.) The following conjecture is the natural generalization of the Shimura-Taniyama conjecture in the setting of abelian varieties of GL2 -type. Conjecture 2.4 (Shimura and Taniyama). If A is an abelian variety of GL2 -type over K of conductor n, then there exists a Hilbert modular form f over K of weight 2 and level n such that ρf,λ ρA,λ for all primes λ of E. If A satisfies the conclusion of Conjecture 2.4, one says that A is modular. Remark. To prove that A is modular, it is enough to show that it satisfies the conclusion of Conjecture 2.4 for a single prime λ of E. Conjecture 2.4 appears to be difficult in general, even with the powerful new techniques introduced by Wiles in [W3]. In connection with equation (1), one is particularly interested in Conjecture 2.4 for hypergeometric abelian varieties. Conjecture 2.5. For all t ∈ Q, the hypergeometric abelian variety J (t) (resp., ± (t), J ± (t)) attached to the equation x p + y p = zp (resp., x p + y p = zr , Jr± (t), Jr,r q,r r r x +y = zp , x p +y q = zr ) is modular over Q (resp., Q(ζr )+ , Q(ζr )+ , Q(ζq , ζr )+ ). 2.2. Modularity of hypergeometric abelian varieties The modularity of J . The modularity of the curves in the Legendre family J follows from Wiles’s proof of the Shimura-Taniyama conjecture for semistable elliptic curves. To prove that J is modular, Wiles begins with the fact that the mod 3 representation J [3] is modular. This follows from results of Langlands and Tunnell on base change; the key fact being that GL2 (F3 ) is solvable. Wiles then shows (at least when the representation J [3] is irreducible and semistable) that every “sufficiently well-behaved” lift of J [3] is also modular. This includes the representation arising from the 3-adic Tate module of J , and hence J itself is modular. ± . When r = 2, the abelian variety J is an elliptic The modularity of Jr± and Jr,r 2 curve (which arises from the universal family on X0 (2)) and its modularity follows from the work of Wiles and its extensions [Di1]. − are elliptic curves, so that Likewise, when r = 3, the abelian varieties Jr± and Jr,r their modularity follows from the Shimura-Taniyama conjecture. It is still conjectural in this case, in spite of the progress made toward the Shimura-Taniyama conjecture − in [Di1] and [CDT]: For many values of t, the conductors of J3± (t) and J3,3 (t) are divisible by 27.


431

For r > 3, the prime 3 is never split in Q(ζr )+ , so that the image of the Galois ± [3] is contained in a product of groups isomorrepresentation acting on Jr± [3] or Jr,r phic to GL2 (F3s ) with s > 1. Because GL2 (F3s ) is not solvable when s > 1, it seems ± [3] and use the prime 3 as difficult to directly prove the modularity of Jr± [3] or Jr,r in Wiles’s original strategy. Consider instead the prime r of norm r. Since GQ fixes r, it acts naturally on ± [r] of r-torsion points of J ± and J ± . Furthermore, these the modules Jr± [r] and Jr,r r r,r modules are 2-dimensional Fr -vector spaces, and the action of GQ on them is Fr linear. − [r] are isomorphic to a quadratic Theorem 2.6. (1) The modules Jr− [r] and Jr,r twist of the mod r representation associated to the Legendre family J . + [r] are reducible Galois representations. (2) The modules Jr+ [r] and Jr,r

Proof. By the same arguments as in the proof of Theorem 1.10, one shows that the − [r] (resp., J + [r] and J + [r]), if irreducible, representations attached to Jr− [r] and Jr,r r r,r are Frey representations associated to the Fermat equation x r + y r = zr which are odd (resp., even). By Theorem 1.5, there is a unique odd Frey representation (up to twisting by a quadratic character) associated to x r + y r = zr , which is the one associated to the r-torsion points on the Legendre family J (t). Part (1) follows. Since there are no even Frey representations associated to x r + y r = zr , the reducibility + [r] follows as well. (Alternately, in [DMs, Prop. 2.3], an explicit of Jr+ [r] and Jr,r r-isogeny from Jr+ (t) to Jr+ (−t) defined over K is constructed, which shows that the corresponding representation is reducible and, in fact, that Jr+ has a K-rational torsion point of order r.) ± be the conductors of the G -representations J ± [r] and J ± [r]. Let Nr± and Nr,r Q r r,r ± [r] arise from a classical Corollary 2.7. The GQ -representations Jr± [r] and Jr,r ± ± modular form f0 on ;0 (Nr ) and ;0 (Nr,r ).

Proof. Since the elliptic curve J : y 2 = x(x − 1)(x − t) is modular for all t ∈ Q, it is associated to a cusp form on ;0 (NJ ) where NJ is the conductor of J (t). The lowering-the-level result of Ribet [Ri2] ensures that there is a form f0 of level Nr− − ) attached to J − [r] (resp., J − [r]). In the case of the even Frey represen(resp., Nr,r r r,r tations, the appropriate modular form f0 can be constructed directly from Eisenstein series. ± [r] to G , Consider now the restriction of the Galois representations Jr± [r] and Jr,r K which we denote with the same symbol by abuse of notation.

Theorem 2.8. There are Hilbert modular forms f over K giving rise to Jr± [r] or

± [r]. Jr,r

Proof. This is a consequence of cyclic base change, taking f to be the base change lift of f0 from Q to K.

432

HENRI DARMON

In light of Theorem 2.8, what is needed now is a “lifting theorem” in the spirit of [TW] and [W3] for Hilbert modular forms over K, which would allow us to ± . The methods of conclude the modularity of the r-adic Tate module of Jr± and Jr,r [TW] are quite flexible and have recently been partially extended to the context of Hilbert modular forms over totally real fields by a number of mathematicians, notably Fujiwara [Fu] and Skinner and Wiles [SW1]–[SW3]. Certain technical difficulties ± in full generality. prevent one from concluding the modularity of Jr± and Jr,r (1) When r does not divide ab, the r-adic Tate module of Jr± is neither flat nor ordinary at r. One needs lifting theorems that take this into account. The work of Conrad, Diamond, and Taylor [CDT] is a promising step in this direction, but many technical difficulties remain to be resolved. Even when r = 3, one cannot yet prove − that the elliptic curves J3± (t) and J3,3 (t) are modular for all t ∈ Q. (2) The reducibility of the representation Jr+ [r] may cause some technical difficulties, although the recent results of Skinner and Wiles [SW1]–[SW3] go a long way toward resolving these difficulties in the ordinary case. As an application of the results of Skinner and Wiles, we have the following theorem. Theorem 2.9. (1) If r divides ab, then the abelian varieties Jr± (a, b, c) are modular. ± (a, b, c) are modular. (2) If r divides c, then the abelian varieties Jr,r ± (a, b, c) have multiplicative reProof. The abelian varieties Jr± (a, b, c) and Jr,r duction at r, by the proof of Proposition 1.16. Hence the r-adic Tate modules Tr± ± of these varieties, viewed as a representation of G , are ordinary at r. Since and Tr,r K + are reducible, the modularity of the residual representations attached to Tr+ and Tr,r the associated r-adic representations follows from [SW3, Sec. 4.5, Th. A]. (Note that the five hypotheses listed in this theorem are satisfied in our setting, with k = 2 and A = 1, since the field denoted there by F (χ1 /χ2 ) is equal to the cyclotomic field − , the associated residual representation is never Q(ζr ).) In the case of Tr− and Tr,r reducible when r > 5 by the work of Mazur, and the modularity of the associated r-adic representations follows from [SW2, Sec. 5, Th. 5.1]. ± . Let K = Q(ζ , ζ )+ , and let q be a prime of K above q. The modularity of Jq,r q r This prime is totally ramified in K/Q(ζr )+ . Denote by q also the unique prime of Q(ζr )+ below q, and let F be the common residue field of Q(ζr )+ and K at q. ± [q ] As in the previous section, one notes that the action of GK on the module Jq,r extends to an F-linear action of GQ(ζr )+ . ± [q] is isomorphic to a quadratic twist of J ± [q] Theorem 2.10. The module Jq,r r as a GQ(ζr )+ -module.

Proof. The proof is exactly the same as the proof of Theorem 2.6. ± [q]. Corollary 2.11. If Jr± is modular, then so is Jq,r


433

Proof. The proof is the same as for Corollary 2.7 and Theorem 2.8; this time applying cyclic base change from Q(ζr )+ to K. Modularity of J [3] | | ↓ Modularity of J ↓ Modularity of J [r] | | ↓

←−

Base change

←−

Wiles lifting

−→

x r + y r = zr

←−

Theorem 2.6 & base change

Theorem 2.6 ↓

Modularity of Jr− [r] − [r ] and Jr,r | | ↓

Modularity of Jr+ [r] + [r ] and Jr,r ←−

Generalized Wiles lifting?

−→

Modularity of Jr− − and Jr,r

Modularity of Jr+ + and Jr,r

↓

↓ Jr− [q]

Modularity of − [q ] and Jr,r | | ↓

−→ ←−

xq + yq

xr + yr

= zr , = zq

Theorem 2.10 & base change

←−

Modularity of Jr+ [q] + [q ] and Jr,r

−→

| | ↓

− [q ] Modularity of Jq,r

| | ↓

+ [q ] Modularity of Jq,r

←−

Generalized Wiles lifting?

−→

− Modularity of Jq,r

| | ↓ + Modularity of Jq,r

↓ Modularity of

| | ↓

↓ − [p ] Jq,r

−→

xp + yq

= zr

←−

+ [p ] Modularity of Jq,r

Figure 1

Corollaries 2.7 and 2.11 suggest an inductive strategy for establishing the modu± , and J ± : combining a series of base changes with successive larity of J , Jr± , Jr,r q,r

434

HENRI DARMON

applications of Wiles-type lifting theorems (at the last step, for Hilbert modular forms over Q(ζq , ζr )+ ). This strategy, and its connections with Fermat’s equation and its variants, is summarized in Figure 1. 3. Lowering the level 3.1. Ribet’s theorem. Let ρ : GK → GL2 (F) be a Galois representation of GK with values in GL2 (F), where F is a finite field. If f is a Hilbert modular form that is an eigenform for the Hecke operators, denote by ᏻf the ring generated by the associated eigenvalues. Definition 3.1. We say that ρ is modular if there exists a Hilbert modular form f over K and a homomorphism j : ᏻf → F such that, for all primes q that are unramified for ρ, trace ρ frobq = j aq(f ) . If f can be chosen to be of weight k and level n, we say that ρ is modular of weight k and level n. The following is a generalization of Serre’s conjectures [Se2] to totally real fields, in a simple special case. Conjecture 3.2. Suppose that ρ : GK −→ GL2 (F) is an absolutely irreducible Galois representation, where F is a finite field of characteristic p. Suppose also that (1) ρ is odd, and its determinant is the cyclotomic character; (2) ρ is finite at all primes p dividing p; (3) the conductor of ρ in the sense of [Se2] is equal to n. Then ρ is modular of weight 2 and level n. This conjecture also seems quite difficult. (For example, the argument in [Se2, Sec. 4, Th. 4] shows that Conjecture 3.2 implies the generalized Shimura-Taniyama Conjecture 2.4.) The following conjecture, which extends a result of Ribet [Ri2] to totally real fields, should be more approachable. Conjecture 3.3. Suppose that ρ satisfies the assumptions of Conjecture 3.2 and that it is modular of weight 2 and some level. Then ρ is modular of weight 2 and level n. The following partial result is proved in [Ja] and [Ra], building on the methods of [Ri2]. Theorem 3.4. Let ρ : GK −→ GL2 (F) be an irreducible mod p representation associated to a Hilbert cuspidal eigenform f of weight 2 and level nλ, where n, λ,


435

and p are mutually relatively prime and λ is a prime of K. If [K : Q] is even, assume that f is either special or supercuspidal at a finite prime q not dividing p and λ. If ρ is unramified at λ, then ρ comes from a Hilbert cuspidal eigenform g of weight 2 and level n. 3.2. Application to x p + y p = zr . In the remainder of this paper, we focus our attention on the equation x p + y p = zr and attack it by studying the representations ρr+ = + (a p /cr ) (and, toward the end, ρr− ) attached to the p-torsion of Jr± (a, b, c). Theorem 3.5. (1) If r divides ab, then ρr+ (resp., ρr− ) comes from a modular form of weight 2 and level dividing r (resp., 2u r, for some u). (2) If r does not divide ab, assume further that Jr± (t) is modular and that Conjecture 3.3 holds for Hilbert modular forms over K. Then ρr+ (resp., ρr− ) comes from a modular form of weight 2 and level dividing r3 (resp., 2u r3 , for some u). Proof. The modularity of Jr± (a, b, c) (which, when r divides ab, follows from Theorem 2.9) implies that ρr± is modular of weight 2 and some level. By Theorem 1.17, ρr+ has conductor dividing r when r | ab and dividing r3 in general, and it satisfies all the other hypotheses in Conjecture 3.2; a similar statement holds for ρr− . Conjecture 3.3 implies the conclusion. Note that, when r | ab, the Hilbert modular form f associated to Jr+ (a, b, c) is special or supercuspidal at r, so that the hypotheses of Theorem 3.4 are satisfied. Hence, Theorem 3.4 can be applied to remove all the unramified primes from the level of the associated modular form, proving part (1) of Theorem 3.5 unconditionally. Remark. Theorem 3.5 suggests that the analysis of the solutions (a, b, c) to x p + y p = zr splits naturally into two cases, depending on whether or not r divides ab. The following definition is inspired by Sophie Germain’s classical terminology. Definition 3.6. A primitive solution (a, b, c) of x p + y p = zr is called a first case solution if r divides ab, and a second case solution otherwise. Remark. As with Fermat’s last theorem, the first case seems easier to deal with than the second case (cf. Theorem 3.22). Our hope is that Theorem 3.5 forces the image of ρr± to be small (at least for some values of r). Before pursuing this matter further, observe that equation (1) has (up to sign) three trivial solutions: (0, 1, 1), (1, 0, 1), and (1, −1, 0). Proposition 3.7. (1) If (a, b, c) = (0, 1, 1) or (1, 0, 1), then Jr+ and Jr− have degenerate reduction, and the representations ρr± are therefore reducible. (2) If (a, b, c) = (1, −1, 0), then Jr± have complex multiplication by Q(ζr ), and hence the image of ρr± is contained in the normalizer of a Cartan subgroup of GL2 (F). Proof. This can be shown by a direct calculation. For example, the curve Cr+ (1, −1, 0) has equation

436

HENRI DARMON

y 2 = x r+1 − 4x. Making the substitution (x, y) = (−1/u, (2v +1)/u(r+1)/2 ), one obtains the equation v 2 + v = ur , and one recognizes this as the equation for the hyperelliptic quotient of the Fermat curve x r + y r = zr that has complex multiplication by Q(ζr ). Proposition 3.7 suggests the following question. Question 3.8. Can one show that the image of ρr± (a, b, c) is necessarily contained in a Borel subgroup or in the normalizer of a Cartan subgroup of GL2 (F)? The case r = 2 and 3. For r = 2 (resp., r = 3) one can answer this question in the affirmative, by noting that ρ2 (a, b, c) (resp., ρ3+ (a, b, c)) is modular of level dividing 32 (resp., 27). (One needs to assume the Shimura-Taniyama conjecture for r = 3.) The space of classical cusp forms of weight 2 and level 32 (resp., 27) is 1-dimensional. In fact, X0 (32) (resp., X0 (27)) is an elliptic curve with complex multiplication by Q(i) (resp., Q(ζ3 )). (It is also a quotient of the Fermat curve x 4 + y 4 = z4 (resp., x 3 + y 3 = z3 ).) So the Galois representations arising from nontrivial primitive solutions of x p + y p = z2 and x p + y p = z3 are either reducible or of dihedral type. This was proved in [Da1] (see also [DMr]). Answering Question 3.8, for specific values of r > 3 and p, requires a computation of all the Hilbert modular forms over K of weight 2 and level dividing r3 . We limit ourselves to the simpler case where K has narrow class number 1. Remark. It is known that K has narrow class number 1 for all r < 100 except r = 29, when the narrow class number is equal to 8. (The author is grateful to Cornelius Greither for pointing out these facts.) We now give a formula for the dimension of S2 (1) and S2 (r k ), with k = 1, . . . , 3 under the narrow class number 1 assumption. To do this we need to introduce some notation. • Recall that d = (r − 1)/2 denotes the degree of K over Q. • Set δ2 = 2 if r ≡ 1 (mod 4) and δ2 = 0 if r ≡ 3 (mod 4). Likewise let δ3 = 2 if r ≡ 1 (mod 3) and δ3 = 0 if r ≡ 2 (mod 3). • Let ζK (s) be the Dedekind zeta-function of K. The main contribution to the dimension of S2 (r k ) is given by the special value ζK (−1), a rational number that can be computed from the formula: (−1)d B2,χ ζK (−1) = , 12 χ 2

r

B2,χ

1 = χ(a)a 2 , r a=1

where the product is taken over all nontrivial even Dirichlet characters χ : (Z/rZ)× / !±1" → C× of conductor r.


437

• Let h− be the minus part of class number of Q(ζr ). This number can be evaluated also as a product of generalized Bernoulli numbers: h− = (−1)d 2r

B1,χ χ

2

r

,

B1,χ =

1 χ(a)a, r a=1

where the product this time is taken over the odd Dirichlet characters √ of conductor r. • Let h(a) be the class number of the quadratic extension K( a), and (for d < 0) √ × × √ let q(a) be the index of ᏻ× K ᏻQ( a) in ᏻK ( a) . One has q(a) = 1 or 2, and q(a) = 1 if r ≡ 3 (mod 4). Only the ratios h(−1)/q(−1) and h(−3)/q(−3) are involved in the formula for the dimension of S2 (r k ). Let χ4 and χ3 denote the nontrivial Dirichlet character mod 4 and 3, respectively. When K has narrow class number 1, these ratios are given by the formulae B1,χχ h(−1) 4 = (−1)d+1 , q(−1) 2 χ

B1,χχ h(−3) 3 = (−1)d+1 , q(−3) 2 χ

where the products are taken over the nontrivial even Dirichlet characters of conductor r. Recall that B1,χχ4

4r 1 = aχχ4 (a), 4r

B1,χχ3

a=1

3r 1 = aχχ3 (a). 3r a=1

Table 1 lists these invariants for the first few values of r. Table 1

Let

r

d

ζK (−1)

h−

5

2

1/30

1

1

1

7 11 13 17 19

3 5 6 8 9

−1/21 −20/33 152/39 18688/51 −93504/19

1 1 1 1 1

1 1 3 8 19

1 1 2 5 9

h(−1)/q(−1) h(−3)/q(−3)

χ (n) = 1 + (−1)d dim S2 (n) .

Under the assumption that K has narrow class number 1, this is the arithmetic genus of the Hilbert modular variety Ᏼd / ;0 (n); cf. [Fr, Ch. II, Sec. 4, Th. 4.8]. Theorem 3.9. Assume that K has narrow class number 1. Then χ(r k ) (and hence, the dimension of S2 (r k )) is given by the formulae

438

HENRI DARMON

χ (1) =

h(−3) ζK (−1) r − 1 − h(−1) h + + , + d−1 2r 4q(−1) 3q(−3) 2

χ (r) = (r + 1)

ζK (−1) r − 1 − h(−1) h(−3) h + δ2 + δ3 , + 2r 4q(−1) 3q(−3) 2d−1

ζK (−1) h(−1) h(−3) χ r k = r k−1 (r + 1) d−1 + δ2 + δ3 . 4q(−1) 3q(−3) 2 Proof. The formula for χ (1) is given in [We, Th. 1.14 and 1.15]. A routine calculation then yields the formula for χ(r k ), after noting that (1) an elliptic fixed point of order 2 (resp., 3) on Ᏼd for the action of SL2 (ᏻK ) lifts to δ2 (resp., δ3 ) elliptic fixed points on Ᏼd / ;0 (r k ) for k ≥ 1; (2) an elliptic fixed point of order r lifts to a unique elliptic fixed point modulo ;0 (r), and there are no elliptic fixed points of order r on Ᏼd / ;0 (r k ) when k > 1. Noting that K has narrow class number 1 when r < 23, Theorem 3.9 allows us to compute the dimensions for the relevant spaces of cusp forms (see Table 2). Table 2 r

dim S2 (1) dim S2 (r) dim S2 r2 dim S2 r3

5

0

0

0

2

7 11 13 17 19

0 0 1 6 12

0 1 4 55 379

1 6 24 879 7300

5 56 290 14895 138790

The case r = 5 and√7. When r = 5, the action of the Hecke operators on the spaces S2 (n) over K = Q( 5) can be calculated numerically by exploiting the JacquetLanglands correspondence between forms on GL2 (K) and on certain quaternion algebras. Let B be the (unique, up to isomorphism) totally definite quaternion algebra over K which is split at all finite places. The algebra B can be identified with the standard Hamilton quaternions over K, since 2 is inert in K:

√ B = x + yi + zj + wk, x, y, z, w ∈ Q 5 . The class number of B is equal to 1: The maximal orders in B are all conjugate to the ring of icosians 1 1 R = Z ω, i, j, k, (1 + i + j + k), (i + ωj + ωk) ¯ , 2 2


439

whose unit group R × is isomorphic to the binary icosahedral group of order 120 (cf., e.g., [CS, Ch. 8, Sec. 2.1]). Let Rn be an Eichler order of level n in R, and write ˆ Rˆ n := Rn ⊗ Z,

ˆ Bˆ = B ⊗ Z.

The Jacquet-Langlands correspondence shows that S2 (n) is isomorphic as a Hecke module to the space L2 Rˆ n× \ Bˆ × /B × , on which the Hecke operators act in the standard way. Table 3 lists the eigenvalues of the Hecke operators Tp acting on S2 (r3 ), for all the primes p of K of norm less than or equal to 50.√It turns out that the two eigenforms in S2 (r3 ) are conjugate to each other over Q( 5), so we have only displayed the eigenvalues of one of the two eigenforms. Table 3 p

(2)

ap(f ) 0

(3) 0

p

(6 + ω)

ap(f )

0

(3 − ω) (4 + ω) √ √ (−1 − 5 5)/2 (−1 + 5 5)/2

(4 − ω)

(5 + ω)

(5 − ω)

0

0

0

(7 + 2ω) (5 − 2ω) (7 + ω) (6 − ω) √ √ √ √ (−11 + 5 5)/2 (−11 − 5 5)/2 (9 + 5 5)/2 (9 − 5 5)/2

(7) 0

Observe that ap(f ) = 0 for all the primes p that are inert in the quadratic extension Q(ζ5 )/Q(ω). This suggests that f is actually of CM type, and it corresponds to an abelian variety of dimension 2 with complex multiplication by Q(ζ5 ). In fact, this can be proved: The abelian variety J5+ (1, −1, 0) = Jac y 2 + y = x 5 has complex multiplication by Q(ζ5 ), and its Hasse-Weil L-function is a product of Hecke L-series attached to Grössen characters of Q(ζ5 ) of conductor (1 − ζ5 )2 . + A direct √ 3 calculation shows √ that J5 (1, −1, 0) is associated to the two eigenforms in S2 ( 5 ) over GL2 (Q( 5)). When r = 7, we did not carry out a numerical investigation of the Hecke eigenforms of level r2 and r3 , but this turns out to be unnecessary in identifying the modular forms that arise in these levels. Let A be the (unique, up to isogeny) √ elliptic curve over Q of conductor 49, which has complex multiplication by Q( −7). It corresponds to a cusp form over Q of level 49. Its base change lift to K = Q(cos(2π/7)) is the unique modular form of level r2 . The space S2 (r3 ) contains a 2-dimensional space of old forms, and hence there are three eigenforms of level r3 . These must consist of the Hilbert modular forms associated to the Fermat quotient J7+ (1, −1, 0) : y 2 + y = x 7 .

440

HENRI DARMON

So when r = 5 and 7, the spaces S2 (r3 ) contain only eigenforms of CM type associated to hyperelliptic Fermat quotients or CM elliptic curves. Hence, we have the following theorem. Theorem 3.10. Let r = 5 or 7, and let (a, b, c) be a nontrivial primitive solution to the equation x p + y p = zr , where p = r is an odd prime. Let p be any prime of K = Q(cos(2π/r)) above p, and write F := ᏻK /p. Then we have the following. (1) If (a, b, c) is a first case solution, the mod p representation associated to Jr+ (a, b, c) is reducible. (2) If (a, b, c) is a second case solution, assume further that Jr+ (a, b, c) is modular and that Ribet’s lowering-the-level theorem (see Conjecture 3.3) holds for Hilbert modular forms over K. Then the mod p representation associated to Jr+ (a, b, c) is either reducible or its image is contained in the normalizer of a Cartan subgroup of GL2 (F). Following [Se1], one can use the fact that Jr+ is semistable to obtain more precise information in the first case. Proposition 3.11. If r = 5 or 7 and (a, b, c) is a first case solution to x p + y p = then Jr+ (a, b, c) is Q-isogenous to an abelian variety having a rational point of order p. zr ,

Proof. Choose a prime p of K above p, and let χ1 : GK −→ F× be the character giving the action of GK on the K-rational 1-dimensional F-vector subspace L of Jr+ [p]. Let χ2 be the character of GK describing its action on Jr+ [p]/L. The local analysis in [Se1, Sec. 5.4., Lem. 6] shows that χ1 and χ2 are unramified outside the primes above p. Also, the set of restrictions {χ1 |Ip , χ1 |Ip } to an inertia group Ip at a prime p above p is equal to {χ, 1}, where χ is the cyclotomic character giving the action of Ip on the pth roots of unity. (Use the corollary to Proposition 13 of [Se1].) Hence, one of χ1 or χ2 is everywhere unramified. (When there is a single prime of K above p, this is immediate. If p is split in K, one observes, by analyzing the image × of the map ᏻ× K → (ᏻK ⊗ Fp ) and using class field theory, that the inertia groups at the various p , in the Galois group of the maximal tamely ramified abelian extension of K unramified outside p, have nontrivial intersection and, in fact, are equal for all but finitely many p.) Since K has class number 1, one of χ1 or χ2 is trivial. If χ1 = 1, then Jr+ [p] has a K-rational point whose trace gives a point of order p in Jr+ (a, b, c). If χ2 = 1, the module L˜ generated by the ᏻK [GQ ]-translates of L is a Q-rational subgroup of Jr+ [p] which is of rank 1 over ᏻK ⊗ Fp . The quotient Jr+ /L˜ has a rational point of order p. Corollary 3.12. If 3 is a prime satisfying 3 #Jr (F3 ). This contradicts Proposition 3.11,


441

since the prime-to-3 part of the torsion subgroup of Jr+ (Q) injects into Jr+ (F3 ) (and likewise for any abelian variety isogenous to Jr+ ). Theorem 3.13. Suppose r = 5 or 7. There exists a constant Cr− depending only on r such that, if p ≥ Cr− and (a, b, c) is a first case solution to x p + y p = zr , the Galois representation ρr− is reducible. (In this case, there is a quotient of Jr− (a, b, c) over Q which has a rational point of order p.) Proof. By Corollary 3.12, if p is large enough, then 2 divides ab, so that the abelian variety Jr− (a, b, c) is semistable at 2 and, hence, everywhere (see the proof of Proposition 1.15). The mod p representation associated to Jr− (a, b, c), if irreducible, is therefore equal to the mod p representation associated to a Hilbert modular form f over K in S2 (2r), by Theorem 3.5. Corollary 3.12 further implies that if 3 ≤ p1/d − 2p 1/2d + 1 is a rational prime, then 3 divides ab, so that Jr− (a, b, c) has multiplicative reduction at any prime λ of K above 3. By using the Tate uniformization of Jr− (a, b, c) at λ, we find that aλ (f ) ≡ norm(λ) + 1 (mod p),

for all 3 ≤ p 1/d − 2p 1/2d + 1.

For each f , there is a constant Cf− such that this statement fails whenever p > Cf− , since the mod p representations attached to f are irreducible for almost all p. Now take Cr− to be the maximum of the Cf− as f runs over the normalized eigenforms in S2 (2r). The statement in parentheses follows by applying to Jr− the same arguments used in the proof of Proposition 3.11. Remark. Although the statement of Theorem 3.13 involves only ρr− , note the crucial role played in its proof by the representation ρr+ via Corollary 3.12. This illustrates how information gleaned from one Frey representation may sometimes be used to yield insights into a second a priori unrelated Frey representation associated to the same generalized Fermat equation. The case r = 11. When r = 11, there is a 44-dimensional space of newforms of level r3 , and studying the equation x p +y p = z11 would require computing the Fourier coefficients associated to these newforms. We content ourselves with the following result, which requires only dealing with S2 (r). Theorem 3.14. Let (a, b, c) be a first case solution to the equation x p +y p = z11 , where p > 19 is prime, and let p be any prime of K = Q(cos(2π/11)) above p. Then + + the mod p representation associated to J11 (a, b, c) is reducible, and in fact J11 (a, b, c) has a rational point of order p. Proof. Let p be any ideal of K above p, and let ρp denote the mod p representation + associated to J11 (a, b, c). Suppose that it is irreducible. By exploiting the action of Gal(K/Q), it follows that ρp is irreducible for all choices of p. Theorem 3.5 implies that ρp is modular of level dividing r. Table 2 shows that the space of cusp forms of this level is 1-dimensional. In fact, the unique normalized eigenform f of level r is

442

HENRI DARMON

the base change lift to K = Q(cos(2π/11)) of the modular form f = η(z)2 η(11z)2 of level 11 associated to the elliptic curve X0 (11). Consider the prime ideal (2) of K above 2, of norm 32. Then a(2) (f) = a32 (f ) = 8. This implies that

+ a(2) := a(2) J11 (a, b, c) ≡ 8 (mod p)

for all primes p above p. Taking norms, one finds p 5 divides normK/Q (a(2) − 8). By the Weil bounds, we have √ 5 | normK/Q (a(2) − 8)| ≤ 2 32 + 8 . √ Since p > 20 > 8(1 + 2), we must have a(2) = 8. But this leads to a contradiction. + For, if 2 divides ab, then J11 (a, b, c) has purely toric reduction at 2 and a(2) = ±1. If + ab is odd, then J11 (a, b, c) has good reduction at (2), and 11 divides norm(32 + 1 − + a(2) ) = 255 since J11 (a, b, c) has a K-rational point of order 11 (by Theorem 2.6). It + follows that the mod p representations associated to J11 are reducible. The proof of + Proposition 3.11 now shows that J11 (Q) has a point of order p, since Q(cos(2π/11)) has class number 1. Corollary 3.15. If 3 is a prime satisfying 3 < p 1/5 − 2p1/10 + 1, then 3 divides ab. The proof of this corollary is the same as for Corollary 3.12. Finally, we record the following theorem. − − such that, if p ≥ C11 and (a, b, c) Theorem 3.16. There exists a constant C11 − p p 11 is a first case solution to x + y = z , the Galois representation ρ11 is reducible. − (In this case, there is a quotient of J11 (a, b, c) over Q which has a rational point of order p.)

The proof is the same as for Theorem 3.13. The case r = 13. When r = 13 there is a unique normalized cusp form of level 1, which is the base change lift of the cusp form associated to the elliptic curve X1 (13). (Note that this curve acquires good reduction over Q(cos(2π/13)).) This modular form does not pose any obstructions to studying first case solutions to x p +y p = z13 , since the representation attached to a solution of the equation is ramified at r. On the other hand, the 2-dimensional space of newforms of level r would have to be studied more carefully in order to understand the (first case) solutions to x p +y p = z13 . The numerical calculation of eigenforms in S2 (r) becomes increasingly difficult as


443

r gets larger, and it has not been carried out even for r = 13. One can go further without such explicit numerical calculations (cf. Theorem 3.22) by studying congruences (modulo r) for modular forms. General r. Let 3 be a rational prime. The 3-adic Tate module T3 (Jr± (t)) ⊗ Q3 is a 2-dimensional K3 := K ⊗ Q3 -vector space. When t is rational, the linear action of GK on this vector space extends to a GQ -action that is GK -semilinear; that is, it satisfies σ (αv) = α σ σ (v), for all α ∈ K3 , σ ∈ GQ . Letting aq(Jr± ) := trace(ρJ,3 (frobq)), it follows that σ aq Jr± = aqσ Jr± .

(13)

This motivates the following definition. Definition 3.17. A Hilbert modular form over K of level n is called a Q-form if for all ideals q of K which are prime to n, it satisfies the relation aq(f )σ = aqσ (f ),

for all σ ∈ GQ .

(In particular, this implies that the Fourier coefficients aq(f ) belong to K.) Equation (13) implies the following lemma, which reflects the fact that the abelian varieties Jr± (t) with t ∈ Q are defined over Q (even though their endomorphism rings are only defined over K). Lemma 3.18. For all t ∈ Q, if the abelian varieties Jr− (t) and Jr+ (t) are modular, then they are associated to a modular Q-form over K. Let f be an eigenform in S2 (n), and let λ be a prime in the ring of Fourier coefficients ᏻf . Denote by ρf,λ the λ-adic representation associated to f by Theorem 2.3 and let V be the underlying Kf,λ -vector space. Choose a GK -stable ᏻf,λ -lattice H in ¯ := H/λH gives a 2-dimensional representation ρ¯f,λ for GK over the V . The space H residue field kf,λ := ᏻf,λ /λ. In general, this representation depends on the choice of lattice, but its semisimplification does not. One says that ρf,λ is residually irreducible if ρ¯f,λ is irreducible for some (and hence all) choices of lattice H. Otherwise one says that ρf,λ is residually reducible. In the latter case, the semisimplification of ρ¯f,λ is a direct sum of two 1-dimensional characters × , χ1 , χ2 : GK −→ kf,λ

whose product is the cyclotomic character × χ : GK −→ (Z/3Z)× ⊂ kf,λ

giving the action of GK on the 3th roots of unity.

444

HENRI DARMON

Let f be a Q-form over K in the sense of Definition 3.17, so that in particular its Fourier coefficients are defined over K. We say that f is r-Eisenstein if its associated r-adic representation ρf,r is residually reducible. Proposition 3.19. There exists a constant Cr+ depending only on r such that, for any first case solution (a, b, c) to equation (1) with p > Cr+ , one of the following holds: (1) the representation ρr+ is reducible, or (2) it is isomorphic to the mod p representation attached to an r-Eisenstein Q-form in S2 (r). Proof. Let g be any eigenform in S2 (r). If g is not a Q-form, then there exists a prime q of ᏻg and a σ ∈ GQ such that aq(g)σ = aqσ (g). If g is a Q-form but is not r-Eisenstein, then there is a prime q of K such that r does not divide aq(g) − N q − 1. In either case, one has aq(g) = aq(f ), for all modular forms f that correspond to a Jr+ (t) with t ∈ Q. Indeed, such an f is a Q-form and is r-Eisenstein by Theorem 2.6. If f ≡ g for some prime p of ᏻg K above p, then taking norms gives p divides NormKg K/Q aq(g) − aq(f ) = 0. Let dg := [Kg : Q]. Applying the Weil bounds, one finds NormK K/Q aq(g) − aq(f ) ≤ 16 NormK/Q (q) (r−1)dg /4 , g so that

(r−1)dg /4 p ≤ Cg := 16 NormK/Q (q) .

In particular, if p > Cg , the representation ρr+ is not equivalent to ρ¯g,p for any prime of ᏻg K above p. Now set Cr+ := maxg Cg , where the maximum is taken over all eigenforms g in S2 (r) which are either not Q-forms or are not r-Eisenstein. If p > Cr+ and (a, b, c) is a first case solution to x p + y p = zr and if the associated representation ρr+ is irreducible, then it is associated by Theorem 3.5 to a Hilbert modular eigenform in S2 (r). This form must be an r-Eisenstein Q-form by the choice of Cr+ . In light of Proposition 3.19, it becomes important to understand whether there exist r-Eisenstein Q-forms in S2 (r). Proposition 3.20. Suppose that r is a regular prime. Then there are no rEisenstein Q-forms over K of level 1 or r. Proof. Suppose on the contrary that f is a Q-form in S2 (r) and that ρf,r is residually reducible. Let χ1 and χ2 be the characters of GK which occur in the ¯ for some (and hence all) GK -stable lattices H in V . Because semisimplification of H,


445

f is a Q-form, it follows that χ1 and χ2 are powers of the cyclotomic character χ with values in !±1" ⊂ F× r . Furthermore, χ1 χ2 = χ. Hence, we may assume without loss of generality that χ1 = 1 and χ2 = χ. By [Ri1, Prop. 2.1], there exists a GK -stable lattice H for which χ A0 1 A0 , (14) ρ¯f,r 1 = 0 χ2 0 χ and it is not semisimple. This implies that A := A0 /χ is a nontrivial cocycle in H 1 (K, Z/rZ(−1)). Proposition 3.20 now follows from the next lemma. Lemma 3.21. The cocycle A is unramified. Proof. The cocycle A is unramified at all places v = r because v does not divide the level of f . It is also unramified at r: If f is of level 1, this is because ρ¯f,r comes from a finite flat group scheme over K. If f is of level r, then, by [W2, Th. 2], the restriction of the representation ρ¯f,r to a decomposition group Dr at r is of the form χ A . ρ¯f,r|Dr 0 1 But the restriction of χ to Dr is nontrivial. Comparing the equation above to equation (14), it follows that the local representation ρ¯f,r|Dr splits. Therefore, the cocycle A is locally trivial at r. This completes the proof of Lemma 3.21. Proposition 3.20 now follows directly: The cocycle A cuts out an unramified cyclic extension of Q(ζr ) of degree r, which does not exist if r is a regular prime. Theorem 3.22. Let r be a regular prime. Then there exists a constant Cr+ (depending only on r) such that, for all p > Cr+ and all first case solutions (a, b, c) to x p + y p = zr , the mod p representation associated to Jr+ (a, b, c) is reducible. Proof. Combine Propositions 3.19 and 3.20. Remarks. (1) The value of the constant Cr+ depends on the structure of the space of Hilbert modular forms over Q(cos(2π/r)) of level r. It would be possible in principle to write down a crude estimate for Cr+ by using the Chebotarev density theorem and known estimates for the size of fourier coefficients of Hilbert modular eigenforms, but we have not attempted to do this. (2) The consideration of r-Eisenstein Q-forms so crucial for the proof of Theorem 3.22 is only likely to be of use in studying first case solutions. Indeed, there typically exist r-Eisenstein Q-forms on S2 (r3 ); for example, the base change lifts from Q to K of certain r-Eisenstein forms on X0 (r 2 ) or (more germane to the present discussion) the form in S2 (r3 ) associated to the CM abelian variety Jr+ (1, −1, 0). (3) The arguments based on r-Eisenstein Q-forms yield no a priori information about the Galois representations ρr− , since the mod r representation attached to Jr− (a, b, c) is irreducible. (It is isomorphic to a twist of the representation coming

446

HENRI DARMON

from the r-torsion of the Frey curve y 2 = x(x − a p )(x + b p ), by Theorem 2.6.) Nonetheless, one can still show the following theorem. Theorem 3.23. Assume further that K has class number 1. Then (1) Jr+ (a, b, c) is isogenous to an abelian variety having a rational point of order p; (2) there exists a further constant Cr− such that if p > Cr− , the abelian variety Jr− (a, b, c) is isogenous to an abelian variety having a rational point of order p. Proof. The proof of (1) is the same as for Proposition 3.11, and (2) follows from the same reasoning as for Theorem 3.13. 4. Torsion points on abelian varieties. Ultimately, one wishes to extract a contradiction from theorems like Theorems 3.10, 3.13, 3.14, 3.16, 3.22, and 3.23 by proving that when p is large enough (relative to r perhaps), the image of ρr± is large; for example, that this image contains SL2 (F) or, at the very least, that the abelian varieties Jr± (a, b, c), when semistable, cannot contain a rational point of order p. The following folklore conjecture can be viewed as a direct generalization of a conjecture of Mazur for elliptic curves. Conjecture 4.1. Let E be a totally real field and K a number field. There exists a constant C(K, E) depending only on K and E, such that for any abelian variety A of GL2 -type with EndK (A) ⊗ Q = EndK¯ (A) ⊗ Q E, and all primes p of E of norm greater than C(K, E), the image of the mod p representation associated to A contains SL2 (F). This conjecture seems difficult. The set of abelian varieties of GL2 -type with End(A) ⊗ Q E is parametrized by a d-dimensional Hilbert modular variety, and very little is known about the Diophantine properties of these varieties. When r = 2 and r = 3, one has K = E = Q since the representations ρr± arise from elliptic curves. Much of Conjecture 4.1 can be proved thanks to the ideas of Mazur [Ma1], [Ma2]. • Theorem 8 of [Ma1] implies that the image of ρr± is not contained in a Borel subgroup of GL2 (Fp ) when p > 5. • A result of Momose [Mo] building on the ideas in [Ma1] implies that this image is not contained in the normalizer of a split Cartan subgroup if p > 17. • Finally, a result of Merel and the author [DMr] implies that the image of ρr+ is not contained in the normalizer of a nonsplit Cartan subgroup. (We were unable to prove a similar result for ρr− .) Combining these results with an ad hoc study (carried out by Bjorn Poonen [Po], using traditional descent methods) of the equations x p + y p = zr (r = 2, 3) for small values of p yields the desired contradiction. Thus, the main result of [DMr] provides an (essentially) complete analogue of Fermat’s last theorem for equation (1) when


447

r = 2 or 3, which one would like to emulate for higher values of r. Theorem 4.2 [DMr]. (1) The equation x p + y p = z2 has no nontrivial primitive solutions when p ≥ 4. (2) Assume the Shimura-Taniyama conjecture. Then the equation x p +y p = z3 has no nontrivial primitive solutions when p ≥ 3. Return now to the case r > 3. The following special case of Conjecture 4.1, which is sufficient for the applications to equation (1), seems more tractable. Conjecture 4.3. There exists a constant Br depending only on r, such that for any t ∈ Q and all primes p of K = Q(ζr )+ of norm greater than Br , the image of the mod p representation of GK associated to Jr± (t) is neither contained in a Borel subgroup nor in the normalizer of a Cartan subgroup of GL2 (F). A natural approach to this conjecture is to study the curves X0± (p), Xs± (p), and which classify the abelian varieties Jr± (t) with a rational subgroup, a “normalizer of split Cartan subgroup structure,” and a “normalizer of nonsplit Cartan subgroup structure” on the p-division points, where p is an ideal of the field K. For the moment, we know very little about the arithmetic of these curves, except when r = 2 and r = 3 when they are closely related to classical modular curves. When r > 3, they appear as quotients of the upper half-plane by certain nonarithmetic Fuchsian groups described in [CW]. We content ourselves here with giving a formula for the genus of these curves. Let J = ±1 be defined by the condition Np ≡ J (mod r). ± (p ) Xns

Lemma 4.4. (1) The genus of X0± (p) is equal to 1 1 2 1 J 1− − Np − 1− . 2 r p 2 r (2) The genus of Xs± (p) is equal to 1 1 2 1 J +1 1− − Np(Np + 1) − 1− + 1. 4 r p 4 r ± (p) is equal to (3) The genus of Xns 1 1 2 1 J −1 1− − Np(Np − 1) + 1− + 1. 4 r p 4 r

Proof. The curves above are branched coverings of the projective line with known degrees and ramification structure. The calculation of the genus follows by a direct application of the Riemann-Hurwitz genus formula. Example. When r = 5 and p = (3), one finds that the curves X0− (3) and X0+ (3) are of genus 1; that is, they are elliptic curves over Q. A direct calculation reveals that X0+ (3) is an elliptic curve of conductor 15, denoted by 15E in Cremona’s tables. √ By looking up the curve 15E twisted by Q( 5), one finds that 15E has finite

448

HENRI DARMON

√ Mordell-Weil group over Q( 5). Does J0+ (p) always have a nonzero quotient with finite Mordell-Weil group over Q(ζr )+ , at least when p is large enough? References [Be] [By] [BR] [Ca] [CW] [CDT] [CCNPW]

[CS] [Da1] [Da2] [DDT]

[DK] [DMr] [DMs] [Di1] [Di2] [Ell] [Fr] [Fre] [Fu] [He]

[Hu] [Ja]

S. Beckmann, On extensions of number fields obtained by specializing branched coverings, J. Reine Angew. Math. 419 (1991), 27–53. G. V. Bely˘i, On extensions of the maximal cyclotomic field having a given classical Galois group, J. Reine Angew. Math. 341 (1983), 147–156. D. Blasius and J. D. Rogawski, Motives for Hilbert modular forms, Invent. Math. 114 (1993), 55–87. H. Carayol, Sur les représentations 3-adiques associées aux formes modulaires de Hilbert, Ann. Sci. École Norm. Sup. (4) 19 (1986), 409–468. P. Cohen and J. Wolfart, Modular embeddings for some nonarithmetic Fuchsian groups, Acta Arith. 56 (1990), 93–110. B. Conrad, F. Diamond, and R. Taylor, Modularity of certain potentially Barsotti-Tate Galois representations, J. Amer. Math. Soc. 12 (1999), 521–567. J. H. Conway, R. T. Curtis, S. P. Norton, R. A. Parker, and R. A. Wilson, Atlas of Finite Groups: Maximal Subgroups and Ordinary Characters for Simple Groups, Clarendon Press, Oxford, 1985. J. H. Conway and N. J. A. Sloane, Sphere Packings, Lattices and Groups, 2d ed., Grundlehren Math. Wiss. 290, Springer, New York, 1993. H. Darmon, The equations x n +y n = z2 and x n +y n = z3 , Internat. Math. Res. Notices 1993, 263–274. , Faltings plus epsilon, Wiles plus epsilon, and the generalized Fermat equation, C. R. Math. Rep. Acad. Sci. Canada 19 (1997), 3–14. H. Darmon, F. Diamond, and R. Taylor, “Fermat’s last theorem” in Current Developments in Mathematics (Cambridge, Mass., 1995), International Press, Cambridge, Mass., 1994, 1–154. H. Darmon and A. Kraus, On the equations x r + y r = zp , in preparation. H. Darmon and L. Merel, Winding quotients and some variants of Fermat’s last theorem, J. Reine Angew. Math. 490 (1997), 81–100. H. Darmon and J-F. Mestre, Courbes hyperelliptiques à multiplications réelles et une construction de Shih, CICMA preprint; to appear in Canad. Math. Bull. F. Diamond, On deformation rings and Hecke rings, Ann. of Math. (2) 144 (1996), 137–166. , The Taylor-Wiles construction and multiplicity one, Invent. Math. 128 (1997), 379–391. J. Ellenberg, Hilbert modular forms and the Galois representations associated to Hilbert-Blumenthal abelian varieties, Ph.D. thesis, Harvard University, 1998. E. Freitag, Hilbert Modular Forms, Springer, Berlin, 1990. G. Frey, “Links between solutions of A−B = C and elliptic curves” in Number Theory (Ulm, 1987), Lecture Notes in Math. 1380, Springer, New York, 1989, 31–62. K. Fujiwara, Deformation rings and Hecke algebras in the totally real case, preprint. E. Hecke, Die eindeutige Bestimmung der Modulfunktionen q-ter Stufe durch algebraische Eigenschaften, Math. Ann. 111 (1935), 293–301; in Mathematische Werke, 3d ed., Vandenhoeck & Ruprecht, Göttingen, 1983, 568–576. B. Huppert, Endliche Gruppen, I, Grundlehren Math. Wiss. 134, Springer, Berlin, 1967. F. Jarvis, On Galois representations associated to Hilbert modular forms, J. Reine Angew. Math. 491 (1997), 199–216.

HILBERT MODULAR FORMS AND FERMAT’S LAST THEOREM [Ka] [Ma1] [Ma2] [Me]

[Mo] [Po] [Ra] [Ri1] [Ri2] [Se1] [Se2] [Se3] [Se4] [SW1] [SW2] [SW3] [TTV] [Tay] [TW] [Vi] [We] [W1] [W2] [W3]

449

N. Katz, Exponential Sums and Differential Equations, Ann. of Math. Stud. 124, Princeton Univ. Press, Princeton, 1990. B. Mazur, Modular curves and the Eisenstein ideal, Inst. Hautes Études Sci. Publ. Math. 47 (1977), 33–186. , Rational isogenies of prime degree, Invent. Math. 44 (1978), 129–162. J.-F. Mestre, “Familles de courbes hyperelliptiques à multiplications réelles” in Arithmetic Algebraic Geometry (Texel, 1989), Progr. Math. 89, Birkhäuser, Boston, 1991, 193–208. F. Momose, Rational points on the modular curves Xsplit (p), Compositio Math. 52 (1984), 115–137. B. Poonen, Some Diophantine equations of the form x n + y n = zm , Acta Arith. 86 (1998), 193–205. A. Rajaei, On levels of mod 3 Hilbert modular forms, Ph.D. thesis, Princeton University, 1998. K. Ribet, A modular construction of unramified p-extensions of Q(µp ), Invent. Math. 34 (1976), 151–162. ¯ , On modular representations of Gal(Q/Q) arising from modular forms, Invent. Math. 100 (1990), 431–476. J.-P. Serre, Propriétés galoisiennes des points d’ordre fini des courbes elliptiques, Invent. Math. 15 (1972), 259–331. , Sur les représentations modulaires de degré 2 de Gal(Q/Q), Duke Math. J. 54 (1987), 179–230. , Topics in Galois Theory, Res. Notes Math. 1, Jones and Bartlett, Boston, 1992. , Galois Cohomology, rev. ed., Springer, Berlin, 1997. C. Skinner and A. Wiles, Ordinary representations and modular forms, Proc. Nat. Acad. Sci. U.S.A. 94 (1997), 10520–10527. , Nearly ordinary deformations of irreducible residual representations, to appear. , Residually reducible representations and modular forms, to appear. W. Tautz, J. Top, and A. Verberkmoes, Explicit hyperelliptic curves with real multiplication and permutation polynomials, Canad. J. Math. 43 (1991), 1055–1064. R. Taylor, On Galois representations associated to Hilbert modular forms, Invent. Math. 98 (1989), 265–280. R. Taylor and A. Wiles, Ring-theoretic properties of certain Hecke algebras, Ann. of Math. (2) 141 (1995), 553–572. M.-F. Vignéras, Arithmétique des algèbres de quaternions, Lecture Notes in Math. 800, Springer, Berlin, 1980. D. Weisser, The arithmetic genus of the Hilbert modular variety and the elliptic fixed points of the Hilbert modular group, Math. Ann. 257 (1981), 9–22. A. Wiles, On p-adic representations for totally real fields, Ann. of Math. (2) 123 (1986), 407–456. , On ordinary λ-adic representations associated to modular forms, Invent. Math. 94 (1988), 529–573. , Modular elliptic curves and Fermat’s last theorem, Ann. of Math. (2) 141 (1995), 443–551.

Department of Mathematics and Statistics, McGill University, 805 Sherbrooke Street West, Montréal, Québec H3A 2K6, Canada; [email protected]

Vol. 102, No. 3


© 2000

ON FAMILY RIGIDITY THEOREMS, I KEFENG LIU and XIAONAN MA 0. Introduction. Let M, B be two compact smooth manifolds, and let π : M → B be a submersion with compact fiber X. Assume that a compact Lie group G acts fiberwise on M, that is, the action preserves each fiber of π. Let P be a family of elliptic operators along the fiber X, commuting with the action of G. Then the family index of P is (0.1)

Ind(P ) = Ker P − Coker P ∈ KG (B).

Note that Ind(P ) is a virtual G-representation. Let chg (Ind(P )) with g ∈ G be the equivariant Chern character of Ind(P ) evaluated at g. In this paper, we first prove a family fixed-point formula that expresses chg (Ind(P )) in terms of the geometric data on the fixed points X g of the fiber of π. Then by applying this formula, we generalize the Witten rigidity theorems and several vanishing theorems proved in [Liu3] for elliptic genera to the family case. Let G = S 1 . A family elliptic operator P is called rigid on the equivariant Chern character level with respect to this S 1 -action, if chg (Ind(P )) ∈ H ∗ (B) is independent of g ∈ S 1 . When the base B is a point, we recover the classical rigidity and vanishing theorems. When B is a manifold, we get many nontrivial higher-order rigidity and vanishing theorems by taking the coefficients of certain expansion of chg . For the history of the Witten rigidity theorems, we refer the reader to [T], [BT], [K], [L2], [H], [Liu1], and [Liu4]. The family vanishing theorems that generalize those vanishing theorems in [Liu3], which in turn give us many higher-order vanishing theorems in the family case. In a forthcoming paper, we extend our results to general loop group representations and prove much more general family vanishing theorems that generalize the results in [Liu3]. We believe there should be some applications of our results to topology and geometry, which we hope to report on a later occasion. This paper is organized as follows. In Section 1, we prove the equivariant family index theorem. In Section 2, we prove the family rigidity theorem. In the last part of Section 2, motivated by the family rigidity theorem, we state a conjecture. In Section 3, we generalize the family rigidity theorem to the nonzero anomaly case. As corollaries, we derive several vanishing theorems. Acknowledgements. We would like to thank W. Zhang for many interesting discussions. Part of this work was done while the second author was visiting the Institut des Hautes Études Scientifiques. He would like to thank Professor J. P. Bourguignon and IHES for their hospitality. Received 23 April 1999. 1991 Mathematics Subject Classification. Primary 14D05; Secondary 57R91. 451

452

LIU AND MA

1. Equivariant family index theorem. The purpose of this section is to prove an equivariant family index theorem. As pointed out by Atiyah and Singer, we can introduce equivariant families by proceeding as in [AS1] and [AS2]. Here we prove it directly by using the local index theory as developed by Bismut. This section is organized as follows: In Section 1.1, we state our main result, Theorem 1.1. In Section 1.2, by using the local index theory, we prove Theorem 1.1. 1.1. The index bundle. Let M, B be two compact manifolds, let π : M → B be a fibration with compact fiber X, and assume that dim X = 2k. Let T X denote the relative tangent bundle. Let W be a complex vector bundle on M and let hW be a Hermitian metric on W . Let hT X be a Riemannian metric on T X and let ∇ T X be the corresponding LeviCivita connection on T X along the fiber X. Then the Clifford bundle C(T X) is the bundle of Clifford algebras over M whose fiber at x ∈ M is the Clifford algebra C(Tx X) of (T X, hT X ). We assume that the bundle T X is spin as a bundle on M. Let = + ⊕ − be the spinor bundle of T X. We denote by c(·) the Clifford action of C(T X) on . Let ∇ be the connection on induced by ∇ T X . Let ∇ W be a Hermitian connection on (W, hW ) with curvature R W . Let ∇ ⊗W be the connection on ⊗W along the fiber X: ∇ ⊗W = ∇ ⊗ 1 + 1 ⊗ ∇ W .

(1.1)

For b ∈ B, we denote by Eb , E±,b the set of Ꮿ∞ -sections of ⊗W , ± ⊗W over the fiber Xb . We regard the Eb as the fiber of a smooth Z2 -graded infinite-dimensional vector bundle over B. Smooth sections of E over B are identified to smooth sections of ⊗ W over M. Let {ei } be an orthonormal basis of (T X, hT X ); let {ei } be its dual basis. Definition 1.1. Define the twisted Dirac operator to be c(ei )∇e⊗W (1.2) . DX = i i

DX

Then is a family Dirac operator that acts fiberwise on the fibers of π. For b ∈ B, DbX , denote the restriction of D X to the fiber Eb . D X interchanges E+ and E− . Let X be the restrictions of D X to E . By Atiyah and Singer [AS2], the difference D± ± bundle over B, X X Ind D X = Ker D+,b (1.3) − Ker D−,b , is well defined in the K-group K(B). Now let G be a compact Lie group that acts fiberwise on M. We consider that G acts as identity on B. Without loss of generality, we can assume that G acts on (T X, hT X ) isometrically. We also assume that the action of G lifts to and W , and that the G-action commutes with ∇ W .

ON FAMILY RIGIDITY THEOREMS, I

453

In this case, we know that Ind(D X ) ∈ KG (B). Now we start to give a proof of a local family fixed-point formula that extends [AS2, Proposition 2.2]. with j = 1, . . . , r, a finite number of sections Proposition 1.1. There exist Vj ∈ G (sij +1 , . . . , sij +1 ) with ij +1 −ij = dim Vj of Ꮿ∞ (B, E− ) such that we can find a basis {ej,l } of Vj , under which the map D +,b : Ꮿ∞ (B, E+,b ) ⊕ ⊕rj =1 Vj → Ꮿ∞ (B, E−,b ) given by X X (1.4) D +,b s + "j,l λj,l ej,l = D+ s + "j,l λj,l sij +l X

is G-equivariant and surjective. The vector spaces Ker D +,b form a G-vector bundle X X Ker D + on B, and the element [Ker D + ] − ⊕rj =1 Vj ∈ KG (B) depends only on D X and not on the choice of {Vj } and the sections {si }. Proof. Given b0 ∈ B, we can find a > 0 and a ball U (b0 ) ⊂ B around b0 , such that for any b ∈ U (b0 ), a is not an eigenvalue of DbX,2 . [0,a[ [0,a[ ⊕E−b be the direct sum of the eigenspaces of DbX,2 associated Let Eb[0,a[ = E+b to the eigenvalues λ ∈ [0, a[. By [BeGeV, Proposition 9.10], E [0,a[ forms a finitedimensional subbundle E [0,a[ ⊂ E over U (b0 ). Clearly, E [0,a[ is a G-vector bundle on U (b0 ). By [S, Proposition 2.2], we have an isomorphism of vector bundles on B, [0,a[ E [0,a[ = ⊕V ∈G (1.5) ⊗V, Hom G V , E denotes the space of all irreducible representations of G. We can also find where G [0,a[ ti,k ∈ Ꮿ∞ (U (b0 ), HomG (V , E− )) such that for b ∈ U (b0 ), the elements ti,l form a [0,a[ basis of HomG (V , E− )b . Let {ei,l } be a basis of Vi . Then we can choose the sections [0,a[ ti,k ei,l ∈ Ꮿ∞ (B, E− ) to be our si . This proves the first part of the proposition locally. The global version now follows easily by extending the above local sections of Ꮿ∞ (U (b0 ), E− ) together with a use of the partition of unity argument. This is essentially the same as the proof of [AS2, Proposition 2.2]. By [S, Proposition 2.2], we have X (1.6) ⊗V Ind D X = ⊕V ∈G Hom G V , Ind D and HomG (V , Ind(D X )) ∈ K(B). We denote by (Ind(D X ))G ∈ K(B) the G-invariant part of Ind(D X ). By composing the action of G and the Chern character of HomG (V , Ind(D X )), we get the equivariant Chern character chg (Ind(D X )) ∈ H ∗ (B). Definition 1.2. We say that the operator D X is rigid on the equivariant Chern character level if chg (Ind(D X )) is constant on g ∈ G. More generally, we say that D X is rigid on the equivariant K-Theory level if Ind(D X ) = (Ind(D X ))G . In the rest of this paper, when we say that D X is rigid, we always mean that D X is rigid on the equivariant Chern character level.

454

LIU AND MA

Now let us calculate the equivariant Chern character chg (Ind(D X )) in terms of the fixed-point data of g. Let T H M be a G-equivariant subbundle of T M such that T M = T H M ⊕ T X.

(1.7)

Let P T X denote the projection from T M to T X. If U ∈ T B, let U H denote the lift of U in T H M, so that π∗ U H = U . Let hT B be a Riemannian metric on B, and assume that W has the Riemannian metric hT M = hT X ⊕ π ∗ hT B . Note that our final results are independent of hT B . Let ∇ T M , ∇ T B denote the corresponding Levi-Civita connections on M and B. Put ∇ T X = P T X ∇ T M , which is a connection on T X. As shown in [B1, Theorem 1.9], ∇ T X is independent of the choice of hT B . Now the connection ∇ T X is well defined on T X and on M. Let R T X be the corresponding curvature. We denote by ∇ and ∇ ⊗W the corresponding connections on and ⊗W induced by ∇ T X and ∇ W . Take g ∈ G and set (1.8)

M g = {x ∈ M, gx = x}.

Then π : M g → B is a fibration with compact fiber X g . By [BeGeV, Proposition 6.14], T X g is naturally oriented in M g . Let N denote the normal bundle of M g ; then N = T X/T Xg . We denote the differential of g by dg, which gives a bundle isometry dg : N → N. Since g lies in a compact abelian Lie group, we know that there is an orthogonal decomposition N = N(π ) ⊕ ⊕0 dim X.

588

EDIDIN AND GRAHAM

Proof. By the generalization of [Se, Theorem 6.1] to algebraic spaces, there is a finite cover f : X → X on which G acts freely. Since G acts freely, CH ∗G (X ) is generated by invariant cycles (see [EG2, Proposition 8]). On the other hand, the proper pushforward f∗ : CH ∗G (X ) → CH ∗G (X) is surjective because f is finite and surjective, and we are using rational coefficients. Therefore, CH ∗G (X) is generated by invariant cycles. The preceding proposition implies that the equivariant Chow groups are complete. Using the fact that I and IG generate the same topology (see Corollary 6.1), we can restate the Riemann-Roch theorem. Corollary 5.1. Suppose that X is separated and G acts with finite stabilizers. There is a map ∼ τ : GG (X) −→ GG (X)IG −→ CH ∗G (X) satisfying properties (a)–(e) of Theorem 3.1. Remark 5.2. When G acts properly on a separated scheme X with reduced stabilizers, then Vistoli [Vi] asserts the existence of a map τX : GG (X) −→ CH ∗ [X/G] ; here [X/G] is the Deligne-Mumford quotient stack. By [EG2, Proposition 14], CH ∗ [X/G] = CH ∗G (X). Vistoli noted that his map need not be an isomorphism, and he made a conjecture about its kernel (see [Vi, Conjecture 2.4]). The conjecture states that if α ∈ ker τX , then ξ α = 0, where ξ is the class of some perfect complex with everywhere nonzero rank. We expect that his map is the same as ours in this case. Unfortunately, because he did not write his map down and did not state whether it satisfied properties (d)(i) and (e) of Theorem 3.1, we cannot positively assert this. However, for our map τX , Vistoli’s conjecture is true and his statement can be refined. Corollary 5.2 (Vistoli’s conjecture). We have α ∈ ker τXG : GG (X) −→ CH ∗ ([X/G]) if and only if there exists a virtual representation 3 ∈ R(G) of nonzero rank such that 3α = 0. Proof. The kernel is just the kernel of the localization map GG (X) → GG (X)IG . 5.1. The case G is diagonalizable. By Remark 5.1, GG (X) is supported as an R(G)-module at a finite number of primes, each of which is maximal. Denote these

589

RIEMANN-ROCH FOR EQUIVARIANT CHOW GROUPS

ideals by P = P0 , P1 , . . . , Pk . Following [Seg], each prime Pi corresponds to a finite subgroup (called the support of Pi ) Hi ⊂ G. It is defined as the minimal element of the set of subgroups H ⊂ G such that Pi ∈ I m(Spec R(H ) → Spec R(G)). Note that different Pi ’s may have the same support. This definition makes sense for any group G, but Hi is only defined up to conjugation as a subgroup of G. In our case, G is abelian so the Hi ’s are uniquely determined. Following [Tho5, Lemma 1.1 and Proposition 1.2], we give an explicit construction of the support H of a prime ideal P ⊂ R(G). Since G is diagonalizable, R(G) = Z[N], where N is a finitely generated abelian group without p-torsion (where p = char k). Given a prime P ⊂ R(G), set KP = {n ∈ N | 1 − n ∈ P }. The equivalence of categories between finitely generated abelian groups (without ptorsion) and diagonalizable groups (see [Bor, Section 8]) means that quotient N/KP determines a unique subgroup H ⊂ G with the property that R(H ) = Z[N/KP ]. When P is maximal, KP has finite index in N and H is a finite group. The representation ring of the quotient G/H is the subring Z[KP ] of Z[N ] = R(G). If I is the augmentation ideal of R(G/H ), then this construction shows that I = P ∩ R(G/H ). Denote by X i the subscheme fixed by Hi . Theorem 5.2. Suppose that G is diagonalizable and acts on a separated space X with finite stabilizers. Then GG (X)Pi CH ∗G/Hi (X i ) ⊗R(G/Hi ) R(G)Pi . In particular, there is an isomorphism CH ∗G/Hi (X i ) ⊗R(G/Hi ) R(G)Pi . GG (X) i

Proof. By the localization theorem for diagonalizable group schemes, GG (X)Pi i Pi (see [Tho5, Theorem 2.1]). Since Hi ⊂ G acts trivially on X , we have

GG (X i )

GG (X i ) GG/Hi (X i ) ⊗R(G/Hi ) R(G) (see [Tho2, Lemma 5.6]). Let I i be the augmentation ideal of G/Hi . Since we have Pi ∩ R(G/Hi ) = I i , G/H i G/H i i (X ) ⊗ i (X ) ⊗ G R(G/Hi ) R(G) P = G I i R(G/Hi ) R(G) P . i

i

Thus, by the Riemann-Roch isomorphism of Corollary 5.1, we have G/H i ∗ i i (X ) ⊗ G I i R(G/Hi ) RG P CH G/Hi (X ) ⊗R(G/Hi ) RG P . i

i

590

EDIDIN AND GRAHAM

∗ ∗ i (Here the Chern character ch : R(G/Hi ) → ∞ i=0 AG/Hi makes CH G/Hi (X ) into an R(G/Hi )-module.) The first statement follows. As noted in Remark 5.1, there is an ideal J ⊂ R(G) such that R(G)/J is supported at a finite number of points and J GG (X) = 0. Hence, GG (X) GG (X) ⊗R(G) R(G)/J. Then J = Q1 ∩ Q2 ∩ · · · ∩ Qk with Qi a Pi -primary ideal. By the Chinese remainder theorem, r r R(G)/Qi = R(G)/J P R(G)/J i=1

so

GG (X)

i=1

i

GG (X)Pi

i

and the second statement follows. Remark 5.3. This result was first obtained by Angelo Vistoli (unpublished). It is also related to the Riemann-Roch theorem for algebraic stacks proved by Toen [To]. 6. More on completions. There are other natural completions of GG (X) and CH ∗G (X) besides those of Section 2. The purpose of this section is to prove that the different definitions give isomorphic completions. As an application of these results, we prove a special case of a conjecture of Köck. To begin, fix an embedding of G into GLn . Then GG (X) is an R(GLn )-module. G (X) was defined to be the completion of GG (X) along the augRecall that G ∗ (X) is the completion of CH ∗ (X) along the mentation ideal I of R(GLn ), and CH G G ∗ augmentation ideal of AG (pt). We refer to these as the “point completions” because they are defined using ideals in the equivariant groups of a point. Let IX ⊂ K G (X) denote the augmentation ideal, that is, the ideal of virtual vector G (X) denote the completion of GG (X) along I . Let bundles of rank zero, and let G X ∗ (X) denote the completion ∗ JX ⊂ AG (X) denote the augmentation ideal, and let CH G of CH ∗G (X) along JX . We refer to these as the “X-completions” because they are defined using ideals in the equivariant groups of X. In the proof below, it is necessary to distinguish between ideals corresponding to different groups. We use a subscript to indicate this, for example, IX,G ⊂ K G (X). Note that the X-completions only depend on G and X, while a priori, the point completions depend in addition on an embedding of G into GLn . Corollary 6.1 implies that the point completion is independent of the embedding. The main result of this section is that the point and X-completions are isomorphic. Theorem 6.1. (a) The I -adic and IX -adic topologies on GG (X) coincide. Hence, we have an isomorphism of completions G (X) G G (X). G


591

(b) The J -adic and JX -adic topologies on CH ∗G (X) coincide. Hence, we have an isomorphism of completions ∗ (X) CH ∗ (X). CH G G

Proof. We only prove (a); the proof of (b) is similar. To show that the filtrations induced by powers of the ideals I and IX induce the same topology, we must check two things. First, we must show that for any n, there exists an r such that I r GG (X) ⊆ IXn GG (X). This is clear because under the map R(GLn ) → K G (X), the image of I is contained in IX , so we can take r = n. Second, we must show that for any n, there exists an r such that IXr GG (X) ⊆ I n GG (X). As above, we do this in steps: first for G = B the group of upper triangular matrices, then for G = GLn , and finally for arbitrary G. Suppose then that B ⊂ GLn is the group of upper triangular matrices. We use the notation of the proof of Theorem 2.2. We know there exists m such that ker km ⊆ IBn GB (X). Since IB and I generate the same topology on R(B), we can assume ker km ⊆ I n GB (X). So we must show that there exists r such that IXr GB (X) ⊆ ker km , that is, such that IXr GB (X × Um ) = 0. Since B acts freely on X × Um , we have GB (X × Um ) G(X ×B Um ). Under this isomorphism, IXr GB (X × Um ) ⊆ ar G X ×B Um , where ar denotes the augmentation ideal of K(X ×B Um ). By Lemma 2.4, we have ar G(X ×B Um ) = 0 for r 0. The analogous statement for Chow groups, that JXr CH ∗B (X × Um ) = 0 for r > dim(X ×B Um ), also holds. Assume now that G = GLn . Then we have GB (X) GG (G/B × X)

o

/ GG (X),

where the two maps are i! and i ! . We have proved that there exists r such that r GB (X) ⊂ I n GB (X). IX,B

Hence,

r i! IX,B GB (X) ⊂ i! I n GB (X) = I n GG (X),

where the last equality follows by the projection formula. Now, we also have r r GG (X) ⊂ IX,B GB (X). i ! IX,G Combining these facts, we see that r GG (X) ⊂ I n GG (X). i! i ! IX,G

592

EDIDIN AND GRAHAM

r GG (X) ⊂ I n GG (X). Since i! i ! is the identity, IX,G Finally, consider the case where G → GLn is any subgroup. By Proposition 3.2, there is an isomorphism of R(G)-modules GG (X) GGLn GLn ×G X .

Under this isomorphism I n GG (X) corresponds to I n GGLn (GLn ×G X). Moreover, r GG (X) corresponds to I r IX,G GGLn (GLn ×G X). This follows from the GLn ×G X,GLn fact that the equivalence of categories obtained from descent of (GLn ×G)-equivariant sheaves on GLn ×X takes locally free sheaves to locally free sheaves of the same rank (see [EGAIV, Proposition 2.5.2]). Because of these correspondences, the theorem follows from the case where G = GLn . Corollary 6.1. If H ⊂ G, then the topology on R(H ) induced by the ideal IG R(H ) is the same as the IH -adic topology. Proof. By embedding H ⊂ G ⊂ GLn , it suffices to prove the result for G ⊂ GLn . The corollary now follows by applying the theorem when X = pt. Remark 6.1. In characteristic zero this is the same as [Seg, Corollary 3.9], but in characteristic p the result is new. For a large class of group schemes over perfect fields Thomason [Tho1, Corollary 3.3] showed that IG -adic and IGLn -adic topologies are the same on the mod l ν equivariant G-theory localized at the Bott element. G Observe that the higher equivariant K-groups GG i (X) (resp., Ki (X)) are also G modules over R(G) and K (X). Thus we can define completions of these groups with respect to the ideals IX and IG . If X is a regular scheme, then KiG (X) = GG i (X) and we can prove a corollary about higher K-theory as well. Part (b) proves a conjecture of Köck [Ko, Conjecture 5.6] for regular schemes over fields.

G G Corollary 6.2. (a) If X is a regular G-scheme, then K i (X) Ki (X). (b) Let f : Y → X be an equivariant proper morphism of regular G-schemes. Then the pushforward f∗ : KiG (Y ) → KiG (X) induces a map of completions G G f∗ : K i (Y ) −→ Ki (X).

Proof. The action of R(G) on KiG (X) factors through the map R(G) → K G (X). Since K G (X) = GG (X), Theorem 6.1 implies that the ideals IX ⊂ K G (X) and the G G IG K G (X) generate the same topology. Thus, K i (X) Ki (X), which proves (a). By the projection formula applied to the commutative triangle /X } } } }} }~ } pt

Y


593

we have f∗ (I k KiG (Y )) = I k f∗ KiG (Y ) ⊂ I k KiG (X). Hence, f∗ is continuous with respect to the I -adic topology, proving (b). Remark 6.2. In its full form, Köck’s conjecture asserts that if G/S is a flat group scheme and if X → Y is any equivariant projective local complete intersection morphism, then there is a pushforward G G f∗ : K i (X) −→ Ki (Y )

of completions. This conjecture is quite subtle because (despite the suggestive notation) the completions are taken with respect to different ideals, and if X and Y are not regular, there is no obvious way of comparing the topologies. Remark 6.3. The K0 version of Köck’s conjecture has been proved [CEPT] for finite group schemes acting on regular projective varieties over rings of integers of number fields. References [AS] [BFM] [Bor] [Bo]

[Br] [CEPT]

[E]

[EG1] [EG2] [F] [FL] [Gi] [SGA]

[EGAII]

M. Atiyah and G. Segal, Equivariant K-theory and completion, J. Differential Geom. 3 (1969), 1–18. P. Baum, W. Fulton, and R. MacPherson, Riemann-Roch for singular varieties, Inst. Hautes Études Sci. Publ. Math. 45 (1975), 101–145. A. Borel, Linear Algebraic Groups, 2d ed., Grad. Texts in Math. 126, Springer, New York, 1991. R. Bott, “The index theorem for homogeneous differential operators” in Differential and Combinatorial Topology (A Symposium in Honor of Marston Morse), Princeton Math. Ser. 27, Princeton Univ. Press, Princeton, 1965, 167–186. M. Brion, Equivariant Chow groups for torus actions, Transform. Groups 2 (1997), 225–267. T. Chinburg, B. Erez, G. Pappas, and M. J. Taylor, Riemann-Roch type theorems for arithmetic schemes with a finite group action, J. Reine Angew. Math. 489 (1997), 151–187. D. Edidin, “Notes on the construction of the moduli space of curves” to appear in Recent Progress in Intersection Theory (Bologna, 1997), ed. A. Vistoli, Birkhäuser, Boston. D. Edidin and W. Graham, Characteristic classes in the Chow ring, J. Algebraic Geom. 6 (1997), 431–443. , Equivariant intersection theory, Invent. Math. 131 (1998), 595–634. W. Fulton, Intersection Theory, Ergeb. Math. Grenzgeb. (3) 2, Springer, Berlin, 1984. W. Fulton and S. Lang, Riemann-Roch Algebra, Grundlehren Math. Wiss. 277, Springer, New York, 1985. H. Gillet, Intersection theory on algebraic stacks and Q-varieties, J. Pure Appl. Algebra 34 (1984), 193–240. A. Grothendieck, Revêtements étales et groupe fondamental, Séminaire de Géométrie Algébrique du Bois-Marie (SGA1), Lecture Notes in Math. 224, Springer, Berlin, 1971. A. Grothendieck and J. Dieudonné, Éléments de géométrie algébrique, II: Étude globale élémentaire de quelques classes de morphismes, Inst. Hautes Études Sci. Publ. Math. 8 (1961).

594 [EGAIV] [Kn] [Ko] [MFK] [Seg] [Ser] [Se] [Tho1] [Tho2] [Tho3]

[Tho4] [Tho5] [To] [T] [Vi]

EDIDIN AND GRAHAM , Éléments de géométrie algébrique, IV: Étude locale des schémas et des morphismes de schémas, II, Inst. Hautes Études Sci. Publ. Math. 24 (1965). D. Knutson, Algebraic Spaces, Lecture Notes in Math. 203, Springer, Berlin, 1971. B. Köck, The Grothendieck-Riemann-Roch theorem for group scheme actions, Ann. Sci. École Norm. Sup. (4) 31 (1998), 415–458. D. Mumford, J. Fogarty, and F. Kirwan, Geometric Invariant Theory, 3d ed., Ergeb. Math. Grenzgeb. (2) 34, Springer, Berlin, 1994. G. Segal, The representation ring of a compact Lie group, Inst. Hautes Études Sci. Publ. Math. 34 (1968), 113–128. J.-P. Serre, Groupes de Grothendieck des schémas en groupes réductifs déployés, Inst. Hautes Études Sci. Publ. Math. 34 (1968), 37–52. C. S. Seshadri, Quotient spaces modulo reductive algebraic groups, Ann. of Math. (2) 95 (1972), 511–556; Errata, Ann. of Math. (2) 96 (1972), 599. R. Thomason, Comparison of equivariant and topological K-theory, Duke Math. J. 53 (1986), 795–825. , Lefschetz-Riemann-Roch theorem and coherent trace formula, Invent. Math. 85 (1986), 515–543. , “Algebraic K-theory of group scheme actions” in Algebraic Topology and Algebraic K-Theory (Princeton, 1983), Ann. of Math. Stud. 113, Princeton Univ. Press, Princeton, 1987, 539–563. , Equivariant algebraic vs. topological K-homology Atiyah-Segal-style, Duke Math. J. 56 (1988), 589–636. , Une formule de Lefschetz en K-théorie équivariante algébrique, Duke Math. J. 68 (1992), 447–462. B. Toen, Riemann-Roch theorems for Deligne-Mumford stacks, preprint, http://xxx.lanl. gov/abs/math.AG/9803076. B. Totaro, The Chow ring of the symmetric group, to appear in Contemp. Math. A. Vistoli, “Equivariant Grothendieck groups and equivariant Chow groups” in Classification of Irregular Varieties (Trento, 1990), Lecture Notes in Math. 1515, Springer, Berlin, 1992, 112–133.

Edidin: Department of Mathematics, University of Missouri, Columbia, Missouri 65211, USA; [email protected] Graham: Department of Mathematics, University of Georgia, Boyd Graduate Studies Research Center, Athens, Georgia 30602, USA; [email protected]

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

Recommend Documents