No title

DUKE MATHEMATICAL JOURNAL c 2002 Vol. 111, No. 1, POTENTIAL ESTIMATES FOR A CLASS OF FULLY NONLINEAR ELLIPTIC EQUATION...

Author: Jonathan Wahl (Academic Editor)

11 downloads 536 Views 4MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

DUKE MATHEMATICAL JOURNAL c 2002 Vol. 111, No. 1,

POTENTIAL ESTIMATES FOR A CLASS OF FULLY NONLINEAR ELLIPTIC EQUATIONS DENIS A. LABUTIN

Abstract We study the pointwise properties of k-subharmonic functions, that is, the viscosity subsolutions to the fully nonlinear elliptic equations Fk [u] = 0, where Fk [u] is the elementary symmetric function of order k, 1 ≤ k ≤ n, of the eigenvalues of D 2 u , F1 [u] = 1u, Fn [u] = det D 2 u. Thus 1-subharmonic functions are subharmonic in the classical sense; n-subharmonic functions are convex. We use a special capacity to investigate the typical questions of potential theory: local behaviour, removability of singularities, and polar, negligible, and thin sets, and we obtain estimates for the capacity in terms of the Hausdorff measure. We also prove the Wiener test for the regularity of a boundary point for the Dirichlet problem for the fully nonlinear equation Fk [u] = 0. The crucial tool in the proofs of these results is the Radon measure Fk [u] introduced recently by N. Trudinger and X.-J. Wang for any k-subharmonic u. We use ideas from the potential theories both for the complex Monge-Ampère and for the p-Laplace equations. Contents 1. Introduction . . . . . . . . . . . . . . . . 2. Estimates in terms of the Wolff potential . . 3. Local behaviour of k-subharmonic functions 4. Capacity and exceptional sets . . . . . . . . 5. Metric estimates for capacity . . . . . . . . 6. Thinness and the Wiener test . . . . . . . . References . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 4 15 21 34 38 47

1. Introduction The notion of k-subharmonic function in Rn related to the fully nonlinear k-Hessian operator Fk , k = 1, . . . , n, was introduced in [39], [40], and [41]. It generalises the classical notion of subharmonic function. The present paper concerns the structural DUKE MATHEMATICAL JOURNAL c 2002 Vol. 111, No. 1, Received 19 November 1999. Revision received 20 December 2000. 2000 Mathematics Subject Classification. Primary 35J60, 35J70, 31C45. 1

2

DENIS A. LABUTIN

properties of k-subharmonic functions. More precisely, we investigate here the basic questions of potential theory for such functions analogous to the questions of the classical potential theory for the subharmonic functions. In the real nonlinear setting, potential theory was previously developed only for the quasilinear equations in divergence form. For fully nonlinear elliptic operators, the (pluri-)potential theory was developed for the complex Monge-Ampère operator in Cn , n ≥ 2. Although we use ideas of E. Bedford and B. Taylor from pluripotential theory, some of our results for the operators Fk are completely different. The reason for this is that we prove a new pointwise estimate that has no counterpart in the pluricomplex case. 2 (), the Let be a domain in Rn , n ≥ 2. For k = 1, 2, . . . , n and u ∈ Cloc k-Hessian operator Fk is defined by Fk [u] = Sk λ(D 2 u) , (1.1) where λ = (λ1 , . . . , λn ) are the eigenvalues of the real symmetric Hessian matrix 2 D u , and where Sk are the elementary symmetric functions X Sk (λ) = λi1 · · · λik . 1≤i 1 n/2, k-subharmonic functions are locally Hölder continuous (see [39]) as in the case of convex functions k = n. We show that a compact set K is removable for bounded k-harmonic functions if and only if it has the k-capacity zero. As in [41] we strongly use ideas of Bedford and Taylor [4] from the pluripotential theory in Cn . In Section 5 we use the Wolff potential estimate to show that, vaguely speaking, the Hausdorff dimension n − 2k, 1 ≤ k ≤ n/2, separates the sets of the capacity zero from the sets of the positive capacity. More precisely, we give a necessary condition and a sufficient condition for a set to be of the k-capacity zero in terms of the Hausdorff measure, where the dimension n − 2k is critical. The results are com-

4

DENIS A. LABUTIN

pletely analogous to the classical comparison theorems for the Laplacian (k = 1). It is well known that there is no such connection between the Hausdorff measure and the capacity from [4] related to the complex Monge-Ampère operator. Section 6 begins with the investigation of the k-thin sets. We prove the Wiener test for a set E to be k-thin at a point x0 ∈ E. Then we investigate the regularity of a boundary point for the Dirichlet problem ( Fk [u] = 0 in , u= f

on ∂

in a bounded domain . The lack of the total ellipticity of Fk , k = 2, . . . , n, implies that even points on the C ∞ -smooth boundary may not be regular for the Perron solution. We make the natural modification in the statement of the Dirichlet problem (the same as in [13], [14]), and we prove the Wiener test for the regularity of x0 ∈ ∂. Previously, the Wiener test was known to hold only for the quasilinear equations in divergence form. Notation. By B(x, r ), we denote the open ball in Rn with the centre x and the radius r . For a set E ⊂ Rn , we denote by E its closure and by |E| its Lebesgue measure. By C, e C1 , . . . , we denote positive constants depending only on the dimension n and k, C, e C1 , . . .) may vary even in the same line. We write k = 1, 2, . . . , n. The value of C (C, A B if A/C ≤ B ≤ C A for some C. For functions u, v defined on a set , we put {u < v} = {x ∈ : u(x) < v(x)}. By “*”, we denote the weak* convergence of Radon measures. 2. Estimates in terms of the Wolff potential In this section we prove a bilateral pointwise estimate for a k-subharmonic function by the Wolff potential of its k-Hessian measure. It has no analogies in pluripotential theory. This estimate is analogous to the estimate of Kilpeläinen and Malý [21] (see also [28, Chap. 2]) for the A -subharmonic functions. Here A is the quasilinear elliptic operator in divergence form with some structural assumptions. Such tools of quasilinear theory as truncations and multiplication by cutoff functions are not available for fully nonlinear and not totally elliptic k-Hessian equations. We employ several estimates from [37] and [40]. Let u be a function in , σ ≥ 0. Following [12], we denote u(x) − u(y) (σ ) σ σ +1 |u|0,1;B R = sup dx u(x) + sup dx,y , |x − y| x∈ x,y∈,x6 = y where dx = dist(x, ∂), dx,y = min{dx , d y }.

POTENTIAL THEORY FOR HESSIAN EQUATIONS

5

If u ∈ 8k (B R ), 1 ≤ k ≤ n, u ≤ 0, satisfies Fk [u] ≡ const ≥ 0

in B R ,

then (see [38], [40]) (n)

|u|0,1;B R ≤ C

Z (2.1)

|u|. BR

For such u, estimate (2.1) and the mean value property of subharmonic functions imply Z 1 u(0) n u (2.2) r Br for, say, 0 < r < 9R/10. If u ∈ 8k (B R ), 1 ≤ k ≤ n, u ≤ 0, then (see [40]) Z 1 Z 1/k 1 Fk [u] ≤C n |u|. R BR R n−2k B9R/10

(2.3)

We use the following Sobolev-type inequalities from [44] and [37] for the functions u ∈ 8k (Bρ ) ∩ C 2 (Bρ ) such that u = 0 on ∂ Bρ : Z Z 1/(k+1) 1 Z 1 1 |u| = u ≤ C Fk [u](−u) . (2.4) n n n−2k ρ Bρ ρ ρ Bρ Bρ The Wolff potential of a Radon measure µ is the integral Z r µ B(x, t) 1/k dt µ Wk (x, r ) = t t n−2k 0 µ

(see [16], [3], [31]). For the purpose of this article, we choose Wk (x, r ) as a notation µ for the Wolff potential instead of W2k/k+1,k+1 (x, r ) in [3]. The following estimate obviously holds for the Wolff potential of µ: ∞ X µ B(x, R2− j ) 1/k µ C ≤ Wk (x, R) (R2− j )n−2k j=1 ∞ 1 X µ B(x, R2− j ) 1/k ≤ . (2.5) C (R2− j )n−2k j=0

∈

The following comparison principle for viscosity solutions is also used. Let g, h ϕ ∈ C(), ϕ ≥ 0. Then

8k () ∩ C(),

g ≤ h on ∂, Fk [g] ≥ ϕ, and Fk [h] ≤ ϕ in =⇒ g ≤ h in (for the proof, see [6], [42], [34]).

(2.6)

6

DENIS A. LABUTIN

THEOREM 2.1 Let u ∈ 8k (B3R ), 1 ≤ k ≤ n/2, u ≤ 0, Fk [u] = µ. Then µ µ C1 Wk (0, R/8) ≤ u(0) ≤ C2 Wk (0, 2R) − C3 sup u.

(2.7)

BR

Remark 2.2 Estimate (2.7) holds also in the case n/2 < k ≤ n. For these k the proof of the left estimate is the same as below. The right estimate in (2.7) for n/2 < k ≤ n is a simple consequence of the much stronger L ∞ -estimate due to Trudinger [37]. The latter generalises the classical Aleksandrov-Bakelman estimate of kuk L ∞ () in terms R of | det D 2 u| (see, e.g., [12, Chap. 9]). Proof of Theorem 2.1 First we prove the lower estimate for the Wolff potential in (2.7). Let ϕ be a raR dial, smooth function with compact support, ϕ ≥ 0, ϕ = 1. By the standard mollification of a locally integrable function g, we mean the convolution g ∗ ϕε , ϕε (·) = (1/ε n )ϕ(·/ε), ε > 0. Consider the standard mollifications u ∗ ϕε of function u from the theorem. The convexity of the functional cone 8k () implies that u ∗ ϕε ∈ 8k (0 ) for 0 ⊂⊂ and small enough ε > 0 (see [40]). For any fixed δ > 0, we can find a sequence {u j }, u j = u ∗ ϕε j , ε j ↓ 0, such that u j ∈ 8k (B3R−δ ) ∩ C ∞ (B 3R−δ ) for all j ≥ 1 and such that uj ↓ u

in B 3R−δ .

The monotonicity here follows from the classical (1-)subharmonicity of u (see [18], [15]). By the weak* continuity of the k-Hessian measures (see [40]), µ j = Fk [u j ] * Fk [u],

j → ∞, in B3R−δ .

Consequently (see, e.g., [26, Chap. 1]), lim inf µ j (Br ) ≥ µ(Br ), j→∞

0 < r < 3R − δ.

(2.8)

Assume for a moment that the lower estimate in (2.7) holds for any u j as above. Then from (2.8) and Fatou’s lemma for the integral over dr we obtain the estimate in its full generality. Hence we need to prove the left estimate in (2.7) only for negative functions u ∈ 8k (B3R−δ ) ∩ C ∞ (B 3R−δ ). To do this, we put R j = 8− j R, j = 1, 2, . . . . We apply (2.3) to the function


7

(u − sup B8R u) considered in B2R j , and we obtain j

µ(B R ) 1/k j R n−2k j

Z u − sup u = C1 1 u − sup u R nj B2R j B2R j B8R j B8R j Z 1 ≤ C1 n u − sup u (2.9) R j B(x,4R j ) B8R 1 ≤ C1 n Rj

Z

j

for all x ∈ B R j . Now we use the following mean value property of the classical subharmonic functions: Z 1 u − sup u ≤ C2 u(x) − sup u (2.10) n R j B(x,4R j ) B8R B8R j

j

for all x ∈ B R j . From (2.9) and (2.10), µ(B R ) 1/k j R n−2k j

≤ C3 inf u(x) − sup u x∈B R j

B8R j

= C3 sup u − sup u . B8R j

BR j

Taking the summation over j ≥ 1, we obtain ∞ X µ(B R j ) 1/k j=1

R n−2k j

≤ C lim sup u − sup u m→∞ B R

B Rm

≤ −C lim sup u = C u(0) .

m→∞ B Rm

Using (2.5), we see that the lower estimate in (2.7) holds. Now we prove the upper estimate in (2.7). Although the inequality lim sup µ j (B) ≤ µ(B)

(2.11)

j→∞

holds for any closed ball B and any sequence µ j * µ, we are not able to use Fatou’s lemma. Thus we cannot reduce the proof to the smooth case as easily as was done in the first part. The proof of the right-hand side inequality in (2.7) consists of three steps. The first step (the first reduction) is to show that it is sufficient to prove the estimate for continuous functions with only a finite number of nonzero terms in the sum in (2.5). The second step (the second reduction) reduces the proof further to an integral estimate for smooth functions. The final step is to establish the estimate for the smooth functions.

8

DENIS A. LABUTIN

First reduction We show that it is enough to prove an inequality similar to the upper estimate in (2.7) for the functions e u ∈ 8k (B2R ) of the following special type. Let us assume that for any e u ∈ 8k (B2R ), e u ≤ 0, such that e u ∈ C(B 2R ), in Br for some 0 < r < R/16,

Fk [e u] = 0 we have

NX (r ) Fk [e u ](B R2− j ) 1/k e − sup e u , u (0) ≤ C (R2− j )n−2k BR

(2.12)

(2.13)

(2.14)

j=1

where N (r ) ∈ N is the minimal integer such that R2−N (r ) < r . We then deduce (2.7) in its full generality. For an arbitrary u ∈ 8k (B3R ), Fk [u] = µ, u ≤ 0, we form the sequence of the standard mollifications u j = u ∗ ϕ R/j ,

j = 1, 2, . . . .

We have u j ∈ 8k (B2R ) ∩ C ∞ (B 2R ) for all j ≥ 1 and, from the subharmonicity of u, uj ↓ u

in B 2R when j → ∞.

Fix any such u j . For m ≥ 4, we construct the “k-harmonic lifting” vm j of u j in the ball B R/(2m ) : • •

in B 2R \B R2−m , we define vm j = u j ; in B R/2m , let vm j be the unique solution of the Dirichlet problem (

Fk [vm j ] = 0

in B R/2m ,

vm j = u j

on ∂ B R/2m .

The existence and uniqueness for this Dirichlet problem is the consequence of the viscosity theory for the operators Fk from [34] (see also [38], [40]). We claim that vm j ∈ 8k (B2R ). We present a detailed proof of the claim because we encounter the same construction several more times in the paper. One proof essentially copies the basic argument in viscosity theory (cf., e.g., [5], [9] [10], [34]), saying that the maximum of two subsolutions to an elliptic equation is again a subsolution. Take


9

any quadratic polynomial q for which vm j − q has a finite local maximum at x0 ∈ B2R . If x0 ∈ / ∂ B R/2m , then Fk [q] ≥ 0 because vm j ∈ 8k (B R/2m ) and vm j = u j in B2R \ B R/2m . If x0 ∈ ∂ B R/2m , then u j − q also has a local maximum at x0 because vm j ≥ u j in B R/2m by the definition of k-subharmonic functions. Hence Fk [q] ≥ 0 because u j ∈ 8k (B2R ). Alternatively, the claim can be proved in the spirit of classical potential theory using the last definition of k-subharmonic functions (see, e.g., [12, Chap. 2] for k = 1). By our assumptions, vm j satisfies (2.12) – (2.14). In [39] and [41] it was proved that for bounded (and, in particular, for continuous) functions g, h ∈ 8k ()∩ L ∞ () in a bounded smooth domain , the following implication holds: Z Z g = h on ∂ and g ≤ h in =⇒ Fk [g] ≥ Fk [h]. (2.15)

For the discontinuous functions, the equality g = h is understood in the limit sense. Hence Fk [vm j ](Bρ ) ≤ Fk [u j ](Bρ ) for all ρ ≤ 2R. Consequently, from (2.14) we obtain X m Fk [u j ](B R2−s ) 1/k vm j (0) ≤ C − sup u j . (R2−s )n−2k BR

(2.16)

s=1

When j → ∞, vm j monotonely decreases to a function Um , Um = u

in B2R \ B R2−m .

The monotonicity here follows directly from the definition of a k-subharmonic function. The function Um is the limit of a decreasing sequence of k-subharmonic functions. Consequently, Um ∈ 8k (B2R ) (see [40], [41]). Passing to the limit as j → ∞ in (2.16), we also obtain the following from (2.11): X m µ(B R2−s ) 1/k Um (0) ≤ C − sup u . (R2−s )n−2k BR s=1

Inequality vm j ≥ v(m+1) j , which again follows directly from the definition of a ksubharmonic function, implies Um ≥ Um+1 . Now we have Um ↓ U as m → ∞, U ∈ 8k (B2R ), and U = u everywhere, except maybe at zero. For any subharmonic function g, we have (see [18], [15]) Z 1 g(x) = lim g. (2.17) ρ→0 B(x, ρ) B(x,ρ)

10

DENIS A. LABUTIN

From (2.17) applied to the subharmonic function U , we conclude that U (0) = u(0) as well. Therefore, for our u ∈ 8k (B3R ), we have X m µ(B R2−s ) 1/k u(0) = lim Um (0) ≤ lim C − sup u m→∞ m→∞ (R2−s )n−2k BR s=1 X ∞ µ(B R2−s ) 1/k =C − sup u (R2−s )n−2k BR s=1 µ ≤ C Wk (0, 2R) − sup u , BR

and the right-hand-side estimate in (2.7) is proved. Second reduction We show that to establish (2.14) for functions satisfying (2.12) and (2.13), it is sufficient to prove that for any v ∈ 8k (Bρ ), ρ > 0, such that ( Fk [v] = f in Bρ , (2.18) v=0 on ∂ Bρ , with f > 0 in B ρ , f ∈ C ∞ (B ρ ), the following estimate holds for all J = 0, 1, . . . : 1 (ρ2−J )n

Z Bρ2−J

J +1 X ν(Bρ2−s ) 1/k v ≤ C , (ρ2−s )n−2k

(2.19)

s=0

where ν = Fk [v] = f (x) d x. We remark that v ∈ C ∞ (B ρ ) from [6] and [36]. In fact, let e u ∈ 8k (B2R ), e u ≤ 0, satisfy (2.12) and (2.13). Estimate (2.14) directly follows from (2.2), (2.13), and inequality NX Z (r ) 1 Fk [e u ](B R2−s ) 1/k − sup e u , (2.20) e u ≤ C r n Br (R2−s )n−2k BR s=1

where r and N (r ) are taken from (2.13) and (2.14). To deduce (2.20) from (2.19) we use a special modification U of e u introduced in [40]. This technical trick plays, to a certain extent, a role similar to multiplication by the cutoff function in quasilinear theory. It is also used further in our paper. Let U =e u in B R , and in the shell B2R \ B R , let U be the solution to the Dirichlet problem     Fk [U ] = 0 in B2R \ B R , U =0   U = e u

on ∂ B2R ,

on ∂ B R .

(2.21)


11

The viscosity solution to the Dirichlet problem (2.21) exists, is unique, and can be obtained by the Perron process (see, e.g., [34]). To see that the Perron process for viscosity solutions works in this situation, note the following. Function b(x) = max{e u (x), M(|x|2 − 4R 2 )}, x ∈ B2R , is k-subharmonic in B2R . Moreover, b|∂ B2R = 0, b|∂ B R = e u , if the constant M > 0 is big enough. The function b serves as the lower barrier in B2R \ B R for problem (2.21). Any upper barrier for the Laplacian at x0 ∈ ∂(B2R \ B R ) is an upper barrier for (2.21) at x0 because any ksubharmonic function is subharmonic in the classical sense. As in the first reduction, U ∈ 8k (B2R ) ∩ C(B 2R ) by the arguments from viscosity theory (or by the arguments based on the definition of k-subharmonic functions). Now we proceed to the proof of (2.20). By construction, Z Z 1 1 e u = U . rn rn Consider the standard mollifications U j = U ∗ ϕ R/j , j = 1, 2, . . . . From the continuity of U , we have Uj ↓ U

uniformly in B 2R−ε , j → ∞.

Let f j = Fk [U j ]. Take any ε > 0. For any j, say j ≥ 10, we define the function u j as the solution to the Dirichlet problem ( Fk [u j ] = f j + ε in B3R/2 , on ∂ B3R/2 .

uj = 0

Thus u j is of the type (2.18), and by our assumption estimate (2.19) holds for u j with ρ = 3R/2. Apply comparison principle (2.6) to u j + min∂ B3R/2 U j and U j in B3R/2 . We obtain 0 ≥ U j ≥ u j + min U j in B3R/2 . ∂ B3R/2

By the uniform convergence of U j , we have min U j min U ∂ B3R/2

∂ B3R/2

for all j sufficiently large. From (2.2) and (2.21), we also have min U max U . ∂ B3R/2

∂ B3R/2

The maximum principle for the subharmonic function U in B3R/2 implies u. 0 ≥ max U ≥ max U = max e ∂ B3R/2

∂ BR

BR

12

DENIS A. LABUTIN

We conclude that 0 ≥ U j ≥ u j + C max e u. BR

Now for r and N (r ) taken from (2.13) and (2.14), we obtain the following from (2.19): Z Z 1 1 U ≤ u u − C max e j j r n Br r n Br BR ≤ − C max e u+C BR

N (r ) X s=1

Fk [u j ](B R2−s ) 1/k (R2−s )n−2k \ B ) 1/k

F [u ](B k j 3R/2 R/2 +C n−2k R N (r ) X Fk [U j ](B R2−s ) + ε|B R2−s | 1/k u+C = − C max e (R2−s )n−2k BR s=1 F [U ](B k j 3R/2 \ B R/2 ) + ε|B3R/2 | 1/k +C . R n−2k Letting ε → 0, we obtain 1 rn

Z Br

N (r ) X Fk [U j ](B R2−s ) 1/k U j ≤ − C max e u+C (R2−s )n−2k BR s=1 F [U ](B k j 3R/2 \ B R/2 ) 1/k +C . n−2k R

We claim that the last term can be estimated from above by C(− max B R e u ). In fact, let us cover the shell B3R/2 \ B R/2 by χ(n) number of balls e Bi ⊂⊂ B2R , i = 1, . . . , χ (n), of radius, say, 11R/10. Then from (2.3), (2.2), and the maximum principle for the subharmonic function U , we obtain the following for all j sufficiently large: χ(n) F [U ](B X 1 Z k j 3R/2 \ B R/2 ) 1/k U ≤ C 1 Rn e R n−2k Bi i=1

≤ C2 max |U | ≤ C20 min |U | ∂ B3R/2

∂ B3R/2

≤ −C3 max e u. BR

Thus

NX Z (r ) Fk [U j ](B R/2s ) 1/k 1 − sup e u . U ≤ C j r n Br (R/2s )n−2k BR s=1

The limit in this inequality when j → ∞, along with (2.11), gives (2.20).


13

Final step: The proof of (2.19) for the solution of (2.18) Let v satisfy (2.18), v ∈ C ∞ (B ρ ). First we prove (2.19) for J = 0. That is, we establish Z 1 Z 1/k 1 v ≤ C F [v] . (2.22) k ρ n Bρ ρ n−2k Bρ For our smooth v, we have (summating over the repeated indices) Z Z Z ρ 2 − |x|2 ρ 2 − |x|2 ij Fk [v] ≥ Fk [v] = c(n, k) Fk [v]Di j v 2 ρ ρ2 Bρ Bρ Bρ Z ρ 2 − |x|2 ij = c(n, k) Di j Fk [v]v ρ2 Bρ Z Z e c(n, k) 2c(n, k) jj Fk [v](−v) = Fk−1 [v](−v) = ρ2 ρ2 Bρ Bρ Z k 1 v , ≥ C nk−n+2k ρ Bρ and (2.22) follows. In this chain of inequalities, the second equality follows from the identity (see [33]) ij 2 D j Fk [g] ≡ 0, g ∈ Cloc (), ij

where Fk [g] denotes the derivative on gi j of the function Fk [g] expressed in terms of the elements of the matrix [gi j ]. The last inequality follows from the Sobolev-type embedding estimate (2.4). Now by the scaling invariance of (2.18) and (2.19), we can assume ρ = 1. From the structure of the sum in (2.19), we can assume in the proof that J ≥ 4, J odd. Let J = 2 p − 1 for some fixed p ≥ 3. We are going to split f from (2.18) in a special way. We consider the partition of B1 into the sequence of the spherical shells {Sm }, m ≥ 1, Sm = B2−m+1 \ B2−m . For 0 < ε < 1/3, define the sets Smε = x ∈ Sm : dist(x, ∂ Sm ) > ε/2m . Consider the radial function η ∈ C ∞ (B 1 ) such that η=1

on S2l−1 , l = 1, 2, . . . , p − 1;

η=1

on B2−J = B2−2 p+1 ;

η=0

ε on S2l , l = 1, 2, . . . , p − 1.

We put f 1 = f η + δ, f 2 = f (1 − η) + δ. Let vi , i = 1, 2, be the (k-subharmonic) solutions of the Dirichlet problems ( Fk [vi ] = f i in B1 , i = 1, 2. (2.23) vi = 0 on ∂ B1 ,

14

DENIS A. LABUTIN

We have vi ∈ C ∞ (B 1 ), i = 1, 2, because δ > 0. Expanding the sum of the principal (k × k)-minors, we have Fk [v1 + v2 ] ≥ Fk [v1 ] + Fk [v2 ] = f + 2δ (see [6], [39]). Thus, due to comparison principle (2.6), we obtain v1 + v2 ≤ v ≤ 0. Consequently, it is enough to establish estimate (2.19) for v1,2 from (2.23) and then let δ → 0 as we did in the second reduction. We prove (2.19) for v1 . The proof for v2 is exactly the same. We consider the ε , l = 1, 2, . . . , p − 1: spheres Pl lying inside the spherical shells S2l Pl = ∂ B3/22l+1 ,

l = 1, 2, . . . , p − 1,

and we denote l = B3/22l+1 ,

∂l = Pl , l = 1, 2, . . . , p − 1.

Let us define the numbers M1 , M2 , . . . , M p−1 and the functions w1 , w1 , . . . , w p−1 in the following inductive way. Let M1 = sup |v1 |, P1

and let w1 be the solution to the Dirichlet problem ( Fk [w1 ] = f 1 in 1 , w1 = −M1

on ∂1 .

For l = 2, 3, . . . , p − 1, let Ml = sup |wl−1 |, Pl

and let wl be the solution of the Dirichlet problem ( Fk [wl ] = f 1 in l , wl = −Ml

on ∂l .

By comparison principle (2.6), w p−1 ≤ · · · ≤ w1 ≤ v1 ≤ 0 Thus

in p−1 .

Z Z 1 v ≤ C w . 1 p−1 −J n (2 ) | p−1 | p−1 B2−J 1

(2.24)


15

From (2.22) applied to (w p−1 + M p−1 ) in p−1 , we obtain Z Z 1/k 1 1 w p−1 ≤ C M p−1 + C f 1 | p−1 | p−1 (2−2 p+2 )n−2k B2−2 p+2 Z 1/k 1 =C f 1 (2−2 p+2 )n−2k B2−2 p+2 p−2 X +C (Ml+1 − Ml ) + C M1 .

(2.25)

l=1

From (2.2) and (2.22) applied to (wl + Ml ) in l , Z 1 |Ml+1 − Ml | ≤ C1 wl + Ml |l | l Z 1/k 1 ≤ C2 f1 . −2l n−2k (2 ) B2−2l From (2.2) and (2.22) applied to v1 in B1 , Z Z 1/k 1 |M1 | ≤ C1 v ≤ C f1 . 1 2 |B1 | B1 B1

(2.26)

(2.27)

From (2.24) – (2.27), Z p−1 Z 1/k X 1 v ≤ C f , 1 1 (2−J )n B2−J (2−2l )n−2k B2−2l 1

J = 2 p − 1.

(2.28)

l=0

Combining (2.28) with the similar estimate for v2 , we conclude that (2.19) holds for solutions of (2.18). 3. Local behaviour of k-subharmonic functions In this section we give the first applications of estimate (2.7). We investigate the behaviour of k-subharmonic functions in the neighbourhood of an interior point. Assertions of Theorems 3.1–3.5 for the p-Laplacian (and more general quasilinear operators) were proved in [21]. In Theorem 3.6 we establish the Serrin-type characterisation of isolated singularities for the equation Fk [u] = 0. We call u satisfying this equation k-harmonic. The equation can equally be understood in the viscosity sense (see [34], [42]), in the weak sense (see [38]), or in the sense of measures (see [39], [40]). 3.1 Suppose that u ∈ 8k (B R ), and let Fk [u] = µ. Then |u(0)| < ∞ if and only if Z ρ µ B(0, r ) 1/k dr < +∞, 0 < ρ < R. r r n−2k 0 THEOREM

16

DENIS A. LABUTIN

Proof This is a direct consequence of Theorem 2.1 THEOREM 3.2 Suppose that u ∈ 8k (B R ), and let Fk [u] = µ. Then u is finite valued and continuous at zero if and only if for each ε > 0 there is r > 0 such that µ

Wk (x, r ) < ε

(3.1)

whenever x ∈ Br . Proof Assume first that for any ε > 0, inequality (3.1) holds for all x ∈ Br . From Theorem 3.1, u(0) > −∞. Because u is upper semicontinuous, we can find ρ > 0 such that u − u(0) − ε < 0 in B10ρ and such that r in (3.1) satisfies r < 10ρ. Now for any x0 ∈ Bρ we apply the right estimate in (2.7) to (u − u(0) − ε) in B(x0 , 6ρ): u(x0 ) − u(0) − ε ≤ C1 Wµ (x0 , 4ρ) − C2 sup u − u(0) − ε k ≤C

B(x0 ,2ρ) µ Wk (x0 , 4ρ) − sup u − u(0) − ε . Bρ

Due to the maximum principle for the subharmonic u, sup u ≥ u(0), Bρ

and finally, for all x ∈ Bρ , u(x) − u(0) ≤ (2C + 1)ε. To prove the converse, suppose that u(0) > −∞, and suppose that u is continuous at zero. For a fixed ε > 0, we choose r > 0 such that for all x ∈ B10r , u(x) − u(0) < ε. Pick any x0 ∈ Br . Then apply the left estimate in (2.7) to (u − sup B4r u) in B(x0 , 3r ), and obtain µ W (x0 , r/8) ≤ C1 u(x0 ) − sup u k

B4r

= C1 u(x0 ) − u(0) + u(0) − sup u ≤ C2 ε. B4r


17

THEOREM 3.3 Let u ∈ 8k (B R ), 1 ≤ k ≤ n/2, u ≤ 0, µ = Fk [u], and let there exist constants M > 0, ε > 0 such that µ B(x, r ) ≤ Mr n−2k+ε (3.2)

for all x ∈ B R/10 , 0 < r < R/10. Then sup |u| ≤ C1 inf |u| + C2r ε/k , Br

Br

(3.3)

where C1 = C1 (n, k), C2 = C2 (n, k, M, ε). Proof For any x ∈ Br , we apply the right estimate in (2.7) to u in B(x, 6r ). Using (3.2) and the maximum principle for the subharmonic function u, we obtain ! Z 4r µ B(x, ρ) 1/k dρ u(x) ≤ C + inf |u| B(x,2r ) ρ ρ n−2k 0 Z 4r ε/k ρ ≤ C M 1/k dρ + C inf |u|, Br ρ 0 and (3.3) follows. COROLLARY 3.4 Let u ∈ 8k (B R ), 1 ≤ k ≤ n/2, µ = Fk [u], and let there exist constants M > 0, ε > 0 such that µ B(x, r ) ≤ Mr n−2k+ε (3.4)

for all x, r such that B(x, 10r ) ⊂⊂ B R . Then for any compact set K ⊂ B R , there exist constants C K > 0, γ > 0 such that u(x) − u(y) ≤ C K |x − y|γ for all x, y ∈ K . Proof Fix any z 0 ∈ B R . From (3.4), inequality (3.3) holds for (u + inf B(z 0 ,10ρ) |u|). We obtain that for all ρ > 0 small enough, Mρ − m 10ρ ≤ C1 m ρ − m 10ρ + C2 ρ ε/k , where Mr = sup |u|, B(z 0 ,r )

m r = inf |u| B(z 0 ,r )

18

DENIS A. LABUTIN

and where C1 > 0 is taken from (3.3). Consequently, C1 (Mρ − m ρ ) ≤ (C1 − 1)Mρ − (C1 − 1)m 10ρ + C2 ρ ε/k ≤ (C1 − 1) M10ρ − m 10ρ + C2 ρ ε/k . We conclude that there exists ρ0 > 0, ρ0 = ρ0 (dist(z 0 , ∂ B R )), such that osc u ≤ C

B(z 0 ,ρ)

osc

B(z 0 ,10ρ)

u + ρ ε/k

for all ρ ∈ (0, ρ0 ) with C = (C1 − 1)/C1 < 1. The corollary now follows from the well-known DeGiorgi lemma on the iterations of monotone functions on the real interval (0, ρ0 ) (see, e.g., [12, Chap. 8], [17, Chap. 6]). The following theorem is, in a sense, converse to Corollary 3.4. THEOREM 3.5 Let u ∈ 8k (B R ), 1 ≤ k ≤ n/2, µ = Fk [u], and let there exist constants C > 0, γ > 0 such that u(x) − u(y) ≤ C|x − y|γ (3.5)

for all x, y ∈ B R/2 . Then there exists M > 0 such that µ(Br ) ≤ Mr n−2k+γ k

(3.6)

for any 0 < r < R/10. Proof Fix any r , 0 < r < R/10. Apply the left estimate in (2.7) to u − sup B3r u in B3r . Using (3.5), we obtain Z r Z r µ(B ) 1/k µ(Bρ ) 1/k dρ µ(Bρ ) 1/k dρ r ≤ C ≤ C 1 2 n−2k ρ ρ r n−2k ρ n−2k r/2 ρ 0 γ ≤ C3 u(0) − sup u ≤ Mr , B3r

and (3.6) follows. Let u ∈ 8k (B R \ {0}) satisfy Fk [u] = 0

in B R \ {0}.

(3.7)

We say that the singularity of u at zero is removable if u can be defined at zero, so that u ∈ 8k (B R ) satisfies Fk [u] = 0 in the entire ball B R .


19

0,1 From (2.1), we see that solutions to Fk [u] = 0 are in Cloc . In contrast to the case k = 1, this is, in general, the best possible local regularity for k ≥ 2. For the Monge-Ampère equation, k = n, this is well known. The examples for any k ≥ 2 can easily be constructed using the Perron solution to the Monge-Ampère equation in Rk and using the extension of functions to Rn as in [42]. The fundamental solutions 0k to the operators Fk are defined by (see [39], [40], [41])   −|x|2−n/k for 1 ≤ k < n/2, x 6= 0,  0k (x) = log |x| (3.8) for k = n/2, x 6 = 0,   |x|2−n/k for k > n/2,

with 0k (0) = −∞ for 1 ≤ k ≤ n/2. We have (see [39], [40]) 0k ∈ 8k (Rn ) and Fk [0k ] = Cδ, where δ is the Dirac measure at zero. The following result is a refined version of Serrin-type characterisation of isolated singularities to (3.7). Such characterisation was established in J. Serrin’s famous paper [32] for the quasilinear equations. For the fully nonlinear uniformly elliptic equations, a similar characterisation was obtained in [25]. THEOREM 3.6 Let u ∈ 8k (B R \ {0}), 1 ≤ k ≤ n/2, u ≤ 0, in B R \ {0}, and let it satisfy (3.7). Then either the singularity of u at zero is removable, or there exist constants C1 > 0, C2 > 0 such that

Fk [u] = C1 δ

in B R ,

(3.9)

u(x) = C2 0k (x) + O(1),

x → 0.

(3.10)

Proof For any ε > 0, the function u ε (x) =

( u(x) + ε0k (x), −∞,

x 6 = 0, x = 0,

belongs to 8k (B R ). This follows from the upper semicontinuity of u ε and from the viscosity definition of the k-subharmonicity. As soon as u ε → u in L 1loc (B R ), ε → 0, we can define u at zero, so that u ∈ 8k (B R ) (see [40]). From (3.7), the equality (3.9) holds with C1 ≥ 0. If C1 = 0, then the singularity at zero is removable. Otherwise, applying Theorem 2.1, we obtain 0k /a ≤ u ≤ a0k

in B R/3

(3.11)

20

DENIS A. LABUTIN

for some constant a > 0. In the remaining part of the proof, we refine (3.11) and prove (3.10). We can assume that max∂ B R/2 u = 0. Consider the sequence {v j }, where v j ∈ k 8 (B R/2 \ B 2− j R ) ∩ C(B R/2 \ B2− j R ) is defined as follows:     Fk [v j ] = 0 in B R/2 \ B 2− j R , (3.12) vj = 0 on ∂ B R/2 ,   v = u on ∂ B − j . j

2

R

Dirichlet problem (3.12) is solvable by exactly the same arguments as for (2.21). Denote m = min∂ B R/2 u. From comparison principle (2.6), or from the definition of k-subharmonic functions, m + vj ≤ u ≤ vj

in B R/2 \ B2− j R .

(3.13)

We claim that there exist a subsequence { jk } and a function U ∈ 8k (B R/2 \ {0}) ∩Cloc (B R/2 \ {0}) such that (3.14) v jk → U in Cloc B R/2 \ {0} . Accepting this assertion, we finish the proof of the theorem. The proof of the claim is given after that. From (3.12) and (3.14), we have m +U ≤ u ≤U

in B R/2 \ {0}.

(3.15)

In particular, U (x) → −∞ when x → 0. From (3.12), (3.14), and the weak* continuity of the Hessian measures, we obtain   in B R/2 \ {0},   Fk [U ] = 0 (3.16) U =0 on ∂ B R/2 ,   U (x) → −∞ if x → 0. Now the rotational invariance of Fk and comparison principle (2.6) allow us to apply the moving plane arguments to solutions of (3.16) to derive that U is radial. In fact, for 0 ≤ λ ≤ R/2, define the plane Pλ = {x ∈ Rn : xn = λ}, and write xλ = (x1 , . . . , xn−1 , 2λ − xn ) to denote the reflection of x in Pλ . Define also E λ = {x ∈ B R/2 : λ ≤ xn ≤ R/2}.


21

In order to follow the classical moving plane arguments (see, e.g., [11]), we only need to show that U (x) ≥ U (xλ ) for all x ∈ E λ and λ ∈ (0, R/2).

(3.17)

Define Fλ = {x : x = yλ for some y ∈ E λ }, V (x) = U (yλ ),

x ∈ Fλ , y ∈ E λ , x = yλ .

For any fixed δ > 0, we apply (2.6) to U in B R/2 \ Bδ , and we obtain U ≤ 0 in B R/2 . To prove (3.17), first take λ ∈ (R/4, R/2); thus 0 ∈ / Fλ . From (2.6), U ≤ V in Fλ . Thus inequality (3.17) holds for λ ∈ [R/4, R/2]. Finally, take λ ∈ (0, R/4). From (2.6), U ≤ V in Fλ \ B δ for all δ > 0 small enough. Letting δ → 0, we obtain U ≤ V in Fλ for λ ∈ [0, R/4], and (3.17) holds in the full generality. Next, solving the corresponding ordinary differential equation (see [37], [40]), we conclude that U = β0k + γ for some β > 0, γ ∈ R. Combining this with (3.15), we derive (3.10). It is left to establish (3.14) for the sequence {v j } given by (3.12). From (3.11) and (3.13), we obtain 0k /a ≤ v j ≤ 0 in B R/2 \ B2− j R with C and M independent of j. The existence of a subsequence {v jk } with property (3.14) now follows directly from the Arzelà-Ascoli lemma and (2.1). Estimate (3.11) holds also for n/2 < k ≤ n with the possible shift u(x) − u 0 , u 0 ∈ R, in (3.11). The proof in this case is slightly different as the fundamental solutions 0k are bounded. To overcome the difficulty, one needs to exploit the viscosity definition of k-harmonic functions in the same way as was done for uniformly elliptic fully nonlinear equations in [25]. Concerning the proof of Theorem 3.6, we remark that the moving plane arguments have been applied (for a different purpose) to the Hessian equations in [41]. 4. Capacity and exceptional sets Let be a bounded open set in Rn . The purpose of this section is to give the potentialtheoretic description of various “small” sets related to functions from 8k (). The main result of this section is Theorem 4.2, giving the description of exceptional sets in terms of a suitable capacity. Investigating the uniqueness for the Dirichlet problem for the equation Fk [u] = µ with a Radon measure µ, Trudinger and Wang [41] introduced capacities for functions

22

DENIS A. LABUTIN

from 8k (), 1 ≤ k ≤ n. These capacities are defined in the same way as the capacity introduced by Bedford and Taylor in [4] for the plurisubharmonic functions (see also [22]). Let us state the definition. Let K ⊂ be a compact set. Note that for the main results we use only the case = B R . The (relative) k-Hessian capacity of K in is defined by nZ o capk (K , ) = sup Fk [u] : u ∈ 8k (), −1 < u < 0 , (4.1) K

1 ≤ k ≤ n. Estimate (2.3) gives capk (K , ) < +∞. For an open set E ⊂ , we define capk (E, ) = sup capk (K , ) : K is compact, K ⊂ E . (4.2) For an arbitrary set E ⊂ , we define capk (E, ) = inf capk (ω, ) : ω is open, ω ⊃ E, ω ⊂ .

(4.3)

The value of (4.1) agrees with (4.3) for compact sets E in the most important, for us, case = B R . This is the content of claim (iv) in the following lemma. LEMMA 4.1 Let ⊂ Rn be an open set, E, E 1 , . . . ⊂ ; then we have the following: (i) if E 1 ⊂ E 2 , then capk (E 1 , ) ≤ capk (E 2 , ); (ii) if 1⊂ 2 , then cap 1 ) ≥ capk (E, 2 ); k (E, S∞ P∞ (iii) capk j=1 E j , ≤ j=1 capk (E j , ); (iv) if = B R , K 1 ⊃ K 2 ⊃ · · · is a sequence of compact subsets of B R , and T K = j K j , then

lim capk (K j , B R ) = capk (K , B R ) = inf capk (ω, ) : ω is open, B R ⊃ ω ⊃ K ,

j→∞

where the capacity of the compacta is defined by (4.1). The first three assertions of the lemma follow easily from (4.1) – (4.3), exactly as in [4] and [22, Chap. 4]. The proof of the last assertion is more involved. We give it later. As in the axiomatic theory, a set E ⊂ is called capacitable if capk (E, ) = sup capk (K , ) : K is compact, K ⊂ E . It follows directly from definitions (4.2) and (4.3) that open sets are capacitable. In Theorem 4.8 we show that capk (·, ) is a generalised capacity in the sense of G. Choquet (see [3, Chap. 2], [18, Chap. 3], [22, Chap. 4]). Then it follows from the axiomatic theory that all Borel sets (more generally, all Suslin sets) are capacitable.


23

Now we define two types of exceptional sets. A set E ⊂ Rn is said to be k-polar if for each point a ∈ E there is a neighbourhood B(a, r ) and a function u ∈ 8k (B(a, r )) such that u| E∩B(a,r ) = −∞. Let us define the other type of exceptional sets. Let U ⊂ 8k () be a family of functions that are locally bounded from above. For z ∈ , we define U (z) = sup u(z) : u ∈ U . The function U is not, in general, k-subharmonic or even upper semicontinuous. However, its upper semicontinuous regularisation U (x) = lim sup U (y) = inf sup u , x ∈ , ε>0 B(x,ε)

y→x

is k-subharmonic. The inclusion U ∈ 8k () can be easily proved in two different ways. The first way is to proceed as in the case of the classical subharmonic functions (see, e.g., [18, Chap. 3]), using the fact that a distribution U ∈ D 0 () is equivalent to a function u ∈ 8k () if and only if n X

ai j Di j U ≥ 0

in

i, j=1

for all constant symmetric matrices [ai j ] from a special matrix cone (see [40]). The second way is to refer to the well-known properties of the viscosity subsolutions (see, e.g., [9, Sec. 8]). A set N is called k-negligible if N ⊂ z ∈ : U (z) < U (z) (4.4) for some family U as above. The inclusions 1, p

8k () ⊂ Wloc (), 8 () ⊂ k

p < nk/(n − k), 1 ≤ k ≤ n,

0,2−n/k Cloc (),

n/2 < k ≤ n,

were established in [39] and [40]. (The case k = 1 is well known; see, e.g., [18].) Thus the exceptional sets are empty for k > n/2. Also, considering functions (3.8), it is easy to see that for n/2 < k ≤ n only the empty set has zero k-Hessian capacity. In this section we always assume that 1 ≤ k ≤ n/2. THEOREM 4.2 Let E ⊂⊂ B R . The following three statements are equivalent: (i) E is k-polar;

24

(ii) (iii)

DENIS A. LABUTIN

E is k-negligible; capk (E, B R ) = 0.

In Corollary 6.2 we show that each of (i) – (iii) in Theorem 4.2 is equivalent to E being k-thin (see Sec. 6 for the definition). The important result in the potential theory is the quasicontinuity of k-subharmonic functions with respect to the capacity (4.1) (see Th. 4.3). It was proved in [41] in connection with the investigation of the uniqueness for the generalised Dirichlet problem. We use Theorem 4.3 in our proofs. For k = 1, this is the classical result of H. Cartan [7] (see also [26]). THEOREM 4.3 Let u ∈ 8k (). For each ε > 0, there exists an open set ω ⊂ such that capk (ω, ) < ε, and the restriction u|\ω is continuous.

Before turning to the proof of Theorem 4.2, we finish the proof of Lemma 4.1. To do this, we introduce the main tool in the potential theory for Fk . This is the relative extremal function defined as (E ⊂⊂ ) Rk (E, )(x) = sup u(x) : u ∈ 8k (), u ≤ 0, u| E ≤ −1 , x ∈ . The regularised relative extremal function is the upper semicontinuous regularisation R k (E, ) of the relative extremal function, R k (E, ) ≥ Rk (E, ) in . As was explained earlier in this section, R k (E, ) ∈ 8k (). From the well-known result for the classical subharmonic functions, Rk (E, ) = R k (E, ) a.e. in (cf., e.g., [18], [15]). For a compact set K ⊂ B R , the construction of R k (K , ) in \ K is nothing but the Perron process. Thus from viscosity theory, (4.5) Fk R k (K , ) = 0 in \ K in the viscosity sense. Equality (4.5) holds also in the sense of weak solutions (see [38]) and in the sense of the Hessian measures (see [39], [40]). From (4.5) and (2.1), R k (K , ) and Rk (K , ) are equal and locally Lipschitz continuous in \ K . Let = B R . Considering the k-subharmonic functions max{u, M(|x|2 − R 2 )} (M > 0 is big enough), we see that lim R k (E, B R )(y) = 0

y→x

for any x ∈ ∂ B R ,


25

E ⊂⊂ B R . More generally, R k (E, )|∂ = 0, provided is a bounded uniformly k-convex domain, ∂ ∈ C 2 (for the definitions and exhaustion functions for the kconvex domains, see [6], [37], [34]). Example 1 We give the explicit formulae for R k (Br , B R ), 0 < r < R. Using fundamental solutions (3.8) and the definition of k-subharmonic functions (see [34], [36], [40]), we immediately obtain for 1 ≤ k < n/2, 1 1 − R k (Br , B R )(x) = max −1, R n/k−2 |x|n/k−2 1 1 −1 (4.6) × n/k−2 − n/k−2 r R and

|x| R −1 R n/2 (Br , B R )(x) = max −1, log log , R r

(4.7)

where x ∈ B R . Proof of Lemma 4.1(iv) First we prove convergence in (iv). We claim that for any x ∈ B R , Rk (K j , B R )(x) ↑ Rk (K , B R )(x) when j ↑ ∞.

(4.8)

Clearly, Rk (K 1 , B R ) ≤ Rk (K 2 , B R ) ≤ · · · ≤ Rk (K , B R ). Hence there exists lim Rk (K j , B R ) ≤ Rk (K , B R ) in B R .

j→∞

On the other hand, from the definition we have Rk (K , )(x) = sup u(x) : u ∈ 8k (), u < 0, u| K < −1 ,

x ∈ .

Take u ∈ 8k (), u < 0, such that u| K < −1. The set {z ∈ B R : u(z) < −1} is open. Therefore u < Rk (K j , B R ) for all j big enough, and u ≤ lim Rk (K j , B R ). j→∞

Consequently, Rk (K , B R ) ≤ lim Rk (K j , B R ), j→∞

26

DENIS A. LABUTIN

and (4.8) holds. Thus R k (K j , B R ) → R k (K , B R ),

in L 1 (B R ) as j → ∞.

The convergence in (iv) now follows from the weak* continuity of the Hessian measures and the following claim: for a compact set K ⊂ B R , Z Z capk (K , B R ) = Fk R k (K , B R ) = Fk R k (K , B R ) . (4.9) BR

K

Here capk (K , B R ) is understood in the sense of (4.1). Let us prove (4.9). The last equality in (4.9) holds due to (4.5). From definition (4.1), Z capk (K , B R ) = (1 − ε)−k sup Fk [v] : v ∈ 8k (B R ), −1 + ε < v < 0 (4.10) K

for any ε > 0. Thus to prove (4.9) it is enough to show that Z Z Fk [v] ≤ Fk R k (K , B R ) K

(4.11)

BR

for any v from (4.10). Consider the decreasing sequence of the open sets { j }, j+1 ⊂ j ⊂⊂ B R , ∞ \ j = K , ∂ j ∈ C ∞ . j=1

From the regularity of ∂ j , we have Rk ( j , B R ) = R k ( j , B R ) = u j ∈ C(B R ). Now we apply (2.15) to the bounded k-subharmonic functions v and u j in B R . Using (4.5), we obtain Z Z Z Z Fk [v] ≤ Fk [v] ≤ Fk [u j ] = Fk [u j ] K

{u j 0. Now (4.11) follows from the weak* continuity of the Hessian measures and (2.11) because u j → R k (K , B R ) in L 1 (B R ) as j → ∞. Finally, the last equality in (iv) in the lemma follows from the first part of (iv). T We just need to take a decreasing sequence {K j }, j K j = K , with K j being the closure of a smooth domain.


27

Now we proceed with the proof of Theorem 4.2. We need several lemmata. First we show that the k-polar sets can be defined globally. This can be established following the proof of the Josefson theorem in pluripotential theory given in [4]. We prefer to give the direct proof based only on the maximum principle and estimate (2.2) for k-harmonic functions. LEMMA 4.4 Let E ⊂ Rn be k-polar. Then there exists U ∈ 8k (Rn ) such that U | E = −∞.

Proof The set E can be covered by a sequence {B j } = {B(q j , r j )} of balls with the following properties. For each j, (q j , r j ) are rational numbers and there exists a function u ∈ 8k (B(q j , 4r j )) such that u j | B j ∩E=−∞ . We construct U ∈ 8k (Rn ), U | E = −∞, as a series with the terms built using {u j }. Let Bm be one of the balls above. We can assume that Bm = B R . Let us first construct a function w such that w ∈ 8k (Rn ),

w| B R ∩E = −∞.

The function u m = u corresponding to Bm = B R is locally bounded from above. Thus for any δ > 0, subtracting possibly a constant, we have u ∈ 8k (B4R ),

u| B R ∩E = −∞, u| B4R−δ ≤ 0.

Using the standard regularisation, we can find a function sequence {e u j } defined in B 3R such that e u j ∈ 8k (B3R ) ∩ C ∞ (B 3R ),

e u j ≤ 0, e u j ↓ u when j → ∞.

For any j = 1, 2, . . . , we define the function v j in B3R as follows: vj = e uj

in B R ;

and in the shell B3R \ B R , we define v j to be the viscosity solution of     Fk [v j ] = 0 in B3R \ B R , vj = 0 on ∂ B3R ,   v = e u on ∂ B . j

j

R

By exactly the same arguments as for problem (2.21), we see that v j exists and v j ∈ 8k (B3R ) ∩ C(B 3R ), j = 1, 2, . . . . Let M j = max |v j |. ∂ B2R

28

DENIS A. LABUTIN

Due to the strong maximum principle for classical subharmonic functions, v j |∂ B5R/2 < 0. Thus for the function  v (x), x ∈ B 2R ,    j 4M j w j (x) = max v j (x), R (|x| − (5R/2)) , x ∈ B 5R/2 \ B 2R ,    4M j (|x| − (5R/2)), x ∈ Rn \ B 5R/2 , R we have w j ∈ 8k (Rn ) ∩ Cloc (Rn ). From (2.2) for all j ≥ 1, M j ≤ C(R)kuk L 1 (B3R ) . Thus due to (2.1) and the Arzelà-Ascoli lemma, there exist a subsequence (we still denote it {w j }) and a function w ∈ 8k (Rn ) such that wj → w

in L 1loc (Rn ) when j → ∞.

From the construction, w j = v j = e u j in B R and, consequently, w| B R ∩E = u| B R ∩E = −∞. Now we construct U ∈ 8k (Rn ) such that U | E = −∞. First assume that E is bounded, E ⊂⊂ Bρ . Let {B j } = {B(q j , r j )} be the sequence of balls covering E described at the beginning of the proof. We can assume that r j ≤ 1 for all j. Thus we can find N = N (ρ) such that Bρ ⊂⊂ B(0, 2 N + j ),

B j ⊂⊂ B(0, 2 N + j ),

j = 1, 2, . . . .

As we already know, for each B j there exists u j ∈ 8k (Rn ) such that u j | B j ∩E = −∞. If {B j } is finite, then we can put X U= u j. j

If {B j } is infinite, then we proceed as follows. For each j = 1, 2, . . . , we define uj − Uj =

2 j u j −

sup B(0,2 N + j )

sup B(0,2 N + j )

, u j L 1 (B(0,2 N + j ))

U j ∈ 8k (Rn ). Let U=

X j

uj

Uj.


29

e a.e. for The series obviously converge in L 1loc (Rn ). Consequently (see [40]), U = U k n e some U ∈ 8 (R ). We also have U j ≤ 0 on Bρ for all j ≥ 1. Thus U is upper semicontinuous in Bρ as the limit of a decreasing sequence of upper semicontinuous e in B R . Due to the construction, U | E = −∞. functions. Consequently, U = U If E is unbounded, we apply a similar construction to the sequence of k-subharmonic functions related to {E ∩ B(0, N )}, N = 1, 2, . . . . LEMMA 4.5 Let E ⊂⊂ B R . Then E is k-polar if and only if R k (E, B R ) = 0.

Proof Let R k (E, B R ) = 0. Fix x0 ∈ B R \ E. Then Rk (E, B R )(x0 ) = R k (E, B R )(x0 ) = 0. From the definition of the relative extremal function, we can find a sequence {u j } such that u j ∈ 8k (B R ), u j ≤ 0, u j | E ≤ −1, −2− j ≤ u j (x0 ) ≤ 0 for any j ≥ 1. For the upper semicontuinuous function u=

∞ X

u j,

j=1

we have u ∈ 8k (B R ), u| E = −∞, −1 ≤ u(x0 ) ≤ 0. Thus E is k-polar. Conversely, if E is k-polar, then by Lemma 4.4 we can find u ∈ 8k (B R ) such that u| E = −∞. For any ε > 0, εu| E ≤ −1. Thus Rk (E, B R )(x) = 0 for any x such that u(x) 6= −∞. Taking x ∈ B R \ E, we also obtain R k (E, B R )(x) = 0. The strong maximum principle for classical subharmonic functions gives R k (E, B R ) = 0 (cf. [18], [15]). LEMMA 4.6 Let E ⊂⊂ B R . Then

Z

Fk R k (E, B R ) .

(4.12)

Proof First we claim that for an open set ω ⊂⊂ B R , Z capk (ω, B R ) = Fk R k (ω, B R ) .

(4.13)

capk (E, B R ) =

BR

BR

30

DENIS A. LABUTIN

To prove (4.13), we consider the increasing sequence of the compact sets {K j }, ∞ [

K j = ω,

∂ K j ∈ C ∞.

j=1

Functions R k (K j , B R ) monotonely decrease to R k (ω, B R ), and (4.9) implies (4.13). Now we prove (4.12) in its whole generality. Let us show that Z capk (E, B R ) ≥ Fk R k (E, B R ) . (4.14) BR

Let ω ⊂⊂ B R be an open set such that E ⊂ ω. Then from (2.15) and (4.13) we have for any ε > 0, Z Z Fk R k (E, B R ) ≤ Fk (1 + ε)R k (ω, B R ) BR

BR

= (1 + ε)k capk (ω, B R ), and (4.14) follows when ε → 0. Let us show the inequality opposite to (4.14). From the Choquet topological lemma, we can find (in the same way as for the classical subharmonic functions) an increasing sequence {v j }, v j ∈ 8k (B R ), −1 ≤ v j ≤ 0, such that lim v j (x) = R k (E, B R )(x) for almost all x ∈ B R

j→∞

(on this construction, see, e.g., [18, Chap. 3], [17, Chap. 8]). Consider open sets ω j = x ∈ B R : (1 + 1/j)v j (x) < −1 . We have E ⊂ ω j and (1 + 1/j)v j ≤ Rk (ω j , B R ) ≤ Rk (E, B R ). Consequently, R k (ω j , B R ) → R k (E, B R ),

in L 1 (B R ) as j → ∞.

From the weak* continuity of the Hessian measures, we obtain Z capk (E, B R ) ≤ lim capk (ω j , B R ) = lim Fk R k (ω j , B R ) j→∞ j→∞ B R Z = Fk R k (E, B R ) , BR

and Lemma 4.6 is proved.


31

Let us consider the following application of Lemma 4.6. Example 2 We compute capk (Br , B R ), 0 < r < R. First we recall a formula (see [35]; see also [37], [39]) which is also used further in the paper. If v ∈ 8k () ∩ C 2 (), v ≡ const on ∂, ∂ ∈ C 2 , then Z Z 1 ∂v k Fk [v] = Sk−1 [∂], (4.15) k ∂ ∂ν where ν is the outer normal to , and where Sk−1 [∂] is the elementary symmetric function of the curvatures of ∂ (see [35], [39], [37]). If = Bρ , then Sk−1 [∂ Bρ ] = ρ n−k ds∂ B1 . From (4.15), using the standard regularisation of the relative extremal functions (4.6) and (4.7), we obtain, after elementary calculations, capk (Br , B R ) = C(n, k)

1

−

1

r n/k−2 R n/k−2 −n/2 capn/2 (Br , B R ) = C(n) log(R/r ) .

−k

for 1 ≤ k < n/2,

(4.16) (4.17)

Proof of Theorem 4.2 We show that (i) ⇒ (ii) ⇒ (iii) ⇒ (i). (i) ⇒ (ii). By Lemma 4.4, we can find u ∈ 8k (B R ) such that u < 0, u| E = −∞. Consider the family u ε = εu, ε > 0, and U = sup u ε . ε>0

Then U (x) = 0 if u(x) 6= −∞, and U | E = −∞. Consequently, U = 0 in B R , and our set E is of the form (4.4). (ii) ⇒ (iii). Let E be of the form (4.4). Due to the subadditivity of capk (·, B R ), the problem is purely local. Thus, as in pluripotential theory (see [4], [18], [22]), we can assume that from (4.4) is B R , U < 0 in B R , functions U and U are equal and continuous outside a compact set K , and E ⊂ K ⊂ B R . By the Choquet lemma (see [18, Chap. 3], [17, Chap. 8]), we can assume that the family U in the definition of k-negligible sets is countable, U = {u j }. By quasicontinuity (see Th. 4.3), for any ε > 0 we can choose an open set ω ⊂⊂ B R such that all u j are continuous on B R \ ω and capk (ω, B R ) < ε. (4.18) For rational numbers r < s ≤ 0, we define K r,s = x ∈ B R \ ω : U (x) ≤ r < s ≤ U (x) .

32

DENIS A. LABUTIN

We have E ⊂ω

[[

K r,s ,

r,s

where the conjunction is taken over all rational r < s ≤ 0. Thus X capk (E, B R ) ≤ capk (ω, B R ) + capk (K r,s , B R ).

(4.19)

r,s

Since U | B R \ω is lower semicontinuous and U is upper semicontinuous, each of the sets K r,s is either compact or empty. For any K r,s = K , we have U/|r | ≤ Rk (K , B R ), and consequently −1 < s/|r | ≤ U /|r | ≤ R k (K , B R ).

(4.20)

Let us show that (4.20) implies that capk (K , B R ) = 0. If this is proved, we conclude the proof by (4.18) and (4.19). Note that (4.20) implies R k (K , B R ) ≥ −1 + 1/j

in B R

for some j ∈ N. Now the definition of the capacity and Lemma 4.6 imply that Z capk (K , B R ) ≥ Fk R k (K , B R ) j/( j − 1) BR j k Z Fk R k (K , B R ) = j −1 BR j k capk (K , B R ). = j −1 This is possible only when capk (K , B R ) = 0. (iii) ⇒ (i). From Lemma 4.6, Fk [R k (E, B R )] = 0. Comparison principle (2.6) implies R k (E, B R ) = 0, and (i) follows from Lemma 4.5. COROLLARY 4.7 Let E ⊂⊂ B R , and let e = x ∈ B R : R k (E, B R )(x) > Rk (E, B R )(x) .

Then capk (e, B R ) = 0. Proof The set e is k-negligible.


33

THEOREM 4.8 The function capk (·, B R ) is a generalised capacity in the sense of Choquet on subsets of B R . This means that (i) capk (∅, B R ) = 0; (ii) if K 1 ⊃ K 2 ⊃ · · · is a sequence of compact subsets of B R , then

lim capk (K j , B R ) = capk

j→∞

(iii)

∞ \

K j , BR ;

j=1

if E 1 ⊂ E 2 ⊂ · · · is a sequence of arbitrary subsets of B R , then lim capk (E j , B R ) = capk

j→∞

∞ [

E j , BR .

j=1

Moreover, all Suslin (in particular, all Borel) subsets of B R are capacitable. Proof It is the classical result (see [8]) that the last assertion of the theorem follows from (i) – (iii) (see also [3, Chap. 2], [18, Chap. 3], [22, Chap. 4]). It is clear that (i) holds. We proved (ii) in Lemma 4.1. It is left to prove (iii). Let F=

∞ [

x ∈ B R : R k (E j , B R )(x) > −1 .

j=1

From Corollary 4.7 and Lemma 4.1, capk (F, B R ) = 0. Let us fix an arbitrary ε > 0. By (4.3), we can find an open set G, F ⊂ G ⊂ B R , such that capk (G, B R ) < ε. Let U j = x ∈ B R : R k (E j , B R )(x) < −1 + ε , and let V j = U j ∪ G. From the upper semicontinuity of the regularised relative extremal function, the sets U j , V j are open. We have R k (E j , B R )/(1 − ε) ≤ R k (U j , B R ). Then by the subadditivity of capk (·, B R ) and Lemma 4.6, Z capk (V j , B R ) ≤ ε + Fk R k (U j , B R ) BR

≤ ε + (1 − ε)−k capk (E j , B R ).

34

DENIS A. LABUTIN

Finally, capk

∞ [

∞ [ E j , B R ≤ capk V j , BR

j=1

= capk

j=1 ∞ [

V j , B R = lim capk V j , B R j→∞

j=1

≤ ε + (1 − ε)−k lim capk (E j , B R ). j→∞

When ε → 0, we obtain (iii). We conclude the section with an application of our potential estimates to removability of singular sets for k-harmonic functions. The following theorem complements Theorem 3.6. THEOREM 4.9 Let K ⊂ B R be a compact set. Any u ∈ 8k (B R \ K ) satisfying

Fk [u] = 0

in B R \ K ,

kuk L ∞ (B R \K ) < +∞

(4.21)

can be extended as a solution to Fk [u] = 0

in B R ,

u ∈ 8k (B R ),

(4.22)

if and only if capk (K , B R ) = 0. Proof If capk (K , B R ) > 0, then K is the nonremovable singular set for R k (K , B R ). Conversely, let capk (K , B R ) = 0. Then K is k-polar due to Theorem 4.2. Consequently, due to Lemma 4.4, we can find a function U ∈ 8k (B R ), U | K = −∞. Now let u satisfy (4.21). We consider u +εU . Arguing as in the proof of Theorem 3.6, we conclude that u ∈ 8k (B R ), kuk L ∞ (B R ) < +∞. A bounded k-subharmonic function cannot put its k-Hessian measure on K because capk (K , B R ) = 0. Thus (4.22) holds. 5. Metric estimates for capacity For the classical capacity, the Hausdorff dimension n − 2, loosely speaking, separates polar from nonpolar sets (see, e.g., [26, Chap. 3]). More precisely, sets with the finite (n − 2)-Hausdorff measure have zero capacity, and, conversely, sets of zero capacity have the (n − 2 + ε)-Hausdorff measure zero for any ε > 0. The same effect takes R place for the capacity related to the variational integral |Du| p d x, 1 < p ≤ n. In this case the critical Hausdorff dimension is n − p (see, e.g., [17, Chap. 2], [28,


35

Chap. 2]). In fully nonlinear pluripotential theory, there is no critical dimension for the Bedford-Taylor capacity (see [4]). In this section we use estimate (2.7) to show that the Hausdorff dimension n − 2k, 1 ≤ k ≤ n/2, is critical for k-polar sets. In the case k = 1 we are dealing with the classical subharmonic functions, and all results are of course well known. In Theorems 5.1 and 5.2 only compact sets are considered. The case of arbitrary E follows then from general facts about the Hausdorff measures and the Choquet capacities (see, e.g., [3, Chap. 5]). Let h be a real-valued, increasing function on [0, 1) with limr →0 h(r ) = h(0) = 0. The h-Hausdorff measure of E is defined by X 3h (E) = lim inf h(r j ) , δ→0

j

where the infimum is taken over all coverings of E by the balls B j with the radii r j not exceeding δ. We remark that our idea of utilising the relative extremal function in the proof of the next theorem works also in the pluricomplex case. In the latter situation this idea allows us to prove the pluripolarity of any set E ⊂ Cn with 3(log(1/r ))−n (E) < +∞, and this result is sharp (see [24]). To prove Theorem 5.1, we also use a construction from [43]. THEOREM 5.1 Let h(r ) = r n−2k for 1 ≤ k < n/2, let h(r ) = (log(1/r ))−n/2 for k = n/2, and let E ⊂ B R be a compactum. Then

3h (E) < +∞ =⇒ capk (E, B R ) = 0. Proof Seeking a contradiction, assume that capk (E, B R ) > 0, and, consequently, assume that R k (E, B R ) 6 = 0. Let E 1 = x ∈ B R : R k (E, B R )(x) = −1 , E 1 ⊂ E. From Corollary 4.7, capk (E 1 , B R ) = capk (E, B R ). The set E 1 is Borel, being the intersection of the family of the open sets V j = x ∈ B R : R k (E, B R )(x) < −1 + 1/j . In particular, E 1 is capacitable. Thus there exists a compact set K ⊂ E 1 such that Z Fk R k (E, B R ) ≥ capk (E, B R )/2 > 0. (5.1) K

36

DENIS A. LABUTIN

We claim that capk (K , ) ≤ c(n, k)3h (K )

(5.2)

for any bounded open set ⊃ K . The claim is established at the end of the proof. Assuming for a moment that (5.2) holds, we now deduce the contradiction. In fact, by the definition, R k (E, B R ) is continuous at x0 , provided R k (E, B R )(x0 ) = −1. Consequently, the compactness of K ⊂ E 1 implies that lim

dist(x,K )→0

R k (E, B R )(x) = −1.

Thus ϕ(δ) osc R k (E, B R ) → 0, δ

δ → 0,

(5.3)

where δ = x ∈ B R : dist(x, K ) < δ . We have −1
0 we can find r x , 0 < r x < ε, such that h(r x ) ≤ µ B(x, r x ) . (5.6) S For a fixed ε > 0, we consider the family x∈E B(x, r x ) covering the set E. Numbers r x are taken from (5.6). Applying the Besicovitch covering theorem to this S family, we extract a sequence {B j } = {B(x j , r j )}, r j = r x j , such that E ⊂ j B j and such that any point in Rn is contained in no more than α(n) balls B j . From (5.6), X h(r j ) ≤ α(n)µ(B R ). j

Thus 3h (E) ≤ α(n)µ(B R ). The functions N h, N > 0, still satisfy (2.3). Consequently, we obtain 3h (E) ≤ α(n)µ(B R )/N . When N → ∞, we obtain (5.5).

38

DENIS A. LABUTIN

6. Thinness and the Wiener test In this section we apply estimate (2.7) and capacity (4.3) to the investigation of the k-thinness for the k-Hessian operators. Our main result is the Wiener test for the regularity of a boundary point for the Dirichlet problem. The classical Wiener test characterises regular points for the Dirichlet problem for the Laplace equation. Since the 1960s, it has been generalised to linear (possibly degenerate) elliptic equations in divergence and nondivergence forms. In his seminal paper [29], V. Maz’ya started the investigation of the Wiener regularity for quasilinear equations. A study of the quasilinear equations was recently completed by Kilpeläinen and Malý in [21]. An important contribution was made earlier by P. Lindqvist and O. Martio [27]. Rather complete results were obtained for the linear heat equation and quasilinear subelliptic equations. We complement the picture by establishing the Wiener test for the elliptic equations nonlinear on the second derivatives. We are not reviewing all interesting literature on the subject. The complete bibliography on the Wiener test before 1994 can be found in [1] and [30]. The history is carefully expounded in [1] (see also D. Adams’s review [2] of the book [28]). The monographs [17] and [28] contain detailed investigation of the regularity of a boundary point for the quasilinear elliptic equations and the bibliography after 1994. We define the k-thin sets in the same way as the thin sets are defined in classical potential theory (see [26]). Set E ⊂ Rn is called k-thin at x0 , 1 ≤ k ≤ n, if either x0 6∈ E or x0 ∈ E, and there exist ε > 0 and u ∈ 8k (B(x0 , ε)) such that lim sup

x→x0 , x∈E\{x0 }

u(x) < u(x0 ).

(6.1)

Set E is k-thin if for any x, x ∈ E, E is k-thin at x. Thus the 1-thinness coincides with the classical thinness. From the continuity of k-subharmonic functions for k > n/2, a set E is k-thin at x0 if and only if x0 6 ∈ E. Also, for k > n/2 only, E = ∅ is k-thin. THEOREM 6.1 Set E ⊂ Rn is k-thin at x0 if and only if

Z 1 capk E ∩ B(x0 , r ), B(x0 , 2r ) 1/k dr < +∞. r r n−2k 0

(6.2)

COROLLARY 6.2 Let E ⊂⊂ B R . Each of the statements (i)–(iii) in Theorem 4.2 is equivalent to the condition E is k-thin.

Now we describe our results concerning the following Dirichlet problem in bounded


domains

(

39

Fk [u] = 0

in ,

u= f

on ∂.

(6.3)

By the Perron solution to (6.3), we mean the continuous function P f (x) = sup u(x) : u ∈ 8k (), u ≤ f on ∂ , x ∈ . It satisfies the equation from (6.3) inside in the viscosity sense (see [10], [9], [34]) and in the sense of measures (see [39], [40], [41]). As in the classical theory, if (6.3) is solvable, then the solution coincides with P f . For 2 ≤ k ≤ n, the Perron solution to (6.3) does not attain the boundary values f ∈ C(∂) even for some domains with ∂ ∈ C ∞ . The situation is completely analogous to the case of the Monge-Ampère equation (k = n), when to solve (6.3) for any f ∈ C(∂) (in the elliptic framework) the convexity of is required. For 1 < k < n, the natural generalisation of the convexity condition is the k-convexity of , ∂ ∈ C 2 (see [6], [36], [34], [39], [40]). We refer to [6] and [36] for the solution of the Dirichlet problem (6.3) with nonzero right-hand side in the uniformly elliptic case (and also in some degenerate cases). To treat problem (6.3) in general domains using the elliptic theory, we follow the approach from [13] and [14]. In [13] the Dirichlet problem (6.3) with nonzero righthand side (and even with a nonlinear right-hand side) was treated for the smooth data. Instead of convexity-type conditions on (in [13], ∂ ∈ C 2 ), a condition on f was imposed. The condition was, roughly speaking, that f is the trace of a k-subharmonic subsolution to the corresponding equation. A boundary point x0 ∈ ∂ of a bounded domain ⊂ Rn is called regular for the Dirichlet problem if for any f ∈ 8k () ∩ C(), the Perron solution P f to (6.3) has the limit value f (x0 ) at x0 . The point is irregular if it is not regular. Theorem 6.5 states that the set of irregular points of ∂ has the capacity zero. Now we formulate the Wiener criterion for regularity. THEOREM 6.3 Let ⊂ Rn be a bounded domain. Point x0 ∈ ∂ is regular for Dirichlet problem (6.3) if and only if Z 1 capk B(x0 , r ) \ , B(x0 , 2r ) 1/k dr = +∞. (6.4) r r n−2k 0

Note that f ∈ 8k () ∩ C() is the lower barrier for the Perron solution of (6.3). For k > n/2, functions (3.8) are the upper barriers. Thus for k > n/2, every point of ∂ is regular for (6.3). The Wiener integral (6.4) always diverges for k > n/2 because, as in Example 2 from Section 4, we have capk (x0 , B(x0 , r )) = c(n, k)r n−2k , k > n/2.

40

DENIS A. LABUTIN

In the proofs of Theorems 6.1 and 6.3, we use ideas from the quasilinear theory. The good references are [17] and [28]. We need the following lemma. LEMMA 6.4 Let K ⊂ B R be a compact set, 1 ≤ k ≤ n/2, −1 < t < 0, K t = x ∈ B R : R k (K , B R )(x) ≤ t .

Then capk (K t , B R ) =

1 capk (K , B R ). |t|k

(6.5)

Proof We consider the decreasing sequence of the open sets { j }, ∞ \

j = K,

∂ j ∈ C ∞ .

j=1

For the corresponding sequence of the relative extremal functions {R k ( j , B R )} and R k ( j , B R ) = Rk ( j , B R ) = u j , we have u j ∈ C(B R ), j ≥ 1, and u j → R k (K , B R ) in L 1 (B R ). For t ∈ (−1, 0), the sets K j = x ∈ B R : u j (x) ≤ t are compact. Due to (4.5) and comparison principle (2.6) for the viscosity solutions in the shell B R \ K j , we have R k (K j , B R ) = max −1, u j /|t| . Thus, from Lemma 4.6 and (2.15), we obtain capk (K j , B R ) = Let F =

T

j

1 capk ( j , B R ). |t|k

K j . We have F = x ∈ B R : Rk (K , B R )(x) ≤ t

because u j ↑ Rk (K , B R ) in B R when j → ∞. Obviously, K t ⊂ F and F \ K t = x ∈ B R : R k (K , B R )(x) > Rk (K , B R )(x) .

(6.6)


41

The last set is k-negligible. Thus, by Corollary 4.7, capk (F \ K t , B R ) = 0 and capk (K t , B R ) = capk (F, B R ). From (6.6) and Theorem 4.8, we have capk (F, B R ) = lim capk (K , j, B R ) = lim j→∞

=

j→∞

1 capk ( j , B R ) |t|k

1 capk (K , B R ), |t|k

and (6.5) holds. Proof of Theorem 6.1 We assume that x0 = 0, E ⊂ B R . In the proof we denote B j = B R2− j , j = 0, 1, . . . (B0 = B R ). First we show that if E is k-thin at zero, then (6.2) holds. We can assume that there exists u ∈ 8k (Rn ) such that lim sup

u(x) < −1 < u(0) < 0.

x→0, x∈E\{0}

Moreover, we may also assume that u < 0 in B R , u < −1 in B R ∩ E. Otherwise, taking r > 0 small enough, we can consider the function ( max {u(x), 0k (x) − 0k (r )} , x ∈ B R , e u (x) = 0k (x) − 0k (r ), x ∈ Rn \ Br , in Bρ for sufficiently small ρ > 0. Let U ⊃ E be the open set U = x ∈ B R : u(x) < −1 . For j ≥ 1, we define w j = R k (U ∩ B j , B j−1 ), cap (U ∩ B , B ) 1/k j j−1 k βj = . capk (B j , B j−1 ) For w1 = R k (U ∩ B1 , B0 ), we have −1 < u(0) ≤ w1 (0) < 0. Consequently, (6.2) is proved if we show that for every j ≥ 1, j X 1 + w1 (0) ≤ exp −C βi = i=1

1 , α1 · · · α j

(6.7)

42

DENIS A. LABUTIN

where αi = exp(Cβi ). Estimate (6.7) is the direct consequence of the estimate sup w j ≤ −Cβ j ≤ −1 + exp(−Cβ j ) = −1 + Bj

1 , αj

j ≥ 1,

(6.8)

and the following inequality, which holds for all j ≥ 2 and any x ∈ B j−1 : (−1 + α1 · · · α j−1 ) + (α1 · · · α j−1 )w1 (x) ≤ w j (x).

(6.9)

Let us prove (6.8). We define e B j = B3R/2 j+1 , B j ⊂ e B j ⊂ B j−1 , M = max w j ,

m = min w j , ∂e Bj

∂e Bj

M ≥ sup B j w j . From (4.5) and (2.2), we have M m. For any j ≥ 1, we approximate the open set U ∩ B j from the interior by compact sets. Then we obtain from Lemma 6.4, capk {w j ≤ m}, B j−1

1 capk (U ∩ B j , B j−1 ). |M|k

From (2.2), we have the Harnack inequality for w j in B j−1 \ e B j . Consequently, {w j ≤ e m} ⊂ B j . Now (6.8) follows because capk (B j , B j−1 ) capk ( e B j , B j−1 ). Finally, we prove by induction that (6.9) holds for every j ≥ 2. For j = 2, we need to show that for any x ∈ B1 , −1 + α1 + α1 w1 (x) ≤ w2 (x).

(6.10)

To see this, we consider the function (−1 + α1 + α1 w1 ) ∈ 8k (B1 ). Note that from (6.8) with j = 1, −1 + α1 + α1 w1 (x) ≤ 0, x ∈ B1 , (6.11) and also −1 + α1 + α1 w1 (x) = −1,

x ∈ U ∩ B2 .

(6.12)

Thus for all x ∈ B1 , −1 + α1 + α1 w1 (x) ≤ R k (U ∩ B2 , B1 )(x) = w2 (x), and (6.10) holds. Assume that (6.9) holds for a fixed j. From (6.7), we obtain, as in (6.11) and (6.12), −1 + α j + α j w j (x) ≤ 0, −1 + α j + α j w j (x) = −1,

x ∈ Bj, x ∈ U ∩ B j+1 ,


43

and, consequently, −1 + α j + α j w j (x) ≤ w j+1 (x),

x ∈ Bj.

(6.13)

Using (6.9) to estimate w j in the left-hand side of (6.13) from below, we obtain (6.9) with ( j + 1) instead of j. Thus (6.9) holds for any j ≥ 2, and (6.2) holds. Now we prove that (6.2) implies that E is k-thin at zero. We may assume that E is open. Otherwise, for every j = 1, 2, . . . , we find an open set U j ⊃ (B j ∩ E) such that capk (B j ∩ U j , B j−1 ) ≤ capk (B j ∩ E, B j−1 ) + (R/2 j )n−k , and we put E = (U1 \ B 2 )

[ [ [ (U1 ∩ U2 \ B 3 ) (U1 ∩ U2 ∩ U3 \ B 3 ) ···

(see [28, Chap. 2]). Let E j = E ∩ B j , j ≥ 0. We fix an integer m ≥ 2, and we put u = R k (E m , Bm−2 ), µ = Fk [u]. To prove the sufficiency assertion of the theorem, we show that for m large enough, u(0) ≥ −

1 > −1. 2

From Lemma 6.4 we obtain, using the exhaustion of E m by compact sets, capk (Bm , Bm−2 ) ≤ capk u < sup u , Bm−2 Bm

−k = sup u capk (E m , Bm−2 ) Bm

−k = sup u µ(Bm−1 ).

(6.14)

Bm

From (6.14), using (4.16) and (4.17), we conclude that µ(B 1/k m−1 ) sup u ≤ C . (R/2m−1 )n−2k Bm

(6.15)

From the monotonicity of the capacity, we also have for j ≥ m − 1, µ(B j ) ≤ capk (E j , Bm−2 ) ≤ capk (E j , B j−1 ).

(6.16)

Finally, from Theorem 2.1, (6.15), and (6.16), we have due to (6.2), ∞ X capk (E ∩ B j , B j−1 ) 1/k 1 u(0) ≤ C ≤ 2 (R/2 j )n−2k j=m−1

for m large enough. We estimated the second term in the right-hand side of (2.7) using (6.15).

44

DENIS A. LABUTIN

Proof of Corollary 6.2 From Theorem 6.1, it follows directly that Theorem 4.2(iii) implies Lemma 4.1(iv). To finish the proof, we show that k-thin sets are k-negligible. Let {B j } be all balls with rational centres and radii such that B j ⊂⊂ B R , B j ∩ E 6 = ∅. Let N j = x ∈ B R : R k (B j ∩ E, B R )(x) > Rk (B j ∩ E, B R )(x) . S According to Corollary 4.7 and Theorem 4.2, the set N = ∞ j=1 N j is k-negligible. Let us show that E ⊂ N . We pick any x0 ∈ E. We can assume that x0 is not an isolated point of E. Because E is k-thin at x0 , we can find u ∈ 8k (B R ), u < 0 in B R , and a ball B j 3 x0 such that u(x) < u(x0 ) − ε, ε > 0, for any x ∈ (E ∩ B j ) \ {x0 }. We introduce the functions u(x) 1 + 0 (x − x0 ). u m (x) = u(x0 ) − ε m k For every m ≥ 1, we have u m ∈ 8k (B R ), u m < 0 in B R , u m | B j ∩E ≤ −1. Now we see that x0 ∈ N j because R k (B j ∩ E, B R )(x0 ) ≥ lim sup sup u m (x) x→x0

m

u(x) u(x0 ) ≥ = lim sup u(x0 ) − ε x→x0 u(x 0 ) − ε > −1 = Rk (B j ∩ E, B R )(x0 ).

Proof of Theorem 6.3 We assume that x0 = 0. From the Perron construction, we always have for the Perron solution of (6.3), lim inf P f (x) ≥ f (0). x→0, x∈

Thus the regularity of zero for (6.3) is equivalent to lim sup P f (x) ≤ f (0).

(6.17)

x→0, x∈

We prove that (6.4) implies (6.17). First we claim that if R k (Bρ \ , B2ρ )(0) = −1

(6.18)

for all sufficiently small ρ > 0, then (6.17) holds for all f ∈ 8k () ∩ C(). In fact, if (6.18) holds, then lim R k (Bρ \ , B2ρ )(x) + 1 = 0 x→0


45

and the function R k (Bρ \ , B2ρ ) + 1 is k-harmonic in ∩ B2ρ . For a given f ∈ 8k () ∩ C() and ε > 0, we choose ρ > 0 such that (6.18) holds and osc( f ) < ε. B2ρ

Due to comparison principle (2.6), the Perron solution P f of (6.3) is dominated in ∩ B2ρ by 2k f k L ∞ () R k (Bρ \ , B2ρ )(x) + 1 + f (0) + ε. Consequently, lim sup P f (x) ≤ f (0) + ε, x→0, x∈

and (6.17) holds. Now, seeking a contradiction, suppose that (6.4) is true but (6.18) does not hold for some ρ > 0. Then the Borel set K = x ∈ Bρ \ : R k (Bρ \ , B2ρ )(x) = −1 is k-thin at zero. From Corollary 4.7, the set (Bρ \ ) \ K has capacity zero. Thus from Theorem 6.1, Z ρ Z ρ capk (Br \ , B2r ) 1/k dr capk (K ∩ Br , B2r ) 1/k dr = < +∞, r r r n−2k r n−2k 0 0 a contradiction with (6.4). The sufficiency of (6.4) for the regularity of x0 is proved. Let us prove that if Z 1 capk (Br \ , B2r ) 1/k dr < +∞, (6.19) r r n−2k 0 then zero is not regular for the Dirichlet problem (6.3). From (6.19) and Theorem 6.1, the set Rn \ is k-thin at zero. Consequently, we can find a ball Br , r > 0, and a function u ∈ 8k (Br ) such that −3/2 −1. Using the function u, we construct the functions f ∈ 8k () ∩ C() and U ∈ 8k () such that U= f

on ∂ \ {0},

lim sup U (x) > f (0). x→0, x∈

(6.20) (6.21)

46

DENIS A. LABUTIN

Assume for a moment that U and f exist. Then we immediately conclude that zero is irregular for the Dirichlet problem. In fact, if we fix C = C() > 0 such that 0k − C ≤ 0 in , then from (6.20), P f (x) ≥ sup U (x) + ε(0k (x) − C) = U (x), x ∈ , ε>0

and (6.17) does not hold due to (6.21). We finish the proof of the theorem constructing f and U . We define f (x) = max 0k (x) − 0k (r/2), −1 , x ∈ Rn , and U (x) =

( max{u(x), f (x)}, f (x),

x ∈ Br/2 , x ∈ Rn \ Br/2 .

In Br/2 , u = −1 on ∂ \ {0}, and we conclude that (6.20) holds. Condition (6.21) also holds because lim sup U (x) ≥ lim sup u(x) = u(0) > −1 = f (0). x→0, x∈

x→0, x∈

The estimates we obtained in the proofs of Theorems 6.1 and 6.3 allow us to investigate the Hölder continuity of the solution to Dirichlet problem (6.3) with Hölder continuous data, provided the domain satisfies the lower capacity density condition (cf. [29], [17]). We do not consider this question in the present paper. We finish the section showing that, for an arbitrary bounded domain, the set of irregular boundary points is small. THEOREM 6.5 Let ⊂⊂ B R be a bounded domain, and let I ⊂ ∂ be the set of points irregular for Dirichlet problem (6.3). Then capk (I, B R ) = 0.

Proof The proof of Theorem 6.3 implies that for any x0 ∈ I , we can find a ball B(q, r ) 3 x0 with rational q and r such that x0 ∈ x : R k B(q, r ) \ , B(q, 2r ) (x) > Rk B(q, r ) \ , B(q, 2r ) (x) . Thus I belongs to a countable union of k-negligible sets and the theorem follows from Theorem 4.2.


47

Acknowledgments. I wish to thank my Ph.D. supervisors N. S. Trudinger and X. J. Wang for their constant interest, sharing of ideas, support, and discussions of their unpublished works. I also wish to thank John Urbas and the referees for their remarks. References [1]

[2]

[3] [4] [5] [6]

[7] [8] [9]

[10]

[11] [12]

[13]

[14] [15] [16]

D. R. ADAMS, “Potential and capacity before and after Wiener” in Proceedings of the

Norbert Wiener Centenary Congress (East Lansing, Mich., 1994), Proc. Sympos. Appl. Math. 52, Amer. Math. Soc., Providence, 1997, 63–83. MR 98k:31003 38 , review of Fine Regularity of Solutions of Elliptic Partial Differential Equations by J. Malý and W. P. Ziemer, Bull. London Math. Soc. 31 (1999), 248–250. 38 D. R. ADAMS and L. I. HEDBERG, Function Spaces and Potential Theory, Grundlehren Math. Wiss. 314, Springer, Berlin, 1996. MR 97j:46024 3, 5, 22, 33, 35, 37 E. BEDFORD and B. A. TAYLOR, A new capacity for plurisubharmonic functions, Acta Math. 149 (1982), 1–40. MR 84d:32024 3, 4, 22, 27, 31, 35 L. A. CAFFARELLI and X. CABRE´ , Fully Nonlinear Elliptic Equations, Amer. Math. Soc. Colloq. Publ. 43, Amer. Math. Soc., Providence, 1995. MR 96h:35046 8 L. CAFFARELLI, L. NIRENBERG, and J. SPRUCK, The Dirichlet problem for nonlinear second-order elliptic equations, III: Functions of the eigenvalues of the Hessian, Acta Math. 155 (1985), 261–301. MR 87f:35098 3, 5, 10, 14, 25, 39 H. CARTAN, Théorie du potentiel newtonien: e´ nergie, capacité, suites de potentiels, Bull. Soc. Math. France 73 (1945), 74–106. MR 7:447h 24 G. CHOQUET, Theory of capacities, Ann. Inst. Fourier (Grenoble) 5 (1953), 131–295. MR 18:295g 33 M. G. CRANDALL, “Viscosity solutions: A primer” in Viscosity Solutions and Applications (Montecatini Terme, Italy, 1995), Lecture Notes in Math. 1660, Springer, Berlin, 1997, 1–43. MR 98g:35034 8, 23, 39 M. G. CRANDALL, H. ISHII, and P.-L. LIONS, User’s guide to viscosity solutions of second order partial differential equations, Bull. Amer. Math. Soc. (N.S.) 27 (1992), 1–67. MR 92j:35050 8, 39 B. GIDAS, W. M. NI, and L. NIRENBERG, Symmetry and related properties via the maximum principle, Comm. Math. Phys. 68 (1979), 209–243. MR 80h:35043 21 D. GILBARG and N. S. TRUDINGER, Elliptic Partial Differential Equations of Second Order, 2d ed., Grundlehren Math. Wiss. 224, Springer, Berlin, 1983. MR 86c:35035 4, 6, 9, 18 B. GUAN, The Dirichlet problem for a class of fully nonlinear elliptic equations, Comm. Partial Differential Equations 19 (1994), 399–416. MR 95c:35100 3, 4, 39 B. GUAN and J. SPRUCK, Boundary-value problems on S n for surfaces of constant Gauss curvature, Ann. of Math. (2) 138 (1993), 601–624. MR 94i:53039 4, 39 W. K. HAYMAN and P. B. KENNEDY, Subharmonic Functions, Vol. I, London Math. Soc. Monogr. 9, Academic Press, London, 1976. MR 57:665 6, 9, 24, 29 L. I. HEDBERG and TH. H. WOLFF, Thin sets in nonlinear potential theory, Ann. Inst.

48

DENIS A. LABUTIN

Fourier (Grenoble) 33 (1983), 161–187. MR 85f:31015 3, 5 [17]

[18] [19]

[20]

[21] [22] [23] [24] [25] [26] [27] [28]

[29]

[30]

[31]

[32]

¨ J. HEINONEN, T. KILPELAINEN, and O. MARTIO, Nonlinear Potential Theory of

Degenerate Elliptic Equations, Oxford Math. Monogr., Oxford Sci. Publ., Oxford Univ. Press, New York, 1993. MR 94e:31003 18, 30, 31, 34, 38, 40, 46 ¨ L. HORMANDER , Notions of Convexity, Progr. Math. 127, Birkhäuser, Boston, 1994. MR 95k:00002 3, 6, 9, 22, 23, 24, 29, 30, 31, 33 N. M. IVOCHKINA, Description of cones of stability generated by differential operators of Monge-Ampère type (in Russian), Mat. Sb. (N.S.) 122 (164), no. 2 (1983), 265–275; English translation in Math. USSR-Sb. 50, no. 1 (1985), 259–268. MR 85g:35043 3 , Solution of the Dirichlet problem for certain equations of Monge-Ampère type (in Russian), Mat. Sb. (N.S.) 128 (170), no. 3 (1985), 403–415, 447; English translation in Math. USSR-Sb. 56, no. 2 (1987), 403–415. MR 88a:35092 3 ¨ T. KILPELAINEN and J. MALY´ , The Wiener test and potential estimates for quasilinear elliptic equations, Acta Math. 172 (1994), 137–161. MR 95a:35050 3, 4, 15, 38 M. KLIMEK, Pluripotential Theory, London Math. Soc. Monogr. (N.S.) 6, Oxford Sci. Publ., Oxford Univ. Press, New York, 1991. MR 93h:32031 22, 31, 33 N. V. KRYLOV, Lectures on fully nonlinear elliptic equations, Rudolph Lipschitz Lectures, Univ. of Bonn, Germany, 1993. 3 D. A. LABUTIN, Pluripolarity of sets with small Hausdorff measure, Manuscripta Math. 102 (2000), 163–167. MR 2001f:32056 35 , Isolated singularities of solutions of fully nonlinear elliptic equations, to appear in J. Differential Equations. 19, 21 N. S. LANDKOF, Foundations of Modern Potential Theory, Grundlehren Math. Wiss. 180, Springer, New York, 1972. MR 50:2520 6, 24, 34, 37, 38 P. LINDQVIST and O. MARTIO, Two theorems of N. Wiener for solutions of quasilinear elliptic equations, Acta Math. 155 (1985), 153–171. MR 87g:35074 38 ´ and W. P. ZIEMER, Fine Regularity of Solutions of Elliptic Partial Differential J. MALY Equations, Math. Surveys Monogr. 51, Amer. Math. Soc., Providence, 1997. MR 98h:35080 4, 35, 38, 40, 43 V. G. MAZ’YA, The continuity at a boundary point of the solutions of quasi-linear elliptic equations (in Russian), Vestnik Leningrad. Univ. 25, no. 13 (1970), 42–55; English translation in Vestnik Leningrad. Univ. Math. 3 (1976), 225–242. MR 43:706 38, 46 , “Unsolved problems connected with the Wiener criterion” in The Legacy of Norbert Wiener: A Centennial Symposium (Cambridge, Mass., 1994), Proc. Sympos. Pure Math. 60, Amer. Math. Soc., Providence, 1997, 199–208. MR 98e:35040 38 V. G. MAZ’YA and V. P. HAVIN, A nonlinear potential theory (in Russian), Uspekhi Mat. Nauk 27, no. 6 (1972), 67–138; English translation in Russian Math. Surveys 27, no. 6 (1972), 71–148. MR 53:13610 3, 5 J. SERRIN, Isolated singularities of solutions of quasi-linear equations, Acta Math. 113 (1965), 219–240. MR 31:494 19


[33]

[34] [35] [36] [37] [38] [39] [40] [41]

[42]

[43] [44]

49

N. S. TRUDINGER, “A priori bounds for graphs with prescribed curvature” in Analysis,

et cetera: Research Papers Published in Honor of Jürgen Moser’s 60th Birthday, Academic Press, Boston, 1990, 667–676. MR 91a:53016 13 , The Dirichlet problem for the prescribed curvature equations, Arch. Rational Mech. Anal. 111 (1990), 153–179. MR 91g:35118 5, 8, 11, 15, 25, 39 , Isoperimetric inequalities for quermassintegrals, Ann. Inst. H. Poincaré Anal. Non Linéaire 11 (1994), 411–425. MR 95k:52013 31 , On the Dirichlet problem for Hessian equations, Acta Math. 175 (1995), 151–164. MR 96m:35113 3, 10, 25, 39 , On new isoperimetric inequalities and symmetrization, J. Reine Angew. Math. 488 (1997), 203–220. MR 99a:35076 4, 5, 6, 21, 25, 31 , Weak solutions of Hessian equations, Comm. Partial Differential Equations 22 (1997), 1251–1261. MR 99a:35077 5, 8, 15, 24 N. S. TRUDINGER and X.-J. WANG, Hessian measures, I, Topol. Methods Nonlinear Anal. 10 (1997), 225–239. MR 2000a:35061 1, 2, 3, 9, 14, 15, 19, 23, 24, 31, 39 , Hessian measures, II, Ann. of Math. (2) 150 (1999), 579–604. MR 2001f:35141 1, 2, 3, 4, 5, 6, 8, 9, 10, 15, 19, 21, 23, 24, 25, 29, 37, 39 , Hessian measures, III, preprint, 2000, Australian National University mathematics research report no. MRR00-016, http://www.math.anu.edu.au/research.reports/00mrr.html 1, 2, 3, 9, 19, 21, 24, 39 J. I. E. URBAS, On the existence of nonclassical solutions for two classes of fully nonlinear elliptic equations, Indiana Univ. Math. J. 39 (1990), 355–382. MR 92h:35074 5, 15, 19 ¨ AL ¨ A ¨ , Capacity and measure, Michigan Math. J. 22 (1975), 1–3. MR 51:8441 J. VAIS 35 X. J. WANG, A class of fully nonlinear elliptic equations and related functionals, Indiana Univ. Math. J. 43 (1994), 25–54. MR 95f:35089 5

Centre for Mathematics and its Applications, Australian National University, Canberra, Australian Capital Territory 0200, Australia; current: Departement Mathematik, Eidgenössische Technische Hochschule, CH-8092 Zürich, Switzerland; [email protected]


TWISTED VERTEX REPRESENTATIONS VIA SPIN GROUPS AND THE MCKAY CORRESPONDENCE IGOR B. FRENKEL, NAIHUAN JING, and WEIQIANG WANG

Abstract We establish a twisted analog of our recent work on vertex representations and the McKay correspondence. For each finite group 0 and a virtual character of 0, we construct twisted vertex operators on the Fock space spanned by the super spin characters of the spin wreath products 0 o e Sn of 0 and a double cover of the symmetric group Sn for all n. When 0 is a subgroup of SL2 (C) with the McKay virtual character, our construction gives a group-theoretic realization of the basic representations of the twisted affine and twisted toroidal Lie algebras. When 0 is an arbitrary finite group and the virtual character is trivial, our vertex operator construction yields the spin character tables for 0 o e Sn . 1. Introduction The connection between the direct sum of Grothendieck groups of the symmetric groups Sn for all n and the theory of symmetric functions (see [M], [Z]) has a simple interpretation in terms of a Heisenberg algebra and vertex operators (see [F2]; see also [J1, Part 1]). In the recent works [W] and [FJW2], we have realized a generalization of such a connection by substituting the symmetric group Sn with the wreath product 0n = 0 o Sn associated to an arbitrary finite group 0. Moreover, we introduced a crucial modification of this connection that, in the case when 0 is a finite subgroup of SL2 (C), yields a group-theoretic realization of the affine Lie algebra b g (see [FK], [Se]) and of the toroidal Lie algebra b g (see [F1], [MRY]), where g is a complex simple Lie algebra of ADE type whose Dynkin diagram is related to 0 via the McKay correspondence (see [Mc]). The main goal of the present work is to extend the above results to realize the basic representation of a twisted affine Lie algebra b g [−1] and its toroidal counterpart by means of a spin cover e 0n of the wreath product 0n associated to a subgroup 0 of DUKE MATHEMATICAL JOURNAL c 2002 Vol. 111, No. 1, Received 15 March 2000. 2000 Mathematics Subject Classification. Primary 17B65, (20C25); Secondary 20C30, 17B67, 05E05, 05E10, 20B30, 14C05, 14C17. Frenkel’s research supported by National Science Foundation grant number DMS-9700765. Jing’s research supported by National Science Foundation grant number DMS-9970493. 51

52

FRENKEL, JING, and WANG

SL2 (C). The twisting of the basic representation of the affine Lie algebra under consideration is determined by multiplication by −1 on the Cartan subalgebra h of g and can be viewed as an odd counterpart of the even (untwisted) case. This twisting was originally introduced as the first step toward the construction of the Moonshine module for the Monster group in [FLM1] and [FLM2]. As in the homogeneous case, one starts with a representation of the Heisenberg subalgebra b h[−1] and reconstructs the rest of the twisted affine Lie algebra b g [−1] using the twisted vertex operators. The representation theory of the spin group e Sn , which is a double cover of the symmetric group Sn , was initiated by I. Schur [S] (also see [Jo] for an exposition). Its connection with vertex operators was further studied in [J1]. These results play an important role in our present work. The representation theory of e 0n was also studied in [HH] from a Hopf algebra viewpoint. In order to work effectively with only the spin representations of e 0n , that is, those that do not factor through 0n , we adopt the approach of [Jo] by introducing a superalgebra structure on the group algebra of e 0n and considering its supermodules. It turns out that the superstructure is preserved under the main operations, such as induction and restriction. The direct sum of the Grothendieck groups of spin supermodules of e 0n carries a natural Hopf algebra, and we remark that a Hopf algebra was constructed in [HH] on a different space. This allows us to realize the vertex operators acting in the twisted vertex representations constructed from the sum of the Grothendieck rings. Our group-theoretic method naturally recovers the basic representations of twisted affine Lie algebras b g [−1] (see [LW], [FLM1], [FLM2]). As in [FJW2], we realize this by introducing a modified bilinear form associated to the McKay virtual character ξ which is twice the trivial character minus the character of the two-dimensional natural representation of 0 in SL2 (C). Much of our construction is valid for an arbitrary finite group 0, and we have introduced the modified bilinear form associated to an arbitrary virtual character ξ of 0 as well. In the special case when ξ is the trivial character, the twisted vertex operators generate an infinite-dimensional generalized Clifford algebra, which recovers the twisted boson-fermion correspondence. We further obtain the super character tables of the spin group e 0n for all n, generalizing the results of [J1]. One may generalize the results of this paper to the quantum case, as was done in [FJW1] for the homogeneous picture of quantum affine algebras (see [FJ]). Our results also suggest that various previous constructions associated to (quantum) vertex representations admit remarkable interpretation via Grothendieck rings of certain finite groups which are variations of wreath products, though every new step in this direction is unpredictable and brings new surprises. It is a very interesting and challenging problem to find such a group-theoretic realization.

TWISTED VERTEX OPERATORS AND MCKAY CORRESPONDENCE

53

The organization of the paper is as follows. In Section 2 we present the representation theory and structures of the spin group e 0n . In Section 3 we review superalgebras and supermodules and define the Hopf algebra of the super spin characters of e 0n . In Section 4 we introduce the weighted bilinear forms in the Grothendieck rings of supermodules and construct basic spin supermodules. In Section 5 we define the twisted Heisenberg algebras and their Fock spaces. In Section 6 we establish the isometry between the sum of Grothendieck rings of supermodules of e 0n and the Fock space of a twisted Heisenberg algebra. In Section 7 we construct twisted vertex operators via the induction and restriction functors on the Grothendieck rings. In Section 8 we obtain the twisted basic representation of the affine Lie algebras b g [−1] and the corresponding toroidal algebras. In Section 9 we derive the super spin character tables of e 0n for all n from the twisted boson-fermion correspondence.

2. A double cover of the wreath product 2.1. The spin group e Sn In this subsection we discuss some of the basic properties of the double covers of the symmetric group, which were introduced by Schur in his seminal paper [S]. We adopt the modern account (see [Jo]) of Schur’s theory. Let Sn be the symmetric group of n letters; we use the convention of multiplying permutations from right to left (different from [S], [Jo]). The spin group e Sn is the finite group generated by z and ti , i = 1, . . . , n − 1 subject to the relations z 2 = 1,

ti2 = (ti ti+1 )3 = z,

ti t j = zt j ti ,

i > j + 1,

zti = ti z.

(2.1) (2.2) (2.3)

Let θn be the homomorphism from e Sn to Sn sending ti to the transposition (i, i + e 1) and z to 1. We see that Sn is a central extension of Sn by the cyclic group Z2 : ι

θn

1 −→ Z2 −→ e Sn −→ Sn −→ 1, where the embedding ι sends the order 2 element in Z2 to z. Schur [S] determined that H 2 (Sn , C∗ ) ' Z2 for n > 3. The group e Sn is one of the two double covers of the symmetric group Sn (n > 3). Our results in this paper can be easily translated to the other double cover (cf. [S], [J1]). The group e Sn has a parity given as follows. Let d be the homomorphism from the free group generated by {ti , z} (i = 1, . . . , n−1) to Z2 by d(ti ) = 1, i = 1, . . . , n−1, and d(z) = 0. It is easily seen that d preserves the relations (2.1)–(2.3). Thus it defines

54


a homomorphism from e Sn to Z2 , which we still denote by d. An element x ∈ e Sn is called even (resp., odd) if d(x) = 0 (resp., d(x) = 1). The parity in e Sn given by d lifts the standard notion of even and odd permutations in the symmetric group Sn . The spin group e Sn has a cycle presentation due to J. H. Conway and others (see [Ws]). Embed e Sn into e Sn+1 by identifying their first n − 1 generators ti , i = 1, . . . , n − 1. For i = 1, . . . , n, we define xi = ti ti+1 · · · tn · · · ti+1 ti ∈ e Sn+1 . For a sequence i 1 , . . . , i m of distinct integers from {1, 2, . . . , n}, we can define cycles in e Sn as ( z, m = 1, [i 1 i 2 · · · i m ] = (2.4) xi1 xim xim−1 · · · xi1 , 1 < m ≤ n. It is known that θn ([i 1 i 2 · · · i m ]) = (i 1 i 2 · · · i m ) and θn+1 (xi ) = (i, n + 1). We list some useful identities for the cycles: x j [i 1 i 2 · · · i m ] = z m−1 [i 1 i 2 · · · i m ]x j ,

j 6 = i s , x 2j = z,

[i 1 i 2 · · · i m ]−1 = [i m · · · i 2 i 1 ], [i 1 i 2 · · · i m ] = z

m−1

[i 2 i 3 · · · i m i 1 ],

[i 1 i 2 · · · i m ][ j1 j2 · · · jk ] = z (m−1)(k−1) [ j1 j2 · · · jk ][i 1 i 2 · · · i m ], [i, i + 1, . . . , i + j − 1] = z

j−1

ti ti+1 · · · ti+ j−2 ,

(2.5) (2.6) (2.7) (2.8) (2.9)

where the cycles [i 1 i 2 · · · i m ] and [ j1 j2 · · · jk ] are disjoint. PROPOSITION 2.1 ([Jo]) Each element of e Sn can be presented as

z p [i 1 i 2 · · · i m ][ j1 j2 · · · jk ] · · · , where {i 1 · · · i m }, { j1 · · · jk }, . . . is a partition of the set {1, 2, . . . , n} and p = 0, 1. 0 If z p c1 c2 · · · cl = z p c10 c20 · · · cl00 are two expressions of the same element in terms of cycles ci and ci0 , then l = l 0 and there is a permutation σ ∈ Sl such that ci0 = cσ (i) z m i ,

m i ≡ |ci | − 1 (mod 2),

where |ci | denotes the length of the cycle ci . Moreover, if [i 1 · · · i k ] = z m [ j1 · · · jk ], then js = σ (i s ) for a cyclic permutation σ of {i 1 , . . . , i k }. Let λ be a partition; we identify λ with its Young diagram consisting of l rows of λ1 , . . . , λl squares, respectively, aligned to the left. A tableau Tλ of shape λ is a numbering of the squares with integers 1, 2, . . . , |λ|, each appearing exactly once. For each tableau Tλ of shape λ with a numbering a11 , . . . , a1λ1 , a21 , . . . , a2λ2 , . . . , al1 , . . . , alλl , we define the element tλ of e Sn to be tλ = [a11 · · · a1λ1 ][a21 · · · a2λ2 ] · · · [al1 · · · alλl ].

(2.10)


55

Q The permutation li=1 (ai1 · · · aiλi ) associated with tλ is denoted by s(λ). It follows from Proposition 2.1 that the general element in e Sn is of the form z p tλ . For a permuQl s tation s ∈ Sn , we also define tλ = i=1 [s(ai1 ) · · · s(aiλi )]. The following can be checked by induction using (2.5) and (2.8). 2.2 For any two elements tλ , tµ in e Sn associated to tableaux Tλ and Tµ , we have LEMMA

s(µ)

tµ tλ tµ−1 = z d(λ)d(µ) tλ

.

2.2. The spin wreath product e 0n In this subsection we introduce e 0n , the main finite group in this work, and we extend our discussion from e Sn to e 0n . Let 0 be a finite group with r + 1 conjugacy classes. We denote by 0 ∗ = {γi }ri=0 the set of complex irreducible characters, where γ0 stands for the trivial character, and by 0∗ the set of conjugacy classes. The character value γ (c) of γ ∈ 0 ∗ at a conjugacy class c ∈ 0∗ yields the character table {γ (c)} of 0. L Let R(0) = ri=0 Cγi be the space of complex-valued class functions on 0. For c ∈ 0∗ , let ζc be the order of the centralizer of an element in the class c, so the order of the class is then |0|/ζc . The usual bilinear form on R(0) is defined as follows: h f, gi0 =

X 1 X f (x)g(x −1 ) = ζc−1 f (c)g(c−1 ), |0| x∈0

c∈0∗

{x −1 , x

where denotes the conjugacy class ∈ c}. Clearly, ζc = ζc−1 . We often write h , i for h , i0 when no ambiguity may arise. It is well known that c−1

hγi , γ j i = δi j , X

γ (c )γ (c−1 ) = δc,c0 ζc , 0

c, c0 ∈ 0∗ .

(2.11)

γ ∈0 ∗

L Thus RZ (0) = ri=0 Zγi endowed with this bilinear form becomes an integral lattice in R(0). Given a positive integer n, let 0 n = 0 × · · · × 0 be the nth direct product of 0, and let 0 0 be the trivial group. The spin group e Sn acts on 0 n through the action of the group Sn by permuting the indices: tλ (g1 , . . . , gn ) = (gs(λ)−1 (1) , . . . , gs(λ)−1 (n) ), and z(g1 , . . . , gn ) = (g1 , . . . , gn ). The wreath product e 0n = 0 o e Sn of 0 with e Sn is defined to be the semidirect product e 0n = 0 n o e Sn = (g, t)|g = (g1 , . . . , gn ) ∈ 0 n , t ∈ e Sn with the multiplication (g, t) · (h, s) = g t (h), ts .

56


Note that e 0n reduces to e Sn when 0 is trivial. Clearly, e 0n is a central extension of 0n e by Z2 and |0n | = 2n!|0|n . We define a a parity for e 0n by extending the parity of e Sn . Let d : e 0n −→ Z2 = e {0, 1} be the homomorphism from 0n to Z2 given by d(g, ti ) = 1,

d(g, z) = 0.

(2.12)

en , where An is the Clearly, the degree zero subset e 0n0 is the wreath product 0 o A 1 alternating group, and the degree one part e 0n is the complementary subset. e Let τ be a section from 0n to 0n such that θτ = 1. An element x ∈ 0n is called split if τ (x) is not conjugate to zτ (x). Otherwise, x is said to be nonsplit. Clearly, this definition does not depend on the choice of the section τ , and two conjugate elements are simultaneously split or nonsplit. A conjugacy class of 0n is called split if its elements are split. We also say that an element x ∈ e 0n is split (resp., nonsplit) if θ (x) is split (resp., nonsplit). Clearly, the class Cρ splits if and only if the preimage θn−1 (Cρ ) splits into two conjugacy classes in e 0n . Any representation π of 0n can be viewed as a representation of e 0n . Such a representation π of e 0n satisfies the property π(z) = Id. A representation π of e 0n is called spin if π(z) = − Id . It follows that the characters of spin representations vanish on nonsplit classes. In this paper we consider only spin representations. We remark that spin representations are sometimes referred to as negative or projective representations in the literature. 2.3. Conjugacy classes of e 0n Let λ = (λ1 , λ2 , . . . , λl ) be a partition of the integer |λ| = λ1 + · · · + λl , where λ1 ≥ · · · ≥ λl ≥ 1. The integer l is called the length of the partition λ and is denoted by l(λ). We identify the partition (λ1 , λ2 , . . . , λl ) with (λ1 , λ2 , . . . , λl , 0, . . . , 0). We also make use of another notation for partitions: λ = (1m 1 2m 2 · · · ), where m i is the number of parts in λ equal to i. The number n(λ0 ) is defined to be P λi 0 i 2 , where λ is the dual partition associated to λ. We use the dominance order on partitions. For two partitions λ and µ, we write λ > µ if λ1 ≥ µ1 , λ1 + λ2 ≥ µ1 + µ2 , and so on. A partition λ is called strict if its parts are distinct integers (excluding the trivial parts of zero), in which case all the multiplicities m i are 1 or 0. We use partitions indexed by 0∗ and 0 ∗ . For a finite set X and ρ = (ρ(x))x∈X a family of partitions indexed by X , we write X ρ(x) . kρk = x∈X


57

It is convenient to regard ρ = (ρ(x))x∈X as a partition-valued function on X . We denote by P (X ) the set of all partitions indexed by X and by Pn (X ) the set of all partitions in P (X ) such that kρk = n. The total number of parts, denoted by P l(ρ) = x l(ρ(x)), in the partition-valued function ρ = (ρ(x))x∈X is called the length of ρ. The dominance order is extended to partition-valued functions as follows. We define ρ > π if ρ(x) > π(x) for each x. We say that ρ π if ρ(x) > π(x) and ρ(x) 6 = π(x) for each x ∈ X . For a partition-valued function ρ, we define X X ρi (c) 0 0 n(ρ ) = n ρ(c) = . (2.13) 2 c c,i

Let OP (X ) be the set of partition-valued functions (ρ(x))x∈X in P (X ) such that all parts of the partitions ρ(x) are odd integers, and let S P (X ) be the set of partition-valued functions ρ : X −→ P such that each partition ρ(x) is strict. When X consists of a single element, we omit X and simply write P for P (X ); thus the notation OP or S P are used similarly. LEMMA 2.3 We have |OP n (X )| = |S P n (X )|.

Proof The generating function of the cardinalities of strict partition-valued functions is ∞ YY

1

2n−1 x∈X n=1 1 − q x

=

∞ YY

1 − qx2n

2n−1 )(1 − qx2n ) x∈X n=1 (1 − q x

=

∞ YY

(1 + qxn ),

x∈X n=1

which is the generating function of S P n (X ). We also define a parity on partitions. For each partition λ we define d(λ) = |λ| −l(λ). P For a partition-valued function ρ = (ρ(x))x∈X , we define d(ρ) = x |ρ(x)| = kρk − l(ρ). It is clear that the conjugacy class of type λ in Sn is even if and only if d(λ) is even. We define the parity of the partition-valued function ρ to be the parity of d(ρ). We define Pn0 (X ) = λ ∈ Pn (X )| d(ρ) ≡ 0 (mod2) , (2.14) Pn1 (X ) = λ ∈ Pn (X )| d(ρ) ≡ 1 (mod2) , (2.15) and we define S P in (X ) = Pni (X ) ∩ S P n (X ) for i = 0, 1. We now recall the description of conjugacy classes of 0n [M]. Let x = (g, σ ) be an element in a conjugacy class of 0n , where g = (g1 , . . . , gn ). For each cycle y = (i 1 i 2 · · · i k ) in the permutation σ , the element g y = gik gik−1 · · · gi1 ∈ 0 is called

58


the cycle-product of x corresponding to the cycle y. For each c ∈ 0∗ and i ≥ 0, let m i (c) be the number of i-cycles in the permutation σ such that the cycle products g y lie in the conjugacy class c. Then c → ρ(c) = (1m 1 (c) 2m 2 (c) · · · ) defines a partitionvalued function on 0∗ . It is known that the partition-valued function (ρ(c))c∈0∗ is in one-to-one correspondence to the conjugacy class of x = (g, σ ) in 0n and is called the type of the conjugacy class. We also say that an element has conjugacy type ρ if this element is contained in the conjugacy class. Let (−1)d be the representation of e 0n given by x 7 −→ (−1)d(x) . A representation π of e 0n is called a double spin representation if (−1)d π ' π. If π 0 = (−1)d π 6= π, then π 0 and π are called associate spin representations of e 0n . The following result was proved in [J1] for a double cover of any finite group. PROPOSITION 2.4 The number of split conjugacy classes of 0n is equal to the number of irreducible spin representations of e 0n .

2.4. Split conjugacy classes of e 0n We fix an order of conjugacy classes of 0: c0 = {1}, c1 , . . . , cr . For each partitionvalued function ρ = (ρ(c)) ∈ Pn (0∗ ), we let tρ(ci ) be the element of e Sn associP j )| +1, . . . , |ρ(c ated to a tableau Tρ(ci ) of shape ρ(ci ) using the numbers j≤i−1 P j j≤i |ρ(c )|, and we define the element tρ to be tρ = tρ(c0 ) tρ(c1 ) · · · tρ(cr ) ,

(2.16)

which depends on the sequence Tρ of the tableaux Tρ(c0 ) , . . . , Tρ(cr ) . We remark that the general element of e 0n is of the form (g, z p tρ ), where ρ is the type of the conjugacy p class of (g, z tρ ). The following theorem is well known in the case of 0 = {1} (cf. [Jo], [St]). 2.5 Let ρ = (ρ(c))c∈0∗ be the type of a conjugacy class Cρ in 0n . Then the preimage θn−1 (Cρ ) splits into two conjugacy classes in e 0n if and only if (1) the class Cρ is even and all the ρ(c) (c ∈ 0∗ ) are partitions with odd integer parts, that is, ρ ∈ OP n (0∗ ); (2) the class Cρ is odd and all the ρ(c) (c ∈ 0∗ ) are strict partitions, that is, ρ ∈ S P 1n (0∗ ). THEOREM

Proof (1) Let d(ρ) be even, and let each partition ρ(c) have odd integer parts. Assume on


59

the contrary that (g, tρ ) and z(g, tρ ) are conjugate in θn−1 (Cρ ), where tρ is associated to a sequence of tableaux (see (2.16)). Then for some (h, tµ ) ∈ e 0n , (h, tµ )(g, tρ )(h, tµ )−1 = h · s(µ)(g) · s(µ)s(ρ)s(µ)−1 (h −1 ), tµ tρ tµ−1 = h · s(µ)(g) · s(ρ)(h −1 ), tµ tρ tµ−1 = (g, ztρ ), (2.17) where we have used the fact that s(ρ)s(µ) = s(µ)s(ρ). It follows from Lemma 2.2 s(µ) s(µ) that ztρ = z d(ρ)d(µ) tρ = tρ , since d(ρ) = 0 (mod 2). Let tρ = c1 c2 · · · cl and s(µ) 0 tρ = c10 c20 · · · cl0 be their cycle representations. Then ci = z m i cν(i) and m i = |ci |−1 (mod 2) for some ν ∈ Sl by Proposition 2.1. Since each cycle length |ci | is odd, all 0 the cycles mutually commute with each other. Substituting ci = z m i cν(i) back and rearranging the cycles, we have 1 = z 1+

P

i

(|ci |−1)

= z 1+d(ρ) = z,

which is a contradiction. Now suppose that for some c ∈ 0∗ there is an even cycle in ρ(c) of the class Cρ of type ρ. That is, there is an element (g, tρ ) ∈ θ −1 (Cρ ) such that tρ = · · · [i 1 i 2 · · · i 2k ] · · · . Consider the element (h, tµ ) ∈ e 0n , where tµ = [i 1 i 2 · · · i 2k ] and h = (h 1 , . . . , h n ) with h j = 1 for j 6 = i s and h is = gis , s = 1, . . . , 2k. We claim that (h, tµ )(g, tρ )(h, tµ )−1 = hs(µ)(g)s(ρ)(h −1 ), tµ tρ tµ−1 = (g, ztρ ), which is shown by two steps. First we consider the jth component of hs(µ)(g) ·s(ρ)(h −1 ) in 0 n . It equals 1·g j ·1 = g j when j 6= i s , and it equals gis gis−1 gi−1 = gis s−1 for j = i s . Second we have tµ tρ tµ−1 = · · · [i 2 i 3 · · · i 2k i 1 ] · · · = ztρ by using (2.7) and d(ρ) ≡ 0 (mod 2) again. Thus (g, tρ ) is conjugate to z(g, tρ ). Therefore all partitions ρ(c) must be from OP (0∗ ) if θ −1 (Cρ ) splits. (2) Let d(ρ) be odd. Assume all partitions ρ(c) are strict partitions. If on the contrary (g, tρ ) is conjugate to z(g, tρ ), then using d(ρ) = 1 we, have as in (2.17), s(µ) ztρ = z d(µ) tρ for the permutation s(µ) ∈ Sn associated to some µ ∈ Pn (0∗ ). s(µ) Let tρ = c1 c2 · · · cl and tρ = c10 c20 · · · cl0 be their cycle representations. Then ci = 0 z |ci |−1 cν(i) for some ν ∈ Sl . Since s(µ) cyclically permutes the indices in each cycle of s(ρ), we have d(µ) = d(ρ). On the other hand, note that each cycle ci corresponds to one part in ρ(c) for some c ∈ 0∗ , and any conjugation of ci still corresponds 0 to a part in the same ρ(c). When we plug the equations ci = z |ci |−1 cν(i) back into s(µ)

ztρ = z d(µ) tρ

, we see that ν is actually the identity since ρ(c) is strict. Therefore

60


P P z 1+ i (|ci |−1) = z d(µ) . Then d(ρ) = i (|ci | − 1) ≡ d(µ) + 1 (mod 2), which is a contradiction. Hence θn−1 (Cρ ) splits. Now suppose θn−1 (Cρ ) splits. If there are two identical parts in ρ(c) for some conjugacy class c ∈ 0∗ , say, tρ = · · · [i 1 · · · i k ][ j1 · · · jk ] · · · for (g, tρ ) ∈ e 0n , then the cycle-products of these two identical parts are conjugate; that is, there exists an element x ∈ 0 such that

xg jk g jk−1 · · · g j1 x −1 = gik gik−1 · · · gi1 .

(2.18)

Consider the element (h, tµ ) such that tµ = [i 1 j1 ] · · · [i k jk ] and h a = 1 for a 6= i s , js , and h is = gis · · · gi1 x(g js · · · g j1 )−1 , h js = g js · · · g j1 x

−1

(gis · · · gi1 )

−1

s = 1, . . . , k, ,

s = 1, . . . , k.

Clearly, h ik = x and h jk = x −1 by (2.18). Therefore we have the following equations for s = 1, 2, . . . , k (mod k): h is = gis h is−1 g −1 js ,

h js = g js h js−1 gi−1 , s

(2.19)

which imply that hs(µ)(g)s(ρ)(h −1 ) = g. Note also that s(ρ)s(µ) = s(µ)s(σ ) and d(µ) = k (mod 2). We see the conjugation (h, tµ )(g, tρ )(h, tµ )−1 = hs(µ)(g)s(ρ)(h −1 ), z k tρs(µ) = (g, z k tρs(µ) ), (2.20) where we used d(ρ) = 1. Observe that by (2.8), tρs(µ) = · · · [ j1 · · · jk ][i 1 · · · i k ] · · · = z (k−1) tρ = z k−1 tρ . 2

Plugging this into (2.20), we obtain (h, tµ )(g, tρ )(h, tµ )−1 = z(g, tρ ), and this contradiction says that each partition ρ(c) must be strict. Let Cρ be a conjugacy class in 0n of type ρ = (ρ(c))c∈0∗ ∈ Pn (0∗ ). We fix an order i of the conjugacy classes of 0 as before: c0 , . . . , cr . Let T ρ(c ) be the special tableau Pi−1 P i j j such that the numbers j=0 |ρ(c )| appear in the natural j=0 |ρ(c )| + 1, . . . , order from left to right and top to bottom in the Young diagram of shape ρ(ci ); thus i t ρ(c ) = 1 + ai−1 , . . . , ρ(ci )1 + ai−1 · · · ρ(ci )1 + · · · + ρ(ci )l−1 + ai−1 , . . . , ρ(ci ) + ai−1 , (2.21) where ai−1 = element t ρ by

Pi−1

j=0 |ρ(c

j )|

and ρ(ci ) = (ρ(ci )1 , . . . , ρ(ci )l ). We define the special t ρ = t ρ(c ) t ρ(c ) · · · t ρ(c ) . 0

1

r

(2.22)


61

For each split conjugacy class Cρ in 0n of type ρ, we define the conjugacy class Dρ+ in e 0n to be the conjugacy class containing the element (g, t ρ ). We also define − Dρ = z Dρ+ . Then θn−1 (Cρ ) = Dρ+ ∪ Dρ− . Let (Dρ+ )−1 = {x −1 |x ∈ Dρ+ }. We remark 0 that (Dρ+ )−1 = z n(ρ ) (Dρ )+ , where n(ρ 0 ) is defined in (2.13) and ρ is the partitionvalued function given by ρ(c) = ρ(c−1 ). Given a partition λ = (1m 1 2m 2 · · · ), we define Y zλ = i m i m i !. i≥1

We note that z λ is the order of the centralizer of an element of cycle type λ in S|λ| . For each partition-valued function ρ = (ρ(c))c∈0∗ , we define Y Zρ = z ρ(c) ζcl(ρ(c)) , c∈0∗

which is the order of the centralizer of an element of conjugacy type ρ = (ρ(c))c∈0∗ (see [M]). 2.6 The order of the centralizer of an element of conjugacy type ρ in e 0n is given by ( 2Z ρ , Cρ split, Z˜ ρ = Z ρ , Cρ nonsplit. PROPOSITION

Proof Let Cρ be a conjugacy class in 0n . If θ −1 (Cρ ) does not split, then θ −1 (Cρ ) is a conjugacy class in e 0n , so its centralizer has the order |e 0n |/|θ −1 (Cρ )| = |0n |/|Cρ | = Z ρ . Otherwise, θ −1 (Cρ ) = Dρ+ ∪ Dρ− , and |e 0n |/|Dρ± | = 2Z ρ . 2.7 ([HH]) The number of conjugacy classes of 0n and 0n 0 are given by the following formulas: |split classes of the group 0n | = 2 S P 1n (0∗ ) + S P 0n (0∗ ) , |split classes of the group 0n 0 | = S P 1 (0∗ ) + 2 S P 0 (0∗ ) .

THEOREM

(1)

n

(2)

n

The number of irreducible double spin representations of e 0n is equal to the number of even strict partition-valued functions on 0∗ , and the number of pairs of irreducible associate spin representations is equal to the number of odd strict partition-valued functions on 0∗ .

62


Proof The first statement in part (1) is a corollary of Theorem 2.5 and Lemma 2.3. To see the second equation in (1), we observe that an irreducible spin representation π decomposes as follows when restricting to the subgroup e 0n0 : ( π1 ⊕ π2 , π a double spin, π|e 0n0 = π, π an associate spin. Moreover, a pair of the associated spin representations, when restricted to e 0n0 , become the same irreducible representation. Applying the counting formulas in (1), we obtain part (2). We remark that the number of split conjugacy classes of 0n contained in 0n 0 is equal to |S P 1n (0∗ )| + |S P 0n (0∗ )| = |OP n (0∗ )| by Theorem 2.5.

3. The Hopf algebra R0− of super spin characters 3.1. Superalgebras and supermodules We basically follow the exposition of [Jo] in this subsection. A complex superalgebra A = A0 ⊕ A1 is a Z2 -graded complex vector space with a binary product A× A −→ A such that Ai A j ⊂ Ai+ j . A vector space V = V0 ⊕ V1 is a supermodule for a superalgebra A = A0 ⊕ A1 if Ai V j ⊂ Vi+ j . Elements of Vi are called homogeneous. A linear map f : M → N between two A-supermodules is a superhomomorphism of degree i if f (M j ) ⊂ Mi+ j , and for any homogeneous element a ∈ A and any homogeneous vector m ∈ M, we have f (am) = (−1)d(a)d( f ) a f (m). Let Hom A (M, N ) = Hom A (M, N )0 ⊕ Hom A (M, N )1 , where Hom A (M, N )i consists of A-superhomomorphisms of degree i from M to N . Let V = V0 ⊕ V1 and W = W0 ⊕ W1 be two supermodules. The tensor product P V ⊗ W is also a supermodule with (V ⊗ W )i = k+l=i (mod 2) Vk ⊗ Wl . Submodules and irreducible or simple supermodules are defined similarly, as usual. Two examples of complex simple superalgebras are given in order. Let r, s ∈ N. We define M(r |s) to be the C-superalgebra of (r + s)-square matri-


63

ces with the grading M(r |s)0 = M(r |s)1 =

A 0

0 C

0 |A ∈ Mr,r (C), D ∈ Ms,s (C) , D B |B ∈ Mr,s (C), C ∈ Ms,r (C) , 0

and the operations are the underlying usual matrix addition and multiplication. As in the ungraded case, M(r |s) can also be viewed as the superalgebra of Z2 -graded linear maps of Cr |s = Cr ⊕ Cs with the usual superpositions of maps. It is easily seen that M(r |s) is a simple C-superalgebra and Cr |s is a simple M(r |s)-supermodule. Another example is the C-superalgebra Q(n). As a supervector space it is defined by A 0 Q(n)0 = |A ∈ Mn,n (C) , 0 A 0 B Q(n)1 = |B ∈ Mn,n (C) . B 0 The superalgebra structure is given by the usual matrix multiplication. The space Cn|n is also a Q(n)-supermodule under the usual matrix multiplication. C. Wall [Wl] showed that these two simple superalgebras are the only two types of simple superalgebras over C. In the sequel we call the supermodule Cr |s of type M if it is considered as a M(r |s)-supermodule and Cn|n of type Q if it is considered as a Q(n)-supermodule. For any finite group G and a subgroup H of index 2, we set the parity of elements of H (resp., G\H ) to be even (resp., odd). The corresponding group superalgebra of G is semisimple (see [Jo]). In the case of the spin wreath product e 0n and the subgroup e 0n0 = 0 n o Añ , this parity agrees with the parity given by the homomorphism d (see (2.12)). As a superalgebra, C[e 0n ] is given by nX o C[e 0n ]0 = ag g|g ∈ e 0n0 , (3.1) g

C[e 0n ]1 =

nX g

and the product is the usual multiplication. PROPOSITION

3.1

o ag g|g ∈ e 0n1 ,

(3.2)

64


There exists an isomorphism of C-superalgebras M M C[e 0n ] ' M(ri |si ) ⊕ Q(n j ). i

j

Any finite-dimensional C[e 0n ]-supermodule is isomorphic to a direct sum of simple supermodules of type M and Q. By the definition of spin representations and Lemma 2.3, we know the number of irreducible spin supermodules for e 0n . PROPOSITION 3.2 The number of irreducible spin supermodules of e 0n is equal to |S P n (0∗ )|, the number of strict partition-valued functions on 0∗ . If V is an irreducible e 0n -supermodule e of type M, then its underlying 0n -module is irreducible. If V is an irreducible e 0n supermodule of type Q, then its underlying e 0n -module decomposes into two irreducible e 0n -modules U and U 0 , where U 0 = U is a vector space and its action is given by a · u = (−1)d(a) au for any homogeneous element a ∈ e 0n .

3.2. Induced supermodules Let G be a finite group with a central involution z and a parity epimorphism d : G −→ Z2 such that d(z) = 0. Let H be a subgroup of G containing z such that the restriction of d on H is not identically zero. Such a pair (G, H ) of finite groups is called an admissible pair of finite groups. Group algebras C[G] and C[H ] become superalgebras with G 0 = ker(d), H 0 = ker(d| H ), and G 1 = G\G 0 , H 1 = G\H 0 . Let W be a C[H ]-supermodule. We define the induced supermodule IndG H W for C[G] by IndG (3.3) H W = C[G] ⊗C[H ] W with the action given by g(h ⊗ w) = gh ⊗ w. Clearly, IndG H W is a spin supermodule if W is a spin supermodule. The following lemma can be checked in the same way as in the ordinary case (see [Sr]). LEMMA 3.3 Let (G, H ) be an admissible pair of finite groups. Let V be a C[G]-supermodule and W a C[H ]-sub-supermodule of V | H . Then V is equal to IndG H W if and only if M V = sW, (3.4) s∈G/H

where sW denotes the subspace x · W (x ∈ s) of the supermodule V .


65

Let (G, K ) be another admissible pair of finite groups. Consider the double cosets H s K of H and K in G. For s ∈ H \G/K the set Hs := s −1 H s ∩ K is a subgroup of K . The following analog of Mackey’s theorem can be proved in the same way as in the ordinary case using Lemma 3.3. 3.4 Let (G, H ) and (G, K ) be admissible pairs of finite groups as above. Then we have M Res K IndG Ind KHs Res Hs W (3.5) H W ' PROPOSITION

s∈H \G/K

as supermodules. 3.3. The space R − (e 0n ) A spin class function on e 0n is a class function map from e 0n to C such that f (zx) = − f (x). Thus spin class functions vanish on nonsplit conjugacy classes. A spin super class function on e 0n is a spin class function f on e 0n such that f vanishes further on odd strict conjugacy classes. In other words, f corresponds to a complex functional on OP n (0∗ ) in view of Theorem 2.5. Let R − (e 0n ) be the C-span of spin super class functions on e 0n . Let R(e 0n ) be the C-span of class functions on e 0n . Let R 0 (e 0n ) be the subspace of the class functions f (x) such that f (zx) = f (x), x ∈ e 0n , and let R 1 (e 0n ) be the space of spin class functions. Then we have R(e 0n ) = R 0 (e 0n ) ⊕ R 1 (e 0n ), R − (e 0n ) ⊂ R 1 (e 0n ),

R 0 (e 0n ) ' R(0n ).

In this paper we focus on the space R − (e 0n ). We remark that R 1 (e 0n ) can be identified as a vector space with the Grothendieck ring of spin representations of e 0n , and it is not difficult to recover R 1 (e 0n ) from R − (e 0n ) using Proposition 3.2. The standard inner product h | i on R(e 0n ) induces an inner product on R − (e 0n ). For two spin super class functions f, g ∈ R − (e 0n ), we define h f, gi = h f, gie 0n X 1 1 = f (x)g(x −1 ) = h f, gie 0n0 , 2 |e 0n | e0

(3.6)

x∈0n

where h f, gie 0n0 is the inner product of f |e 0n0 and g|e 0n0 in the space of class functions on 0 e the subgroup 0n . Since even split conjugacy classes of e 0n have the form {Dρ+ }∪{Dρ− }

66


and f (Dρ+ ) = − f (Dρ− ), we can rewrite the inner product by using Proposition 2.6: h f, gi =

X ρ∈OP n (0∗ )

1 f (ρ)g(ρ), Zρ

(3.7)

where f (ρ) = f (Dρ+ ) and ρ is defined as in Section 2.4. In the sequel we fix the value of a class function at ρ ∈ OP n (0∗ ) to be the value at the conjugacy class Dρ+ . Let V = V0 ⊕ V1 be a e 0n -supermodule. We define the character of V as the function χV : x 7 → tr (x), x ∈ e 0n . Clearly, χV (e 0n1 ) = 0. 3.5 The characters of irreducible spin e 0n -supermodules form a C-basis of R − (e 0n ). Let φ and γ be two irreducible characters of spin supermodules; then    1 if φ ' γ , type M, hφ, γ i = 2 if φ ' γ , type Q, (3.8)   0 otherwise. PROPOSITION

Conversely, if h f, f i = 1 for f ∈ R − (e 0n ), then ± f affords an irreducible spin e 0n -supermodule of type M. If h f, f i = 2, then either ± f is the character of an irreducible spin supermodule of type Q or f is a sum or difference of two irreducible characters of spin supermodules of type M. Proof Let ξc be the characteristic function on the conjugacy class c. Then R − (e 0n ) is spanned by ξc , where c ranges over the set of split even classes. Thus dim(R − (e 0n )) ≤ |OP n (0∗ )|. On the other hand, we see that the characters of spin supermodules are class functions in R − (e 0n ) since the trace of any odd endomorphism is zero. Let φ and γ be the characters of two irreducible spin supermodules of e 0n . It follows from Proposition 3.2 that the underlying module of φ or γ is either an irreducible module or the sum of two associated irreducible modules according to their types, which implies immediately the orthogonality relation (3.8). Therefore the matrix of the inner product is orthogonal on the set of super spin characters. Then by Lemma 2.3 and Proposition 3.2, dim R − (e 0n ) ≥ S P n (0∗ ) = OP n (0∗ ) . Thus the two inequalities above become equal, and so the irreducible characters of spin e 0n -supermodules form a Z-basis in R − (e 0n ). The last characterization of irreducible supermodules follows from the semisimplicity of the superalgebra C[e 0n ] and the usual orthogonality of ordinary irreducible characters.


67

3.4. Hopf algebra structure on R0− We now define one of our main objects: M R0− = R − (e 0n ). n≥0

˜e 0m be the direct product of e 0n and e 0m with a twisted multiplication Let e 0n × (t, t 0 ) · (s, s 0 ) = (tsz d(t )d(s) , t 0 s 0 ), 0

˜e where s, t ∈ e 0n , s 0 , t 0 ∈ e 0m are homogeneous. Note that |e 0n × 0m | = |e 0n ||e 0m |. We define the spin product of e 0n and e 0m (see [HH]) by e ê ˜e 0n × 0m = e 0n × 0m / (1, 1), (z, z) , (3.9) which can be embedded into the spin group e 0n+m canonically by letting (g, ti0 ), 1 7 → (g, ti ), 1, (g, t 00j ) 7 → (g, tn+ j ),

(3.10)

ê where i = 1, . . . , n − 1, j = 1, . . . , m − 1. We identify e 0n × 0m with its image in e e e ê 0m+n and regard it as a subgroup of 0m+n . Clearly, θn+m (0n × 0m ) is the pullback of 0n × 0m . Remark 3.6 Partition {1, 2, . . . , n + m} into a disjoint union of subsets I and J with |I | = n and ê |J | = m. Then e 0n × 0m can be embedded into e 0n+m using a map similar to (3.10) by mapping the generators of e Sn and e Sm to the generators of e Sn+m indexed by I and ê J , respectively. One can check that all such embeddings of e 0n × 0m are conjugate e subgroups in 0n+m . ê The subgroup e 0n × 0m has a distinguished subgroup of index 2 consisting of even ê elements given by d. We define R − (e 0n × 0m ) to be the space of spin class functions ê ê on e 0n × 0m that vanish on odd conjugacy classes of e 0n × 0m . e For two spin supermodules U and V of 0n and e 0m , we define the super (outer) tensor product U ⊗ V by (t, s) · (u ⊗ v) = (−1)d(s)d(u) (tu ⊗ sv), where s and u are homogeneous elements. We see immediately that (z 0 , z 00 ) · (u ⊗ v) = (−u) ⊗ (−v) = u ⊗ v, (z, 1) · (u ⊗ v) = −(u ⊗ v). ê This says that U ⊗ V is a spin (e 0n × 0m )-supermodule. The following proposition is a direct generalization of a result in [Jo] for trivial 0.

68


PROPOSITION 3.7 Let U and V be simple supermodules for e 0n and e 0m , respectively. Then we have the following: ê (1) if both U and V are of type M, then U ⊗ V is a simple (e 0n × 0m )-supermodule of type M; ê (2) if U and V are of different type, then U ⊗V is a simple (e 0n × 0m )-supermodule of type Q; ê (3) if both U and V are type Q, then U ⊗ V ' N ⊕ N for some simple (e 0n × 0m )supermodule N of type M.

The super (outer) tensor product defines an isometric isomorphism φ

n,m ê R − (e 0n ) ⊗ R − (e 0m ) → R − (e 0n × 0m ),

(3.11)

which is actually an isomorphism over Q by Proposition 3.7. The space R0− carries a multiplication defined by the composition φn,m

Ind

ê m : R − (e 0n ) ⊗ R − (e 0m ) −→ R − (e 0n × 0m ) −→ R − (e 0n+m )

(3.12)

and a comultiplication defined by the composition Res

n M

φ −1

m=0 n M

1 : R − (e 0n ) −→ −→

ê R − (e 0n−m × 0m ) (3.13) R (e 0n−m ) ⊗ R (e 0m ). −

−

m=0

Here Ind and Res denote the induction (see (3.3)) and restriction functors, respecL −1 tively. The isomorphism φ −1 is equal to 0≤m≤n φn−m,m (see (3.11)). 3.8 The above operations define a Hopf algebra structure for R0− . THEOREM

Proof Using Remark 3.6 twice, we observe that the following two embeddings give rise to two conjugate subgroups in e 0n+m+l (see Remark 3.6): ê ê ˆ e ê (e 0n × 0m )× 0l ,→ e 0n+m+l ←- e 0n ×( 0m × 0l ). Using this and Lemma 3.3, we can easily check the associativity of the product. For a simple supermodule V , we define ( 0 if V is type M, c(V ) = (3.14) 1 if V is type Q.


69

Let V1 , V2 , and V3 be simple supermodules for C[e 0n ], C[e 0m ], and C[e 0l ], respectively. It is easy to see that c(V1 , V2 ) = c(V1 )c(V2 ) satisfies the cocycle condition c(V1 , V2 ) + c(V1 ⊗ V2 , V3 ) = c(V2 , V3 ) + c(V1 , V2 ⊗ V3 ).

(3.15)

Therefore we can define c(V1 ⊗ V2 ⊗ V3 ) to be either of the above expressions. Using the cocycle c, we prove the coassociativity as follows. Let U be a C[e 0n ]L supermodule, and suppose that Rese ê ê i Ui (m, l, k) as an irreducible 0m × 0l × 0k U = decomposition. Then we have M (1 ⊗ 1)1(U ) = (1 ⊗ φ −1 )φ −1 Rese ê ê 0m × 0l × 0k U m+l+k=n −c(Ui (m,l,k))

M

=

2

Ui (m, l, k)

m+l+k=n,i

M

= (φ −1 ⊗ 1)φ −1

Rese ê ê 0m × 0l × 0k U

m+l+k=n

= (1 ⊗ 1)1(U ), where we used the cocycle condition in the third equation, and the notation P c(U ) 2 i Ui stands for the multiplicity-free summation of the irreducible components (cf. Proposition 3.7 and the definition of φ). Finally, we look at the compatibility of multiplication and comultiplication. Fix m and n; it follows from Proposition 3.4 that M e 0m+n −1 1(U · V ) = φk,l Rese φm,n (U ⊗ V ) ê 0k × 0l Inde ê 0 × 0 m

k+l=m+n

=

M

M

k+l=m+n

s

−1 k ×0l φk,l Ind0(e ê 0 × 0 e ê m

n )s

n

Res(e ê 0m × 0n )s φm,n (U ⊗ V )s ,

ê ê where s runs through the double cosets e 0m × 0n \e 0m+n /e 0k × 0l . Notice that the double cosets are in one-to-one correspondence with the double cosets 0m ×0n \0m+n / 0k ×0l . Again by the cocycle property of c and counting the double cosets, we can check that the last summation is exactly 1(U ) · 1(V ). Remark 3.9 Our Hopf algebra is different from that of [HH], where a bigger space than our R0− was used. The standard bilinear form in R0− is defined in terms of those on R − (e 0n ) as follows: X hu n , vn ie hu, vi = 0n , n≥0

70


where u =

P

n

u n and v =

n vn

P

with u n , vn ∈ e 0n .

4. Basic spin representations of e 0n 4.1. A weighted bilinear form on R(0) and R − (e 0n ) In [FJW2] we introduced the notion of weighted bilinear forms on R(0) and coherently combined several examples in this concept. Here we similarly define weighted bilinear forms on the space R − (e 0n ). Let ξ be a self-dual class function in R(0); that is, ξ(c) = ξ(c−1 ) for all c ∈ 0 ∗ . Let ∗ denote the product of two characters in R(0), which is afforded by the tensor product. Let ai j ∈ C be the (virtual) multiplicities of γ j in ξ ∗ γi : ξ ∗ γi =

r X

ai j γ j .

(4.1)

j=0

We denote further by A the ((r + 1) × (r + 1))-matrix (ai j )0≤i, j≤r . Then the weighted bilinear form h f, giξ is defined by h f, giξ = hξ ∗ f, gi0 ,

f, g ∈ R(0).

Alternatively, it can be explicitly given by 1 X ξ(x) f (x)g(x −1 ) |0| x∈0 X = ζc−1 ξ(c) f (c)g(c−1 )

h f, giξ =

(4.2)

c∈0∗

=

X

ζc−1 ξ(c) f (c−1 )g(c).

(4.3)

c∈0∗

In particular, (4.1) is equivalent to hγi , γ j iξ = ai j .

(4.4)

The self-duality implies that A is a symmetric matrix. Note that the weighted bilinear form becomes the standard bilinear form when ξ = γ0 , the trivial character of 0. Let V be a spin supermodule for e 0n , and let W be a module for 0n . As a Z2 graded vector space, W ⊗ V = W ⊗ V0 ⊕ W ⊗ V1 , and the action of e 0n is defined by (g, z p tρ )(w ⊗ v) = g, s(ρ) · w ⊗ (g, z p tρ ) · v,

g ∈ 0 n , σ ∈ Pn (0∗ ). (4.5)


71

It is easy to check that the tensor product V ⊗ W is a spin e 0n -supermodule. This construction defines a morphism ∗

R(0n ) ⊗ R − (e 0n ) −→ R − (e 0n ).

(4.6)

Let us recall the construction of character ηn (ξ ) in [W] and [FJW2]. Let γ be an irreducible character of 0 afforded by the 0-module V . The tensor product V ⊗n is naturally a 0n -module by the direct product action of 0 n composed with permutation action of the symmetric group Sn . The resulting character of 0n is denoted by ηn (γ ). Furthermore, we can extend ηn from 0 ∗ to R(0). The character value of ηn (ξ ) at the class ρ = (ρ(c)) is given by Y ηn (ξ )(ρ) = ξ(c)l(ρ(c)) . (4.7) c∈0∗

It is clear that the class function ηn (ξ ) is self-dual as long as ξ is. We now introduce a weighted bilinear form on R − (e 0n ) by letting h f, giξ,e 0n = hηn (ξ ) ∗ f, gie 0n ,

f, g ∈ R − (e 0n ),

where we used the map (4.6). The self-duality of ηn (ξ ) implies that the bilinear form h , iξ is symmetric. Remark 4.1 When n = 1, this weighted bilinear form obviously reduces to the weighted bilinear form on R(0) defined in (4.2) and (4.3). R − (e 0n ) is given by X hu, viξ = hu n , vn iξ,e 0n ,

The bilinear form on R0− =

L

u n and v =

P

n

n≥0

where u =

P

n

n vn

with u n , vn ∈ R − (e 0n ).

4.2. Basic spin representations Let the Pauli spin matrices be 0 1 1 0 , σ0 = , σ1 = 0 1 1 0 √ 0 − −1 1 0 √ σ2 = , σ3 = . −1 0 0 −1 Let C2k be the Clifford algebra generated by e1 , e2 , . . . , e2k with relations {ei , e j } = ei e j + e j ei = −2δi j .

(4.8)

72


Thus e2j = −1. The Clifford algebra C2k is endowed with a natural superalgebra structure by letting the parity of ei be odd for each i. When k = 1, one has that C2 ' M(1|1) = End(C1|1 ) and the action of C2 on C1|1 is given by the Pauli spin matrices: √ √ e2 7→ −1σ2 . e1 7 → −1σ1 , More generally, we have C2k = End(⊗k C1|1 ) ' M(2k−1 |2k−1 ). The tensor product ⊗k C1|1 admits a canonical supermodule structure for the Clifford algebra C2k under the action √ ⊗( j−1) ⊗(k− j) e2 j−1 −→ −1σ3 ⊗ σ1 ⊗ σ0 , j = 1, . . . , k, (4.9) √ ⊗( j−1) ⊗(k− j) e2 j −→ −1σ3 ⊗ σ2 ⊗ σ0 , j = 1, . . . , k. (4.10) The above formulas define explicitly the structure of a simple C2k -supermodule on ⊗k C1|1 . Let C2k+1 be the Clifford algebra generated by ei , i = 1, . . . , 2k+1 with relations √ like those in (4.8). We embed C1 into End(C1|1 ) by 1 7 → Id, e1 7→ −1σ1 . Then C2k+1 ' C2k ⊗ C1 ,→ End(⊗k+1 C1|1 ) gives a C2k+1 -supermodule structure on ⊗k+1 C1|1 . The explicit action is given by the same formulae, (4.9) and (4.10), except that j = 1, . . . , k + 1 in (4.9). Observe that C2k+1 ' Q(2k ). It is well known (see, e.g., [Jo]) that there exists an embedding of e Sn into the multiplicative Clifford group of units in Cn−1 . Therefore ⊗dn/2e C1|1 can be regarded as an e Sn -supermodule, which is called the basic spin supermodule for e Sn . More explicitly, we have the following proposition. PROPOSITION 4.2 ([S], [Jo]) The basic spin supermodule for e Sn is ⊗[n/2] C1|1 with the action s s j +1 j −1 t j 7→ ej − e j−1 , j = 1, . . . , n − 1. 2j 2j

Here we take e0 = 0. Its character χn is given by  l(α)/2  if α ∈ OP n , n even,  2 (l(α)−1)/2 χn (α) = 2 if α ∈ OP n , n odd,   0 otherwise. In particular, χn (1) = 2dn/2e . Here dae denotes the largest integer ≤ a.

(4.11)

(4.12)


73

4.3 ([S], [Mo]) Let n ≥ 1 be an odd integer. The basic spin supermodule ⊗(n−1)/2 C1|1 is an irreducible e Sn -module under action (4.11). Its character χn is given by the second equation of (4.12). In particular, χn (1) = 2(n−1)/2 . Let n ≥ 1 be an even integer. The basic spin supermodule is a reducible e Sn module under action (4.11) and decomposes into two irreducible e Sn -modules whose characters χn± are given by   2(l(α)−2)/2 if α ∈ OP n ,   r  √ n χn± (α) = ±( −1)n/2 (4.13) if α = (n),  2    0 otherwise.

PROPOSITION

(1)

(2)

In particular, χn± (1) = 2(n−2)/2 . 4.3. The spin character πn (γ ) of e 0n Let V be a 0-module afforded by the character γ ∈ R(0), and let U be a spin supermodule (resp., module) of e Sn with the character π. The tensor product V ⊗n ⊗ U has a canonical spin supermodule (resp., module) structure for e 0n as follows (cf. (4.5)). n p For any g = (g1 , . . . , gn ) ∈ 0 , let (g, z tρ ) be an element in e 0n . The supermodule (resp., module) structure is defined by (g, z p tρ )·(vi ⊗ · · · ⊗ vn ⊗ u) = g1 vs(ρ)−1 (1) ⊗ · · · ⊗ gn vs(ρ)−1 (n) ⊗ (z p tρ u). We denote by πn (γ ) the character of the constructed spin supermodule (resp., module). Recall that the conjugacy class Dρ+ contains an element (g, t ρ ) (see (2.3)). 4.4 Let π be the character of a spin e Sn -supermodule. Then the character values of πn (γ ) at the conjugacy classes Dρ± (ρ ∈ OP n (0∗ )) are given by PROPOSITION

πn (γ )(Dρ± ) = ±π(t ρ )

Y

γ (c)l(ρ(c)) .

(4.14)

c∈0∗

Proof Consider (g, z p tρ ) ∈ e 0n , where g = (g1 , . . . , gn ) ∈ 0 n and tρ is an n-cycle, say, P tρ = [12 · · · n]. Denote by e1 , . . . , ek a basis of Vγ , and write ge j = i ai j (g)ei ,

74


ai j (g) ∈ C. It follows that (g, z p tρ )(e j1 ⊗ · · · ⊗ e jn ⊗ u) = g1 (e jn ) ⊗ g2 (e j1 ) ⊗ · · · ⊗ gn (e jn−1 ) ⊗ z p tρ (u). Thus we obtain πn (γ )(z p tρ ) = trace a(gn )a(gn−1 ) · · · a(g1 )π(z p tρ ) = trace a(gn gn−1 · · · g1 )π(z p tρ ) = γ (c)π(z p tρ ), where we notice that gn gn−1 · · · g1 lies in c ∈ 0∗ . ˆ ∈e ˆ = Given x ×y 0n , where x ∈ e 0r and y ∈ e 0n−r , we clearly have πn (γ )(x ×y) + e πn (γ )(x)πn (γ )(y). Thus it follows that for the conjugacy class Dρ ∈ 0n of type ρ, we have Y γ (c)l(ρ(c) , πn (γ )(Dρ± ) = ±π(t ρ ) c∈0∗

where ||ρ|| = n. Since the sign character is trivial at even classes, we can naturally extend πn to a map from R(0) to R − (e 0n ) (cf. [W], [FJW2]). For two 0-characters β and γ , we define πn (β − γ ) =

n X m=0

0n (−1)m Inde 0 e

ê 0m n−m ×

πn−m (β) ⊗ πm (γ ) .

(4.15)

When n is even, the character χn of the basic spin supermodule (see Section 4.2) decomposes into the sum of irreducible characters χn± of e 0n -modules. For each c ∈ 0∗ , we define the special partition-valued function c(n) ∈ P (0∗ ) such that c(n) (c) = (n),

c(n) (c0 ) = ∅ for c0 6= c.

(4.16)

The following corollary is an immediate consequence of Propositions 4.4 and 4.3. 4.5 The character value of χn (γ ) at the conjugacy class Dρ+ of type ρ is

COROLLARY

(1)

χn (γ )(ρ) =

( Q 2(l(ρ)−n)/2 c∈0∗ γ (c)l(ρ(c)) , 0

ρ ∈ OP n (0∗ ), otherwise,

where n is 0 or 1 depending on whether n is even or odd.

(4.17)


(2)

75

Let n be an even positive integer. The character values of χn± (γ ) at the conjugacy class Dρ+ of type ρ are  Q  2(l(ρ)−2)/2 c∈0∗ γ (c)l(ρ(c)) , ρ ∈ OP n (0∗ ),   r  √ n ± n/2 χn (γ )(ρ) = ±( −1) γ (c), ρ = c(n) ,  2    0 otherwise.

(4.18)

4.4. Two specializations Let di = γi (c0 ) be the dimension of the irreducible representation of 0 afforded by the character γi . Let A be the matrix of the bilinear form h | i on R(0) with respect to the basis γi . Observe that the vector t vi = γ0 (ci ), γ1 (ci ), . . . , γr (ci ) , i = 0, . . . , r is an eigenvector of the matrix A with eigenvalue ξ(ci ). Two special choices of the weight function ξ are our prototypical examples. The first choice is that ξ = γ0 , the trivial character. Let π be the character of the two-dimensional representation of 0 given by the embedding of 0 in SL2 (C). Let ξ = 2γ0 − π. Then the weighted bilinear form h , iξ on R0 becomes positive semidefinite. The radical of this bilinear form is one-dimensional and spanned by the character of the regular representation of 0: r X δ= di γi . i=0

The following is the well-known list of finite subgroups of SL2 (C): the cyclic, binary dihedral, tetrahedral, octahedral, and icosahedral groups. McKay observed that they are in one-to-one correspondence to simply laced Dynkin diagrams of affine types (see [Mc]): aii = 2 for all i; if 0 6 = Z/2Z and i 6 = j, then ai j = 0 or −1. If 0 = Z/2Z, then a01 = −2.

5. Twisted Heisenberg algebras and e 0n 5.1. Twisted Heisenberg algebra b h0,ξ [−1] Let b h0,ξ [−1] be the infinite-dimensional Heisenberg algebra over C, associated with a finite group 0 and a self-dual class function ξ ∈ R(0), with generators am (γ ), m ∈

76


2Z + 1, γ ∈ 0 ∗ , and a central element C subject to the relations m am (γ ), an (γ 0 ) = δm,−n hγ , γ 0 iξ C, m, n ∈ 2Z + 1, γ , γ 0 ∈ 0 ∗ . (5.1) 2 Pr We extend am (γ ) to all γ = i=0 si γi ∈ R(0) (si ∈ C) by linearity: am (γ ) = P i si am (γi ). The Heisenberg algebra may contain a large center because the bilinear form h·, ·iξ may be degenerate. The center of b h0,ξ [−1] is spanned by C together with 0 am (γ ), m ∈ 2Z + 1, γ ∈ R , the radical of the bilinear form h·, ·iξ in R(0). For m ∈ 2Z + 1, c ∈ 0∗ , we introduce another basis for b h0,ξ [−1]: X am (c) = γ (c−1 )am (γ ). (5.2) γ ∈0 ∗

The orthogonality of the irreducible characters of 0 (see (2.11)) implies that X am (γ ) = ζc−1 γ (c)am (c). c∈0∗

PROPOSITION 5.1 The commutation relations for the new basis for b h0,ξ are given by

m −1 am (c0 ), an (c) = δm,−n δc0 ,c ζc ξ(c)C, 2

c, c0 ∈ 0∗ ,

where m, n ∈ 2Z + 1. Proof The proof is similar to the untwisted case [FJW2]. 5.2. Action of b h0,ξ [−1] on S0− and S 0 − Denote by S0 the symmetric algebra generated by a−n (γ ), n ∈ 2Z+ + 1, γ ∈ 0 ∗ . There is a natural degree operator on S0− , deg a−n (γ ) = n, n ∈ 2Z+ + 1, which makes S0− into a Z+ -graded space. We define an action of b h0,ξ [−1] on S0− as follows: a−n (γ ), n > 0, acts as a multiplication operator on S0− , and C acts as the identity operator; an (γ ), n > 0, acts as a derivation of the symmetric algebra an (γ ) · a−n 1 (α1 )a−n 2 (α2 ) · · · a−n k (αk ) =

k X i=1

δn,n i hγ , αi iξ a−n 1 (α1 )a−n 2 (α2 ) · · · aˇ −n i (αi ) · · · a−n k (αk ).


77

Here n i > 0, αi ∈ R(0) for i = 1, . . . , k, and aˇ −n i (αi ) means the very term is deleted. In other words, the operator an (γ ), n > 0, γ ∈ R 0 , acts as zero, and an (γ ), n > 0, γ ∈ R(0) − R 0 , acts as a certain nonzero differential operator. Note that S0− is not an irreducible representation over b h0,ξ in general since the bilinear form h , iξ may be degenerate. Denote by S00 the ideal in the symmetric algebra S0− generated by a−n (γ ), n ∈ N, γ ∈ R 0 . Denote by S 0 the quotient S0− /S00 . It follows from the definition that S00 is a subrepresentation of S0− over the Heisenberg algebra b h0,ξ [−1]. In particular, this induces a Heisenberg algebra action on S 0 which is irreducible. The unit 1 in the symmetric algebra S0− is the highest weight vector. We also denote by 1 its image in the quotient S 0 . 5.3. The bilinear form on S0− The space S0− admits a bilinear form h , i0ξ determined by h1, 1i0ξ = 1,

an (γ )∗ = a−n (γ ),

n ∈ 2Z + 1.

(5.3)

Here an (γ )∗ denotes the adjoint of an (γ ). For any partition λ = (λ1 , λ2 , . . . ) ∈ OP and γ ∈ 0 ∗ , we define a−λ (γ ) = a−λ1 (γ )a−λ2 (γ ) · · · . For ρ = (ρ(γ ))γ ∈0 ∗ ∈ OP (0 ∗ ), we define Y a−ρ = a−ρ(γ ) (γ ). γ ∈0 ∗

It is clear that a−ρ , ρ ∈ OP (0 ∗ ), form a basis for S0− . Similarly, we define a−λ (c) = a−λ1 (c)a−λ2 (c) · · · for any partition λ = (λ1 , λ2 , . . .) ∈ OP and c ∈ 0∗ . For any ρ = (ρ(c))c∈0∗ ∈ OP (0∗ ), we further define Y 0 a−ρ(c) (c). a−ρ = c∈0∗ 0 , ρ ∈ OP (0 ), provide a new C-basis for S − . The elements a−ρ ∗ 0 Recall that ρ ∈ OP (0∗ ) is given by assigning to c ∈ 0∗ the partition ρ(c−1 ), which is the composition of ρ with the involution on 0∗ given by c 7→ c−1 . It follows from Proposition 5.1 that 0 0 0 ha−ρ 0 , a−ρ iξ = δρ 0 , ρ

Zρ Y ξ(c)l(ρ(c)) , 2l(ρ) c∈0∗

ρ 0 , ρ ∈ OP (0∗ ).

(5.4)

78


Remark 5.2 S00 can be characterized as the radical of the bilinear form h , i0ξ in S0− . Thus the bilinear form h , i0ξ induces a bilinear form on S 0 which is denoted by the same notation.

6. Isometry between R0− and S0− 6.1. The characteristic map ch We define a C-linear map ch : R0− −→ S0− by letting ch( f ) =

X ρ∈OP (0∗ )

2l(ρ)/2 0 f (ρ)a−ρ , Zρ

(6.1)

where f (ρ) = f (Dρ+ ). The map ch is called the characteristic map (cf. [S], [Jo] for 0 trivial). Fix n ∈ 2Z+ + 1 in this paragraph. Denote by Dn (c)+ , c ∈ 0∗ , the conjugacy class in e 0n of elements (x, ts ) ∈ 0n such that s is an n-cycle and the cycle product of (x, ts ) is c. Then set Dn (c)− = z Dn (c)+ . Thus Dn (c)± are the associated split conjugacy classes of type c(n) (see √ (4.16)). Denote by σn (c) the super class function on e 0n which takes value ±(n/ 2)ζc on elements in the conjugacy classes Dn (c)± Q and zero elsewhere. For ρ = {i m i (c) } ∈ OP n (0∗ ), σρ = i∈2Z+ +1,c∈0∗ σi (c)m i (c) is the class function of e 0n which takes value ±2−l(ρ)/2 Z ρ on the conjugacy classes Dρ± and zero elsewhere. Given denote by σn (γ ) the class function on e 0n √ γ ∈ R(0), we ± which takes value ±(n/ 2)γ (c) on Dn (c) , c ∈ 0∗ , and zero elsewhere. The following lemma is not difficult to verify. LEMMA 6.1 0 . In particular, it sends σ (c) to a (c) in S − and σ (γ ) The map ch sends σρ to a−ρ n −n n 0 to a−n (γ ) for n ∈ 2Z + 1.

In Section 9.2, we see that the space S0− has another distinguished basis consisting of generalized Schur Q-functions, which give rise to some integral basis in R − (e 0n ). 6.2. The image of χn (γ ) under ch Recall that we have defined a map from R(0) to R − (e 0n ) (see Section 4.3). PROPOSITION

6.2


79

For any γ ∈ R(0), we have X n≥0

X 2 2n/2 ch χn (γ ) z n = exp a−n (γ )z n , n

(6.2)

n≥1,odd

where n is 0 or 1 according to whether n is even or odd. Proof The character value of χn (γ ) is given in Corollary 4.5, and we have X X Y 0 2n/2 ch χn (γ ) z n = 2l(ρ) Z ρ−1 γ (c)l(ρ(c)) a−ρ(c) z ||ρ|| ρ

n≥0

=

c∈0∗

Y X c∈0∗

λ

(2ζc−1 γ (c))l(λ) z λ−1 a−λ (c)z |λ|

X 2 X = exp ζc−1 γ (c)a−n (c)z n n n≥1 c∈0∗ X 2 = exp a−n (γ )z n . n n≥1

Let β, γ be the characters of two representations of 0. It follows from (4.15) that X

2n/2 ch χn (β − γ ) z n

n≥0

=

X

X n/2 2n/2 ch χn (β) z n · 2 ch χn (γ ) (−z)n

n≥0

n≥0

X 2 = exp a−n (β − γ )z n . n n≥1,odd

Therefore the proposition holds for β − γ , and so for any element γ ∈ RZ (0). COROLLARY 6.3 The formula (4.17) holds for any γ ∈ R(0). In particular, χn (ξ ) is self-dual if ξ is self-dual.

Componentwise, we obtain X 2l(ρ) ch χn (γ ) = 2−n/2 a−ρ (γ ), zρ ρ where the sum runs through all the partitions ρ of n into odd integers.

80


6.3. Isometry between R0− and S0− It is well known that there exists a natural Hopf algebra structure on the symmetric algebra S0− with the usual multiplication and the comultiplication 1 characterized by 1 a−n (γ ) = a−n (γ ) ⊗ 1 + 1 ⊗ a−n (γ ),

n ∈ 2Z+ + 1.

(6.3)

Recalling the Hopf algebra structure on R0− defined in Section 3.4, we can easily verify the following proposition as we did in the untwisted case. PROPOSITION 6.4 The characteristic map ch : R0− −→ S0− is an isomorphism of Hopf algebras.

Proof By counting dimensions of homogeneous degree subspaces, it is easy to see that ch is an isomorphism of vector spaces. The algebra isomorphism follows simply from the Frobenius reciprocity. To check the coalgebra isomorphism, we use Proposition 6.2 to pass from the generators an (γ ) to the character χn (γ ). It is then a simple calculation to verify that χn (γ ) is grouplike under the comultiplication (3.13), and this shows that ch is a Hopf algebra isomorphism by using (6.3). Recall that we have defined a bilinear form h , iξ on R0− and a bilinear form on S0− denoted by h , i0ξ . The next lemma follows from our definition of h , i0ξ and the comultiplication 1. 6.5 The bilinear form h , i0ξ on S0− can be characterized by the following two properties: (1) ha−n (β), a−m (γ )i0ξ = (n/2)δn,m hβ, γ i0ξ , β, γ ∈ 0 ∗ , m, n ∈ 2Z+ + 1; (2) h f g, hi0ξ = h f ⊗ g, 1hi0ξ , where f, g, h ∈ S0− , and the bilinear form on the right-hand side of (2), which is defined on S0− ⊗ S0− , is induced from h , i0ξ on S0− . LEMMA

THEOREM 6.6 The characteristic map ch is an isometry from the space (R0− , h , iξ ) to (S0− , h , i0ξ ).

Proof Let f and g be any two super class functions in R − (e 0n ). By the definition of ch (see


81

(6.1)), it follows that

0 ch( f ), ch(g) ξ 2(l(ρ)+l(ρ ))/2 0 0 0 f (ρ)g(ρ 0 )ha−ρ , a−ρ 0 iξ Z ρ Z ρ0 0

=

X ρ,ρ 0 ∈OP n (0∗ )

=

X ρ,ρ 0 ∈OP n (0∗ )

=

X ρ∈OP n (0∗ )

0 Z ρ0 Y 2(l(ρ)+l(ρ ))/2 f (ρ)g(ρ 0 ) l(ρ 0 ) ξ(c)l(ρ(c)) δρ,ρ 0 Z ρ Z ρ0 2

1 f (ρ)g(ρ) Zρ

c∈0∗

Y

ξ(c)l(ρ(c))

c∈0∗

= h f, giξ , where we have used the inner product identity (5.4). Remark 6.7 We can also prove it by showing that the characteristic map preserves the inner product of basis elements σρ ∈ R0− and that of a−ρ = ch(σρ ) ∈ S0− , as in [FJW2]. From now on we identify the inner product h , iξ on R0− with the inner product h , i0ξ on S0− . As a special case, the standard Hermitian form on R − (0n ), and therefore on R0− , is compatible via the characteristic map ch with the Hermitian form characterized by (5.3) on S0− . 7. Vertex operators and R0− 7.1. A central extension of RZ (0)/2RZ (0) From now on we assume that ξ is a self-adjoint virtual character of 0, and thus RZ (0) is an integral lattice under the symmetric bilinear form h , iξ . Let 2RZ (0) be the sublattice of RZ (0) consisting of elements 2α, α ∈ RZ (0). The quotient RF2 (0) = RZ (0)/2RZ (0) has an induced abelian group structure, and it can also be viewed as an (r + 1)-dimensional vector space over F2 = Z/2Z. We denote by α the natural image of α in RF2 (0). Define c1 to be the alternating form: RF2 (0) × RF2 (0) → F2 given by c1 (α, β) = hα, βiξ + hα, αiξ hβ, βiξ (mod 2), and let r0 be its rank over F2 . The alternating form c1 gives rise to a central extension Rˆ F−2 (0) of the abelian group RF2 (0) by the two-element group h±1i (see [FLM1]), ˘ 1 → h±1i ,→ Rˆ F−2 (0) → RF2 (0) → 1,

(7.1)

82


˘ ˘ b) such that aba −1 b−1 = (−1)c1 (a, , a, b ∈ Rˆ F−2 (0). The elements of Rˆ F−2 (0) can be presented as ±eα , where α ∈ RZ (0), which implies that dim( Rˆ − (0)) = 2r +2 . We note that eα ∈ Rˆ − (0) satisfies (eα )2 = 1. F2

F2

Let 8 be a subgroup of RZ (0) which is maximal such that the alternating form c1 vanishes on 8/2RZ (0). A variant of the following lemma was given in [FLM2].

LEMMA 7.1 There are 2(r +1−r0 ) irreducible Rˆ F−2 (0)-module structures on the space C[RZ (0)/8] such that −1 ∈ Rˆ − (0) acts faithfully and F2

eα eβ = eβ eα (−1)c1 (α,β)

(7.2)

as operators on C[RZ (0)/8]. The dimension of C[RZ (0)/8] is equal to 2(1/2)r0 . We denote the elements of C[RZ (0)/8] by e[α] , where [α] = α + 8 ∈ RZ (0)/8. Clearly, e2[α] = 1, e[α+β] = e[α] e[β] . For α, β ∈ RZ (0), we write the action of Rˆ F−2 (0) on C[RZ (0)/8] as eα · e[β] = (α, β)e[α+β] .

(7.3)

Then one can check that is a well-defined cocycle map from RZ (0) × RZ (0) → h±1i. One also has (α, β) = (α, −β). 7.2. Twisted vertex operators X (γ , z) Fix an irreducible Rˆ F−2 (0)-module structure on C[RZ (0)/8] as described in (7.3). We extend the actions of eα to the space of tensor product F0− = R0− ⊗ C[RZ (0)/8]

by letting them act on the R0− part trivially. Introduce the operators H±n (γ ), γ ∈ R(0), n > 0, as the following compositions of maps: H−n (γ ) : R − (e 0m )

2n/2 χn (γ )⊗

−→

Res

Ind

ê R − (e 0n × 0m ) −→ R − (e 0n+m ),

ê Hn (γ ) : R − (e 0m ) −→ R − (e 0n × 0m−n )

h2n/2 χn (γ ),·iξ

−→

R − (e 0m−n ).

Define H+ (γ , z) =

X n>0

H−n (γ )z n ,

H− (γ , z) =

X n>0

Hn (γ )z −n .


83

We now define the twisted vertex operators X n (γ ), n ∈ Z, γ ∈ R0 , by the following generating functions: X + (γ , z) ≡ X (γ , z) X = X n (γ )z −n n∈Z

= H+ (γ , z)H− (γ , −z)eγ .

(7.4)

We also denote X − (γ , z) ≡ X (−γ , z) = X (γ , −z) X = X n− (γ )z −n . n∈Z

The operators X n (γ ) are well-defined operators acting on the space F0− . We extend the bilinear form h , iξ on R0− to F0− by letting h f e[α] , ge[β] iξ = h f, giξ δ[α],[β] ,

f, g ∈ R0− , α, β ∈ RZ (0).

We extend the Z+ -gradation from R0− to F0− by letting dega−n (γ ) = n,

degeγ = 0.

Similarly, we extend the bilinear form h , iξ to the space V0− = S0− ⊗ C RZ (0)/8 and extend the Z+ -gradation on S0 to a Z+ -gradation on V0− . The characteristic map ch is extended to an isometry from F0− to V0− by fixing the subspace C[RZ (0)/8]. We denote this map again by ch. 7.3. Twisted Heisenberg algebra and R0− We define e a−n (γ ), n ∈ 2Z+ + 1, to be a map from R0− to itself by the composition σn (γ )⊗

R − (e 0m ) −→ R − (e 0n )

O

Ind

R − (e 0m ) −→ R − (e 0n+m ).

We also define e an (γ ), n ∈ 2Z+ + 1, to be a map from R0− to itself as the composition Res R − (e 0m ) −→ R − (e 0n )

O

R − (e 0m−n )

hσn (γ ),·iξ

−→

R − (e 0m−n ).

We denote by R00 the radical of the bilinear form h , iξ in R0− , and we denote by R 0 the quotient R0− /R00 , which inherits the bilinear form h , iξ from R0− .

84


THEOREM 7.2 R0− is a representation

of the twisted Heisenberg algebra b h0,ξ [−1] by letting an (γ ) (n ∈ 2Z + 1) act as e an (γ ) and C act as 1. R00 is a subrepresentation of R0− over b h0,ξ [−1], and the quotient R 0 is irreducible. The characteristic map ch is an isomorphism of R0− (resp., R00 , R 0 ) and S0− (resp., S00 , S 0 ) as representations over b h0,ξ [−1]. 7.4. The characteristic map of twisted vertex operators We extend the characteristic map ch to a linear map ch: End(R0− ) → End(S0− ) by ch( f ) · ch(v) = ch( f · v),

f ∈ End(R0− ), v ∈ R0− .

(7.5)

The relation between the vertex operators defined in (7.4) and the Heisenberg algebra b h0,ξ is revealed in the following theorem. THEOREM 7.3 For any γ ∈ R(0) we have

2 a−n (γ )z n , n n≥1, odd X 2 an (γ )z −n . ch H− (γ , z) = exp n

X ch H+ (γ , z) = exp

n≥1, odd

Proof Observe that the operator H+ (γ , z) is the adjoint operator of H− (γ , z −1 ) with respect to the bilinear form h , iξ . Then the theorem follows from Lemma 6.1 and Proposition 6.2 by invoking the characteristic map. As a consequence we have X ch X (γ , z) = exp n≥1, odd

X 2 a−n (γ )z n exp − n

n≥1, odd

2 an (γ )z −n eγ . n

Thus the characteristic map identifies the twisted vertex operators X (γ , z) defined via finite groups e 0n with the usual twisted vertex operators of [FLM1], [FLM2].

8. Vertex representations and the McKay correspondence 8.1. Product of two vertex operators The normal ordered product :X (α, z)X (β, w) :, α, β ∈ R(0), of two vertex operators is defined as follows: :X (α, z)X (β, w): = H+ (α, z)H+ (β, w)H− (α, −z)H− (β, −w)eα+β .


85

In the following theorem and later, the expression ((z − w)/(z + w))hα,βiξ represents the power series expansion in the variable w/z. THEOREM 8.1 For α, β ∈ R(0), one has the following operator product expansion identity for twisted vertex operators:

z − w hα,βiξ X (α, z)X (β, w) = (α, β):X (α, z)X (β, w): . z+w Proof It suffices to compute that ch H− (α, −z)H+ (β, w) X 2 X 2 an (α)z −n exp a−n (β)wn = exp − n n n≥1, odd n≥1, odd X 2 z −n wn = ch H+ (β, w)H− (α, −z) exp −hα, βiξ n n≥1, odd z − w hα,βiξ = ch H+ (β, w)H− (α, −z) . z+w The following proposition is easy to check. PROPOSITION 8.2 Given α ∈ R(0), β ∈ RZ (0), and n ∈ 2Z + 1, we have an (α), X (β, z) = hα, βiξ X (β, z)z n .

8.2. Twisted affine Lie algebra b g [−1] and twisted toroidal Lie algebra b g [−1] Let g be a rank r complex simple Lie algebra of ADE type, and let 1 be the root system generated by a set of simple roots α1 , . . . , αr . Let αmax be the highest root. The Lie algebra is generated by the Chevalley generators eαi , e−αi , h i = h αi . We normalize the invariant bilinear form on g by (αmax , αmax ) = 2. Let θ be an automorphism of g of order k, and let ω = exp(2πi/k). The automorphism θ induces a Z/kZ-gradation for g: M g= gi , gi = g ∈ g|θ(g) = ωi g , i∈Z/kZ

and [gi , g j ] ⊂ gi+ j .

86


The twisted affine Lie algebra b g [θ] is the graded vector space b g [θ ] =

k−1 M

gi ⊗ t i C[t k , t −k ]

M

CC

(8.1)

i=0

with the commutating relations n a(n), b(m) = [a, b](n + m) + δn,−m (a|b)C, k C, a(n) = 0, a, b ∈ g, n, m ∈ Z,

(8.2) (8.3)

where we used the notation a(n) = a ⊗ t n ,

a ∈ g, n ∈ Z.

When θ is the identity, b g [1] becomes the (untwisted) affine Lie algebra b g. Let A = (ai j )0≤i, j≤r be the affine Cartan matrix associated to b g. The submatrix (ai j )1≤i, j≤r is the Cartan matrix of g. The linear map eαi −→ e−αi , h αi −→ −h αi defines an involution of the Lie algebra g. We denote the associated twisted affine Lie algebra by b g [−1]. Let kα = (1/2)(eα + e−α ) and pα = (1/2)(eα − e−α ), where α ∈ 1. It is easily seen that M g0 = Ckα , M M g1 = C pα ⊕ Ch α . The basic twisted representation V of b g [−1] is the irreducible highest weight representation generated by a highest weight vector which is annihilated by a(n), n ∈ Z+ , a ∈ g, and C acts on V as the identity operator. We now introduce the complex twisted toroidal Lie algebra b g [−1] (associated to g) with the following presentation. The generators are C,

h i (m),

xn (±αi ),

m ∈ 2Z + 1, n ∈ Z, i = 0, . . . , r ;


87

and the relations are given by the following: C is central and xn (αi ) = (−1)n xn (−αi ), m h i (m), h j (m 0 ) = ai j δm,−m 0 C, 2 h i (n), xm (α j ) = ai j xn+m (α j ), xn (αi ), xn 0 (−αi ) = 8 h i (n + n 0 ) + nδn,−n 0 C ,

(8.4)

ai j

X ai j xn+s (αi ), xn 0 +ai j −s (α j ) = 0 s

if ai j ≥ 0,

−ai j (−1) xn+s (αi ), xn 0 −ai j −s (α j ) = 0 s

if ai j < 0,

s=0

−ai j

X s=0

s

where n, n 0 ∈ Z, m, m 0 ∈ 2Z + 1, i, j = 0, 1, . . . , r , and h i (2n) = 0 for n ∈ Z. The twisted toroidal algebra is the (q → 1)-limit of the twisted quantum current algebra in [J2] (see a slightly different form in [DI]). Set ki (2n) = (1/4)x2n (αi ) and pi (2n + 1) = (1/4)x2n+1 (αi ). One can check g [−1] for the that the relations given in (8.4) are consequences of the twisted algebra b case of θ = −1 defined in (8.2) (cf. the proof of Theorem 8.3). 8.3. Realization of twisted vertex representations Let 0 be a finite subgroup of SL2 (C), and let the virtual character ξ be twice the trivial character minus the character of the two-dimensional defining representation of 0 ,→ SL2 (C). It is known (see [Mc]) that the matrix A = (ai j )0≤i, j≤r in Section 4.4 is the Cartan matrix for the corresponding affine Lie algebra b g. The following theorem provides a finite group realization of the vertex representation of the twisted toroidal Lie algebra b g [−1] on F0− . THEOREM 8.3 A vertex representation of the twisted toroidal Lie algebra b g [−1] is defined on the space F0− by letting

xn (αi ) 7→ X n (γi ),

xn (−αi ) 7 → (γi , γi )X n (−γi ),

h i (m) 7→ am (γi ),

C 7 → 1,

where n ∈ Z, m ∈ 2Z + 1, 0 ≤ i ≤ r . Proof All the commutation relations without binomial coefficients are easy consequences of Proposition 8.2 and Theorem 8.1 by the usual vertex operator calculus in the twisted

88


picture (see [FLM2], [J1]). The corresponding relations with binomial coefficients in V0− are equivalent to (z + w)ai j X (γi , z), X (γ j , w) = 0, ai j ≥ 0, (z − w)−ai j X (γi , z), X (γ j , w) = 0, ai j < 0. This is again proved by using Theorem 8.1 with the same method as in the quantum vertex operators (see [J2]). P Recall that δ = ri=0 di γi generates the one-dimensional radical RZ0 of the bilinear form h , iξ in RZ (0), where di is the degree of the irreducible character γi of 0. The lattice RZ (0) in this case can be identified with the root lattice for the corresponding affine Lie algebra. The quotient lattice RZ (0)/RZ0 inherits a positive definite integral ∗ bilinear form. Denote by 0 the set of nontrivial irreducible characters of 0: ∗

0 = {γ1 , γ2 , . . . , γr }. ∗

∗

∗

Let RZ (0 ) be the sublattice of RZ (0) generated by 0 . Denote by Sym(0 ) the symmetric algebra generated by a−n (γi ), n ∈ 2Z+ + 1, i = 1, . . . , r . Equipped with the ∗ bilinear form h , iξ , Sym(0 ) is isometric to S 0 , which is in turn isometric to R 0 as ∗ well. The irreducible Rˆ − (0)-module C[RZ (0)/8] induces an irreducible Rˆ − (0 )F2

F2

∗

module structure on C[RZ (0 )/8] given by the restriction of the alternating form c1 . We let r 0 denote the rank of the restriction of c1 . Then the statement of Lemma 7.1 ∗ ∗ also holds for the sublattice RF2 (0 ) and Rˆ F−2 (0 ). In this case, if the determinant of ∗

the Cartan matrix is an odd integer, then r 0 = 0 and the space C[RZ (0 )/8] is trivial. We define ∗ ∗ ∗ V 0 = S 0 ⊗ C RZ (0 )/8 ∼ = Sym(0 ) ⊗ C RZ (0 )/8 , ∗ F 0 = R 0 ⊗ C RZ (0 )/8 . Obviously ch, when restricted to F 0 , is an isometric isomorphism onto V 0 . The space F0− associated to the lattice RZ (0) is isomorphic to the tensor product of the space F 0 associated to R Z (0) and the space associated to the rank 1 lattice Zδ equipped with the zero bilinear form. The identity for a product of vertex operators X (γ , z) associated to γ ∈ 1 (cf. Theorem 8.3) implies that V 0 provides a realization of the vertex representation of b g [−1] on V 0 (cf. [FLM1]). The following theorem establishes a direct link from the finite group 0 ∈ SL2 (C) to the affine Lie algebrab g [−1]. This gives a twisted version of the new form of the McKay correspondence given in [FJW2]. 8.4 The operators X n (γ ), γ ∈ 1, an (γi ), i = 1, 2, . . . , r, n ∈ Z, define an irreducible THEOREM


89

representation of the affine Lie algebra b g [−1] on F 0 isomorphic to the twisted basic representation. 9. Vertex operators and irreducible characters of e 0n In this section we specialize ξ to be the trivial character γ0 of 0. We describe how to obtain the character table for the spin supermodules of e 0n from our vertex operator approach, generalizing [J1]. 9.1. Algebra of vertex operators for ξ = γ0 In this case the weighted bilinear form reduces to the standard one h , i and RZ (0) is isomorphic to the lattice Zr +1 with the standard integral bilinear form. Recall that hγi , γ j i = δi j . For simplicity we consider only the vertex representations on the space R0 . In the following result the bracket { , } denotes the anticommutator. THEOREM 9.1 The operators X n+ (γi ), X n− (γi ) (n ∈ Z, 0 ≤ i ≤ r ) generate a generalized Clifford algebra: + X n (γi ), X n+0 (γ j ) = 0, i 6 = j, − X n (γi ), X n−0 (γ j ) = 0, i 6 = j, + X n (γi ), X n+0 (γi ) = 2(−1)n δn,−n 0 , − X n (γi ), X n−0 (γi ) = 2(−1)n δn,−n 0 , + X n (γi ), X n−0 (γ j ) = 0, i 6 = j, + X n (γi ), X n−0 (γi ) = 2δn,−n 0 . (9.1)

Proof The proof follows from the standard vertex operator calculus (cf. [J1]) by using Theorem 8.1. Therefore we see that R0− is isomorphic to the tensor product of r + 1 copies of the space R − , the sum of Grothendieck groups of spin characters of e Sn -supermodules. Remark 9.2 Let A = (1 − δi j )(r +1)×(r +1) , the matrix of the alternating form c1 over F2 ; then A2 = r I . Here r = 0 if r is even and 1 if r is odd. Consequently, it follows from Lemma 7.1 that there are exactly 2r +1 irreducible Rˆ F−2 (0)-module structures on the 2d(r +1)/2e -dimensional space C[RZ (0)/8]. One of the (at most) two irreducible module structures is given by the cocycle (γi , γ j ) = 1, for i ≤ j, and (γi , γ j ) = −1, for i > j. Then the vertex operators X n± (γi ) generate the twisted Clifford algebra on the

90


space F0− defined by {X n± (γi ), X n±0 (γi )} = 2(−1)n δi j δn,−n 0 and {X n+ (γi ), X n−0 (γi )} = 2δi j δn,−n 0 . 9.2. Super spin character tables of e 0n and vertex operators We now use the twisted vertex operators to construct all irreducible characters of spin supermodules of e 0n for all n. − e Let RZ (0n ) be the lattice generated by the characters of spin irreducible e 0n supermodules. Then RZ− (e 0n ) ⊗ C ' R − (e 0n ). We first construct a special orthonormal basis in R0− and then use it to give irreducible characters of spin e 0n -supermodules. The vertex operator X n (γ ) is defined as in (7.4) except that we drop eγ . The following lemma is easily seen (cf. [J1]). LEMMA 9.3 For n ∈ Z, α ∈ R(0), and γ ∈ 0 ∗ , we have

X −n (±γ ) · 1 = δn,0 ,

n ≥ 0.

For an m-tuple index φ = (φ1 , . . . , φm ) ∈ Zm , we denote X φ (γ ) = X φ1 (γ ) · · · X φm (γ ) · 1, xφ (γ ) = X φ1 (γ ) · 1 · · · X φm (γ ) · 1 . We also define the raising operator Ri j by Ri j (φ1 , . . . , φm ) = (φ1 , . . . , φi + 1, . . . , φ j − 1, . . . , φm ). Then we define the action of the raising operator Ri j on X φ (γ ) or xφ (γ ) by X Ri j φ (γ ) or x Ri j φ (γ ). Given λ ∈ OP (0 ∗ ), we define Y Xλ = X −λ(γ ) (γ ). γ ∈0 ∗

Similarly, we define xλ =

Q

γ ∈0 ∗

xλ(γ ) (γ ).

9.4 The vectors X λ for λ = (λ(γ ))γ ∈0 ∗ ∈ S P (0 ∗ ) form an orthogonal basis in the vector space R0− with hX λ , X µ i = 2l(λ) δλ,µ . Moreover, we have THEOREM

Y Y 1 − Ri j X λ (γ ) 1 + Ri j γ ∈0∗ i< j X cλ,µ xµ , = xλ +

Xλ =

λµ

(9.2) (9.3)


91

where cλ,µ ∈ Z, and Ri j is the raising operator. Proof The generalized Clifford algebra structure (9.1) implies that the nonzero elements Y X −n 1 (γ ) · · · X −nl (γ ) · 1 γ ∈0 ∗

of distinct indices generate a spanning set in the space R0− . To see that they satisfy the raising operator expansion, we compute Y zi − z j X (γ , z 1 ) · · · X (γ , zl ) · 1 =: X (γ , z 1 ) · · · X (γ , zl ) : . zi + z j i< j

Using the result in [J1], it follows that this is exactly the generating function of the raising operator expansion at the case 0 is trivial under the isomorphism ch. In other words, (9.2) is true when λ is a characteristic partition-valued function. Next the orthogonality follows from the generalized Clifford algebra commutation relations in Theorem 9.1. The orthogonality relations show that the raising operator is not affected by the character γ ; hence the general case follows by multiplying the raising operator formula for each γ . The corresponding basis in S0− is the classical symmetric function called Schur’s Qfunction (see [J1]). For any strict partition λ = (λ1 , . . . , λl ), Schur’s Q-function Q λ is determined by (see [S], [M]) X Y yσ (i) + yσ ( j) Q λ (y1 , . . . , yn ) = 2l yσλ1(1) · · · yσλl(l) , (9.4) yσ (i) − yσ ( j) σ ∈Sn /Sn−l

λi >λ j

where Sn−l acts on yl+1 , . . . , yn and we allow λ j = 0 for j = l +1, . . . , n. It is known (see, e.g., [M], [J1]) that Q λ , λ ∈ S P n , form a basis in the subring of symmetric functions generated by the power sums p1 , p3 , p5 , . . . . We can think of a−n (γ ), n > 0, γ ∈ 0 ∗ , as the nth power sum in a sequence of variables yγ = (yiγ )i≥1 . In this way we identify the space S0− with a distinguished subspace of symmetric functions generated by odd degree power sums indexed by 0 ∗ . In particular, given a strict partition λ, we denote by Q λ (γ ) the Schur’s Q-function associated to yγ . We also denote by Q λ (γ ) the corresponding element in S0− by the identification of S0− and R0− . For λ ∈ P (0 ∗ ), we denote Y Qλ = Q λ(γ ) (γ ) ∈ S0− . (9.5) γ ∈0 ∗

For λ ∈ S P (0 ∗ ) we define Q λ = 2−(l(λ)−d(λ))/2 Q λ ,

(9.6)

92


where d(λ) is the parity of λ (see (2.14)). Similarly, we define Y q λ = 2−(l(λ)−d(λ))/2 qλi (γ ) (γ ), γ ∈0 ∗ ,i

where qm (γ ) = Q (m) (γ ). In particular, we have ch χn (γ ) = q n = 2−n/2 qn (γ ) = 2−n/2 Q (n) (γ ). The following result immediately follows from Theorem 9.4 combined with the characteristic map ch and (9.6). PROPOSITION 9.5 For a strict partition-valued function λ ∈ S P (0 ∗ ), we have

hQ λ , Q µ i = 0 if λ 6= µ, ( 1 if λ is even, hQ λ , Q λ i = 2 if λ is odd.

(9.7)

The following result generalizes a similar result of [Jo] for trivial 0. LEMMA 9.6 Under the characteristic map ch, the symmetric function q λ corresponds to a character in RZ− (e 0n ), and Q λ corresponds to a virtual character in RZ− (e 0n ).

Proof Observe that the tensor product of two irreducible supermodules of type Q is a sum of two irreducible supermodules of type M (see Proposition 3.7). Computing its inner product, we know that for positive odd integers m and n the character χn (γ )χm (γ ) is twice some irreducible character. Then by induction we see that ch−1 (q λ ) is a character in RZ− (e 0n ). From Theorem 9.4 we see that Q λ is a Z-linear combination of q µ with λ µ; hence Q λ is a virtual character of e 0n . ˆ ···× ê For λ ∈ S P n (0 ∗ ) we define e 0λ = e 0λ(γ0 ) × 0λ(γr ) . For a partition µ and an irreducible character γ of 0, we define the spin character χµ (γ ) of e 0µ to be χµ1 (γ )⊗ · · · ⊗ χµl (γ ) (see Corollary 4.5). THEOREM 9.7 For each strict partition-valued function λ ∈ S P n (0 ∗ ), the vector Q λ corresponds, under the characteristic map ch, to the irreducible character χλ of the spin e 0n supermodule given by a Z-linear combination of 0n χρ(γ0 ) (γ0 ) ⊗ · · · ⊗ χρ(γr ) (γr ), Inde 0 e

ρ

(9.8)


93

where ρ 6 λ and the first summand is ρ = λ with multiplicity one. The parity of χλ is equal to d(λ) = n − l(λ) (mod 2). Its character at the conjugacy class of type µ ∈ OP n (0∗ ) is equal to the matrix coefficient 0 2(l(µ)−l(λ)+d(λ))/2 hX λ , a−µ i.

(9.9)

Moreover, the degree of the character is equal to 2b(n−l(λ))/2c n!

Y γ ∈0 ∗

Y λi (γ ) − λ j (γ ) deg(γ )|λ(γ )| , λi (γ ) + λ j (γ ) 1≤i≤l(λ(γ )) λi (γ )!

Q

(9.10)

i< j

where bac denotes the smallest integer greater than or equal to a. Proof Suppose we know that Q λ corresponds to the character of an irreducible e 0n supermodule. By (5.4) and the definition of the characteristic map, we see immediately that the linear combination (9.8) is given by the vertex operator structure and the matrix coefficient (9.9) gives the character table of all irreducible supermodules. First we observe that the number of irreducible spin supermodules of type M is equal to the number of even strict partition-valued functions, which are realized by the vectors 2−(l(λ)−d(λ))/2 X λ (γ ) (λ ∈ S P 0 (0 ∗ )) up to signs. As for the vectors 2−(l(λ)−d(λ))/2 X λ (γ ) with (λ ∈ S P 1 (0 ∗ )), it follows from Theorem 9.4 and Proposition 3.5 that each of such vectors corresponds to a virtual irreducible character in RZ− (e 0n ) of type Q since the case of sum or difference of two irreducible characters of type M is ruled out by the orthogonality. To show that they correspond to actually irreducible characters, it is sufficient to show that the value of ch−1 2−(l(λ)−d(λ))/2 X λ (γ ) at the conjugacy class of the identity element of e 0n is positive. 0 Let c ∈ 0∗ be the class consisting of the identity in 0. The type of the identity element in e 0n is the partition-valued function ρ such that ρ(c0 ) = (1n ) and ρ(c) = 0 P for c 6 = c0 . Recall from (5.2) that am (c0 ) = γ ∈0 ∗ deg(γ )am (γ ). By comparing weights and using orthogonality (see Theorem 9.4), we have

0 n hX λ , a−ρ i = X λ(γ ) , a−1 (c0 ) Y X n = X λ(γ ) (γ ), (degγ )a−1 (γ ) γ ∈0 ∗

= n!

γ ∈0 ∗

Y (degγ )|λ(γ )| Y

|λ(γ )| X λ(γ ) (γ ), a−1 (γ ) . λ(γ ) ! γ ∈0 ∗ γ ∈0 ∗

94


By the result of [J1, (6.51)], we have

|λ(γ )| X λ(γ ) (γ ), a−1 (γ )

=

λ(γ ) ! λ1 (γ )! · · · λl(λ(γ )) (γ )!

Y λi (γ ) − λ j (γ ) i< j

λi (γ ) + λ j (γ )

,

which implies (9.10); thus the theorem is proved. The irreducible spin e 0n -supermodules can be described easily as follows. For each irreducible character γ ∈ 0 ∗ , let Uγ be the irreducible 0-module affording γ . For each strict partition ν, let Vν be the corresponding irreducible spin supermodule of e Sn . Using the construction of Section 4.3, we see that Uγ⊗n ⊗Vν is a spin e 0n -supermodule. PROPOSITION 9.8 For each strict partition-valued function λ = (λ(γ )) ∈ S P n (0 ∗ ), with m of the partitions λ(γ ) being odd, the super tensor product Y (Uγ⊗l(λ(γ )) ⊗ Vλ(γ ) ) γ ∈0 ∗

decomposes completely into 2dm/2e copies of an irreducible spin e 0λ -supermodule. e 0n Denote this irreducible module by Wλ . Then the induced supermodule Inde W is 0λ λ the irreducible spin e 0n -supermodule corresponding to λ, and it is of type M or Q according to whether d(λ) = n − l(λ) is even or odd. Proof Let Vλ be the irreducible spin e 0n -supermodule corresponding to λ. It follows from e 0n Theorem 9.7 that Vλ is an irreducible component of Inde Wλ . 0 ⊗l(λ(γ ))

λ

Note that the supermodule Uγ ⊗ Vλ(γ ) is irreducible and is of type M (or type Q) according to d(λ(γ )) = |λ(γ )| − l(λ(γ )) even (or odd). Let λ(γi0 ), . . . , λ(γim−1 ) be odd partitions, and let λ(γim ), . . . , λ(γir ) be even partitions. Then the parity of λ equals the parity of m; that is, d(λ) = n − l(λ) ≡ m (mod 2). Q ⊗l(λ(γ )) It follows from Proposition 3.7 that γ ∈0 ∗ Uγ ⊗ Vλ(γ ) decomposes comdm/2e pletely into 2 copies of the irreducible supermodule Wλ of e 0λ , and Wλ is of e 0n type M if m is even and of type Q otherwise. The degree of Inde W equals to 0λ λ (|e 0n |/|e 0λ |)deg(Wλ ), and we have Y deg(Wλ ) = 2−dm/2e deg(γ )l(λ(γ )) deg(Vλ(γ ) ) γ ∈0 ∗

= 2−dm/2e

Y deg(γ )l(λ(γ )) |λ(γ ))|! Y λi (γ ) − λ j (γ ) 2bd(λ(γ ))/2c Q , λi (γ ) + λ j (γ ) 1≤i≤l(λ(γ )) λi (γ )! ∗

γ ∈0

i< j


95

where we have used the degree formula (9.10) for the special case of e Sn . The exponents of 2 in the first factor and the product sum up to r X d λ(γis ) + 1 X X l m m m−1 d λ(γis ) m−m d λ(γ ) − = + − 2 2 2 2 s=m γ ∈0 ∗ s=0 j n − l(λ) k n − l(λ) m + = , = 2 2 2 where m = 0 or 1 according to whether m is even or odd. Thus the degree of e e 0n 0n Inde W is exactly the one given by (9.10). Hence Inde W is the irreducible spin 0λ λ 0λ λ e 0n -supermodule Vλ corresponding to λ. References [DI]

J. DING and K. IOHARA, Generalization of Drinfeld quantum affine algebras, Lett.

[F1]

I. B. FRENKEL, “Representations of Kac-Moody algebras and dual resonance models”

Math. Phys. 41 (1997), 181–193. MR 98e:17019 87

[F2] [FJ] [FJW1]

[FJW2]

[FK] [FLM1]

[FLM2] [HH]

[J1]

in Applications of Group Theory in Physics and Mathematical Physics (Chicago, 1982), ed. M. Flato, P. Sally, and G. Zuckerman, Lectures in Appl. Math. 21, Amer. Math. Soc., Providence, 1985, 325–353. MR 87b:17010 51 , course on infinite dimensional Lie algebras, Yale Univ., New Haven, Conn., 1986. 51 I. B. FRENKEL and N. JING, Vertex representations of quantum affine algebras, Proc. Nat. Acad. Sci. U.S.A. 85 (1988), 9373–9377. MR 90e:17028 52 I. B. FRENKEL, N. JING, and W. WANG, Quantum vertex representations via finite groups and the McKay correspondence, Comm. Math. Phys. 211 (2000), 365–393. MR CMP 1 754 520 52 , Vertex representations via finite groups and the McKay correspondence, Internat. Math. Res. Notices 2000, 195–222. MR 2001c:17042 51, 52, 70, 71, 74, 76, 81, 88 I. B. FRENKEL and V. G. KAC, Basic representations of affine Lie algebras and dual resonance models, Invent. Math. 62 (1980), 23–66. MR 84f:17004 51 I. B. FRENKEL, J. LEPOWSKY, and A. MEURMAN, “An E 8 -approach to F1 ” in Finite Groups: Coming of Age (Montréal, 1982), Contemp. Math. 45, Amer. Math. Soc., Providence, 1985, 99–120. MR 87e:20038 52, 81, 84, 88 , Vertex Operator Algebras and the Monster, Pure Appl. Math. 134, Academic Press, Boston, 1988. MR 90h:17026 52, 82, 84, 88 P. N. HOFFMAN and J. F. HUMPHREYS, Hopf algebras and projective representations of G o Sn and G o An , Canad. J. Math. 38 (1986), 1380–1458. MR 88h:20014 52, 61, 67, 69 N. JING, Vertex operators, symmetric functions, and the spin group 0n , J. Algebra 138 (1991), 340–398. MR 92e:17033 51, 52, 53, 58, 88, 89, 90, 91, 94

96

[J2]

[Jo]

[LW] [M] [Mc]

[MRY] [Mo] [S]

[Se] [Sr] [St] [Ws] [Wl] [W] [Z]


, “New twisted quantum affine algebras” in Representations and Quantizations (Shanghai, 1998), China High. Educ. Press, Beijing, 2000, 263–274. MR CMP 1 802 177 87, 88 ´ T. JOZEFIAK , Characters of projective representations of symmetric groups, Exposition. Math. 7 (1989), 193–247. MR 90f:20018 52, 53, 54, 58, 62, 63, 67, 72, 78, 92 (1) J. LEPOWSKY and R. L. WILSON, Construction of the affine Lie algebra A1 , Comm. Math. Phys. 62 (1978), 43–53. MR 58:28089 52 I. G. MACDONALD, Symmetric Functions and Hall Polynomials, 2d ed., Oxford Math. Monogr., Oxford Univ. Press, New York, 1995. MR 96h:05207 51, 57, 61, 91 J. MCKAY, “Graphs, singularities, and finite groups” in The Santa Cruz Conference on Finite Groups (Santa Cruz, Calif., 1979), Proc. Sympos. Pure Math. 37, Amer. Math. Soc, Providence, 1980, 183–186. MR 82e:20014 51, 75, 87 R. V. MOODY, S. E. RAO, and T. YOKONUMA, Toroidal Lie algebras and vertex representations, Geom. Dedicata 35 (1990), 283–307. MR 91i:17032 51 A. O. MORRIS, The spin representation of the symmetric group, Proc. London Math. Soc. (3) 12 (1962), 55–76. MR 25:133 73 ¨ I. SCHUR, Uber die Darstellung der symmetrischen und der alternierenden Gruppe durch gebrochene lineare Substitutionen, J. Reine Angew. Math. 139 (1911), 155–250. 52, 53, 72, 73, 78, 91 G. SEGAL, Unitary representations of some infinite-dimensional groups, Comm. Math. Phys. 80 (1981), 301–342. MR 82k:22004 51 J.-P. SERRE, Représentations linéaires des groupes finis, 3d ed., Hermann, Paris, 1978. MR 80f:20001 64 J. R. STEMBRIDGE, Shifted tableaux and the projective representations of symmetric groups, Adv. Math. 74 (1989), 87–134. MR 90k:20026 58 D. B. WALES, Some projective representations of Sn , J. Algebra 61 (1979), 37–57. MR 81f:20015 54 C. T. C. WALL, Graded Brauer groups, J. Reine Angew. Math. 213 (1964), 187–199. MR 29:4771 63 W. WANG, Equivariant K-theory, wreath products, and Heisenberg algebra, Duke Math. J. 103 (2000), 1–23. MR 2001b:19005 51, 71, 74 A. V. ZELEVINSKY, Representations of Finite Classical Groups: A Hopf Algebra Approach, Lecture Notes in Math. 869, Springer, Berlin, 1981. MR 83k:20017 51

Frenkel Department of Mathematics, Yale University, New Haven, Connecticut 06520, USA; [email protected] Jing Department of Mathematics, North Carolina State University, Raleigh, North Carolina 27695-8205, USA; [email protected]


Wang Department of Mathematics, North Carolina State University, Raleigh, North Carolina 27695-8205, USA; [email protected]

97


RIEMANNIAN MANIFOLDS WITH UNIFORMLY BOUNDED EIGENFUNCTIONS JOHN A. TOTH and STEVE ZELDITCH

Abstract The standard eigenfunctions φλ = eihλ,xi on flat tori Rn /L have L ∞ -norms bounded independently of the eigenvalue. In the case of irrational flat tori, it follows that L 2 normalized eigenfunctions have uniformly bounded L ∞ -norms. Similar bases exist on other flat manifolds. Does this property characterize flat manifolds? We give an affirmative answer for compact Riemannian manifolds with quantum completely integrable Laplacians. 0. Introduction This paper is concerned with the relation between the dynamics of the geodesic flow G t on the unit sphere bundle S ∗ M of a compact Riemannian manifold (M, g) and the growth rate of the L ∞ -norms of its L 2 -normalized 1-eigenfunctions (or “modes”) {φλ }. Let Vλ := {φ : 1φλ = λφλ } denote the λ-eigenspace for λ ∈ Sp(1), and define L ∞ (λ, g) =

sup ||φ|| L ∞ ,

φ∈Vλ ||φ|| 2 =1 L

`∞ (λ, g) =

inf

ONB{φ j }∈Vλ

sup

||φ j || L ∞ .

j=1,...,dim Vλ

(1) The universal bound

L ∞ (λ, g) = 0(λ(n−1)/4 )

holds for any (M, g) in consequence of the local Weyl law (see [Ho]): N (T, x) :=

X j:λ j ≤T

|φλ j (x)|2 =

1 vol(M, g)T n/2 + O(T (n−1)/2 ). (2π )n

The error is attained in the case of the standard (S n , can) (by the zonal spherical harmonics) but is far off in the case of irrational flat tori (T n , ds 2 ) where L ∞ (λ, g) = DUKE MATHEMATICAL JOURNAL c 2002 Vol. 111, No. 1, Received 31 March 2000. Revision received 29 January 2001. 2000 Mathematics Subject Classification. Primary 58F07, 58G25. Toth’s work partially supported by Natural Sciences and Engineering Research Council grant number OGP0170280. Zelditch’s work partially supported by National Science Foundation grant number DMS-0071358. 97

98

TOTH and ZELDITCH

O(1). These cases represent the extremes, and the problem arises of characterizing the manifolds with extremal growth rates of L ∞ -norms of eigenfunctions. In this article we are interested in the case of minimal growth. Problem Determine the (M, g) for which `∞ (λ, g) = O(1) and those for which L ∞ (λ, g) = O(1). The same kind of problem may be posed in the more general setting of semiclassical Schrödinger operators ~2 1 + V . The eigenvalue problem (~2 1 + V )φ j = E j (~)φ j now depends on ~, and we are interested in the behaviour of eigenfunctions φ j in the semiclassical limit ~ → 0. The spectrum becomes dense around each regular value E of the classical Hamiltonian H (x, ξ ) = |ξ |2g + V (x) on T ∗ M, and for any 0 < δ < 1, the asymptotics of spectral data from an interval [E − c~1−δ , E + c~1+δ ] around E reflect the dynamics of the classical Hamiltonian flow 8tE on the energy surface X E = {H (x, ξ ) = E}. We fix E and 0 < δ < 1 and consider the eigenvalues E j (h) ∈ [E − c~1−δ , E + c~1−δ ]. Denote by VE j (~) the eigenspace of eigenvalue E j (h), put L ∞ ~, E j (~); g, V = sup ||φ|| L ∞ , φ∈V E (~) j ||φ|| 2 =1 L

`∞ h, E j (~); g, V =

inf

ONB{φ j }∈VE j (~)

sup

||φ|| L ∞ ,

(2)

j=1,...,dim VE j (~)

and pose the analogous questions. Problem Determine the (M, g, V ) for which there exists a regular energy level E such that `∞ (~, E j (~); g, V ) = O(1) and the (M, g, V ) for which L ∞ (~, E j (~); g, V ) = O(1) as ~ → 0 with E j (~) ∈ [E − c~1−δ , E + c~1−δ ] for some c > 0. The problem on Laplace operators is the same as the problem on Schrödinger operators in the case V = 0, for any value of E > 0. The problems about `∞ ask which Laplacians or Schrödinger operators possess an orthonormal basis of eigenfunctions (ONBE) of minimal growth. The problems about L ∞ ask which ones have the property that every ONBE has minimal growth. Obviously, the distinction between `∞ and L ∞ arises only when the spectrum of 1 is multiple. At the opposite extreme, one may ask which (M, g) possess eigenfunctions that achieve the maximal rate of growth, but we do not discuss that problem here (see [SZ]). One may also pose quantitative problems of giving upper and lower bounds on

UNIFORMLY BOUNDED EIGENFUNCTIONS

99

`∞ (λ, g), L ∞ (λ, g), and their L p -analogues, under various dynamical hypotheses. Some results on such quantitative problems will be given in a subsequent article [TZ]. The known connections between (~2 1 + V )-eigenfunctions and the dynamics of 8tE are not strong enough at present to answer these questions in the general setting of compact Riemannian manifolds. If, however, the systems are assumed to be completely integrable geodesic flows, then much more can be said. Let us assume, in fact, that 1 is quantum completely integrable (QCI) in the (well-known) sense that there exist P1 , . . . , Pn ∈ 9 1 (M) (n = dimM) satisfying • [Pi , P j ] = 0; • d p1 ∧ d p2 ∧ · · · ∧ dpn 6= 0 on a dense open set ⊂ T ∗ M − 0 of finite complexity (see below); √ ˆ • 1 = K (P1 , . . . , Pn ) for some polyhomogeneous function Kˆ on Rn − 0. Here 9 m (M) is the space of mth-order pseudodifferential operators over M, and pk = σ Pk is the principal symbol of Pk . Since σ[Pi ,P j ] = { pi , p j } (the Poisson bracket), it follows that the p j ’s generate a homogeneous Hamiltonian action 8t of t ∈ Rn on T ∗ M − 0 with moment map P : T ∗ M − 0 → Rn ,

P = ( p1 , . . . , pn ).

We denote the image P (T ∗ M − 0) by B and denote by Breg (resp., Bsing ) the regular values (resp., singular values) of the moment map. By finite complexity we mean the following. For each b = (b(1) , . . . , b(n) ) ∈ B, let m cl (b) denote the number of Rn -orbits of the joint flow 8t on the level set P −1 (b). Then ∃M : m cl (b) < M (∀b ∈ B) (finite complexity condition). (3) When b ∈ Breg , then P −1 (b) is the union of m cl (b) isolated Lagrangian tori. If b ∈ Bsing , then P −1 (b) consists of a finite number of connected components, each of which is a finite union of orbits. These orbits may be Lagrangian tori, singular compact tori (i.e., compact tori of dimension less than n), or noncompact orbits consisting of cylinders or planes. We also make the following assumption on the quantum level: (bounded eigenvalue multiplicity). (4) With this assumption, L ∞ is bounded by a constant times `∞ , so all ONBE’s are uniformly bounded if and only if one is. Without assumption (4), it is simple to construct an ONBE that is not uniformly bounded. We recall the construction in §4 and discuss some open problems in which the bounded eigenvalue multiplicity is dropped. The Hamiltonian v uX u n |ξ |g = t g i j (x)ξi ξ j ∃M 0 : m(λ) ≤ M 0

(∀λ; m(λ) = dimVλ )

i, j=1

100

TOTH and ZELDITCH

is then given by |ξ |g = K ( p1 , . . . , pn ), where K is the homogeneous term of order 1 of Kˆ . Hence, the geodesic flow commutes with a Hamiltonian Rn -action; that is, it is completely integrable. We assume throughout the following properness assumption. Our main result is the following rigidity theorem. 0.1 Suppose that 1 is a quantum completely integrable Laplacian on a compact Riemannian manifold (M, g), and suppose that the corresponding moment map satisfies (3). Then (a) if L ∞ (λ, g) = O(1), (M, g) is flat; (b) if `∞ (λ, g) = O(1) and if (4) holds, (M, g) is flat. More generally, suppose that ~2 1 + V is a quantum completely integrable Schrödinger operator and that the corresponding moment map P is proper and satisfies (3). Assume there exists an energy level E such that (a0 ) L ∞ (~, E j (~)); g, V ; ) = O(1) as ~ → 0; 0 (b ) `∞ (h, E j (~); g, V ) = O(1) as ~ → 0, and (4) holds. Then E > max V, and (M, (E − V )g) is flat. If (a) (or (b)) holds for all energy levels E in an interval E 1 < E < E 2 , then (M, g) is flat and V is constant. THEOREM

As mentioned above, (a) and (b) are equivalent, as are (a0 ) and (b0 ), so we consider only (a), (a0 ) henceforth. We recall that flat manifolds are manifolds carrying a flat metric. By the Bieberbach theorems (see [W, Ths. 3.3.1 and 3.3.2] ), a flat manifold (M, g) may be expressed as the quotient M = Rn / 0 of Rn by a discrete (crystallographic) subgroup of Euclidean motions 0 ⊂ E(n). The subgroup 0 ∗ := 0 ∩ Rn is normal and of finite index in 0, so there exist a flat torus T n = Rn / 0 ∗ and a finite normal Riemannian cover π : T n → M with deck transformation group G = 0/ 0 ∗ . For each n > 0, there are only finitely many affine equivalence classes of flat compact connected (M, g) of dimension n (affinely equivalent equals same fundamental group), and in low dimensions they have been classified (cf. [W]). The eigenfunctions φλ of 1g on (M, g) may be lifted to G-invariant eigenfunctions π ∗ φλ on T n , and hence the eigenspace E λ (M, g) may be identified with the G-invariant eigenspace E λ (T n , gT )G . The latter eigenfunctions may be written as sums of exponential functions. Let us outline the proof of Theorem 0.1 in the simplest case of toric integrable systems (see §1 for background) and then explain what more is involved in the case of general integrable systems. By definition, the geodesic flow G tg : T ∗ M → T ∗ M of a compact Riemannian manifold (M, g) is toric integrable if it commutes with a Hamiltonian action of the n-torus Rn /Zn . Equivalently, if there exist global action variables {(I j , θ j ) : j = 1, . . . , n} for the geodesic flow, that is, functions of ( p1 , . . . , pn )


101

whose Hamilton flows are 2π-periodic, the level sets TI := I −1 (I ) of the moment map I = (I1 , . . . , In ) : T ∗ M − 0 → Rn

are then orbits Rn /Zn · (xo , ξo ) of the torus action and hence are tori. The image B of T ∗ M − 0 under I is a convex polyhedral cone, and I is a Lagrangian torus bundle over its interior. Such moment maps I are the cotangent bundle analogues of toric varieties in algebraic geometry. In the toric case it is always possible to quantize the action variables as first-order pseudodifferential action operators Iˆj which commute with 1. The actions define a (projective) action of Rn /Zn by Fourier integral operators, or equivalently, the joint spectrum Sp( Iˆ1 , . . . , Iˆ1 ) is contained in an (off-centered) lattice Zn + µ. The joint eigenfunctions ( Iˆ1 , . . . , Iˆn )φλ = λφλ (λ ∈ Rn ) are therefore quantizations of the invariant Lagrangian tori Tλ with integral actions λ ∈ Zn + µ. In particular, eigenfunctions {φλ } localize on the invariant tori in the semiclassical limit in the sense that for any zeroth-order pseudodifferential operator A (with symbol σ A ), Z σ A dµλ + O(k −1 ), (5) (Aφkλ , φkλ ) = Tλ

where dµλ is the normalized Lebesgue (probability) measure on Tλ . Hence |φλ (x)|2 measures the density of the natural projection πλ : Tλ → M at x. The proof of Theorem 0.1 in the toric case is based on the following simple lemmas. First we have the following (see Proposition 3.1). Suppose that G t is toric integrable and that L ∞ (M, g) = O(1). Then every invariant torus Tλ has a nonsingular projection to M. The proof uses the fact that for any invariant torus TI , there exists a sequence of joint eigenfunctions {φλ } of the quantum torus action which localizes on TI . Uniform boundedness of the eigenfunctions then implies regular projection of the tori. The second ingredient in the proof of the main theorem in the case of toric integrable systems is the following purely geometric statement, which follows from the recently proved Hopf conjecture (cf. [BI], [CK]). Suppose that (M, g) is a compact Riemannian manifold with toric integrable geodesic flow, and suppose that all the invariant tori project regularly to M. Then (M, g) is a flat manifold (see Lemma 3.5).

102

TOTH and ZELDITCH

By “projecting regularly” we mean that the projection has no singular values and hence (in view of the dimensions) is a covering map. The proof of Theorem 0.1 in the case of general Hamiltonian Rn -actions is basically similar, but there are some new complications to handle. Geometrically, the new features are that the fibers P −1 (b) may have several components (“geometric multiplicity”), that there may exist noncompact orbits (e.g., embedded cylinders), and that there may exist singular orbits lying over the interior of the image of T ∗ M − 0 under P . Analytically, the main new feature is that modes need not localize on individual components of P −1 (b). What do localize on individual tori are quasimodes, that is, semiclassical Lagrangian distributions that approximately solve the eigenvalue problem. In the toric case, modes and quasimodes are the same, but this is not the case in general. As originally stressed by V. Arnold [A], and as is evident from simple examples such as the symmetric double well potential, eigenfunctions may be linear combinations of quasimodes with very close quasieigenvalues, and in the classical limit their mass concentrates in some way on the union of the components. How the mass is distributed involves the question of whether the tori are resonant or not, and whether or not there is tunnelling between tori. We will discuss such relations between modes and quasimodes in detail in [TZ] and elsewhere, where we prove (among other things) that quasimodes have uniformly bounded sup norms when modes do and where we determine precisely how modes blow up around singular orbits. In this paper we take a softer approach via quantum limits of eigenfunctions and semiclassical trace formulae.

1. Background 1.1. Completely integrable systems By a completely integrable system on T ∗ M we mean a set of n independent C ∞ functions p1 , . . . , pn on T ∗ M satisfying • { pi , p j } = 0 for all 1 ≤ i, j ≤ n; • d p1 ∧ d p2 ∧ · · · ∧ dpn 6= 0 on an open dense subset of T ∗ M. The associated moment map is defined by P = ( p 1 , . . . , p n ) : T ∗ M → B ⊂ Rn .

(6)

We refer to to the set B as the “image of the moment map.” The Hamiltonians generate an action of Rn defined by 8t = exp t1 4 p1 ◦ exp t2 4 p2 ◦ · · · ◦ exp tn 4 pn . We often denote 8t -orbits by Rn · (x, ξ ). The isotropy group of (x, ξ ) is denoted by I(x,ξ ) . When Rn · (x, ξ ) is a compact Lagrangian orbit, then I(x,ξ ) is a lattice of full


103

rank in Rn and is known as the “period lattice” since it consists of the “times” T ∈ Rn such that 8T |3( j) (b) = Id. We need the following. Definition 1.1 We say that • b ∈ Bsing if P −1 (b) is a singular level of the moment map, that is, if there exists a point (x, ξ ) ∈ P −1 (b) with dp1 ∧ · · · ∧ dpn (x, ξ ) = 0; such a point (x, ξ ) is called a singular point of P ; • a connected component of P −1 (b) (b ∈ Bsing ) is a singular component if it contains a singular point; • an orbit Rn · (x, ξ ) of 8t is singular if it is non-Lagrangian, that is, has dimension less than n; • b ∈ Breg and that P −1 (b) is a regular level if all points (x, ξ ) ∈ P −1 (b) are regular, that is, if dp1 ∧ · · · ∧ d pn (x, ξ ) 6 = 0; • a component of P −1 (b) ( b ∈ Bsing ∪ Breg ) is regular if it contains no singular points. By the Liouville-Arnold theorem (see [AM]), the orbits of the joint flow 8t are diffeomorphic to Rk × T m for some (k, m), k + m ≤ n. By the properness assumption on P , a regular level has the form P −1 (b) = 3(1) (b) ∪ · · · ∪ 3(m cl ) (b)

(b ∈ Breg ),

(7)

where each 3(l) (b) ' T n is an n-dimensional Lagrangian torus. The classical (or geometric) multiplicity function m cl (b) = #P −1 (b), that is, the number of orbits on the level set P −1 (b), is constant on connected components of Breg , and the moment map (6) is a fibration over each component with fiber (7). In sufficiently small neighbourhoods (l) (b) of each component torus 3(l) (b), the Liouville-Arnold theorem (l) (l) (l) (l) also gives the existence of local action-angle variables (I1 , . . . , In , θ1 , . . . , θn ) in terms of which the joint flow of 4 p1 , . . . , 4 pn is linearized (see [AM]). For con(l) (l) (l) venience we henceforth normalize the action variables I1 , . . . , In , so that I j = 0, j = 1, . . . , n, on the torus 3(l) (b). When b ∈ Breg , the Lagrangian tori 3( j) (b) of P −1 (b) carry two natural measures, which we take some care to distinguish. Definition 1.2 We define ( j) • the Lebesgue measure dµb = (2π)−n dθ1 ∧ · · · ∧ dθn on 3( j) (b) as the normalized (mass one) 8t -invariant measure on this orbit;

104

•

TOTH and ZELDITCH ( j)

the Liouville measure dωb on 3( j) (b) as the surface measure induced by the moment map P , that is, ( j)

dωb =

dV , dp1 ∧ · · · ∧ d pn

where d V is the symplectic volume measure on T ∗ M. By the Liouville mass of 3( j) (b) we mean the integral Z ( j) ω( j) (b) := dωb . 3( j) (b)

The Liouville mass of a compact Lagrangian orbit 3( j) (b) has a simple dynamical interpretation: it is the Euclidean volume of the fundamental domain of the common ( j) ( j) period lattice Ib = I(x,ξ ) of points (x, ξ ) ∈ 3( j) (b), that is, ( j)

ω( j) (b) = Vol(Rn /Ib ).

(8)

Indeed, by writing Liouville measure in local action-angle variables, we see that ( j) ( j) dωb = det T`k (b) dµb ,

where T`k =

∂ Ik . ∂ p`

(9)

( j)

It is clear from the definition of the action-angle variables that Ib is generated by the rows (T1k , . . . , Tnk ); hence the determinant is the covolume of the period lattice. We now turn to singular levels. When b ∈ Bsing , we first decompose the singular level r [ ( j) P −1 (b) = 0sing (b) (10) j=1 ( j)

into connected components 0sing (b) and then decompose each component ( j)

0sing (b) =

p [

Rn · (xk , ξk )

(11)

k=1

into orbits. Both decompositions can take a variety of forms. The regular components ( j) 0sing (b) must be Lagrangian tori by the properness assumption. A singular component consists of finitely many orbits by the finite complexity assumption. The orbit Rn · (x, ξ ) of a singular point is necessarily singular, hence has the form Rk ×T m for some (k, m) with k+m < n. Regular points may also occur on a singular component, whose orbits are Lagrangian and can take any one of the forms Rk × T m for some (k, m) with k + m = n. We need the following result in the proof of Theorem 0.1.


105

PROPOSITION 1.3 ( j) A singular component 0sing (b) ⊂ P −1 (b) (with b ∈ Bsing ) must contain a compact singular orbit Rn · (x, ξ ) ' T k , k < n.

Proof It follows by a standard averaging argument (see [M1]) that the set M I ( j) of invariant 0sing ( j) probability measures supported on 0sing is nonempty. For any probability measure µ0 ( j) supported on 0sing , the set of weak* limit points of the set of finite time averages

µT =

1 vol{|t| ≤ T }

Z

(8t )∗ µ0 dt |t|≤T ( j)

gives at least one nontrivial element of M I ( j) . Since 0sing consists of only finitely 0sing

many orbits, any invariant measure in M I ( j) is a finite sum of (ergodic) measures, 0sing

each supported on just one orbit. The noncompact orbits Rk × T m obviously cannot carry invariant probability measures; hence at least one orbit must be compact. We need a further result on Hamiltonian Rn -actions 8t . We define a nonzero period ( j) of 8t to be a time T ∈ Ib − {0} for some (b, j) and denote the set of periods by T. PROPOSITION 1.4 There exists a constant C > 0, which depends on the Riemannian manifold (M, g), such that inf{T ∈T } |T | ≥ C.

Proof In the case of a Hamiltonian flow with Hamilton vector field 4, this is a case of the Yorke theorem [Y]. In fact, C = 2π/L, where L = ||d4||∞ . In the case of Rn actions, we can apply the Yorke theorem to any one-parameter subgroup. 1.2. Hamiltonian torus actions In special cases (see [D] for the geometric conditions), the Hamiltonian Rn -action descends to the Hamiltonian action of the torus Rn /Zn on T ∗ M. Such Hamiltonian torus actions are the cotangent space analogues of toric varieties in algebraic geometry. In this case there exist generators I := (I1 , . . . , In ) : T ∗ M → B ⊂ Rn

of the Hamiltonian Rn -action, so that each I j generates a 2π-periodic Hamiltonian flow. The components I j are called global action variables, and I is called a toric

106

TOTH and ZELDITCH

moment map. In the toric case, B is a convex polyhedral cone, Breg is simply the interior of B, Bsing = ∂ B (its boundary), and m cl (b) ≡ 1. Since tori are now labelled by actions, we write TI := I −1 (I ). Singular orbits Rn · (x, ξ ) are obviously compact non-Lagrangian tori, and singular levels consist of just one singular orbit. Examples (i) M = Rn /Zn , I j = ξ j , the usual linear coordinates on T ∗ (Rn /Zn ). (ii) M = S2 , I1 = pθ , I2 = |ξ |0 , where pθ (x, ξ ) = ξ(∂/∂θ) (the infinitesimal generator of rotations around the z-axis) and where |ξ |0 is the length function of the standard metric. 1.3. Riemannian manifolds with completely integrable geodesic flow Now suppose that g is a Riemannian metric on M, and let H (x, ξ ) = |ξ |g denote the associated length function on covectors. The Hamilton flow G t of H on T ∗ M − 0 is homogeneous of degree 1 with respect to the natural R+ -action and is referred to as the geodesic flow. It leaves invariant the cosphere bundles S ∗ M E = {H = E}, and the flows G tE on S ∗ M E are all equivalent under dilation (x, ξ ) → E(x, ξ ) to G t1 . The geodesic flow G t is called integrable if it commutes with a homogeneous Hamiltonian action of Rn . We may then put H = p1 . It is called toric integrable if it commutes with a homogeneous Hamiltonian action of Rn /Zn . Because m cl (b) ≡ 1 in this case, there exists a homogeneous function K on B such that H = K (I ). Examples (i) M = Rn /Zn and g is flat. Then (M, g) is toric integrable. (ii) M = S2 and g is a rotationally invariant metric. If g is of “simple type” (e.g., convex), then (M, g) is toric integrable (see [CV3]). (iii) M = S2 and g is the metric for which (S2 , g) is an ellipsoid. (iv) M = R2 /Z2 and g is a Liouville metric (cf. [BKS], [KMS]). (v) Bi-invariant metrics on compact Lie groups: geodesic flow on SO(3) is known as the Euler top. 1.4. Manifolds without conjugate points A Riemannian manifold (M, g) is said to be without conjugate points if there exists ˜ g) a unique geodesic between each two points of its universal Riemannian cover ( M, ˜ or, equivalently, if every exponential map expx : Tx M → M is nonsingular. We need the following geometric theorems on manifolds without conjugate points. THEOREM 1.5 ([M2]) Let (M, g) be a compact Riemannian manifold with (co)geodesic flow G t : T ∗ M −


107

0 → T ∗ M − 0. Suppose that G t preserves a (nonsingular) Lagrangian foliation L of T ∗ M − 0; that is, suppose that G t L = L for all leaves L of L . Then (M, g) has no conjugate points. The Hopf conjecture on tori without conjugate points was proved by D. Burago and S. Ivanov. THEOREM 1.6 ([BI]) Suppose that g is a metric on the n-torus T n without conjugate points. Then g is flat.

1.5. Integrable Newtonian flows on cotangent bundles We also consider Newtonian flows, that is, flows of classical Hamiltonians H (x, ξ ) = |ξ |2 + V (x) on cotangent bundles T ∗ M. Such Hamiltonians and their flows G t are no longer homogeneous. The invariant energy surfaces X E = {H = E} and the restricted flows G tE of G t to X E may change drastically with E. In particular, they may be completely integrable for some values of E and not others. Examples (i) The spherical pendulum. M = S2 , H = |ξ |2 + cos φ; |ξ |2 corresponds to the round metric, and φ is the azimuthal angle. P (ii) The C. Neumann oscillator on T ∗ Sn , H = |ξ |2 + nj=1 α j x 2j on T ∗ Sn . Here 0 < α1 < · · · < αn are constants, (x1 , . . . , xn ) are Cartesian coordinates on Rn+1 , and |ξ |2 corresponds to the usual round metric. (iii) The Kowalevsky and Chaplygin tops. (See [He].) We note that in the nonhomogeneous case, the joint flows 8tE on each energy level are distinct systems and may be integrable only for some values of E. An interesting case is the Chaplygin top (see [He]), which is integrable only when the angular momentum integral is put equal to zero. 1.6. Rigidity theorems for Newtonian flows We need a generalization of R. Mañe´ ’s rigidity theorem to Newtonian flows on tori. The following combines some ideas of M. Bialy and L. Polterovich [BP] and A. Knauf [K] to give a rigidity result when M is a torus and H is completely integrable with only compact regular orbits. In fact, it is more general. PROPOSITION 1.7 Suppose that g is a metric and V (x) is a potential on the n-torus Tn such that the Hamiltonian flow G tE of H (x, ξ ) on X E preserves a C 1 -Lagrangian foliation by tori

108

TOTH and ZELDITCH

which project regularly to Tn . Then E > max V and (E − V )g is a flat metric. Proof By [K, Th. 2] no such invariant foliation exists unless E > max V , so we may assume this is the case. The Jacobi metric (E − V )g is then a well-defined metric on Tn . We denote by |ξ |2J,E the associated homogeneous Hamiltonian (length squared of a covector). Since the sets {H = E} and {|ξ |2J,E = 1} are the same, the latter carries a Lagrangian torus foliation which projects regularly to Tn . Since the geodesic flow G tJ,E of (E − V )g on {|ξ |2J,E = 1} coincides (up to a time reparametrization) with G tE , this foliation is invariant under G tJ,E . Now let Dr : T ∗ M − 0 → T ∗ M − 0 be the dilation Dr (x, ξ ) = (x, r ξ ). Then Dr : {|ξ |2J,E = 1} → {|ξ |2J,E = r 2 } intertwines the geodesic flows on these sphere bundles (up to constant time reparametrization). Since Dr is conformally symplectic, it also carries the invariant Lagrangian torus foliation of {|ξ |2J,E = 1} to an invariant Lagrangian torus foliation of {|ξ |2J,E = r 2 }. It follows that T ∗ M − 0 carries a Lagrangian torus foliation invariant under the geodesic flow of the Jacobi metric. By the Mañe´ theorem, the geodesic flow has no conjugate points, and so by the BuragoIvanov theorem, (E − V )g must be flat. COROLLARY 1.8 With the same notation as above, suppose that there exists an interval [E 0 −, E 0 +] such that, for all E ∈ [E 0 − , E 0 + ], G tE preserves a Lagrangian torus foliation which projects regularly to Tn . Then g is flat and V is constant.

Proof The assumption implies that (E − V )g is flat for all E in the interval. Let R E denote the curvature tensor of (E − V )g. It is clearly a real analytic function of E. Since R E ≡ 0 in [E 0 − , E 0 + ], it must vanish identically. Therefore the Newton flow 8t on T ∗ T n has no conjugate points. By [BP, Rem. 1.C and Th. 1.B], it follows that g is flat and V is constant. 1.7. Semiclassical quantum integrable systems: Semiclassical calculus We now provide the necessary background on quantum integrable systems. Since we wish to include quantizations of possibly inhomogeneous Hamiltonians, the proper framework is that of semiclassical pseudodifferential operators. First, we introduce symbols. On a given open U ⊂ Rn , we say that a(x, ξ ; ~) ∈ ∞ C (U × Rn ) is in the symbol class S m,k (U × Rn ), provided α β ∂ ∂ a(x, ξ ; ~) ≤ Cαβ ~−m 1 + |ξ | k−|β| . x ξ


109

We say that a ∈ Sclm,k (U × Rn ), provided there exists an asymptotic expansion a(x, ξ ; ~) ∼ ~−m

∞ X

a j (x, ξ )~ j ,

j=0

with a j (x, ξ ) ∈ S 0,k− j (U × Rn ). The associated ~-quantization Op~ (a) is defined locally by the standard formula Z −n Op~ (a)(x, y) = (2π~) ei(x−y)ξ/~ a(x, ξ ; ~) dξ. Rn

By using a partition of unity, one constructs a corresponding class Op~ (S m,k ) of properly supported ~-pseudodifferential operators acting globally on C ∞ (M); as is well known, it is independent of the choice of partition of unity. Given a ∈ S m 1 ,k1 and b ∈ S m 2 ,k2 , the composition is given by Op~ (a) ◦ Op~ (b) = Op~ (c) + O (~∞ ) in L 2 (M), where locally, c(x, ξ ; ~) ∼ ~−(m 1 +m 2 )

∞ X (−i~)|α| α (∂ξ a) · (∂xα b). α!

|α|=0

Definition 1.9 We say that the operators P j~ ∈ Op~ (Sclm,k ), j = 1, . . . , n, generate a semiclassical quantum completely integrable system on M if for each ~, Pn ~∗ ~ ∗ • j=1 P j P j is jointly elliptic on T M, • [Pi~ , P j~ ] = 0 (∀1 ≤ i, j ≤ n) and the respective semiclassical principal symbols p1 , . . . , pn generate a classical integrable system on T ∗ M with dp1 ∧ dp2 ∧ · · · ∧ dpn 6 = 0 on a dense open subset of T ∗ M. We also assume that the finiteness condition (3) is satisfied. 1.7.1. Examples The basic examples we have in mind are where P1~ = ~2 1 + V ∈ Op~ (Scl0,2 ) is a Schrödinger operator over a compact manifold M. Examples include the following. • Quantum integrable Laplacians 1 such as Laplacians of Liouville metrics on the sphere or torus (see [BKS], [KMS]), or of the ellipsoid (see [T1]). • Toric integrable Laplacians such as the flat Laplacian on Tn , or Laplacians for surfaces of revolution of “simple type” (see below and [CV3]). • The quantum spherical pendulum ~2 1 + cos φ. M = S 2 , 1 is the standard Laplacian, and V = cos φ where φ is the azimuthal angle. The commuting operator is ~(∂/∂θ), the generator of rotations around the z-axis. • The C. Neumann oscillator on Sn . Here the quantum Hamiltonian is the P Schrödinger operator ~2 1 + nj=1 α j x 2j acting on C ∞ (Sn ). Here 1 is the

110

•

TOTH and ZELDITCH

spherical, constant curvature Laplacian, and the potential is the one described above. For the quantized C. Neumann system, one can construct quantum integrals that are all second-order, real-analytic, semiclassical partial differential operators on the sphere (see [T1]). The quantized Euler, Lagrange, and Kowalevsky tops. The Euler and Lagrange cases are classical (see [He]), while the quantum Kowalevsky top was shown to be QCI (quasiconformal instanton) recently by G. Heckman [He]. Here the integrals are semiclassical differential operators in the enveloping algebra of so(3) F R3 defined as follows: Let E 1 , E 2 , E 3 be the standard Pauli basis of so(3, R), and let L 1 , L 2 , L 3 be the corresponding left-invariant vector fields defined by d f (x exp t E i ) t=0 . dt 3 Fix a unit vector e ∈ R , and define the C ∞ -functions on SO(3) by L i ( f )(x) :=

Q i (x) := hxei , ei. Then the space of operators generated by Q 1 , Q 2 , Q 3 , L 1 , L 2 , L 3 can be identified with so(3) F R3 . Two of the quantum integrals are the quantized energy Schrödinger operator, P1 := (1/4)~2 (L 21 + L 22 + 2L 23 ) − Q 1 , and the quantized P momentum operator, P2 = ~ 3j=1 Q j L j . In analogy with the classical case, the third quantum integral is a fourth-order partial differential operator defined as follows. Put K := ~2 (L 1 + i L 2 )2 + 4(Q 1 + i Q 2 ). Then, in terms of K , P3 = K K ∗ + K ∗ K − 8~4 (L 21 + L 22 ). Homogeneous quantum completely integrable systems are the special case where ~ occurs with the same power in each term and where homogeneous symbols √ the usual √ of the operators are all of order 1, for example, ~ 1 or ~ 1 + V . In this case, one could remove ~ and use the homogeneous symbolic calculus. However, it is often more convenient to convert homogeneous systems P1 , . . . , Pn into semiclassical ones by introducing a semiclassical parameter ~ (with values in some sequence {~k ; k = 1, 2, 3, . . .} with ~k → 0) and semiclassically scaling the P j ’s: P j~ := ~P j ,

j = 1, 2, . . . , n. (12) √ When P1 = 1, P2 , . . . , Pn are classical pseudodifferential operators of order 1, then P j~ := ~P j ∈ Op(Scl0,1 ) generate the semiclassical quantum integrable system in the sense of Definition 1.9. 1.8. Quantum torus actions (See [GS2] for many details on this case.) Classical torus actions can always be quantized and produce the simplest examples of toric quantum integrable systems. The


111

classical actions {I j } can be quantized as commuting pseudodifferential operators Iˆ1 , . . . , Iˆn whose joint spectrum Sp( Iˆ1 , . . . , Iˆn ) = 3 ⊂ (Zn + ν) ∩ B is a lattice (translated by a Maslov index). The simplest case is that of the torus, where Iˆj = ∂/∂θ j (with θ j denoting the usual angular coordinates). The operators √ 1 + 1/4, ∂/∂θ on S 2 provide another example. Less obviously, any convex surface of revolution has a toric integrable Laplacian (cf. [CV3]). Just as the classical multiplicity m cl (b) ≡ 1 in the toric case, so also the multiplicity m(λ) of the joint eigenvalues is 1 for |λ| sufficiently large (see [CV3]). Hence, up to a finite dimensional subspace, there is a unique (up to unit scalars) orthonormal basis of joint eigenfunctions Iˆj φλ = λ j φλ ,

λ = (λ1 , . . . , λn ) ∈ 3.

1.9. Joint eigenvalue ladders In the next section we study the localization of sequences of eigenfunctions on level sets of the moment map. To obtain sequences that localize on a given level P −1 (b), it is necessary to choose the corresponding joint eigenvalues to tend in an appropriate sense to b. Roughly speaking, such joint eigenvalues form an “eigenvalue ladder.” The term comes from the toric case, where the joint spectrum 3 of the action operators is a semilattice (i.e., the set of lattice points in a cone). We define ladders (or rays) in a direction λ by Nλ = {kλ + ν, k = 0, 1, 2, . . . } ⊂ 3.

(13)

In the case of quantizations of torus and other Hamiltonian compact group actions, semiclassical limits are essentially the same as limits along ladders (cf. [GS2], [CV3]). In the Rn case, there is usually no optimal choice of the generators P j , and their joint spectrum is quite far from a lattice. We therefore define a homogeneous ladder of eigenvalues in the direction b = (b(1) , b(2) , . . . , b(n) ) ∈ Rn to be a sequence satisfying n o λk (1) (n) λk := (λk , . . . , λk ) ∈ Spec(P1 , . . . , Pn ); ∀ j = 1, . . . , n, lim = b , k→∞ |λk | (14) q (1) 2 (n) 2 where |λk | := |λk | + · · · + |λk | . Finally, we introduce a notion of semiclassical ladders. We fix 0 < δ < 1,

112

TOTH and ZELDITCH

b = (b(1) , b(2) , . . . , b(n) ) ∈ Rn and define the set (1) (2) (n) Lb;δ (~) := b j (~) := b j (~), b j (~), . . . , b j (~) ∈ Spec(P1 , . . . , Pn ); |b j (~) − b| ≤ C~1−δ .

(15)

(1)

Here b j (~) = E j (~). Taking a sequence ~ → 0, the joint eigenvalues in Lb;δ (~) form a sequence tending to b which is the analogue of a homogeneous ladder. 2. Localization on tori One of the main inputs in the proof of Theorem 0.1 is the localization of a ladder of joint eigenfunctions of a quantum completely integrable system in a regular direction b ∈ Breg on the level set P −1 (b) of the moment map. In this section we prove the relevant localization results. We first consider toric systems, where level sets are regular and connected and eigenfunctions necessarily localize on individual tori. In the general Rn case, ladders of eigenfunctions localize on the possibly disconnected level set P −1 (b), and it is a complicated problem to determine how the limit eigenfunction mass (or “charge”) is distributed among the components. To deal with this problem, we define a notion of the charge of a component and prove that every compact component of P −1 (b) is charged by some sequence of eigenfunctions. This result plays an important role in the proof of the theorem. 2.1. Toric integrable systems Let A ∈ 9 o (M) denote any zeroth-order pseudodifferential operator, and let dµλ denote the Lebesgue measure on the Lagrangian torus Tλ . In the toric case we have the following localization result. PROPOSITION 2.1 ([Z]) For any ladder {kλ + ν : k = 0, 1, 2, . . . } of joint eigenvalues, we have Z (Aφkλ , φkλ ) = σ A dµλ + O(k −1 ). Tλ

We thus have the following. 2.2 For any invariant torus Tλ ⊂ S ∗ M, there exists a ladder {φkλ , k = 0, 1, 2, . . . } of eigenfunctions localizing on Tλ . COROLLARY

2.2. Rn -integrable systems The proper generalization of the toric localization result Proposition 2.1 to Rn -actions says that ladders of joint eigenfunctions localize on level sets of the moment map


113

rather than on individual tori. This result is more or less a folk theorem in the physics literature (see [E], [Be1], [Be2]), and the rigorous result is in principle known to experts. However, we were unable to find the result in the literature, so we sketch the proof here. It uses some material on quantum Birkhoff normal forms from [CV1]. Let b be a regular value of the moment map P , let P −1 (b) = 3(1) (b) ∪ · · · ∪ 3(m cl ) (b),

where the 3(l) (b), l = 1, . . . , m, are n-dimensional Lagrangian tori, and let dµ3( j) (b) denote the normalized Lesbegue measure on the torus 3( j) (b). Let b j (~) ∈ Lb,δ (~), and define

cl ~; b j (~) := Op~ (χl )φb j (~) , φb j (~) , l = 1, . . . , m cl (b). (16) We recall that χl is a cutoff function that is equal to 1 in the neighbourhood (l) (b) S of the torus 3(l) (b) and vanishes on k6=l (k) (b). 2.3 Let b ∈ Breg , and let {φb j (~) } be a sequence of L 2 -normalized joint eigenfunctions of P1 , . . . , Pn with joint eigenvalues in the ladder Lb,δ (~) of (15). Then, for any a ∈ S 0,−∞ , we have that as ~ → 0, PROPOSITION

Op~ (a)φb j (~) , φb j (~) =

m X l=1

cl ~; b j (~)

Z 3(`) (b)

a dµ3(`) (b) + O (~1−δ ).

Here dµ3(`) (b) denotes the Lebesgue measure on 3(`) (b). Proof Let L (l) be the pullback of the Maslov line bundle over 3(l) to the affine torus (l) (l) given by I1 = · · · = In = 0, and let (l) be a sufficiently small neighbourhood of 3(l) on which there exist action-angle variables (θ (l) , I (l) ). According to the quantum Birkhoff normal form (QBNF) construction (see [CV1]), for l = 1, . . . , k (l) and j = 1, . . . , n, there exist ~-Fourier integral operators Ub,~ : C ∞ (M) → (l)

C ∞ (Tn ; L (l) ), microlocally elliptic on (l) , together with C ∞ symbols f j (x; ~) ∼ P∞ (l) k k=0 f jk (x)~ , with f j0 (0) = 0 such that (l)∗ (l)

(l)

Ub,~ f j (P1 − b(1) , . . . , Pn − b(n) ; ~)Ub,~ =(l) 0

(l)

~ ∂ . i ∂θ j

(17)

Moreover, when P1 , . . . , Pn are self-adjoint, the operator Ub can be taken to be microlocally unitary.

114

TOTH and ZELDITCH

We now observe that the space of admissible (see [CP]) solutions of the microlocal eigenfunction equation (k)

Pk φb j (~) =(l) (b) b j (~)φb j (~)

(18)

is 1-dimensional. Indeed, such solutions are the same as solutions of (l)

(l)

(1)

(n)

f k (P1 − b(1) , . . . , Pn − b(n) ; ~)φ j =(l) (b) f k (b j − b(1) , . . . , b j − b(n) ; ~)φ j . We conjugate this equation to Birkhoff normal form (17) and use the fact that the microlocal solutions of the model equation ~ ∂ uj = mj uj i ∂θ are just multiples of exp[i(n + πγ /4)θ], where γ is the Maslov index and n ∈ Z. Thus the joint eigenfunctions φb j (~) are given microlocally by q (l) (19) φb j (~) =(l) (b) cl ~; b j (~) Ub;~ ei(n j +πγ /4)θ . The right sides of (19) are the usual quasimodes or semiclassical Lagrangian distributions (see [CV1]). Now let χl (x, ξ ) ∈ C0∞ (T ∗ M), l = 1, . . . , m cl (b), be a cutoff function that is identically equal to one on the neighbourhood (l) (b) and vanishes on (k) (b) for k 6= l. For ~ sufficiently small, we then have

cl (b) mX

Op~ (a)φb j (~) , φb j (~) = Op~ (a) ◦ Op~ (χl )φb j (~) , φb j (~) + O (~∞ ).

l=1

It follows by (19), the semiclassical Egorov theorem, and a Taylor expansion about the Lagrangian torus I (l) = 0 that

Op~ (a) ◦ Op~ (χl )φb j (~) , φb j (~)

(l) (l) = cl ~; b j (~) Op~ (a) ◦ Op~ (χl )Ub;~ (ei(n j +π γ /4)θ ), Ub;~ (ei(n j +πγ /4)θ ) (l)∗ (l) = cl ~; b j (~) Ub;~ Op~ (a) ◦ Op~ (χl )Ub;~ ei(n j +πγ /4)θ , ei(n j +πγ /4)θ Z = (2π )−n cl ~; b j (~) a dµl + e(~) + O (~), (20) 3(l)

where e(~) = hOp~ (r )u ~ , u ~ i for some function r ∈ C0∞ (Tn × D1 ) satisfying r (θ, I ) = O (|I |). (Recall that we have normalized the action variables so that I (l) = 0 on the torus 3(l) (b).) Here u ~ (θ) = exp[i(m 1 θ1 + · · · + m n θn )] with m j (~) = O (~1−δ ). An integration by parts in the I1 , . . . , In variables shows that Op~ (r )u ~ , u ~ = O (~1−δ ), and the proposition follows.


115

2.3. Charge of compact Lagrangian orbits We now investigate the coefficients c j (~) in Proposition 2.3 for “ladders” of eigenfunctions. Our purpose is to show that there exist ladders for which the limit as ~ → 0 of c j (~) is bounded below by a positive geometric constant. It is convenient at this point to introduce the language of quantum limits. 2.3.1. Quantum limits Let (P1 , . . . , Pn ) denote a quantum integrable system with classical integrable flow 8t . Fix E, and let M IE denote the set of invariant probability measures for 8tE on X E . For instance, M IE includes the orbital averaging measures µz , defined by Z Z 1 f 8t (z) dt. f dµz = lim n T →∞ T max |t j |≤T XE In the case of compact (torus) orbits, µz is the Lebesgue probability measure on the orbit of z. By the set Q E of “quantum limit” measures of the quantum integrable system at energy level E, we mean the set of weak* limits (as ~ → 0) of the measures d8b j (~) , defined by Z

(1) Op~ (a)φb j (~) , φb j (~) = a d8b j (~) (b j → E). (21) XE

We write d8b j (~) → dµ ∈ Q E for weak* convergence to the limit as ~ → 0. It is an easy consequence of the semiclassical Egorov theorem that Q E ⊂ M IE . When dµ equals the Lebesgue probability measure on an orbit, we say that the sequence {φb j (~) } localizes on the orbit. For background, terminology, and references in a closely related context, we refer to [JZ]. We now consider quantum limits of eigenfunctions corresponding to a ladder of joint eigenvalues. Put Vb,δ (~) = φb j (~) : b j (~) ∈ Lb;δ (~) . (22) There are many possible weak* limit points of the set following.

~∈[0,~0 ] Vb,δ (~).

S

We say the

Definition 2.4 For b ∈ Breg , a ladder of eigenfunctions is a sequence Eb := {φb j (~)} of joint eigenfunctions with the following properties: • b j (~) ∈ Lb,δ (~) as ~ → 0 forms an eigenvalue ladder; • d8b j (~) has a unique weak limit d8Eb as ~ → 0.

116

TOTH and ZELDITCH

For a ladder of eigenfunctions, lim~→0 c` (~; b j (~)) exists for each ` in Proposition 2.3. Definition 2.5 Given b ∈ Breg , we say that the ladder Eb = {φb j (~) } gives charge cl (Eb ) := lim~→0 c` (~; b j (~)) to the component torus 3(l) (b) and that it charges 3(l) (b) if cl (Eb ) > 0. The limit in Definition 2.5 clearly depends on the ladder Eb . For instance, there could be sequences of joint eigenfunctions localizing on each single component of P −1 (b). To obtain an invariant of the Lagrangian orbits which is independent of the ladder, we say the following. Definition 2.6 The charge c(3(l) (b)) of a component torus 3(l) (b) ⊂ P −1 (b) is defined by the formula c 3(l) (b) = sup cl (Eb ), Eb

where cl is the coefficient in the sum of Proposition 2.3. A useful formula for the charge is the following. PROPOSITION

We have

2.7

c 3(l) (b) = lim sup ~→0

max

φb j (~) ∈Vδ (~)

Op~ (χl )φb j (~) , φb j (~) .

Proof (i) ≥: By definition,

cl ~; b j (~) = Op~ (χl )φb j (~) , φb j (~) , where χl is a cutoff to l . Since Vb,δ (~) is a finite set for each ~, there exists φbmax ∈ j (~) Vb,δ (~) such that

Op~ (χl )φbmax , φbmax = max Op~ (χl )φb j (~) , φb j (~) . j (~) j (~) φb j (~) ∈Vδ (~)

We form the sequence {φbmax }~∈{~k } and then choose a subladder Ebmax with a unique j (~)


117

quantum limit. Then

max c 3(l) (b) ≥ lim Op~ (χl )φbmax , φ (~) b (~) j j ~→0

≥ lim sup max Op~ (χl )φb j (~) , φb j (~) . ~→0

φb j (~) ∈Vδ (~)

(ii) ≤: It is clear that for each ladder Eb we have

Op~ (χl )φb j (~) , φb j (~) . cl (Eb ) ≤ lim sup max ~→0

φb j (~) ∈Vδ (~)

Therefore the same holds after taking the supremum over Eb . The following lemma is the main result of this section. LEMMA 2.8 Let ω(l) (b) denote Liouville measure of the Lagrangian torus 3(l) (b) l 1, . . . , m cl (b). Then for all (b, l) ∈ Breg × {1, . . . , m cl (b)} we have

=

ω(l) (b) c 3(l) (b) ≥ Pm (b) . cl ( j) (b) ω j=1 Proof Fix ζ ∈ S (Rn ) with ζ ≥ 0, ζˇ ∈ C0∞ (Rn ), and ζˇ (0) = 1. Assume, moreover, that 0 ∈ Rn is the only point of intersection of supp ζ with the joint periods of the joint flow 8t . Let K be a fixed compact neighbourhood of b = (b(1) , . . . , b(n) ) and a ∈ S 0,−∞ . Consider the localized semiclassical trace X

b j (~) − b Tra (ζ ) := Op~ (a)φb j (~) , φb j (~) ζ . (23) ~ b j (~)∈K

The localized semiclassical trace formula for commuting operators (see [Ch]) implies that for any a ∈ S 0,−∞ and ζ ∈ S (Rn ) as above, Z Tra (ζ ) = (2π)−n a dω(l) (b) + O (~). (24) P −1 (b)

So in particular, putting a(x, ξ ) = χl (x, ξ ), we have Z Trχl (ζ ) = (2π )−n χl dω(l) (b) + O (~) = (2π)−n ω(l) (b) + O (~) 3(l) (b)

(25)

since χl = 1 on the torus 3(l) (b). On the other hand, since ζ ∈ S (Rn ), it follows that X

b j (~) − b Trχl (ζ ) = Op~ (χl )φb j (~) , φb j (~) ζ + O (~∞ ). (26) ~ {b j (~)∈Lb,δ (~)}

118

TOTH and ZELDITCH

Thus, by the definition (2.6) of the charge c(3(l) (b)) and the fact that ζ ≥ 0, we have

max Op~ (χl )φb j (~) , φb j (~) | Trχl (ζ )| ≤ (2π)−n {b j (~)∈Lb,δ (~)}

X

·

ζ

{b j (~)∈Lb,δ (~)}

b (~) − b j + O (~∞ ). ~

(27)

Next, by applying the trace formula once again, we get X

ζ

{b j (~)∈Lb,δ (~)}

mX cl (b) b (~) − b j = (2π )−n ω( j) (b) + O (~). ~

(28)

j=1

Substituting (28) in (27) yields the estimate Trχ (ζ ) ≤ (2π )−n l

max

{b j (~)∈Lb,δ (~)}

cl (b) mX Op~ (χl )φb j (~) , φb j (~) · ω( j) (b) + O (~).

j=1

(29) The lemma then follows by combining (29) and (25) and letting ~ → 0. This yields a generalization of Corollary 2.2. 2.9 (`) For any b ∈ Breg , and for any 1 ≤ ` ≤ m cl (b), there exists a ladder Eb = {φb j (~) } such that ω(`) (b) (`) c` (Eb ) ≥ Pm (b) . cl ( j) (b) ω j=1 COROLLARY

Thus, every regular torus orbit is charged by some ladder. This follows from Lemma 2.8, Proposition 2.7, and Proposition 2.3. 2.3.2. Charge of compact singular orbits Our next step is to prove that some compact singular orbits are also charged. To be precise, we have so far only defined the notion of charge for regular levels of the moment map (see Definition 2.5). The analogous definition in the case of a singular S ( j) value bs ∈ Bs is as follows. Let P −1 (bs ) = rj=1 0sing (bs ) be the decomposition of (10) into connected components. Definition 2.10 When bs ∈ Bsing , we define an eigenfunction ladder Ebs to be a sequence of joint eigenfunctions with joint eigenvalues satisfying b j (~) − bs = o(1) as ~ → 0 and


119

R with unique limit measure d8Ebs . We say that Ebs gives charge 0 ( j) (b ) d8Ebs to sing s R ( j) the component 0sing . Similarly, we say that it gives charge 3( j) (b ) d8Ebs to any ( j) ( j) orbit 3sing (bs ) on 0sing (bs ) (see (11)). Finally, the charge c(3( j) (bs )), of a component, respectively, an orbit on the

sing

s

c(0 ( j) (b

s )), respectively, component, is the supre-

mum of the same over all ladders Ebs . We then have the following. 2.11 ( j) Let bs ∈ Bsing , and let {0sing (b)} denote the singular components of P −1 (bs ). Then LEMMA

( j)

there exists j such that c(0sing (b)) > 0. Further, there exists a compact singular orbit ( j)

3( j) (bs ) ⊂ 0sing (b) such that c(3( j) (bs )) > 0. Proof S ( j) Let Using be a 8t -invariant neighbourhood of rj=1 0sing (b). Let {bn } ⊂ Breg be a sequence of regular points such that bn → bs . For each j and sufficiently large n, there exists at least one component 3(`) (bn ) of P −1 (bn ) such that 3(`) (bn ) ⊂ Using . By Lemma (2.8), 3(`) (bn ) is charged by an amount greater than or equal to ω(`) (bn ) . Pm cl (bn ) ( j) ω (bn ) j=1 We now break up the discussion into two cases. S ( j) Case 1: All Rn -orbits of rj=1 0sing (b) are compact. In this case we just need a positive lower bound for the quotient ω(`) (bn ) Pm cl (bn ) ( j) ω (bn ) j=1 as n → ∞. A lower bound for the numerator is given by the minimal period of the Yorke theorem (see Proposition 1.4). Since all orbits (including the limit) are compact, the masses in the denominator have uniform upper bounds. Indeed, by (8) the masses are the covolumes of the period lattices of 3(`) (b). Since the period vectors generating the lattices are uniformly bounded as n → ∞, the volumes are also uniformly bounded above. Hence, the denominator is bounded above, and therefore the quotient is bounded below by a positive constant. S ( j) Case 2: There exists a noncompact orbit in rj=1 0sing (b). In this case the denominator tends to infinity, so we need a better lower bound on the numerator. We claim that

120

TOTH and ZELDITCH

there exists ` such that 3(`) (bn ) ⊂ Using and c(3(`) (bn )) ≥ 1/(m cl (bn )). To prove this it suffices to find ` such that ω(`) (bn ) P

j:3( j) (bn )⊂Using

ω( j) (bn )

≥

1 #{ j :

3( j) (b

n)

⊂ Using }

.

(30)

The natural candidate is to choose ` such that ω(`) (bn ) =

max

{ j:3( j) (bn )⊂Using }

ω( j) (bn ).

(31)

We now prove that this choice of ` satisfies (30). We write mX cl (bn ) j=1

ω( j) (bn ) =

X j:3( j) (bn )⊂Using

ω( j) (bn ) +

X

ω( j) (bn ).

j:3( j) (bn )∩Using =∅

The second term is bounded above by a constant C independent of n. The first term S ( j) tends to infinity since rj=1 0sing (b) contains a noncompact orbit. Indeed, at least one vector of the period lattice of 3( j) (bn ) must tend to infinity as n → ∞ since the limit ( j) orbit is noncompact. It follows that the set of period lattices Ib is noncompact in the manifold of lattices of full rank of Rn . Now according to Mahler’s theorem, any set 0 ⊂ Rn | ||γ || ≥ C, (γ ∈ 0 − {0}), and Vol(Rn / 0) ≤ K is compact. By the Yorke theorem in [Y], the minimal period stays bounded below, so noncompactness of the lattices forces some volume ω(`) (bn ) → ∞ as n → ∞. It follows that when a noncompact orbit exists in P −1 (bs ), then for each `, ω(`) (bn ) ω(`) (bn ) =P + o(1) as n → ∞. Pm cl (bn ) ( j) ω( j) (bn ) ( j) (b )⊂U ω (b ) j:3 n n sing j=1 Then (30) follows if we select ` as in (31). We now complete the proof of Lemma 2.11. By the finite complexity condition, we have found 3( jn ) (bn ) ⊂ Using such that c(3( jn ) (bn )) ≥ c := 1/M > 0. Further, for each n there exists a ladder Ebn which gives charge greater than or equal to c to 3( jn ) (bn ) ⊂ P −1 ∩ Using . Let d8Ebn denote the unique weak limit measure of the ladder. Then let ν denote any weak* limit of the sequence {d8Ebn }. It follows that ν S ( j) is an invariant probability measure supported on rj=1 0bs . Indeed, its support must be contained in the set of limit points of the sequence of orbits {3( jn ) (bn )} and hence in P −1 (bs ) ∩ Using . Since Q is closed in the weak* topology (since it is a set of limit points), it follows further that ν ∈ Q . Hence, there exists a ladder Ebs such that


121

S ( j) 8Ebs → ν and which charges rj=1 0bs by an amount c > 0. This proves the first part of Lemma 2.11. The second statement is an immediate consequence of PropoS ( j) ( j) sition 1.3: There must exist at least one compact singular orbit 3bs ⊂ rj=1 0bs . Since ν is an invariant probabililty measure, it must be supported on union of the compact singular orbits and hence must charge at least one such orbit. 3. Proof of Theorem 0.1 We break up the proof into three steps. Step 1 shows that the uniform boundedness assumption implies that all regular tori project without singularities to the base. Step 2 shows that there are no singular tori. Step 3 is a geometric argument showing that any completely integrable system with no singular tori and with all tori projecting regularly to the base is flat. 3.1. Step 1: Regular tori project regularly We first consider the simplest case of toric systems. 3.1.1. Toric integrable systems PROPOSITION 3.1 Suppose that (M, g) is toric integrable and that L ∞ (E, g) = O(1). Then every orbit of the torus action has a nonsingular projection to M. In particular, the orbit foliation is a nonsingular Lagrangian foliation. Proof The assumption implies that the joint eigenfunctions {φλ } of the quantum torus action have uniformly bounded sup-norms. By Proposition 2.1, for every invariant torus Tλ , there exists a ladder {kλ, k = 1, 2, . . . } of joint eigenvalues such that for all V ∈ C ∞ (M) we have Z Z 2 lim V (x) φkλ (x) dvol = V πλ∗ dµλ . k→∞ M

M

If we have ||φkλ ||∞ ≤ C for all (k, λ), then Z 2 V (x)|φ (x)| dvol ≤ C||V || L 1 kλ

(∀k)

M

and hence

Z V (x)|φkλ (x)|2 dvol ≤ C||V || L 1 . lim

k→∞

Therefore

M

Z V πλ∗ dµλ ≤ C||V || L 1 , M

(32)

122

TOTH and ZELDITCH

which implies that πλ∗ dµλ is a continuous linear functional on L 1 (M) and hence belongs to L ∞ (M). That is, we may write πλ∗ dµλ = f λ dvol, with || f λ ||∞ ≤ C. If πλ had a singular value, it is easy to check that πλ∗ dµλ would blow up there. Hence πλ is a nonsingular projection. Now we turn to the general case. 3.1.2. Rn -actions PROPOSITION 3.2 All regular tori project diffeomorphically to the base. Proof Since by Lemma 2.8 a regular torus 3(l) (b) has charge ω(l) (b) c 3(l) (b) ≥ Pm (b) > 0, cl ( j) j=1 ω (b) it follows by Corollary 2.9 that there exists a ladder of joint eigenfunctions {φb j (~) } ⊂ Eb with the property that hV φb j (~) , φb j (~) i =

mX cl (b) l=1

where

cl (Eb )

Z 3(`) (b)

V dµ3(`) (b) + o(1),

ω(l) (b) cl (Eb ) ≥ Pm (b) >0 cl ( j) (b) ω j=1

and ck (Eb ) ≥ 0 for k 6= l. Thus, we have (as in the toric case) Z cl (Eb ) V π∗ dµ3( j) (b) ≤ C ||V || L 1 , M

where we can take C = cl (Eb ) · L ∞ (~, b j (~); g, V ). Since cl (Eb ) > 0, we can cancel it to find that the torus projects regularly. As an immediate consequence of Proposition 3.2 we have the following. COROLLARY 3.3 Let {π∗ dµ3 } denote the set of projections to M of normalized Lebesgue measures on compact Lagrangian tori 3 ⊂ X E . Then, under the assumptions of Theorem 0.1, the family is uniformly bounded as linear functionals on L 1 (M).


123

3.2. Nonexistence of singular levels We have the following. LEMMA 3.4 Under the assumptions of Theorem 0.1, P has no singular levels; all orbits are Lagrangian.

Proof Existence of a compact singular orbit contradicts the uniform boundedness of eigenfunctions assumption. Indeed, it follows from Lemma 2.11 that, for any V ∈ C ∞ (M), (l) there exist a compact, singular orbit 3sing and L 2 -normalized joint eigenfunctions (l)

{φb j (~) } such that for some c(3sing ) > 0, (l) c(3sing )

Z ( j)

3sing

V π∗ dνl ≤ CkV k L 1 (M) .

(33)

However, the estimate in (33) cannot hold since by definition compact singular orbits (l) have dimension dim 3sing < n. Therefore, there cannot exist singular levels of the moment map P . 3.3. Completion of proof of Theorem 0.1 We first complete the proof of Theorem 0.1 for general metrics with quantum completely integrable Laplacians. Subsequently, we take up the case of Schrödinger operators. The first step is to consider projections of regular Lagrangian tori. By Proposition 3.2, the assumption of uniformly bounded eigenfunctions then applies to show that all Lagrangian torus orbits must project regularly to M. Furthermore, by Lemma 3.4 we know that under the finite geometric multiplicity condition (3) and uniform boundedness condition on the eigenfunctions, there do not exist any singular leaves of the moment map. Consequently, the proof of Theorem 0.1 in the case of Laplacians is a direct consequence of the following. 3.5 Suppose that the geodesic flow G t of (M, g) commutes with a Hamiltonian Rn -action. Suppose that there are no singular levels of the moment map, and suppose that each regular Lagrangian orbit Rn · (x, ξ ) has a nonsingular projection to M. Then (M, g) is a flat manifold. LEMMA

Proof We give two proofs of the lemma.

124

TOTH and ZELDITCH

First proof. The first proof uses the Mañe´ theorem (see Theorem 1.5). Since the foliation by orbits has no singular leaves, the Mañe´ theorem implies that (M, g) has no conjugate points. Since each leaf is compact, it must be a torus which covers M. Thus, there exists a cover p : T n → M. Lift the metric to p ∗ g on T n . The lifted metric must have no conjugate points since the universal covering metric is the same. By the Burago-Ivanov theorem (see Theorem 1.6), the metric is flat. In the second proof we do not use the Mañe´ theorem, and we directly relate the condition on torus projections to nonexistence of conjugate points. Second proof. As above, let π : T ∗ M − 0 → M denote the natural projection, and let π I = π|TI . Since each π I : TI → M is nonsingular, and dim TI = dim M, π I must be a covering map. 3.3.1. Case 1: M is a torus Let us first assume that M is a torus, that is, diffeomorphic to Rn /Zn ; we make no assumptions on the metric. From the fact that p I is a covering map, it follows by a result of F. Lalonde and J.-C. Sikorav [LaS] that the degree of π I : TI → M equals 1 for all I . Since π I is a diffeomorphism, there are well-defined inverse maps π I−1 : M → TI with K (I ) = 1. They define sections of π : S ∗ M → M and hence are given by graphs of 1-forms α I : M → S ∗ M. Thus, |α I (x)| ≡ 1 where | · | is the cometric. We have π I−1∗ α = α ∗I α = α I where α is the canonical 1-form. Since the tori TI are Lagrangian and since dα = ω, the 1-forms are closed; that is, dα I = 0. Now let p : M˜ → M denote the universal cover of M, and let Zn denote the deck transformation group, with generators α1 , . . . , αn . The metric g lifts to a Zn -periodic ˜ We note that the corresponding geodesic flow G˜ t is also completely metric g˜ on M. integrable. Indeed, the cover p induces the universal cover p1 : T ∗ M˜ → T ∗ M, whose deck transformation group we continue to denote by Zn . Then G˜ t commutes with the T n -action on T ∗ M − 0 generated by the lifted action integrals I˜j = p1∗ I j . The invariant tori TI therefore lift to G˜ t -invariant level sets T˜I of ( I˜1 , . . . , Iñ ). ˜ They Furthermore, the 1-forms α I lift to Zn -invariant closed 1-forms α˜I on M. ˜ are exact on M˜ and hence have the form d B I for some “potential” B I ∈ C ∞ ( M). ˜ Since |d B I | ≡ 1, we have The gradient ∇ B I is then a Zn -invariant vector field on M. |∇ B I | ≡ 1. We now claim that the integral curves of ∇ B I are lifts of geodesics on TI . To see this, we recall that the generator 4 H of the geodesic flow lies tangent to each torus TI . Hence for each I it projects from TI to a nonsingular vector field


125

π I ∗ 4 H = 4 I on M. We have

h∇ B I , 4 I i = d B I (4 I ) = α I (4 I ) = α|TI , 4 H |TI = 1 since 4 H is a contact vector field for (S ∗ M, α). Since |∇ B I | = 1, it follows that ˜ and hence the integral curves of ∇ B I = 4 I . This relation holds for the lifts to M, ∇ B I are the lifts of the geodesics on TI . We now claim that g has no conjugate points, that is, that each geodesic of g˜ on ˜ M is length minimizing between each two points on it. This follows by a well-known ˜ let v˜ ∈ Sx˜ M, ˜ and let γv˜ be the geodesic of g˜ in argument. Let x˜ be any point of M, the direction v. ˜ To see that γv˜ is length minimizing between x˜ and any other point γv˜ (to ), we project it to S ∗ M. The image lies in one of the (possibly singular) invariant tori TI , and by the above, γv˜ is an integral curve of ∇ B I . If it is not length minimizing to γv˜ (to ), then there exist so < to and a second geodesic α with α(0) = x, ˜ α(so ) = γv˜ (to ). This leads to a contradiction since Z so Z to

B I α(so ) = ∇ B I , α 0 (s) ds = ∇ B I , γv˜0 (s) ds = to > so 0

but

0

Z to =

so 0

∇ B I , α 0 (s) ds ≤ so

as |∇ B I | = 1. Therefore, (T n , g) is a torus without conjugate points. Theorem 1.0 then follows in this case from the recent proof by Burago and Ivanov [BI] of the Hopf conjecture that a metric on T n with no conjugate points is flat. 3.3.2. Case 2: The general case We now consider the general case where M is only covered by a torus T n (namely, TI for each I ). We denote by p : T n → M a fixed d-fold covering map. For notational clarity we denote the metric on M by g M . By Lemma 3.1, there is a Hamiltonian torus action on T ∗ M − 0 with the property that every orbit projects nonsingularly to M. Let gT = p ∗ g M be the metric induced on T n by the cover. We claim that gT is a flat metric. Since p : (T n , gT ) → (M, g) is a Riemannian cover, this implies that g M is a flat metric and concludes the proof. To prove gT is flat, we lift the torus foliation of T ∗ M − 0 to T ∗ T n − 0. Given a metric g on a manifold X, we denote by g˜ : T X → T ∗ X the induced bundle map g(X ˜ ) = g(X, ·). We also consider the bundle map dp : T T n → T M. Since dpx is a fiber-isomorphism for each x ∈ T n , p is a d-fold covering map. It follows that F : T ∗ (T n ) → T ∗ M,

F := g˜M dρ g˜ T−1

is also a d-fold covering map. Let T denote the foliation of T ∗ M − 0 by orbits of the torus action. We define F −1 T to be the foliation of T ∗ T n − 0 whose leaves are given

126

TOTH and ZELDITCH

by T˜I := F −1 TI where {TI } are the leaves of T . (The associated involutive distribution of the n-planes T˜x,ν ⊂ Tx,ν T ∗ T n −0 is defined by d F(T˜x,ν ) = TF(x,ν) TI (F(x,ν)) .) This foliation could also be defined as orbits of the commuting Hamiltonians F ∗ I j on T ∗ T n − 0. Each of the leaves is compact, hence a torus. We note that F : T˜I → TI is always a smooth covering map. We then have the commutative diagrams F T˜I → TI π↓ ↓π

Tn

p

→

(34)

M

We claim that the map π : T˜I → T n is nonsingular. If not, the map π ◦ F : T˜I → M would be singular. But as observed above, it is a covering map. It further follows by the result of [LaS] that π : T˜I → T n has degree 1 and hence is a diffeomorphism. We have now reduced to the previous case of the torus: the metric gT must be a flat metric, hence g M must be flat. This completes the second proof of Theorem 0.1 in the case of torus actions. 2 3.4. Proof of Theorem 0.1 for Schrödinger operators We now consider the case of semiclassical Schrödinger operators ~2 1 + V. Our proof in the homogeneous case (i.e., V = 0) was based on the use of semiclassical pseudodifferential operators, so it generalizes with little change. Proof We fix an energy level E and consider eigenvalues of ~2 1 + V lying in [E − C~1−δ , E + C~1−δ ] for some fixed C > 0. The eigenfunctions we consider are the joint eigenfunctions of P1 , . . . , Pn with joint eigenvalues (E j (~) = (1) (n) (1) b j (~), . . . , b j (~)), respectively, satisfying b j (~) ∈ [E − C~1−δ , E + C~1−δ ] for some 0 < δ < 1. We recall that b = (b(1) = E, b(2) , . . . , b(n) ) and E corresponds to the energy shell X E of the classical Hamiltonian |ξ |2g + V corresponding to the quantum Hamiltonian P1 = ~2 1 + V . By assumption, the eigenfunctions corresponding to these joint eigenvalues are uniformly bounded independently of ~ ≤ ~0 . By Proposition 3.2, it follows that all Lagrangian torus orbits of 8tE on X E project regularly to the base. Indeed, the proof that the torus 3( j) (b) projects regularly only involves trace formula and quantum limits over joint eigenvalues in the (1) (2) (n) set {(E j (~) = b j (~), b j (~), . . . , b j (~) ) : |b j (~) − b| ≤ ~1−δ }. Hence our assumption on uniform boundedness of the eigenfunctions of P1 = ~2 1 + V with eigenvalues in the interval [E − c~1−δ , E + c~1−δ ] is sufficient to obtain the result of Proposition 3.2 for the tori on the energy shell X E .


127

Hence, by a simple covering space argument, we can without loss of generality assume that the base manifold is a torus. By Lemma 3.4, there are no singular levels of the moment map P | X E . Hence, X E has a smooth Lagrangrian foliation invariant under 8tE . By Proposition 1.7, we must have that E > Vmax and the Jacobi metric (E − V )g is flat. If we additionally assume that the sup-norms are bounded independently of ~ and E in some interval [E 0 − , E 0 + ], then the Jacobi metrics (E − V )g are flat for all E in this interval, and it follows by Corollary 1.8 that g is flat and V is constant. 4. Problems and Conjectures We conclude with some problems and conjectures on integrable systems and their eigenfunctions. 4.1. Symplectic geometry of toric integrable systems Some of the ideas of this paper are relevant to purely geometric problems. 4.1 Suppose that g is a metric on Rn /Zn which is toric integrable. Then g is flat. CONJECTURE

This follows from the solution of the Hopf conjecture and from the following. 4.2 Up to symplectic equivalence, the only homogeneous Hamiltonian torus action on T ∗ (Rn /Zn ) is the standard one (8t (x, ξ ) = (x + tξ, ξ )). CONJECTURE

Indeed, the geodesic flow of (Rn /Zn , g) preserves the Lagrangian foliation defined by orbits of 8t , and hence by the Mañe´ theorem g has no conjugate points. Since the time of the original submission of this article, these conjectures have been proved by E. Lerman and N. Shirokova [LeS] (see also [L1], [L2]). 4.2. Eigenfunctions We assume throughout that the Laplacian or Schrödinger operator is quantum completely integrable. It is natural to ask if the hypothesis can be weakened to classical integrability. 4.3 Suppose that (M, g) is a compact Riemannian manifold with completely integrable geodesic flow. Suppose that 1g + V is a Schrödinger operator on (M, g), all of whose ONBE’s have uniformly bounded sup-norms. Then (M, g) is flat. CONJECTURE

128

TOTH and ZELDITCH

Without the assumption of quantum complete integrability, it is not even known whether eigenfunctions localize on level sets of the classical moment map. There are also interesting problems in the converse direction. We explain the difficulty of the next conjecture when we come to multiplicities. 4.4 Suppose that 1g + V is a Schrödinger operator on a flat manifold (M, g). Then for generic V , are the eigenfunctions uniformly bounded? CONJECTURE

We further note that all the questions about sup-norms are equally reasonable in the noncompact case. 4.3. KAM and classically nonintegrable systems We now consider the extent to which even classical complete integrability can be dropped. It is plausible that sup-norm blow-up occurs whenever there exists a stable elliptic orbit of the geodesic flow. In that case one can construct quasimodes associated to the orbit which do blow up. The relation between modes and quasimodes can be quite complicated in general, but it is plausible that there should exist a sequence of modes which also blows up. Kolomogorov-Arnold-Moser (KAM) systems always contain such stable elliptic orbits. We plan to consider these issues in a future article. 4.4. Multiplicities and sup-norms There are (well-known) relations between eigenvalue multiplicities and sup-norm blow-up of eigenfunctions. If there exists a sequence of eigenvalues of unbounded multiplicity, then there exists an ONBE with unbounded sup-norms. Indeed, for each x, consider the eigenfunction 5 E (x, ·) where 5 E is the orthogonal projection onto √ the eigenspace VE . Then 5 E (x, )˙ has L 2 -norm equal to 5 E (x, x). So the normal√ ized eigenfunction is φ Ex (·) := 5 N (x, ·)/ 5 N (x, x). It is well known and easy to see (by the Schwartz inequality) that φ Ex (·) has its maximum at x, where it equals R √ 5 N (x, x). Since M 5 N (x, x)dvol(x) = m(E) (with dvol(x) the volume form), √ there must exist x such that 5 N (x, x) ≥ m(E). Hence ||φ Ex (·)||∞ ≥ m(E). When (M, g) is a rational torus, L ∞ (λ, M, g) therefore grows at a polynomial rate while `∞ (λ, M, g) stays bounded. For instance, on a flat torus Rn /L, an ONBE of the standard Laplacian 10 is given by the exponentials eihλ,xi , with λ ∈ 3 := L ∗ , the dual lattice to L. The associated eigenvalue is E = |λ|2 , and √ its multiplicity m(E) is the number of lattice points of 3 on the sphere of radius E. Counting this number is a well-known problem in number theory when the lattice is rational. When L = Zn , for instance, the multiplicity function m(E|) has logarithmic growth for n = 2 and polynomial growth in


129

higher dimensions. Under a perturbation by a potential V , there exists a smoothly varying orthonormal basis of eigenfunctions (sometimes called the Kato-Rellich basis). It is possible that for some potential V on Rn /Zn , the Kato-Rellich basis for the perturbation 10 + V may be a smooth deformation of the eigenfunctions just described with high sup-norms. If so, it is then possible that even if the multiplicity is broken and all eigenvalues become simple, the eigenfunctions can still have unbounded sup-norms. Conjecture 4.4 states that such potentials should be sparse. It would be of some interest to understand if there exist any potentials for which sup-norm blow-up occurs. The most extreme case of multiplicity is of course that on the standard sphere (S 2 , g0 ). At the time of writing, it remains an open problem whether `∞ (λ, g) = O(1) on the standard sphere. The best result to date is the upper bound of J. VanderKam [V], that for a “random” ONBE of eigenfunctions {φλ }pthe sup-norms satisfy p ||φλ ||∞ /||φλ || L 2 = O( log λ), that is, `∞ (λ, S 2 , g0 ) = O( log λ). Our methods do not apply to this problem. 4.5. Quantitative problems Can one weaken the hypothesis of uniform boundedness of eigenfunctions in L ∞ in the rigidity results? It is plausible that our rigidity results hold as long as L ∞ (λ, M, g) lies below some threshold. One may ask the same question for the analogous L p quantities L p (λ, M, g). In [TZ] (see also [T2], [T3]), we analyse sup-norm blow-up of eigenfunctions near singular levels (among other things). We also study some cases of sup-norm blow-up near singular projections of regular levels. To obtain a threshold of some generality, one needs to estimate the minimal blow-up corresponding to the possible types of singular behaviour. Acknowledgments. We thank Bruce Kleiner for pointing out the paper [M2], Leonid Polterovich for helpful comments on [BP], and François Lalonde for helpful comments on an earlier version of the paper. We would especially like to thank the referee of this paper and Y. Colin de Verdière for pointing out that one of our original (nondegeneracy) hypotheses could be removed from the proof of Theorem 0.1 and for several other corrections and improvements. To clarify the ingredients in the proof, we cut the original manuscript (which appears on the Los Alamos National Laboratory archive as arXiv:math-ph/0002038) into two parts, the present qualitative one and the subsequent quantitative one (see [TZ]). References [AM]

R. ABRAHAM and J. E. MARSDEN, Foundations of Mechanics, 2d ed.,

Benjamin/Cummings, Reading, Mass., 1978. MR 81e:58025 103

130

[A]

[Be1] [Be2] [BP] [BKS]

[BI] [Ch]

[CV1] [CV2] [CV3] [CP]

[CK] [D] [E] [GS1] [GS2] [He] [H] [Ho]

TOTH and ZELDITCH

V. I. ARNOLD, Modes and quasimodes (in Russian), Funktsional. Anal. i Priloˇzen. 6,

no. 2 (1972), 12–20; English translation in Funct. Anal. Appl. 6 (1972), 94–101. MR 45:6331 102 M. V. BERRY, Regular and irregular semiclassical wavefunctions, J. Phys. A 10 (1977), 2083–2091. MR 58:8961 113 , Semi-classical mechanics in phase space: A study of Wigner’s function, Philos. Trans. Roy. Soc. London Ser. A 287 (1977), 237–271. MR 58:8886 113 M. BIALY and L. POLTEROVICH, Hopf-type rigidity for Newton equations, Math. Res. Lett. 2 (1995), 695–700. MR 96m:58077 107, 108, 129 P. BLEHER, D. KOSYGIN, and YA. G. SINA˘I, Distribution of energy levels of quantum free particle on the Liouville surface and trace formulae, Comm. Math. Phys. 179 (1995), 375–403. MR 96c:58173 106, 109 D. BURAGO and S. IVANOV, Riemannian tori without conjugate points are flat, Geom. Funct. Anal. 4 (1994), 259–269. MR 95h:53049 101, 107, 125 A.-M. CHARBONNEL, Comportement semi-classique du spectre conjoint d’opérateurs pseudodifférentiels qui commutent, Asympt. Anal. 1 (1988), 227–261. MR 89j:35100 117 ` , Quasi-modes sur les variétés riemanniennes, Invent. Math. Y. COLIN DE VERDIERE 43 (1977), 15–52. MR 58:18615 113, 114 , Sur le spectre des opérateurs elliptiques a` bicaractéristiques toutes périodiques, Comment. Math. Helv. 54 (1979), 508–522. MR 81a:58052 , Spectre conjoint d’opérateurs pseudo-différentiels qui commutent, II: Le cas intégrable, Math. Z. 171 (1980), 51–73. MR 81i:58046 106, 109, 111 ´ ` Y. COLIN DE VERDIERE and B. PARISSE, Equilibre instable en régime semi-classique, I: Concentration microlocale, Comm. Partial Differential Equations 19 (1994), 1535–1563. MR 96b:58112 114 C. CROKE and B. KLEINER, On tori without conjugate points, Invent. Math. 120 (1995), 241–257. MR 96j:53037 101 J. J. DUISTERMAAT, On global action-angle coordinates, Comm. Pure Appl. Math. 33 (1980), 687–706. MR 82d:58029 105 A. EINSTEIN, Zum Quantensatz von Sommerfeld und Epstein, Verh. Deutsch. Phys. Ges. 19 (1917), 82–92. 113 V. GUILLEMIN and S. STERNBERG, Geometric Asymptotics, Math. Surveys 14, Amer. Math. Soc., Providence, 1977. MR 58:24404 , Homogeneous quantization and multiplicities of group representations, J. Funct. Anal. 47 (1982), 344–380. MR 84d:58034 110, 111 G. J. HECKMAN, Quantum integrability for the Kovalevsky top, Indag. Math. (N.S.) 9 (1998), 359–365. MR 2000c:37082 107, 110 B. HELFFER, Semi-classical Analysis for the Schrödinger Operator and Applications, Lecture Notes in Math. 1336, Springer, Berlin, 1988. MR 90c:81043 ¨ L. HORMANDER , The Analysis of Linear Partial Differential Operators, IV: Fourier Integral Operators, Grundlehren. Math. Wiss. 275, Springer, Berlin, 1985. MR 87d:35002b 97


[JZ]

[K] [KMS]

[LaS]

[L1] [L2] [LeS] [M1] [M2]

[SZ] [T1] [T2] [T3] [TZ] [V]

[W] [Y] [Z]

131

D. JAKOBSON and S. ZELDITCH, “Classical limits of eigenfunctions for some

completely integrable systems” in Emerging Applications of Number Theory (Minneapolis, 1996), IMA Vol. Math. Appl. 109, Springer, New York, 1999. MR 2001g:58053 115 A. KNAUF, Closed orbits and converse KAM theory, Nonlinearity 3 (1990), 961–973. MR 91f:58075 107, 108 D. KOSYGIN, A. MINASOV, and YA. G. SINA˘I, Statistical properties of the spectra of Laplace-Beltrami operators on Liouville surfaces (in Russian), Uspekhi Mat. Nauk 48, no. 4 (1993), 3–130; English translation in Russian Math. Surveys 48, no. 4 (1993), 1–142. MR 95e:58174 106, 109 F. LALONDE and J.-C. SIKORAV, Sous-variétés lagrangiennes et lagrangiennes exactes des fibrés cotangents, Comment. Math. Helv. 66 (1991), 18–33. MR 92f:58060 124, 126 E. LERMAN, A convexity theorem for torus actions on contact manifolds, preprint, arXiv:math.SG/0012017 127 , Contact toric manifolds, preprint, arXiv:math.SG/0107201 127 E. LERMAN and N. SHIROKOVA, Toric integrable geodesic flows, preprint, arXiv:math.DG/0011139 127 ˜ E´ , Ergodic Theory and Differentiable Dynamics, Ergeb. Math. Grenzgeb. (3) 8, R. MAN Springer, Berlin, 1987. MR 88c:58040 105 , “On a theorem of Klingenberg” in Dynamical Systems and Bifurcation Theory (Rio de Janeiro, 1985), Pitman Res. Notes Math. Ser. 160, Longman Sci. Tech., Harlow, England, 1987, 319–345. MR 88k:58129 106, 129 C. D. Sogge and S. Zelditch, Riemannian manifolds with maximal eigenfunction growth, to appear in Duke Math. J. 98 J. A. TOTH, Various quantum mechanical aspects of quadratic forms, J. Funct. Anal. 130 (1995), 1–42. MR 96k:58231 109, 110 , Eigenfunction localization in the quantized rigid body, J. Differential Geom. 43 (1996), 844–858. MR 98a:58165 129 , On the quantum expected values of integrable metric forms, J. Differential Geom. 52 (1999), 327–374. MR 2001j:58048 129 J. A. TOTH and S. ZELDITCH, L p -estimates of eigenfunctions in the completely integrable case, preprint, 2000. 99, 102, 129 J. M. VANDERKAM, L ∞ norms and quantum ergodicity on the sphere, Internat. Math. Res. Notices 1997, 329–347, MR 99d:58175; Correction, Internat. Math. Res. Notices 1998, 65. MR 99d:58176 129 J. A. WOLF, Spaces of Constant Curvature, 5th ed., Publish or Perish, Houston, 1984. MR 88k:53002 100 J. A. YORKE, Periods of periodic solutions and the Lipschitz constant, Proc. Amer. Math. Soc. 22 (1969), 509–512. MR 39:7222 105, 120 S. ZELDITCH, Quantum transition amplitudes for ergodic and for completely integrable systems, J. Funct. Anal. 94 (1990), 415–436. MR 92b:58181 112

132

TOTH and ZELDITCH

Toth Department of Mathematics and Statistics, McGill University, Montreal, Canada H3A-2K6; [email protected] Zelditch Department of Mathematics, Johns Hopkins University, Baltimore, Maryland 21218, USA; [email protected]


UPPER AND LOWER BOUNDS AT s = 1 FOR CERTAIN DIRICHLET SERIES WITH EULER PRODUCT GIUSEPPE MOLTENI

Abstract in the range |s − 1| 1/ log R Estimates of the form L ( j) (s, A ) , j,DA RA A for general L-functions, where RA is a parameter related to the functional equation of L(s, A ), can be quite easily obtained if the Ramanujan hypothesis is assumed. We prove the same estimates when the L-functions have Euler product of polynomial type and the Ramanujan hypothesis is replaced by a much weaker assumption about the growth of certain elementary symmetrical functions. As a consequence, we obtain an upper bound of this type for every L(s, π), where π is an automorphic cusp form on GL(d, A K ). We employ these results to obtain Siegel-type lower bounds for twists by Dirichlet characters of the third symmetric power of a Maass form. 1. Definitions and results We consider the class of functions satisfying the following axioms. (A1) Euler product. Let A = {A p } p , with p prime, be a sequence of complex square matrices of order d, with monic characteristic polynomial Pp (x) = PpA (x) ∈ C[x] and roots α j ( p) = α A j ( p). We define the general L-function L(s, A ) as L(s, A ) =

d YY

1 − α j ( p) p −s

−1

p j=1

=

∞ X

an n −s ,

n=1

and we suppose the series absolutely convergent for σ > 1. (A2) Continuation. There exists m = m(A ) ∈ N such that (s − 1)m L(s, A ) has entire continuation over C as a function of finite order. (A3) Growth. There exists 0 < δ < 1/2 such that L(s, A ) exp exp(|t|), ∀ > 0 as |t| → ∞, uniformly in −δ < σ < 1 + δ. (A4) Functional equation. There exists a second sequence of complex square matrices A ∗ = (A p∗ ) p satisfying axioms (A1), (A2), (A3), and there exist Q A , DUKE MATHEMATICAL JOURNAL c 2002 Vol. 111, No. 1, Received 5 September 2000. Revision received 15 December 2000. 2000 Mathematics Subject Classification. Primary 11M41. 133

134

GIUSEPPE MOLTENI

λ1 , . . . , λr ∈ R+ , µ1 , . . . , µr ∈ C with 0, ∀i = 1, . . . , d2 , p ∈ PA ,

such that if we define L(s, A ⊗ A¯) =

d Y Y

1 − αi ( p)α¯ j ( p) p −s

−1

,

p i, j=1 2

P(s, A ⊗ A¯) =

d Y Y

−1 1 − γi ( p) p −s 1 − δi ( p) p −s ,

p∈PA i=1

L(s, A^ ⊗ A¯) = P(s, A ⊗ A¯)L(s, A ⊗ A¯), then L(s, A^ ⊗ A¯) satisfies axioms (A1) – (A4). By abuse of notation, we denote by ^ R ¯ the main parameter of L(s, A ⊗ A¯), and we assume that A ⊗A

X p∈PA

1 log RA ⊗A¯,

log RA ⊗A¯ log RA .

(1) (2)

It is important to remark that we do not assume anything about the size of the eigenvalues α j ; in particular, the Ramanujan hypothesis is not assumed. This is the

BOUNDS FOR DIRICHLET SERIES WITH EULER PRODUCT

135

only difference from a class of functions introduced and widely studied in E. Carletti, G. Monti Bragadin, and A. Perelli [3], so we refer to that paper for the basic properties of the functions L(s, A ) and for comments about the axioms; here we recall only that the exceptional set PA of axiom (A5) is related to the existence of ramified primes. Remark. For well-known classes of L-functions, the main parameter RA captures, although in a quite rough form, most of the algebraic information about L(s, A ): for example, for the Dirichlet L(s, χ)-functions, R q, that is, the modulus of χ ; for Hecke L-functions related to holomorphic cuspidal forms, R k N , that is, the product of the level N and the weight k; for Maass L-functions, R λN , that is, the product of the level N and the eigenvalue λ; and in general, for the L-functions associated with cuspidal automorphic representations of the groups GL(d), R is of the order of the analytic conductor introduced by H. Iwaniec and P. Sarnak in [13]. For this reason, our results are uniform in the R -aspect but depend on all other parameters appearing in axioms. It is useful, therefore, to introduce the following notation: let DA be the set DA = m A , m A ⊗A¯, rA , rA ⊗A¯, dA , λ j (A ), λ j (A ⊗ A¯) ; that is, DA contains the order of pole m A , the number of gamma factors rA , the degree dA , and the λ j coefficients both of L(s, A ) and of L(s, A ⊗ A¯). In this way we say our results are RA -independent and DA -dependent. The aim of this paper is to prove an upper bound for L(1, A ) and, more generally, for L ( j) (s, A ) of type L ( j) (s, A ) , j,DA RA

for |s − 1| 1/ log RA .

(3)

Such estimates are an essential tool for a general Siegel-type estimate, as we see in Section 2. Obtaining these estimates is quite easy when the Ramanujan hypothesis is assumed (see [3]). However, this hypothesis has been proved only for a limited class of functions (the Hecke L-functions, the Artin L-functions, and the L-functions coming from the cuspidal holomorphic forms for congruence groups; see P. Deligne [4]), although it is generally believed that all the L-functions appearing in number theory should satisfy the Ramanujan hypothesis. For example, it is conjectured to hold for the L-functions associated with cuspidal automorphic representations. In general, only partial and rather poor estimates for the coefficients are at our disposal; hence it is interesting to consider the possibility of obtaining (3) without the Ramanujan hypothesis. For Maass forms (d = 2), this was done first by Iwaniec [12] (in the Q-aspect, P in that paper). He remarks that a preliminary estimate for S(x) = n≤x |an |/n of the

136

GIUSEPPE MOLTENI

form S(x) Q c x for some constant c and for every > 0 can be proved in a standard way. Then the multiplicative properties of the coefficients of these functions can be employed to obtain an estimate for S(x)2 in terms of S(x 2 ). By iterating this relation, the fundamental estimate S(x) (Qx) is deduced, and the claim easily follows. This method has also been used by J. Hoffstein and P. Lockhart [10] to get an analogous estimate in the case of the symmetric square L-functions (d = 3) associated with Maass forms. We modify Iwaniec’s idea in such a way as to obtain the required estimate for functions of any degree d; in this sense the most original part of our work is Section 2, where the basic Proposition 1 is proved. We introduce the following notation: s j ( p) denotes the jth elementary symmetric function of the roots α1 ( p), . . . , αd ( p); that is, X αi1 ( p) · · · αi j ( p). s j ( p) = 1≤i 1 0 such that |α j ( p)| ≤ p 1−ρ for every prime p and the estimate HYPOTHESIS

s j ( p) p j/2 ,

∀ 2 ≤ j ≤ d, uniformly over R ,

holds. Remark. The first part of the hypothesis means that the local components are convergent when σ > 1 − ρ. Moreover, the estimate on the elementary symmetric functions


137

is satisfied in an obvious way when the estimate an n 1/2 holds, uniformly on R . Nevertheless, Hypothesis (R) does not assume anything about the first symmetric function s1 ( p); in this way Hypothesis (R) is satisfied as well in some cases where the global estimate an n 1/2 is not known. It is also interesting to remark that the quality of the estimate assumed in (R) does not depend on the degree d, as a consideration of similar situations might suggest. Our main result is the following. THEOREM 1 Let L(s, A ) satisfy axioms (A1) – (A5) and Hypothesis (R), and define f (s) = (s − 1)m A L(s, A ). Then

f ( j) (s) ,D ( j + 1)c j R log j R +

j! 2jR

for |s − 1| 1/ log R ,

(5)

uniformly on j, for a suitable positive constant c = c(D ), independent of R . Remark. Under the same hypotheses and with some minor change to the proof of this theorem, we can obtain upper bounds of similar type for the values of L(s, A ) when s is centered at a different point of the line σ = 1; for example, for θ 6= 0 we have L(s + iθ, A ) ,D R (1 + |θ|) for |s − 1| 1/ log R (1 + |θ|) . (6) It is interesting to remark that this more general result is already included in Theorem 1 when L(s, A ) is entire. In fact, in this case the shifted L-function L(s, A (θ )) = L(s +iθ, A ) satisfies axioms (A1) – (A5) and (R) with log RA (θ) DA log RA + log(1 + |θ |), so that (5) for L(s, A (θ)) gives (6) for L(s, A ). For the applications it is important to know estimate (3) for the generic tensor product function dA Y dB YY −1 L(s, A ⊗ B ) = 1 − αi ( p)β j ( p) p −s p i=1 j=1

as well; to deal with this function, axiom (A5) has to be modified in the following way. (A50 ) There exists a finite, possibly empty set of primes PA ,B and complex numbers γi ( p), δi ( p) ∈ C for i = 1, . . . , dA dB , p ∈ PA ,B , with γi ( p) , δi ( p) ≤ p 1−ρ for some ρ > 0, ∀i = 1, . . . , dA dB , p ∈ PA ,B ,

138

GIUSEPPE MOLTENI

such that if we define P(s, A ⊗ B ) =

Y

dA dB Y

p∈PA ,B

i=1

−1 1 − γi ( p) p −s 1 − δi ( p) p −s ,

L(s, A^ ⊗ B ) = P(s, A ⊗ B )L(s, A ⊗ B ), then L(s, A^ ⊗ B ) satisfies axioms (A1) – (A4). As before, by abuse of notation we denote by RA ⊗B the main parameter of L(s, A^ ⊗ B ), and we assume that X 1 log RA ⊗B , (7) p∈PA ,B

log RA ⊗B log RA + log RB .

(8)

Remark. Under hypotheses (7) and (8), the estimate (RA RB )− P(1, A ⊗ B ) (RA RB ) holds, so that any upper bound of type (3) for L(s, A^ ⊗ B ) gives a similar upper bound for L(s, A ⊗ B ), and vice versa. Moreover, in the context of L-functions from representations of GL(d), L(s, A^ ⊗ B ) is the Rankin-Selberg convolution and estimates (1), (2), (7), and (8) about the ramified primes are satisfied. An upper bound for L(s, A^ ⊗ B ) can be deduced by Theorem 1, but in this case we have to assume axiom (A5) for L(s, A^ ⊗ B ), that is, the validity of (A1) – (A4) ¯ ¯ for L(s, A ⊗ A ⊗ B ⊗ B ). This fact agrees with well-known conjectures, but it is not proved in general; hence it is important for applications to obtain an upper bound for L(s, A ⊗ B ) under a different set of hypotheses. Theorems 2 and 3 achieve this purpose. 2 Assume that both L(s, A ) and L(s, B ) satisfy axioms (A1) – (A5) and (A50 ). Moreover, assume that both L(s, A ⊗ A¯) and L(s, B ⊗ B¯ ) satisfy Hypothesis (R) and that QdA Q B | i=1 αi ( p)|, | dj=1 β j ( p)| ≤ 1 for every p. Define f (s) = (s − 1)m A ⊗B L(s, A ⊗ B ). Then THEOREM

f ( j) (s) ,DA ,DB ( j + 1)c j (RA RB ) log j (RA RB ) +

j! 2 j RA

RB

for |s − 1| 1/ log(RA RB ), uniformly on j, for a suitable positive constant c = c(DA , DB ), independent of RA , RB .


139

Nevertheless, there are cases where (R) is not proved for L(s, A ⊗ A¯) but the following stronger hypothesis about L(s, A ) holds. (R0 ) There exists ρ > 0 such that |α j ( p)| ≤ p 1−ρ for every prime p and the estimate HYPOTHESIS

s j ( p) p j/4 ,

∀ 2 ≤ j ≤ d, uniformly over R ,

holds. We can prove upper bounds for L(s, A ⊗ B ) under this hypothesis as well. 3 Assume that both L(s, A ) and L(s, B ) satisfy axioms (A1) – (A5) and (A50 ). Moreover, assume that both L(s, A ) and L(s, B ) satisfy Hypothesis (R0 ) and that QdA Q B | i=1 αi ( p)|, | dj=1 β j ( p)| ≤ 1 for every p. Then, defining f (s) = (s − 1)m A ⊗B ·L(s, A ⊗ B ), we have THEOREM

f ( j) (s) ,DA ,DB ( j + 1)c j (RA RB ) log j (RA RB ) + for |s − 1| 1/ log(RA RB ),

j! 2 j RA

RB

(9)

uniformly on j, for a suitable positive constant c = c(DA , DB ), independent of RA , RB . According to Iwaniec’s original idea, Theorems 1 – 3 are deduced in a standard way from suitable upper bounds for the coefficients of L(s, A ) and L(s, A ⊗ B ); we state here explicitly the upper bound giving Theorem 1 since it has an interest of its own; the similar upper bounds for the coefficients of L(s, A ⊗ B ) under the hypotheses of Theorems 2 and 3 are contained in their proofs. THEOREM 4 Let L(s, A ) satisfy axioms (A1) – (A5) and Hypothesis (R). Then

X |an | n≤x

n

,D (R x) ,

∀ > 0.

(10)

Examples Let α( p) and α −1 ( p) be the coefficients of the Euler product of the L-function L(s, f ) associated with a Maass form f . Then d = 2 and s2 ( p) = 1, so (R) is satisfied for L(s, f ); hence for this function estimate (5) holds, as already proved by Iwaniec [12].

140

GIUSEPPE MOLTENI

Moreover, H. Kim and F. Shahidi [15] proved that p −5/34 < |α( p)| < p 5/34 . The square-symmetric L-function L(s, sym2 f ) associated with f satisfies axioms (A1)– (A5) by the works of G. Shimura [20], S. Gelbart and H. Jacquet [5], and C. Moeglin and J. Waldspurger [17]. It has an Euler product of degree 3, and it has α 2 ( p), 1, and α −2 ( p) as coefficients; thus s3 ( p) = 1 and |s2 ( p)| = |α 2 ( p)+1+α −2 ( p)| ≤ 3 p 5/17 . Hence (R) is satisfied and the estimate L(1, sym2 f ) R holds by Theorem 1. This has been proved by Hoffstein and Lockhart [10]. It is easy to verify that the estimate of Kim and Shahidi for the coefficients of a Maass form is sufficiently strong to prove that L(s, sym2 f ⊗ sym2 f ) satisfies (R), so by Theorem 2 it follows that ress=1 L(s, sym2 f ⊗ sym2 f ) R , again a result proved in [10]. (In that paper the upper bound for ress=1 L(s, sym2 f ⊗ sym2 f ) was not deduced by Theorem 2 but by a specific version of Theorem 3 because only the weaker estimate p −1/5 < α( p) < p 1/5 was known at that time.) Finally, Maass functions are examples of Langlands L-functions, a general and extremely important class of functions. This theory associates an L-function, L(s, π), to every automorphic cuspidal representation π of GL(d, A K ), where A K is the Adèle ring of a global field K (see [6]). These functions satisfy axioms (A1) – (A5) by the work of many authors (see [7], [19], and [17]); by W. Luo, Z. Rudnick, and Sarnak [16], the local coefficients of these functions satisfy α j ( p) < p 1/2−1/(d2 +1) for any j = 1, . . . , d; hence all the hypotheses of Theorem 1 are satisfied (but it is interesting to remark that the old result of Jacquet and J. Shalika [14] asserting the bound |α j ( p)| < p 1/2 is sufficient for this purpose), so the next corollary immediately follows. COROLLARY

Let π be an automorphic cuspidal representation of GL(d, A K ), and let L(s, π ) be the associate L-function. Let f (s) = (s − 1)m L(s, π), where m is the order of the pole of L(s, π) at s = 1; then ( j) f (s) ≤ c R for |s − 1| 1/ log R , for a suitable c = c(, j, m, d) > 0. (For the sake of simplicity, we ignore here uniformity on the order j.) An unconditional and general statement corresponding to Theorem 2 or 3 is not possible for the Rankin-Selberg convolution L(s, π ⊗π 0 ) since it is not known if this function satisfies axiom (A5) or if L(s, π) satisfies Hypothesis (R0 ).


141

Siegel-type lower bounds Upper bounds of Theorems 1–3 are necessary ingredients for the proof of Siegel-type theorems, that is, for the proof of lower bounds of the form L(1, A ) ,D R − ,

∀ > 0;

(11)

the importance of such a result justifies the previous researches about lower bounds for L-functions whose coefficients are not known to satisfy Ramanujan hypothesis. A careful analysis of Siegel-type lower bounds is carried out in [18], where the results of E. Golubeva and O. Fomenko [8], [9] on a lower bound for holomorphic cusp forms, the well-known estimate (11) for symmetric square power of a Maass form proved by Hoffstein and Lockhart [10], and similar results are deduced as consequences of a coherent and axiomatic approach. In [18] new results are deduced as well; in the following we show how some of these results, that is, estimates (12) – (14), can be proved. Let f ∈ S0 (00 (N ), ψ); that is, let f be a Maass cusp form for the Hecke congruence subgroup of level N with the real character ψ modulus N as multiplier. As we have shown in previous examples, both L(s, f ) and L(s, sym2 f ) satisfy (R0 ). Moreover, let χ and χ 0 be different real primitive and nonprincipal characters modulo q and q 0 , respectively. It is known that L χ (s, f ) = L(s, χ ⊗ f ), L χ (s, sym2 f ) = L(s, χ ⊗ sym2 f ), and L χ (s, f ⊗ sym2 f ) = L(s, f χ ⊗ sym2 f ) are entire functions satisfying axioms (A1) – (A5) (by [17]) and that log Rχ ⊗ f log R f + log q; the same is true for Rχ⊗sym2

f

and R fχ ⊗sym2 f . We consider

F(s) = L s, (1 + χ ) ⊗ (1 + χ 0 ) ⊗ ( f + sym2 f ) ⊗ ( f + sym2 f ) = ζ (s)L(s, sym2 f )L(s, sym2 f ⊗ sym2 f )L 2 (s, f ⊗ sym2 f ) · L(s, χ )L χ (s, sym2 f )L χ (s, sym2 f ⊗ sym2 f )L 2χ (s, f ⊗ sym2 f ) · L(s, χ 0 )L χ 0 (s, sym2 f )L χ 0 (s, sym2 f ⊗ sym2 f )L 2χ 0 (s, f ⊗ sym2 f ) · L(s, χ χ 0 )L χ χ 0 (s, sym2 f )L χ χ 0 (s, sym2 f ⊗ sym2 f )L 2χχ 0 (s, f ⊗ sym2 f ). This function has a representation as a Dirichlet series with positive coefficients since it can be verified by considering its logarithmic derivative. It has a double pole at s = 1 and is divisible by L χ (s, f ⊗sym2 f )L χ 0 (s, f ⊗sym2 f ) with multiplicity two. By Theorems 1–3, upper bounds of the form f, (qq 0 ) follow for the derivatives of every order of any function appearing into F(s). Therefore, by the standard approach to Siegel-type theorems (see [8], for example), we prove that L χ (1, f ⊗ sym2 f ) f, q − ,

∀ > 0.

142

GIUSEPPE MOLTENI

Since L χ (s, f ⊗ sym2 f ) = L(s, f )L χ (s, sym3 f ), where L(s, sym3 f ) is the Lfunction investigated by Kim and Shahidi [15], we deduce that L χ (1, sym3 f ) f, q − ,

∀ > 0.

(12)

L χ (1, f ⊗ sym2 g) f,g, q − ,

(13)

L χ (1, sym2 f ⊗ sym2 g) f,g, q − ,

(14)

In a similar way we can prove that

for f and g both Maass forms, f 6= g. We recall that βA , the largest real zero of L(s, A ), is called the Siegel zero of L(s, A ) if 1 − βA 1/ log RA and that (11) is equivalent to − 1 − βA ,DA RA .

W. Banks [1] and Hoffstein and D. Ramakrishnan [11] have proved that L-functions related to automorphic cuspidal representations π of GL(2) and GL(3) have no Siegel zero, so that for these functions the stronger estimate L(1, π )

1 log R

holds. However, estimates (12)–(14), involving functions conjecturally related to automorphic representations of GL(4), GL(6), and GL(9), are nontrivial.

2. Algebraic relations 2.1. Local relations Let {bh }∞ h=0 be a sequence with b0 = 1, extended to h < 0 by setting bh = 0 in this range. We assume that the sequence bh verifies the relation b1 bh = bh+1 +

d X

xuu bh+1−u ,

(15)

u=2

where d ≥ 1 is a fixed integer, the degree of the sequence, and {xu }du=2 are nonnegative real parameters. Remark. When d = 1, the relation is reduced to b1 bh = bh+1 . With these assumptions we prove the following.


143

PROPOSITION 2 There exist polynomials Pu,l ∈ N[x2 , · · · , xd ] such that

bl bh =

dl X

Pu,l (x) bh+l−u ,

∀l, h ≥ 0.

(16)

u=0

Moreover, if we assume P0,0 = 1 and Pu,l = 0 in the cases when u < 0, l < 0, u > dl, or u = 1, such polynomials satisfy the recursion Pu,l+1 (x) = Pu,l (x) +

d X

g x g Pu−g,l (x) − Pu−g,l+1−g (x) .

(17)

g=2

Proof We introduce the variables x0 and x1 since in this way we can write b1 bh = Pd u u=0 x u bh+1−u and recover relation (15) by setting x 1 = 0. We prove the claim recursively on l. By (15), it is true when l = 1, for every h. Assume now the claim for the values less than or equal to l and for every h; we verify it for l + 1, for every h again. We have b1 (bl bh ) =

dl X

Pu,l (x) b1 bh+l−u =

u=0

=

dl X

Pu,l (x)

u=0

dl X d X

g

d(l+1) X γ =0

u=0 g=0 d(l+1) X γ =1

g

x g bh+l+1−u−g

g=0

x g Pu,l (x) bh+l+1−u−g =

= bh+l+1 +

d X

X

g x g Pu,l (x) bh+l+1−γ

g+u=γ 0≤g≤d

g x g Pu,l (x) bh+l+1−γ ;

X g+u=γ 0≤g≤d

moreover, (b1 bl )bh =

d X

xuu bl+1−u bh = bl+1 bh +

u=0

= bl+1 bh +

xuu bl+1−u bh

u=1 d d(l+1−u) X X u=1

= bl+1 bh +

d X

dl X γ =1

xuu Pρ,l+1−u (x) bh+l+1−ρ−u

ρ=0

X u+ρ=γ 1≤u≤d 0≤ρ≤d(l+1−u)

xuu Pρ,l+1−u (x) bh+l+1−γ .

144

GIUSEPPE MOLTENI

Comparing these identities, we get bl+1 bh = bh+l+1 +

d(l+1) X γ =1

·

X

g

x g Pu,l (x) −

g+u=γ 0≤g≤d

h

i g x g Pu,l+1−g (x) δγ ≤dl bh+l+1−γ ,

X

g+u=γ 1≤g≤d 0≤u≤d(l+1−g)

(18) where δγ ≤dl is 1 or 0 according to whether or not γ ≤ dl. The existence of the polynomials Pu,l+1 follows from (18), completing the proof of (16). Moreover, h i X g X g Pγ ,l+1 (x) = x g Pu,l (x) − x g Pu,l+1−g (x) δγ ≤dl g+u=γ 0≤g≤d

= Pγ ,l (x) +

g+u=γ 1≤g≤d 0≤u≤d(l+1−g)

X g+u=γ 2≤g≤d

g

x g Pu,l (x) −

h

X

i g x g Pu,l+1−g (x) δγ ≤dl ,

g+u=γ 2≤g≤d 0≤u≤d(l+1−g)

(19) where the assumption x1 = 0 has been introduced. Since Pu,l = 0 for u < 0, d < 0, or u > dl, (19) implies (17). The particular form of (17) suggests the analysis of the polynomials Dl,h = Pu,l − Pu,l−1 ; we get the following proposition. 3 The polynomial sequence Du,l = Pu,l − Pu,l−1 verifies the recursion  Pd g Pg−1    Du,l+1 (x) = g=2 x g ρ=1 Du−g,l+1−ρ (x), D0,0 = 1,    D = 0 when u < 0, u > dl, or l < 0. PROPOSITION

(20)

u,l

Moreover, Du,l ∈ N[x2 , . . . , xd ] and it is homogeneous of degree u; that is, Du,l (λx) = λu Du,l (x) for every λ and x. Proof The recursive relation (20) is deduced from (17). The other claim is an easy consequence of (20).


145

We need another property of the sequence Du,l . 4 If l ≥ u + d, then Du,l = 0. PROPOSITION

Proof We prove the claim recursively on u; it is true when u = 0 or u = 1, for every l > 0. We suppose the claim to hold up for u, and we prove it for u + 1. Let l ≥ u + 1 + d. By the identity g−1 d X X g Du+1,l = xg Du+1−g,l−ρ , g=2

ρ=1

the claim is the same as proving that every Du+1−g,l−ρ is zero. The inductive hypothesis assures it if l − ρ ≥ u + 1 − g + d, and this inequality holds since we assumed l ≥ u + 1 + d and ρ < g. Remark. Let l0 (u) be the smallest value such that Du,l = 0 when l ≥ l0 (u). Proposition 4 states that l0 (u) ≤ u + d, but it is easy to verify that the inequality is not sharp. Nevertheless, the exact value of l0 (u) is not necessary here. Every Du,l (x) ∈ N[x]; therefore Pu,l1 (x) ≤ Pu,l2 (x) when l1 ≤ l2 (recall that we asP sume xi ∈ R+ ) since Pu,l = lg=0 Du,g ; nevertheless, the sequence Pu,l is bounded on l since by Proposition 4 the following series is finite: Pu (x) =

∞ X

Du,g (x).

g=0

Thus Pu,l (x) ≤ Pu (x),

∀u, l, ∀ xi ∈ R+ .

(21)

The existence of the polynomials Pu giving an upper bound for Pu,l , uniform on l, is an essential fact; we summarize their principal properties in the following proposition. PROPOSITION 5 The sequence Pu (x) satisfies the recurrence  Pd g   Pu (x) = g=2 (g − 1)x g Pu−g (x) Pu = 0   P = 1, P = 0. 0

1

if u ≥ 2, if u < 0,

(22)

146

GIUSEPPE MOLTENI

Moreover, Pu belongs to N[x2 , . . . , xd ], is homogeneous with degree u, and satisfies the estimate √ u Pu (x) ≤ 2(x2 + · · · + xd ) . (23) Proof P∞ By definition, Pu = l=0 Du,l ; therefore Pu belongs to N[x] and is homogeneous with degree u because the same properties hold for every Du,l . The recursive relation (22) is recovered from (20). In order to prove (23), we first remark that if set f M (y) =

M h X

yi PM

j=1

i=1

ii yj

,

then f M (y) ≤ 1 when yi ∈ R+ for every i; we prove this fact inductively over M. It is true for M = 1. By the inductive hypothesis f M−1 (y1 , . . . , y M−1 ) ≤ 1, we get y1 1 ≥ f M−1 (y1 + y M , y2 , . . . , y M−1 ) = P M

j=1

y1 ≥ PM

j=1

yj

h y M + PM

iM

j=1

+

yj

M−1 Xh

yi PM

j=1

i=2

yj

yM + PM

j=1

ii yj

+ yj

M−1 Xh i=2

yi PM

j=1

ii yj

= f M (y1 , y2 , . . . , y M ).

Now we can prove (23), once again by induction. It is true when u = 0, and it is trivial when u < 0. By the recursive law (22), we have Pu (x) =

d X

(g

g − 1)x g

g=2

≤

d hX g=2

d g (g − 1)x g i √ X u Pu−g (x) ≤ ( 2 xj) √ Pd g j=2 x j ) g=2 ( 2 j=2

g

xg

P ( dj=2 x j )g

d hX

d d i√ X √ X ( 2 x j )u ≤ ( 2 x j )u , j=2

j=2

where we have used the inequality g−1 ≤ 2g/2 and the previous estimate f M (y) ≤ 1.

Remark. Since every Du,l is homogeneous, Pu,l is homogeneous too (as directly verified by (17)). This immediately implies that there exists a constant c = c(l) such that u Pu,l (x) ≤ c(l)(x2 + · · · + xd ) , for every u. Obtaining (23) is important since it shows that c(l) is actually independent of l. 2.2. Euler product: Proof of Proposition 1 Now we use the results of the previous section for the study of the coefficients of an Euler product. Let {α j }dj=1 be complex numbers, and let {bh }∞ h=0 be the sequence


147

generated by the relation ∞ X

h

bh x =

h=0

d Y

(1 − α j x)−1 .

j=1

Let {s j }dj=1 be the elementary symmetric polynomials of the variables α j ; then the sequence bh satisfies the recursion ( bh − s1 bh−1 + s2 bh−2 + · · · + (−1)d sd bh−d = 0, ∀h > 0, b0 = 1. But s1 = b1 , so the recursive relation can be formulated as b1 bh−1 = bh + s2 bh−2 + · · · + (−1)d sd bh−d . Comparing this relation with (15), by (16), (21), and (23) we get |bl bh | ≤ |bl+h | +

d min {l,h} X

d √ X R |bl+h−u | with R = 2 |s j |1/j . u

u=2

j=2

Proposition 1 follows from this estimate by multiplicativity. 2.3. A useful proposition For the proof of the theorems we need the following proposition establishing an inequality between the coefficients of the series L(s, A ) and those of the convolution series L(s, A ⊗ A¯). PROPOSITION

Let L(s, A ) = for every n.

6 P∞

n=1 an n

−s , and let

L(s, A ⊗ A¯) =

P∞

n=1

An n −s . Then An ≥ |an |2

P 2 −s holds when the degree is 2 by The identity L(s, A ⊗ A¯) = ζ (2s) ∞ n=1 |an | n a formula of Ramanujan, and in this case the statement easily follows by this fact because ζ (2s) is a Dirichlet series with nonnegative coefficients. For larger degrees, P 2 −s again holds for a suitable Dirichlet the relation L(s, A ⊗ A¯) = F(s) ∞ n=1 |an | n series F(s), but in this case the coefficients of F(s) can be negative, so that the general inequality of Proposition 6 cannot be proved in this way. Proof Both L(s, A ) and L(s, A ⊗ A¯) have a representation as an Euler product; thus it is

148

GIUSEPPE MOLTENI

sufficient to prove the claim for the local components, that is, to verify that if f (x) =

d Y

(1 − α j x)−1 =

j=1

and

d Y

F(x) =

∞ X h=0

(1 − αi α¯ j x)

−1

=

i, j=1

then Ah ≥ |ah

|2 .

∞ X

Ah x h ,

h=0

We remark that

∞ X

d

hah x h = x

h=0

X αjx d f (x) = f (x) dx 1 − αjx j=1

= f (x)

d X ∞ X

α hj x h

=

j=1 h=1

where we have set τh = ∞ X

ah x h

Pd

h Ah x h = x

h=0

h j=1 α j ;

∞ X

h

ah x ·

h=0

∞ X

τh x h ,

h=1

in a similar way, for F(x) we have

d X αi α¯ j x d F(x) = F(x) dx 1 − αi α¯ j x i, j=1

= F(x)

d X ∞ X

αih α¯ hj x h

i, j=1 h=1

Therefore, the following recursions hold: ( Ph hah = l=1 τl ah−l , ∀h > 0, a0 = 1,

=

∞ X

h

Ah x ·

h=0

(

h Ah =

∞ X

|τh |2 x h .

h=1

Ph

2 l=1 |τl | A h−l ,

∀h > 0,

A0 = 1.

(24) These relations prove that for every h ≥ 0 there exists a polynomial Ph ∈ N[x1 , . . . , x h ] such that h! ah = Ph (τ1 , . . . , τh ), h! Ah = Ph |τ1 |2 , . . . , |τh |2 ; therefore, proving Ah ≥ |ah |2 is the same as showing that 2 h! Ph |τ1 |2 , . . . , |τh |2 ≥ Ph (τ1 , . . . , τh ) .

(25)

Finally, we observe that Ph (1, . . . , 1) = h!,

(26)


149

since if we set τl = 1 for every l, then the recursion (24) is solved by Ah = 1 for every h. We know that Ph ∈ N[x1 , . . . , x h ]; hence by (26) we can consider Ph as a sum of h! monomials, every one with unitary coefficient, so that (25) follows from the Ph! Ph! Cauchy-Schwarz inequality h! i=1 |yi |2 ≥ | i=1 yi |2 . Remark. Proposition 6 also follows from Littlewood’s lemma quoted in Section 3.3, but we believe our proof is conceptually easier.

3. Proof of theorems 3.1. Proof of Theorem 4 Let S be a finite set of primes, independent of R , which we choose later on. We follow the notation of the axioms, and we consider the quantity SS (x) =

X |an | . n n≤x

(n,S)=1

Step 1. We prove that there exists a constant c1 = c1 (D ) > 0, independent of R , such that SS (x) ,D R c1 x , ∀ > 0. (27) By the Stirling asymptotic formula, when λ > 0, 0. By axiom (A5) a similar estimate holds for L(s, A^ ⊗ A¯), and by assumptions (1) and (2) about the exceptional set PA , the same estimate holds for L(s, A ⊗ A¯); that is, c (s − 1)m A ⊗A¯ L(s, A ⊗ A¯) D RA (1 + |t|) 1 .

(29)

150

GIUSEPPE MOLTENI

Therefore, Z X An X An n r r 2r r ! +i∞ L(s + 1, A ⊗ A¯)(2x)s ≤ 1− 2 = ds, n n 2x 2πi −i∞ s(s + 1) · · · (s + r ) n≤x n≤2x

where r = r (c1 ) is a large parameter assuring the convergence of the integral; by (29) we get X An ,D R c1 x , n n≤x

∀ > 0, for some c1 = c1 (D ) > 0.

By Proposition 6 we have |an | ≤ 1 + |an |2 ≤ 1 + An , so that SS (x) ≤ An )/n, and (27) immediately follows from (30).

(30) n≤x (1

P

+

Step 2. Iwaniec’s work suggests that we proceed by showing next that SS2 (x) ≤ c2 x SS (x 2 ) for some c2 = c2 (, D ) > 0.

(31)

By Proposition 1 we have SS2 (x) =

X n,m≤x (nm,S)=1

≤

X n,m≤x (nm,S)=1

≤

|an am | nm 1 mn

X∗

R(d)|anm/d | =

X

X∗ R(d) |anm/d | d nm/d d

n,m≤x d|(n,m) (nm,S)=1 d|nm

d|(n,m)d d|nm

X∗ R(d) X |a | A max # (n, m) : n, m < x, nm = AD , 2 d A A,D≤x 2 2 d≤x (d,S)=1

A≤x (A,S)=1

so that SS2 (x) log x

X∗ R(d) SS (x 2 ). d 2

(32)

d≤x (d,S)=1

Now we estimate the sum in (32). Recalling that R is a completely multiplicative P function and that ∗ denotes a sum over the square-full divisors only, we get ∞ Y X∗ R(d) Y R 2 ( p) R 3 ( p) R 2 ( p) X R( p) j ≤ 1+ + +· · · ≤ 1+ . d p p2 p3 p2 2 2 2

d≤x (d,S)=1

p≤x p6 ∈S

p≤x p6∈S

j=0

By Hypothesis (R) and the definition of R( p), there exists c3 dependent on D but independent of R such that R( p) ≤ c3 p 1/2 for every prime. Hence, the series in the


151

infinite product is always convergent if a convenient set S is assumed, for example, S = { p : p ≤ c32 + 1}. With this choice of S the estimate becomes X∗ R(d) Y Y c6 c5 R 2 ( p) D ≤ 1 + D logc6 x, 1+ d p p2 2 2 2

d≤x (d,S)=1

p≤x

p≤x p6 ∈S

and (31) is proved in the stronger form SS2 (x) D logc6 +1 x SS (x 2 ). Step 3. By iterating (31) and using (27), we obtain, for every M > 0, M

SS2 (x) ≤ c M x 2

M

M

SS (x 2 ) ≤ c R c1 x 2

M

for some c = c(, D , M) > 0;

hence, taking the 2 M th root with a suitable M = M(c1 , ) = M(, D ), we get SS (x) ,D (R x) .

(33)

P h Since the local factors are convergent for σ > 1 − ρ by (R), the series ∞ h=0 |a p h |/ p converges for every p and is bounded by a constant independent of R . Hence, from (33) we have X |an | n≤x

n

≤

∞ X |a | Y X n |a ph |/ p h SS (x) ,D (R x) . n n≤x p∈S h=0

(n,S)=1

3.2. Proof of Theorem 1 Theorem 1 is deduced from Theorem 4 in the following way. From (33) we get X n≤x

log j n

X |an | |an | ≤ log j x ,D (R x) log j x, n n n≤x

uniformly on j. Let I ( j) (x) be defined by I ( j) (x) =

1 2πi

Z

2+i∞

2−i∞

f ( j) (s + 1)x s ds s(s + 1) · · · (s + r )

(34)

with r = r (D ), independent of j and sufficiently large to assure the convergence of the integral. We remark that P j,m (s, log n) d j sm = j s ds n ns

152

GIUSEPPE MOLTENI

with P j,m (x, y) a polynomial bounded by m j+1 ( j + 1)x m y j , uniformly on m and j. Moreover, 1 2πi

Z

2+i∞ 2−i∞

sm x s ds s(s + 1) · · · (s + r ) n  0 m m r =  r 1 + n r ≤ r 2 , uniformly on r , r! x r!

if n > x, if n ≤ x;

hence, I ( j) (x) m A ( j + 1) j+1

r m A 2r X |an | log j n ,D ( j + 1)c j (R x) log j x, r ! n≤x n

uniformly on j, for some c = c(D ) > 0. Moreover, by (28) and the Cauchy theorem about the derivative of a holomorphic function, we get ( j) c f ( j) (s) = (s − 1)m A L(s, A ) D j! 2− j R (1 + |t|) 1 , uniformly on j, with a < σ < b. We move the integration line in (34) to σ = −1/2, so that by the residue theorem we obtain j!R c1 I ( j) (x) = f ( j) (1)/r ! + O j √ ; 2 x choosing x = R 2c1 +2 , we get the estimate f ( j) (1) ,D ( j + 1)c j R log j R +

j! 2jR

,

uniformly on j,

with some c = c(D ) > 0. Finally, the power series expansion f ( j) (s) =

∞ X f ( j+u) (1) (s − 1)u u! u=0

gives (5) in the range |s − 1| 1/ log R . 3.3. Proofs of Theorems 2 and 3 The following algebraic lemma (see D. Bump [2, Section 2.2]) is necessary for the proofs. LEMMA (E. D. Littlewood) Qd 0 d Let {αi }i=1 and {β j }dj=1 be nonzero complex numbers, and define A = i=1 αi ,


B=

Qd0

j=1 β j .

d Y d Y

153

Then

(1 − αi β j x)−1

i=1 j=1 ∞ X

= (1 − ABx d )−1

Sk1 ,...,kd−1 (α)Sk1 ,...,kd−1 (β)x k1 +2k2 +3k3 +···+(d−1)kd−1

k1 ,...,kd−1 =0

if d = d0 , and 0

d Y d Y

(1 − αi β j x)−1

i=1 j=1

=

∞ X

Sk1 ,...,kd−1 (α)Sk1 ,...,kd ,1,...,1 (β)Akd x k1 +2k2 +3k3 +···+dkd

k1 ,...,kd =0

if d < d0 , with d−1+kd−1 +···+k1 α 1 d−2+kd−1 +···+k2 α1 .. Sk1 ,...,kd−1 (α) = . 1+kd−1 α 1 1

... ... .. . ... ...

d−1+kd−1 +···+k1 d−1 αd α1 d−2+kd−1 +···+k2 d−2 αd α1 . .. / . . . 1+kd−1 α1 αd 1 1

αdd−1 αdd−2 .. . . αd 1

... ... .. . ... ...

The polynomials Sk1 ,...,kd−1 (α) are called Schur’s polynomials. Let A(n) and B(n) be the totally multiplicative functions defined by A( p) = QdA QdB i=1 αi ( p) and B( p) = j=1 β j ( p). By Littlewood’s lemma we have Y p

∞ A( p) 2 A( p) 2 X Y An 1− d s = 1 − d s L(s, A ⊗ A¯) ns p A p A p n=1 ∞ a(n 1 , . . . , n d −1 ) 2 X A = , 2 · · · n dA −1 )s (n n 1 n 1 ,...,n d −1 =1 2 dA −1

(35)

A

Y p

∞ B( p) 2 X Y Bn 1− d s = 1− ns p B p n=1

=

B( p) 2

L(s, B ⊗ B¯ ) p dB s ∞ b(n 1 , . . . , n d −1 ) 2 X B

n 1 ,...,n dB −1 =1

−1 s (n 1 n 22 · · · n ddB ) B −1

,

(36)

154

GIUSEPPE MOLTENI

and Y p

A( p)B( p) 1− L(s, A ⊗ B ) = p ds

∞ X

a(n 1 , . . . , n d−1 )b(n 1 , . . . , n d−1 )

n 1 ,...,n d−1 =1

s (n 1 n 22 · · · n d−1 d−1 )

(37) if dA = dB = d; we also have ∞ X

a(n 1 , . . . , n dA −1 )b(n 1 , . . . , n dA , 1, . . . , 1)A(n dA )

n 1 ,...,n dA =1

(n 1 n 22 · · · n ddA )s A

L(s, A ⊗ B ) =

(38) if dA < dB . The coefficients a(·) and b(·) are multiplicative in every entry and are defined as a( p k1 , . . . , p kd−1 ) = Sk1 ,...,kd−1 α( p) , b( p k1 , . . . , p kd−1 ) = Sk1 ,...,kd−1 β( p) . Proof of Theorem 2 In this case we assume that L(s, A ⊗ A¯) and L(s, B ⊗ B¯ ) satisfy (R); hence we can prove that X An ,DA (RA x) , n n≤x

X Bn ,DB (RB x) n n≤x

by (30) and using Iwaniec’s idea. We can assume that dA ≤ dB . By (35) and (36) and by the assumptions of Theorem 2 giving |A(n)|, |B(n)| ≤ 1 for every n, we have P a(n 1 , . . . , n d −1 ) 2 X d|n Ad X A ≤ n n 1 n 2 · · · n dA −1 d −1 n≤x n 1 n 22 ···n dA −1 ≤x A

2

dA −1

log x

X An ,DA (RA x) n n≤x

(39)

and b(n 1 , . . . , n d , 1, . . . , 1) 2 A

X d n 1 n 22 ···n dA A

≤x

n 1 n 22 · · · n ddA A

b(n 1 , . . . , n d

B −1

X

≤

d

−1

n 1 n 22 ···n dB −1 ≤x B

≤

X

P

d|n

2 )

−1 n 1 n 22 · · · n ddB B −1

Bd

n X Bn log x ,DB (RB x) . (40) n n≤x n≤x


155

Suppose now that dA = dB = d. By (37), (39), and (40),

X

log j n 1 n 22 · · · n d−1 d−1

n 1 n 22 ···n d−1 d−1 ≤x j

≤ log x

a(n 1 , . . . , n d−1 ) 2 1/2

X n 1 n 22 ···n d−1 d−1 ≤x

·

a(n 1 , . . . , n d−1 )b(n 1 , . . . , n d−1 ) n 1 n 22 · · · n d−1 d−1

n 1 n 22 · · · n d−1 d−1

X

n 1 n 22 ···n d−1 d−1 ≤x

b(n 1 , . . . , n d−1 ) 2 1/2 n 1 n 22 · · · n d−1 d−1

,DA ,DB (RA RB x) log j x,

(41)

uniformly on j. We define L ∗ (s, A ⊗ B ) =

Y A( p)B( p) 1− L(s, A ⊗ B ). p ds p

The function L(s, A ⊗B ) has a polynomial behavior on the strip 1−1/(2d) < σ < 2, Q and the same holds for (1 − A( p)B( p)/ p ds ) since |A( p)|, |B( p)| ≤ 1 and d > 1 by hypothesis. Therefore, L ∗ (s, A ⊗ B ) also has a polynomial behavior in that strip. We define ( j) Z 2+i∞ m A ⊗B ∗ s L (1 + s, A ⊗ B ) x s 1 ( j) I (x) = ds 2πi 2−i∞ s(s + 1) · · · (s + r ) with r = r (DA , DB ) independent of j and suffciently large to assure the convergence of the integral. As for Theorem 1, from (37) and (41) we get I ( j) (x) ,DA ,DB ( j + 1)c j (RA RB x) log j x,

(42)

uniformly on j. Moving the integration line to σ = −1/2d, we have I

( j)

( j) j!(R R )c1 s m A ⊗B L ∗ (1 + s, A ⊗ B ) |s=0 A B (x) = +O , r! 2 j x 1/2d

(43)

for some positive constant c1 = c1 (DA , DB ). Choosing x = (RA RB )2d(c1 +1) , the claim follows by (42), (43), and some easy algebraic manipulations.

156

GIUSEPPE MOLTENI

Suppose now that dA < dB . Then, by (38) and (40), we get X log j n 1 n 22 · · · n ddA A d

n 1 n 22 ···n dA ≤x A

·

≤ log j x

a(n 1 , . . . , n dA −1 )b(n 1 , . . . , n dA , 1, . . . , 1)A(n dA ) n 1 n 22 · · · n ddA A a(n 1 , . . . , n d −1 ) 2 1/2 X A d

n 1 n 22 ···n dA ≤x A

·

X

n 1 n 22 · · · n ddA A b(n 1 , . . . , n d , 1, . . . , 1) 2 1/2 A n 1 n 22 · · · n ddA A

d

n 1 n 22 ···n dA ≤x A

,DA ,DB (RA RB x) log j x, uniformly on j. The claim of Theorem 2 again follows from this estimate. Proof of Theorem 3 P −s Following the notation of the axioms, let L(s, A ) = ∞ n=1 an n , and consider SS (x) =

X |an |2 , n n≤x

(n,S)=1

as we did for the proof of Theorem 1. By Proposition 6 we have |an |2 ≤ An , so that c SS (x) ,DA RA x holds by (30) for every > 0, for some c = c(D ) > 0. As in (32), in this case we obtain X∗ R 2 (d) SS2 (x) log3 x SS (x 2 ), d 2 d≤x (d,S)=1

P where ∗ is convergent by (R0 ); from this recursive upper bound and with a suitable choice of S, we get X |an |2 ,DA (RA x) ; (44) n n≤x therefore, under the hypotheses of Theorem 3, estimates (39) and (40) are easy consequences of (44), and upper bounds (9) follow from these, as in Theorem 2.


157

Acknowledgments. This paper is a part of my Ph.D. thesis; I would like to thank A. Perelli, my Ph.D. advisor. Moreover, I thank the referee for careful reading and for suggestions that have considerably improved this paper. References [1]

W. D. BANKS, Twisted symmetric-square L-functions and the nonexistence of Siegel

[2]

D. BUMP, “The Rankin-Selberg method: A survey” in Number Theory, Trace

zeros on GL(3), Duke Math. J. 87 (1997), 343–353. MR 98:11054 142

[3] [4] [5]

[6]

[7] [8]

[9]

[10] [11] [12] [13]

[14]

Formulas and Discrete Groups (Oslo, 1987), Academic Press, Boston, 1989, 49–109. MR 89m:11003 152 E. CARLETTI, G. MONTI BRAGADIN, and A. PERELLI, On general L-functions, Acta Arith. 66 (1994), 147–179. MR 95e:11096 135 ´ P. DELIGNE, La conjecture de Weil, I, Inst. Hautes Etudes Sci. Publ. Math. 43 (1974), 273–307. MR 49:5013 135 S. GELBART and H. JACQUET, A relation between automorphic representations of ´ GL(2) and GL(3), Ann. Sci. Ecole Norm. Sup. (4) 11 (1978), 471–542. MR 81e:10025 140 I. GELFAND and I. PIATESKY-SHAPIRO, Automorphic functions and the theory of representations (in Russian), Tr. Mosk. Mat. Obs. 12 (1963), 389–412. MR 28:3115 140 R. GODEMENT and H. JACQUET, Zeta Functions of Simple Algebras, Lecture Notes in Math. 260, Springer, Berlin, 1972. MR 49:7241 140 E. P. GOLUBEVA and O. M. FOMENKO, Values of Dirichlet series associated with modular forms at the points s = 1/2, 1 (in Russian), Zap. Nauchn. Sem. Leningrad Otdel. Mat. Inst. Steklov. (LOMI) 134 (1984), 117–137; English translation in J. Soviet Math. 36 (1987), 79–93. MR 86b:11041 141 , Behavior of the L−functions of parabolic forms at s = 1 (in Russian), Zap. Nauchn. Sem. S.-Petersburg Otdel. Mat. Inst. Steklov. (POMI) 204 (1993), 37–54; English translation in J. Math. Sci. 79 (1996), 1293–1303. MR 94f:11042 141 J. HOFFSTEIN and P. LOCKHART, Coefficients of Maass forms and the Siegel zero, Ann. of Math. (2) 140 (1994), 161–181. MR 95m:11048 136, 140, 141 J. HOFFSTEIN and D. RAMAKRISHNAN, Siegel zeros and cusp forms, Internat. Math. Res. Notices 1995, 279–308. MR 96h:11040 142 H. IWANIEC, Small eigenvalues of Laplacian for 00 (N ), Acta Arith. 56 (1990), 65–82. MR 92h:11045 135, 139 H. IWANIEC and P. SARNAK, “Perspectives on the analytic theory of L-functions” in GAFA 2000: Visions in Mathematics, Toward 2000 (Tel Aviv, 1999), Part II, Geom. Funct. Anal. 2000, special volume, Birkhäuser, Basel, 2000, 705–741. MR CMP 1 826 269 135 H. JACQUET and J. A. SHALIKA, On Euler products and the classification of automorphic representations, I, II, Amer. J. Math. 103 (1981), 499–558, 777–815. MR 82m:10050a; MR 82m:10050b 140

158

GIUSEPPE MOLTENI

[15]

H. KIM and F. SHAHIDI, Symmetric cube L-functions for GL2 are entire, Ann. of

[16]

W. LUO, Z. RUDNICK, and P. SARNAK, On the Generalized Ramanujan Conjecture for

Math. (2) 150 (1999), 645–662. MR 2000k:11065 140, 142

[17] [18] [19] [20]

GL(n), Proc. Sympos. Pure Math. 66, Part 2, Amer. Math. Soc., Providence, 1999. MR 2000e:11072 140 ´ C. MOEGLIN and J. L. WALDSPURGER, Le spectre résiduel de GL(n), Ann. Sci. Ecole Norm. Sup. (4) 22 (1989), 605–674. MR 91b:22028 140, 141 G. MOLTENI, L-functions: Siegel-type theorems and structure theorems, Ph.D. thesis, University of Milan, Milan, 1999. 141 F. SHAHIDI, On certain L-functions, Amer. J. Math. 103 (1981), 297–355. MR 82i:10030 140 G. SHIMURA, On the holomorphy of certain Dirichlet series, Proc. London Math. Soc. (3) 31 (1975), 79–98. MR 52:3064 140

Dipartimento di Matematica, Università di Milano, Via Saldini 50, 20133 Milano, Italy; [email protected]


ON ALGEBRAIC FIBER SPACES OVER VARIETIES OF MAXIMAL ALBANESE DIMENSION JUNGKAI A. CHEN and CHRISTOPHER D. HACON

Abstract We study algebraic fiber spaces f : X −→ Y where Y is of maximal Albanese dimension. In particular, we give an effective version of a theorem of Y. Kawamata: If Pm (X ) = 1 for some m ≥ 2, then the Albanese map of X is surjective. Combining this with [1], it follows that X is birational to an abelian variety if and only if P2 (X ) = 1 and q(X ) = dim(X ). 0. Introduction In this paper we combine the generic vanishing theorems of [5] and [6], the techniques of [3], and the results of [10] and [11] to answer a number of natural questions concerning the geometry and birational invariants of irregular complex algebraic varieties. Throughout the paper we are motivated by the following conjecture. CONJECTURE K (Ueno) Let X be a nonsingular projective algebraic variety such that κ(X ) = 0, and let alb X : X −→ Alb(X ) be the Albanese map. Then (1) alb X is surjective and has connected fibers; that is, alb X is an algebraic fiber space; (2) if F is a general fiber of alb X , κ(F) = 0; (3) there is an e´ tale covering B −→ Alb(X ) such that X ×Alb(X ) B is birationally equivalent to F × B over B.

The main evidence toward this conjecture is given by the following theorem. (Kawamata) Conjecture K (1) is true (see [7, Th. 1]).

THEOREM

(1)

DUKE MATHEMATICAL JOURNAL c 2002 Vol. 111, No. 1, Received 8 November 2000. Revision received 22 December 2000. 2000 Mathematics Subject Classification. Primary 14D06,14J10. Chen’s work partially supported by National Science Council, Taiwan, grant number NSC-89-2115-M-194-029.

159

160

CHEN and HACON

(2)

If F has a good minimal model (i.e., a model with only canonical singularities and such that m K ∼ 0 for some positive integer m), then Conjecture K is true (see [8]).

We remark here that in the proofs of our statements, we make use of [7, Th. 1] in an essential way. As a consequence of [7, Th. 1], one sees the following corollary. COROLLARY (Kawamata [7]) If κ(X ) = 0, then q(X ) ≤ dim(X ). Moreover, if q(X ) = dim(X ), then alb X : X −→ Alb(X ) is a birational morphism.

There has also been considerable interest in effective versions of this result. Kollár has shown the following theorem. (Kollár [12]) If P3 (X ) = 1, then the Albanese map is surjective. Moreover, if P4 (X ) = 1 and q(X ) = dim(X ), then alb X : X −→ Alb(X ) is a birational morphism.

THEOREM

(1) (2)

(Kollár [13, (18.13)]) X is birational to an abelian variety if and only if q(X ) = dim(X ) and Pm (X ) = 1 for some m ≥ 2. CONJECTURE

Our first result is the following theorem. THEOREM 1 If for some integer m ≥ 2 the mth plurigenera Pm (X ) equals 1, then the Albanese map of X is surjective.

Combining Theorem 1 with the results from [1], we obtain the following corollary. COROLLARY 2 Kollár’s conjecture above (see [13, (18.13)]) is true.

We are also able to generalize the corollary of Kawamata’s theorem. COROLLARY 3 If Pm (X ) = P2m (X ) = 1 for some m ≥ 2, then q(X ) ≤ dim(X ) − κ(X ). If q(X ) = dim(X ) − κ(X ), then the general fiber of the Iitaka fibration of X is birationally

ALGEBRAIC FIBER SPACE

161

equivalent to a fixed e´ tale cover A˜ of A := Alb(X ). Next, we study algebraic fiber spaces f : X −→ Y where X, Y are smooth projective varieties and Y is of maximal Albanese dimension. The generic vanishing theorems of M. Green and R. Lazarsfeld are a very effective technique in the study of irregular varieties. In §2 we prove a more general version of Theorem 2.1 which applies to irregular varieties not necessarily of maximal Albanese dimension. Using this result, we are able to show the following theorem. THEOREM 4 If κ(X ) = 0, let a := alb X : X → Alb(X ) be the Albanese map. Then (1) a∗ ω X is either a zero sheaf or a torsion line bundle; (2) P1 (FX/Alb(X ) ) ≤ 1; (3) if P1 (X ) = 1, then a∗ ω X = OAlb(X ) ; (4) there is a generically finite cover X˜ −→ X with κ(X ) = κ( X˜ ) such that alb X˜ ∗ (ω X˜ ) = OAlb X˜ .

Using a well-known result of T. Fujita (see [14, (4.1)]), for any X with κ(X ) = 0, there exists a generically finite cover X˜ −→ X such that κ( X˜ ) = 0 and P1 ( X˜ ) = 1 as in Theorem 4(3). We explain the significance of Theorem 4(4). For any algebraic fiber space f : X −→ Y , the rank at a generic point of the sheaves f ∗ (ω⊗m X/Y ) corresponds to the plurigenera Pm (FX/Y ) of a generic geometric fiber. It is expected that the positivity of the sheaves f ∗ (ω⊗m X/Y ) measures the birational variation of the geometric fibers. Our methods unluckily cannot be applied to the case m ≥ 2 because of the lack of a suitable geometric interpretation of the sheaves f ∗ (ω⊗m X/Y ). If F = FX/Alb(X ) has a good minimal model, then by Kawamata’s result there is an e´ tale covering B −→ Alb(X ) such that X ×Alb(X ) B is birationally equivalent to F × B over B. Since κ(F) = 0, by Fujita’s lemma again, there exists a generically ˜ = 0 and P1 ( F) ˜ = 1. Since F has a good minimal finite cover F˜ such that κ( F) model, we may assume in fact that (for an appropriate birational model with canonical singularities) K F˜ = 0. Let X˜ = F˜ × B. Then, for all m ≥ 1, π B ∗ (ω⊗m ˜ ) = OB , X

where π B : X˜ −→ B is the projection to the second factor. The following lemma is useful. LEMMA 5 Let f : X −→ Y be an algebraic fiber space with Y of maximal Albanese dimension. If κ(X ) = 0, then Y is birational to an abelian variety.

162

CHEN and HACON

THEOREM 6 Let f : X −→ Y be an algebraic fiber space with Y of maximal Albanese dimension. (1) If κ(X ) = κ(Y ), then P1 (FX/Y ) ≤ 1. (2) If P1 (FX/Y ) ≥ 1, then κ(X ) ≥ κ(Y ).

The above statements are closely connected to the following well-known conjecture. CONJECTURE (C n,m ) Let f : X −→ Y be an algebraic fiber space, let dim(X ) = n, and let dim(Y ) = m. Then κ(X ) ≥ κ(Y ) + κ(FX/Y ).

This conjecture is true when FX/Y has a good minimal model (see [8]). If one could generalize the generic vanishing theorem to the sheaves of the form ωY ⊗ f ∗ ωm X/Y for m ≥ 2, then using the same techniques, the Cn,m conjecture would follow for all algebraic fiber spaces f : X −→ Y with Y of maximal Albanese dimension. Finally, we prove a generalization of [7, Th. 15]. THEOREM 7 Suppose X is a variety with κ(X ) = 0 and dim(X ) ≤ 2q(X ). If P1 (X ) > 0, then n−q n−q h 0 (X, X ) > h 0 (Alb, Alb ). In particular, if dim(X ) = q(X ) + 1, then P1 (X ) = 0.

We then illustrate how one can use this result to recover Conjecture K in the case when dim(X ) = q(X ) + 1 (see [7, Th. 15]). Conventions and notation (0.1) Throughout this paper we work over the field of complex numbers C. X denotes a smooth projective algebraic variety. (0.2) A general point of X denotes a point in the complement of a countable union of proper Zariski closed subsets of X . (0.3) Let f : X −→ Y be an algebraic fiber space, that is, a surjective morphism with connected fibers. Then FX/Y denotes the fiber over a general geometric point of Y . (0.4) For D1 , D2 Q-divisors on a variety X , we write D1 ≡ D2 if D1 and D2 are numerically equivalent. (0.5) For a real number a, let bac be the largest integer less than or equal to a, and let dae be the smallest integer greater than or equal to a. For a Q-divisor P P P D = ai Di , let bDc = bai cDi and dDe = dai eDi . We say that a Q-


(0.6)

163

divisor 1 is Kawamata log terminal (klt) if 1 has normal crossings support and b1c = 0. We refer to [13, Def. 10.1(5)] for the general definition of a klt divisor. Let F be a coherent sheaf on X ; then h i (X, F ) denotes the complex dimension of H i (X, F ). In particular, the plurigenera h 0 (X, ω X ⊗m ) are denoted by Pm (X ) and the irregularity h 0 (X, 1X ) is denoted by q(X ).

1. Preliminaries 1.1. The Iitaka fibration Let X be a smooth complex projective variety with κ(X ) > 0. Then a nonsingular representative of the Iitaka fibering of X is a morphism of smooth complex projective varieties f 0 : X 0 −→ V such that X 0 is birational to X , dim(V ) = κ(X ), and κ(X v0 ) = 0, where X v0 is a general geometric fiber of f 0 . Since our questions are birational in nature, we may always assume that X = X 0 . Let A := Alb(X ), and let Z denote the image of X in A. Let Z 0 denote an appropriate desingularization of Z ; we may assume that X −→ Z factors through Z 0 . By [7] the images a(X v ) = K v are translates of abelian subvarieties of A. Since A contains at most countably many abelian subvarieties, we may assume that K v are all translates of a fixed abelian subvariety K ⊂ A. Let S := A/K , and let W denote the image of Z in S. Let W 0 be an appropriate desingularization of W . We may assume that the induced morphism π : X −→ W factors through a morphism π 0 : X −→ W 0 . Consider now a birational model of the Iitaka fibration f : X −→ V . CLAIM

We may assume that the map π 0 factors through f and a morphism q 0 : V −→ W 0 . To see this, note that by construction there is an open dense subset U of V and a map U −→ S. However, by a standard argument this must complete to a rational map V −→ S (see, e.g., [7, Lem. 14]). Since the problem is birational, we may assume that V −→ S is in fact a morphism that factors through W 0 . The above maps fit in the following commutative diagram: ⊂

X −−−−→ Z 0 −−−−→ Z −−−−→   fy q0

⊂

A  p y

V −−−−→ W 0 −−−−→ W −−−−→ S

164

CHEN and HACON

1.2. Fourier-Mukai transforms Let A be an abelian variety, and denote the corresponding dual abelian variety by ˆ Let P be the normalized Poincaré bundle on A × A. ˆ For any point y ∈ A, ˆ let A. P y denote the associated topological trivial line bundle. Define the functor Sˆ of O A -modules into the category of O Aˆ -modules by ∗ Sˆ(M) = π A,∗ ˆ (P ⊗ π A M).

The derived functor R Sˆ of Sˆ then induces an equivalence of categories between the ˆ In fact, by [15]: There are isomorphisms of two derived categories D(A) and D( A). functors R S ◦ R Sˆ ∼ = (−1 A )∗ [−g] and R Sˆ ◦ R S ∼ = (−1 Aˆ )∗ [−g], where [−g] denotes “shift the complex g places to the right.” The index theorem (IT) is said to hold for a coherent sheaf F on A if there exists an integer i(F ) such that for all j 6 = i(F ), H j (A, F ⊗ P) = 0 for all P ∈ Pic0 (A). The weak index theorem (WIT) holds for a coherent sheaf F if there exists an integer, which we again denote by i(F ), such that for all j 6 = i(F ), R j Sˆ(F ) = 0. It is easily seen that the IT implies the WIT. We denote the coherent sheaf R i(F ) Sˆ(F ) on Aˆ by Fˆ . It follows, for example, that given any coherent sheaf F , if h i (A, F ⊗ P) = 0 for all i and all P ∈ Pic0 (A), then F = 0. Consequently, if F −→ G is an injection of sheaves which induces isomorphisms in cohomology ∼ =

H i (A, F ⊗ P) − → H i (A, G ⊗ P) for all i and all P ∈ Pic0 (A), then F ∼ = G. PROPOSITION 1.2.1 If F is a coherent sheaf on A such that for all P ∈ Pic0 (A) we have h 0 (A, F ⊗ P) = 1 and h i (A, F ⊗ P) = 0 for all i > 0, then F is supported on an abelian subvariety of A.

Proof F satisfies the IT and Fˆ is a line bundle M on Pic0 (A) = Aˆ such that M has index i(M) = dim(A), and Mˆ = (−1 A )∗ F . Any line bundle with i(M) = dim(A) is negative semidefinite; that is, the induced Hermitian form has eigenvalues λi ≤ 0. It is well known that there exists a morphism of abelian varieties b : Pic0 (A) −→ A0 such that M = b∗ M 0 for some negative definite line bundle M 0 on A0 . It follows that Mˆ and hence F are supported on the image of b∗ : Aˆ 0 −→ A.


165

2. Relative generic vanishing theorems We start by recalling some facts on cohomological support loci. Let π : X → A be a morphism from a smooth projective variety X to an abelian variety A. If F is a coherent sheaf on X , then one can define the cohomological support loci by V i (X, A, F ) := P ∈ Pic0 (A)|h i (X, F ⊗ π ∗ P) 6= 0 . In particular, if π = alb X : X → Alb(X ), then we simply write V i (X, F ) := P ∈ Pic0 (X )|h i (X, F ⊗ P) 6 = 0 . We say that X has maximal Albanese dimension if dim(alb X (X )) = dim(X ). The geometry of the loci V i (X, ω X ) defined above is governed by the following theorem. 2.1 (Generic vanishing theorem) Any irreducible component of V i (X, ω X ) is a translate of an abelian subvariety of Pic0 (X ) and is of codimension at least i − (dim(X ) − dim(alb X (X ))). Let P ∈ T be a general point of an irreducible component T of V i (X, ω X ). Suppose that v ∈ H 1 (X, O X ) ∼ = TP Pic0 (X ) is not tangent to T . Then the sequence

THEOREM

(1) (2)

∪v

∪v

H i−1 (X, ω X ⊗ P) −→ H i (X, ω X ⊗ P) −→ H i+1 (X, ω X ⊗ P)

(3)

is exact. If v is tangent to T , then the maps in the above sequence vanish. If X is a variety of maximal Albanese dimension, then Pic0 (X ) ⊃ V 0 (X, ω X ) ⊃ V 1 (X, ω X ) ⊃ · · · ⊃ V n (X, ω X ) = {O X }.

(4)

Every irreducible component of V i (X, ω X ) is a translate of an abelian subvariety of Pic0 (X ) by a torsion point.

Proof For (1) and (2), see [5] and [6]. For (3), see [3], and for (4), see [16]. In [3], L. Ein and Lazarsfeld provide various examples in which the geometry of X can be recovered from information on the loci V i (X, ω X ). In this section we prove a relative version of Theorem 2.1. Let π : X −→ Y be a surjective map of smooth projective varieties. Assume that Y has maximal Albanese dimension. Let n and m be the dimensions of X and Y , respectively. We wish to study the geometry of the loci V i (Y, R j π∗ ω X ). By a result of C. Simpson [16], the irreducible components of these loci are torsion translates of abelian subvarieties of

166

CHEN and HACON

Pic0 (Y ). Therefore, their geometry is completely determined by their torsion points. Recall (see [11, Cor. 3.3]) that for any torsion Q ∈ Pic0 (X ), one has X R · π∗ (ω X ⊗ Q) ∼ R i f ∗ (ω X ⊗ Q)[−i]. = In particular, h p (X, ω X ⊗ Q) =

P

h i (Y, R p−i f ∗ (ω X ⊗ Q)).

2.2 For π : X −→ Y as above, PROPOSITION

Pic0 (Y ) ⊃ V 0 (Y, R j π∗ ω X ) ⊃ V 1 (Y, R j π∗ ω X ) ⊃ · · · ⊃ V m (Y, R j π∗ ω X ). Proof We prove that h i (Y, R j π∗ ω X ⊗ P) > 0 implies h i−1 (Y, R j π∗ ω X ⊗ P) > 0 for all i > 0. By Simpson’s result mentioned above, it suffices to prove the assertion for torsion elements P ∈ Pic0 (Y ). Fix a very ample line bundle L on Y , and for 1 ≤ α ≤ m, let Dα be sufficiently general divisors in |L|. Denote the preimage by D˜ α = π −1 (Dα ). Consider the sequence of varieties defined by X 0 = X , X α+1 = X α ∩ D˜ α+1 . Let Yα := π(X α ). We may assume that the Dα are chosen so that X α , Yα are smooth for 0 ≤ α ≤ m. The corresponding morphisms π| X α : X α → Yα are also denoted simply by π. By [11, Cor. 3.3], for any torsion P ∈ Pic0 (Y ), we may identify H i (Yα , R j π∗ ω X α ⊗ P) as a subgroup of H i+ j (X α , ω X α ⊗ π ∗ P). CLAIM

There exists a surjective map H 0 (Yi , R j π∗ ω X i ⊗ P) −→ H i (Y, R j π∗ ω X ⊗ P). Proof We prove the assertion for P = OY . In the general case the proof proceeds analogously. Consider the exact sequence of sheaves 0 −→ ω X t −→ ω X t ( D˜ t+1 ) −→ ω X t+1 −→ 0. This induces sequences of sheaves 0 −→ R j π∗ ω X t −→ R j π∗ ω X t ( D˜ t+1 ) −→ R j π∗ ω X t+1 −→ 0. By step 4 of the proof of [10, Th. 2.1 (iii)], for appropriately chosen Dα , the above sequence is exact and equivalent to R j π∗ ω X t ⊗ 0 −→ OYt −→ OYt (Dt+1 ) −→ OYt+1 (Dt+1 ) −→ 0 .


167

By [10], H k (Yt , R j π∗ ω X t (Dt+1 )) = 0 for all k > 0. We therefore have an exact sequence 0 −→ H 0 (Yt , R j π∗ ω X t ) −→ H 0 Yt , R j π∗ ω X t (Dt+1 ) −→ H 0 (Yt+1 , R j π∗ ω X t+1 ) −→ H 1 (Yt , R j π∗ ω X t ) −→ 0 and isomorphisms H k (Yt+1 , R j π∗ ω X t+1 ) ∼ = H k+1 (Yt , R j π∗ ω X t ) for k = 1, . . . , m − t − 1. The claim now follows. Moreover, the map H 0 (Yi , R j π∗ ω X i ) −→ H i (Y, R j π∗ ω X ) −→ H i+ j (X, ω X ) is induced by the inclusion H 0 (Yi , R j π∗ ω X i ) −→ H j (X i , ω X i ) and by the coboundary maps j

j+1

δi−1

δi−2

j+i

δ0

δ : H j (X i , ω X i ) −−→ H j+1 (X i−1 , ω X i−1 ) −−→ · · · −−→ H i+ j (X, ω X ). The map δ is dual to the map δ ∗ : H n−i− j (X, O X ) −→ H n−i− j (X 1 , O X 1 ) −→ · · · −→ H n−i− j (X i , O X i ) induced by successive restrictions. In turn, this map is complex conjugate to the map n−i− j

δ ∗ : H 0 (X, X

n−i− j

) −→ H 0 (X 1 , X 1

n−i− j

) −→ · · · −→ H 0 (X i , X i

)

n−i− j

induced by successive restrictions. Let V be the subspace of H 0 (X, X ) corresponding to the complex conjugate of H i (Y, R j π∗ ω X )∗ . It follows from the above n−i− j claim that δ ∗ induces an injection V ,→ H 0 (X i , X i ). Fix a general smooth point p of π(X i ). Let a : Y −→ A be the Albanese map. For a general point p ∈ Yi , one may assume that z 1 , . . . , z m are local holomorphic coordinates for a(Y ) at a( p), where z 1 , . . . , z m−i are local holomorphic coordinates for a(Yi ) at a( p). We may assume, furthermore, that the pullbacks of the z i , which we denote by xi , give rise to local holomorphic coordinates for Y and Yi at p. We may assume that d xl ( p) = 0 in 1Yi ⊗ C( p) for all m − i + 1 ≤ l ≤ m (i.e., d xl is conormal to Yi at p). Fix p˜ ∈ π −1 ( p). Let ω ∈ H 0 (A, 1A ) such that ω( p) = dz m . Let y1 , . . . , yn be local holomorphic coordinates on X centered at p˜ ∈ π −1 ( p) such n− j−i that yi = xi ◦ π for all 1 ≤ i ≤ m. Let v be any element in V ⊂ H 0 (X, X ). Then X v( p) ˜ = a K dy K , where the sum runs over all multi-indices of length n − i − j, that is, K = (k1 , . . . , kn−i− j ) and dy K = dyk1 ∧ · · · ∧ dykn−i− j . Since V injects in

168

CHEN and HACON n−i− j

H 0 (X i , X i

), for any fixed v ∈ V , by genericity of the choice of the point p, n−i− j

we may assume that v( p) ˜ 6= 0 in X i ⊗ C( p). ˜ In particular, there exists a multiindex K¯ such that a K¯ ( p) ˜ 6= 0, and for all 1 ≤ l ≤ n − i − j, k¯l does not belong n− j+1 to {m − i + 1, . . . , m}. Therefore, v ∪ ω ∈ H 0 (X, X ) is nonzero since the coefficient of dy K¯ ∪{m} is nonzero. Composing again with complex conjugation and Serre duality, we see that there is a nonzero element in H i−1 (Y, R j π∗ ω X ) ⊂ H j+i−1 (X, ω X ). The following notation is convenient. For any line bundle L on Y and v ∈ H 1 (Y, OY ), we denote by K Li ,Y,v and B i+1 L ,Y,v the kernel and the image of the map ∧v

H i (Y, L) −→ H i+1 (Y, L). Let H Li ,Y,v := K Li ,Y,v /B iL ,Y,v . The subscripts Y and v are dropped when no confusion is likely. Let τ : H 1 (X, O X ) → Pic0 (X ) be the map induced by the exponential sheaf sequence. Let 1 ⊂ Spec C[t] be a neigborhood of zero. For P ∈ Pic0 (X ) and v ∈ H 1 (X, O X ), let L be a line bundle on X × 1 such that L | X ×t ∼ = P ⊗ τ (tv). Let p X : X × 1 → X and p1 : X × 1 → 1 be the projections to the first and second factors. We need the following theorem due to Green and Lazarsfeld. THEOREM 2.3 ([6]) There is a neighborhood of zero for which

R i p1∗ ( p ∗X ω X ⊗ L ) ∼ = (Kωi X ⊗P ⊗ O1/m ) ⊕ (Hωi X ⊗P ⊗ O1 ). Proof This is a generalization of [6, Th. 3.2] which follows from the comments preceding [6, Th. 6.1] (see also [2]). COROLLARY 2.4 Let φ ∈ Kωi X ⊗P ⊂ H i (X, ω X ⊗ P). Let 8 ∈ R i p1∗ ( p ∗X ω X ⊗ L ) be a section such that (8)0 = φ. Assume that 8|(1−0) = 0. Then φ = γ ∪ v for an appropriate γ ∈ H i−1 (X, ω X ⊗ P).

Proof Since 8|(1−0) = 0, there exists an integer k ≥ 0 such that t k 8 = 0 ∈ R i p1∗ ( p ∗X ω X ⊗L ). Therefore, φt k + φ1 t k+1 + φ2 t k+2 + · · · = 0 ∈ (Kωi X ⊗P ⊗ O1/m ) ⊕ (Hωi X ⊗P ⊗ O1 ).


169

If k = 0, then φ = 0 ∈ Kωi X ⊗P , and there is nothing to show. If k ≥ 1, then φ = 0 ∈ Hωi X ⊗P , and hence φ = γ ∪ v for an appropriate γ ∈ H i−1 (X, ω X ⊗ P). PROPOSITION 2.5 Let π : X → Y be an algebraic fiber space with Y of maximal Albanese dimension. If H i (Y, π∗ ω X ⊗ P) = 0 for all i and all P in a punctured neighborhood of a torsion P0 ∈ Pic0 (Y ), then, for any v ∈ H 1 (Y, OY ), the complex ∪v

∪v

H i−1 (Y, π∗ ω X ⊗ P0 ) −→ H i (Y, π∗ ω X ⊗ P0 ) −→ H i+1 (Y, π∗ ω X ⊗ P0 )

(∗)

is exact. Proof We use the notation of the proof of Proposition 2.2. Let us first consider the following diagram, which is commutative for each square: p1 ∗ ( p ∗X ω X i ⊗ L )   r est=0 y

δ1

−−−−→ R i p1∗ ( p ∗X ω X ⊗ L )   r est=0 y δX

∪π ∗ v

δY

∪v

H 0 (X i , ω X i ⊗ π ∗ P0 ) −−−−→ H i (X, ω X ⊗ π ∗ P0 ) −−−−→ H i+1 (X, ω X ⊗ π ∗ P0 ) x x x    k ∪ ∪ H 0 (Yi , π∗ ω X i ⊗ P0 ) −−−−→ H i (Y, π∗ ω X ⊗ P0 ) −−−−→ H i+1 (Y, π∗ ω X ⊗ P0 ) Let f ∈ H i (Y, π∗ ω X ⊗P0 ) such that f ∪ v = 0 ∈ H i+1 (Y, π∗ ω X ⊗P0 ). By abuse of notation, we also denote by f the corresponding element in H i (X, ω X ⊗ π ∗ P0 ). Since δ X is surjective, we take f˜ ∈ H 0 (X i , ω X i ⊗ π ∗ P0 ), a lift of f . We remark that also f˜ ∪ π ∗ v = 0. Let F˜ ∈ p1 ∗ ( p ∗X ω X i ⊗ L ) be the section corresponding to f˜ + 0t + 0t 2 + · · · , and let F ∈ R i p1∗ ( p ∗X ω X ⊗ L ) be the corresponding section under the coboundary map. One has F|t=0 = f ∈ H i (X, ω X ⊗ π ∗ P0 ). By the claim in the proof of Proposition 2.2, the condition H i (Y, π∗ ω X ⊗ P) = 0 is equivalent to the vanishing of the following map: H 0 (Yi , π∗ ω X i ⊗ P) ∼ = H 0 (X i , ω X i ⊗ π ∗ P) −→ H i (X, ω X ⊗ π ∗ P).

(∗∗)

For all P in a punctured neighborhood of P0 , the map (∗∗) vanishes. Therefore, F|(1−0) = 0. By Corollary 2.4, f = γ ∪ π ∗ v for an appropriate γ ∈ H i−1 (X, ω X ⊗ π ∗ P0 ). P Following [11], write γ = g j , where g j ∈ H i−1− j (Y, R j π∗ ω X ⊗ P0 ); then ∗ i− j j g j ∪ v ∈ H (Y, R π∗ ω X ⊗ P0 ), so f = g0 ∪ v, and therefore (∗) is exact.

170

CHEN and HACON

3. Proof of theorems Proof of Theorem 1 By [7, Th. 1], we may assume that κ(X ) > 0. Let f : X −→ V be a birational model of the Iitaka fibration, and let π : X −→ W and S be as in §1.1. Assume that m = 2 (the proof proceeds analogously for any m ≥ 2). Fix H an ample divisor on S. For a fixed r 0, after replacing X by an appropriate birational model, we may assume that |r K X − π ∗ H | = |Mr | + Fr , where |Mr | is nonempty and free, and Fr has simple normal crossings. Let B be a general divisor of the linear series |r K X −π ∗ H |. We may assume again that B has normal crossing support. Define L := O X (K X −bB/r c). We have that L ≡ (π ∗ H/r )+{B/r } is numerically equivalent to the sum of the pullback of a numerically effective (nef) and big Q-divisor on W and a klt Q-divisor on X . As in the proof of [1, Lem. 2.1], it is possibile to arrange that |2K X | = |K X + L| + bB/r c. In particular, h 0 (X, ω X ⊗ L) = 1. Since π : X −→ W is a surjective map, by [13, Cor. 10.15] π∗ (ω X ⊗ L) is a torsion-free coherent sheaf on W , and h i (W, π∗ (ω X ⊗ L) ⊗ P) = 0 for all i > 0 and P ∈ Pic0 (S). It follows that h 0 W, π∗ (ω X ⊗ L) ⊗ P = χ W, π∗ (ω X ⊗ L) ⊗ P = χ W, π∗ (ω X ⊗ L) = h 0 W, π∗ (ω X ⊗ L) = 1 for all P ∈ Pic0 (S). By Proposition 1.2.1, π∗ (ω X ⊗ L) is supported on an abelian subvariety S 0 of S. Since π∗ (ω X ⊗ L) is torsion free and the image of X generates S, we see that S 0 = S, and hence X −→ A is surjective. Proof of Corollary 3 If κ(X ) = 0, this is a result of Kawamata. We may therefore assume that κ(X ) > 0. ∗ By Theorem 1, X −→ Alb(X ) is surjective. By [1, Lem. 2.1], h 0 (X, ω⊗m X ⊗ π P) > 0 0 for all P ∈ Pic (S), where S is defined as in §1.1. From the map of linear series |m K X + Q| × |m K X − Q| −→ |2m K X |, it follows that P2m (X ) ≥ dim(S) + 1, and therefore dim(S) = 0. The general geometric fiber X v of the Iitaka fibration has dimension dim(X ) − κ(X ). Since dim(S) = 0, by construction it follows that alb X (X v ) = Alb(X ) and hence dim(X ) − κ(X ) ≥ q(X ).


171

Suppose now that dim(X ) − κ(X ) = q(X ). Then the map X v −→ Alb(X ) is birationally e´ tale, and we may assume that X v is birational to A˜ for a fixed abelian ˜ variety A. Remark. Under the same hypothesis one can show that in fact q(V ) = 0. In particular, if Pm (X ) = P2m (X ) = 1 for some m ≥ 2 and q(X ) = dim(X ) − 1, κ(X ) = 1, then V = P1 . 3.1 If κ(X ) = 0, then V 0 (X, ω X ) contains at most one point. LEMMA

Proof Assume that there are two points P, Q in V 0 (X, ω X ). By Theorem 2.1, we may assume that P, Q ∈ Pic0 (X ) are torsion elements. Pick any m > 0 such that P ⊗m = Q ⊗m = O X ; then if P 6= Q, we have h 0 (X, ω⊗m X ) > 1, which is impossible. Therefore P = Q. Proof of Theorem 4 (1) Let A := Alb(X ). By [15], a∗ ω X is zero if and only if V i (A, a∗ ω X ) is empty for all i. By Proposition 2.2, this is equivalent to V 0 (A, a∗ ω X ) being empty. Thus if a∗ ω X 6= 0, by Lemma 3.1 we may assume that V 0 (A, a∗ ω X ) consists of exactly one (torsion) point, say, P and h 0 (A, a∗ ω X ⊗ P) = 1. We shall prove that the injection O A −→ a∗ ω X ⊗ P is in fact an isomorphism of sheaves. Therefore, a∗ ω X ∈ Pic0 (A). To this end, we consider complex D(v), ∪v

∪v

. . . H i−1 (A, a∗ ω X ⊗ P) −→ H i (A, a∗ ω X ⊗ P) −→ H i+1 (A, a∗ ω X ⊗ P) . . . . By Proposition 2.5, this is exact for all v ∈ H 1 (A, O A ). Step 1. Let V ∗ = H 1 (A, O A ) and P = P(V ). There is an exact sequence of vector bundles on P: 0 −→ H 0 (A, a∗ ω X ⊗ P) ⊗ OP (−q) −→ H 1 (A, a∗ ω X ⊗ P) ⊗ OP (−q + 1) −→ · · · −→ H q (A, a∗ ω X ⊗ P) ⊗ OP −→ 0.

(K • )

To see this, it is enough to check exactness on each fiber (see [3] for a similar argument). A point in P corresponds to a line in H 1 (A, O A ) containing a point, say, v. On the fibers above [v], the sequence of vector bundles corresponds to the complex D(v) which is exact. Similarly, there is an exact sequence of vector bundles on P: 0 −→ H 0 (A, O A ) ⊗ OP (−q) −→ H 1 (A, O A ) ⊗ OP (−q + 1) −→ · · · −→ H q (A, O A ) ⊗ OP −→ 0.

(K0• )

172

CHEN and HACON

There is an injection of sheaves i : O A −→ a∗ ω X ⊗ P determined by the choice of a section of a∗ ω X ⊗ P. This induces a map of complexes i • : K0• −→ K • . Step 2. We have that i ∗ : H i (A, O A ) −→ H i (A, a∗ ω X ⊗ P) is an isomorphism for 0 ≤ i ≤ q. We proceed by induction. By assumption, H 0 (A, O A ) ∼ = H 0 (A, a∗ ω X ⊗ P). i i ∼ Assume now that i ∗ : H (A, O A ) = H (A, a∗ ω X ⊗ P) for all i < r . We must show that i ∗ : H r (A, O A ) −→ H r (A, a∗ ω X ⊗ P) is also an isomorphism. Twisting the complexes K0• , K • by OP (−r ) and taking cohomology, we have ···   y

···   y

i∗ −−−− → H r −1 (A, a∗ ω X ⊗ P) ⊗ H q−1 OP (−q − 1) H r −1 (A, O A ) ⊗ H q−1 OP (−q − 1) − ∼ =     y y i∗ H r (A, O A ) ⊗ H q−1 OP (−q) −−−−− → H r (A, a∗ ω X ⊗ P) ⊗ H q−1 OP (−q)     y y 0

0

An easy spectral sequence argument implies that for r > 1, the vertical lines are exact. By the five lemma, we obtain the required isomorphism. Step 3. We have a∗ ω X ⊗ P = O A . Let R be the cokernel of i : O A ,→ a∗ ω X ⊗ P. For all i ≥ 0 and Q ∈ {Pic0 (A) − O A }, h i (A, O A ⊗ Q) = h i (A, a∗ ω X ⊗ P ⊗ Q) = 0. By Step 2, it follows that h i (A, R ⊗ Q) = 0 for all i ≥ 0 and Q ∈ Pic0 (A). By §1.2, R = 0. This completes the proof of (1). (2) This follows from (1) since the generic rank of a∗ ω X corresponds to P1 (FX/Alb(X ) ). (3) This also follows immediately from (1). (4) By Fujita’s lemma (see [14, (4.1)]) there is a smooth projective variety X˜ and a generically finite surjective morphism ν : X˜ −→ X such that κ( X˜ ) = κ(X ) = 0 and P1 ( X˜ ) > 0. Hence P1 ( X˜ ) = 1. The Albanese map of X˜ is surjective by [7], and (4) now follows from (3). Proof of Lemma 5 Let f ∗ : Alb(X ) −→ Alb(Y ) be the map induced from f . Since κ(X ) = 0, by [7, Th. 1] alb X : X −→ Alb(X ) is an algebraic fiber space. It follows easily that f ∗ and f ∗ ◦ alb X are surjective maps. Consider the Stein factorization Alb(X ) −→ A0 −→ Alb(Y ) of the map f ∗ . Then Alb(X ) −→ A0 is an algebraic fiber space and


173

A0 −→ Alb(Y ) is an e´ tale map of abelian varieties. It follows also that a 0 : X −→ A0 is an algebraic fiber space and the fibers FX/Y are contracted by a 0 . By [7] there exists an induced (generically finite) rational map Y 99K A0 . It follows that the generic degree of the surjective map Y −→ A0 is 1. Therefore, Y −→ Alb(Y ) is birationally e´ tale. Proof of Theorem 6 To prove (1), we consider the generically finite map albY : Y −→ albY (Y ) ⊂ Alb(Y ). Step 1. If κ(X ) ≥ 0, then the Iitaka model of X dominates the Iitaka model of Y . Therefore κ(X ) ≥ κ(Y ). Let X −→ V and Y −→ W be appropriate birational models of the Iitaka fibrations of X and Y , respectively. Since κ(FX/V ) = 0, it follows by Lemma 5 that f (FX/V ) is birational to an abelian variety. And the map f (FX/V ) −→ albY f (FX/V ) is birationally e´ tale. Therefore, it is easy to see (following the proof of [7, Th. 13]) that f (FX/V ) is contained in the fibers of the Iitaka fibration Y −→ W . Therefore, there exists a rational map V 99K W . By changing birational models, we may assume that it is a morphism. Step 2. If κ(X ) = κ(Y ), then P1 (FX/Y ) ≤ 1. Since X −→ Y and Y −→ W are algebraic fiber spaces and dim(V ) = κ(X ) = κ(Y ) = dim(W ), it follows that V −→ W is birational. FX/V = FX/W −→ FY/W is also an algebraic fiber space with generic fiber FX/Y . One sees that κ(FX/V ) = 0 and FY/W is of maximal Albanese dimension, and hence FY/W is birational to an abelian variety by Lemma 5. It follows by Theorem 4 that P1 (FX/Y ) ≤ 1. We now prove (2). Assume that κ(Y ) = 0. Since h 0 (FX/Y , ω X | FX/Y ) > 0, then f ∗ ω X 6= 0. If h 0 (X, ω X ⊗ f ∗ P) = 0 for all P ∈ Pic0 (Y ), then by Proposition 2.2, h i (Y, f ∗ ω X ⊗ P) = 0 for all i and all P ∈ Pic0 (Y ), and hence f ∗ ω X = 0, a contradiction. Therefore, h 0 (X, ω X ⊗ f ∗ P) > 0 for some P ∈ Pic0 (Y ). By Theorem 2.1.4 we may assume that P ⊗r = OY for some appropriate integer r > 0. Therefore, Pr (X ) > 0 and κ(X ) ≥ 0. If κ(Y ) > 0, then following [7, Th. 13], there exists an e´ tale cover Y˜ −→ Y which is birational to 0×P with 0 of general type (and maximal Albanese dimension) and P an abelian variety. Consider the corresponding e´ tale cover X˜ := X ×Y Y˜ −→ X and the induced algebraic fiber space f˜ : X˜ −→ Y˜ . Since κ( X˜ ) = κ(X ) and κ(Y˜ ) = κ(Y ), it suffices to show that κ( X˜ ) ≥ κ(Y˜ ) = κ(0). Since 0 is of general type, by a theorem of E. Viehweg (see, e.g., [14, Sec. 6.2(d)]), we have κ( X˜ ) ≥ κ(0)+κ(FX˜ / 0 ).

174

CHEN and HACON

We then consider the fiber space FX˜ / 0 −→ FY˜ / 0 . Since κ(FY˜ / 0 ) = κ(P) = 0, one sees that κ(FX˜ / 0 ) ≥ κ(FY˜ / 0 ) = 0 by the preceding case. The assertion now follows. Proof of Theorem 7 If dim(X ) = q(X ) + 1, this is [7, Th. 15]. Let A = Alb(X ), and let q := q(X ) = dim(A). We have already seen that O A is an isolated point of V 0 (X, A, ω X ). Therefore, proceeding as in the proof of Theorem 4, we have h q (A, a∗ ω X ) = h 0 (A, a∗ ω X ) = 1. By [10, Prop. 7.6], R n−q a∗ ω X = ω A . Therefore, by Hodge symmetry, Serre duality, and [11], n−q

h 0 (X, X ) = h q (X, ω X ) ≥ h q (A, a∗ ω X ) + h q−(n−q) (A, R n−q a∗ ω X ) n−q

> h 2q−n (A, ω A ) = h n−q (A, O A ) = h 0 (A, A ). If n = q + 1 and P1 (X ) = 1, then we would have h 0 (X, 1X ) > h 0 (A, 1A ), which is impossible. COROLLARY 3.2 ([7, Th. 15]) Let κ(X ) = 0, and let dim(X ) = q(X ) + 1. Then Conjecture K holds.

Proof By Fujita’s lemma (see [14, (4.1)]), there is a smooth projective variety Y , and a generically finite surjective morphism ν : Y −→ X such that κ(Y ) = κ(X ) = 0 and P1 (Y ) = 1. One sees that Alb(Y ) −→ Alb(X ) is surjective, and hence q(Y ) ≥ q(X ) = dim(X ) − 1. Therefore, by Theorem 7, q(Y ) ≥ dim(Y ), and hence by Kawamata’s theorem, Y −→ Alb(Y ) is birational. The fibers of Y −→ Alb(X ) are translates of a fixed elliptic curve E. The corollary now follows. Acknowledgment. We are in debt to P. Belkale, A. Bertram, L. Ein, J. Kollár and R. Lazarsfeld for valuable coversations. References [1]

J. A. CHEN and C. D. HACON, Characterization of abelian varieties, Invent. Math. 143

[2]

H. CLEMENS and C. D. HACON, Deformations of the trivial line bundle and vanishing

[3]

L. EIN and R. LAZARSFELD, Singularities of theta divisors and birational geometry of

(2001), 435–447. MR CMP 1 835 393 159, 160, 170 theorems, preprint, arXiv:math.AG/0011244, to appear in Amer. J. Math. 168 irregular varieties, J. Amer. Math. Soc. 10 (1997), 243–258. MR 97d:14063 159, 165, 171


[4]

[5]

[6] [7] [8] [9] [10] [11] [12] [13] [14]

[15] [16]

175

H. ESNAULT and E. VIEHWEG, “Revêtements cycliques (autour du théorème

d’annulation de J. Kollár), II” in Géométrie algébrique et applications (La Rábida, Spain, 1984), II, Travaux en Cours 23, Hermann, Paris, 1987, 81–96. MR 89h:14014 M. GREEN and R. LAZARSFELD, Deformation theory, generic vanishing theorems, and some conjectures of Enriques, Catanese and Beauville, Invent. Math. 90 (1987), 389–407. MR 89b:32025 159, 165 , Higher obstructions to deforming cohomology groups of line bundles, J. Amer. Math. Soc. 4 (1991), 87–103. MR 91i:32021 159, 165, 168 Y. KAWAMATA, Characterization of abelian varieties, Compositio Math. 43 (1981), 253–276. MR 83j:14029 159, 160, 162, 163, 170, 172, 173, 174 , Minimal models and the Kodaira dimension of algebraic fiber spaces, J. Reine Angew. Math. 363 (1985), 1–46. MR 87a:14013 160, 162 , Pluricanonical systems on minimal algebraic varieties, Invent. Math. 79 (1985), 567–588. MR 87h:14005 ´ , Higher direct images of dualizing sheaves, I, Ann. of Math. (2) 123 J. KOLLAR (1986), 11–42. MR 87c:14038 159, 166, 167, 174 , Higher direct images of dualizing sheaves, II, Ann. of Math. (2) 124 (1987), 171–202. MR 87k:14014 159, 166, 169, 174 , Shafarevich maps and plurigenera of algebraic varieties, Invent. Math. 113 (1993), 177–215. MR 94m:14018 160 , Shafarevich Maps and Automorphic Forms, M. B. Porter Lectures, Princeton Univ. Press, Princeton, 1995. MR 96i:14016 160, 163, 170 S. MORI, “Classification of higher-dimensional varieties” in Algebraic Geometry (Brunswick, Maine, 1985), Proc. Sympos. Pure Math. 46, Amer. Math. Soc., Providence, 1987, 269–331. MR 89a:14040 161, 172, 173, 174 S. MUKAI, Duality between D(X ) and D( Xˆ ), with its application to Picard sheaves, Nagoya Math. J. 81 (1981), 153–175. MR 82f:14036 164, 171 ´ C. SIMPSON, Subspaces of moduli spaces of rank one local systems, Ann. Sci. Ecole Norm. Sup. (4) 26 (1993), 361–401. MR 94f:14008 165

Chen Department of Mathematics, National Chung Cheng University, Ming-Hsiung, Chia-Yi 621, Taiwan, Republic of China; [email protected] Hacon Department of Mathematics, University of California, Riverside, Riverside, California 92521, USA; [email protected]


ENERGY QUANTIZATION FOR HARMONIC MAPS ` FANG-HUA LIN and TRISTAN RIVIERE

Abstract In this paper we establish the higher-dimensional energy bubbling results for harmonic maps to spheres. We have shown in particular that the energy density of concentrations has to be the sum of energies of harmonic maps from the standard 2dimensional spheres. The result also applies to the structure of tangent maps of stationary harmonic maps at either a singularity or infinity. 0. Introduction Let M, N be smooth, compact Riemannian manifolds without boundary. Suppose u : M → N is a smooth harmonic map such that the homotopy class, [u], of u is not trivial. Then it follows easily from the small energy regularity theorem of R. Schoen and K. Uhlenbeck (cf. [Sc]) that the total energy of the map u is Z E(u) = |∇u|2 d x ≥ ε0 (M, N ) > 0. (0.1) M

We should view estimate (0.1) as the simplest energy quantization phenomenon in harmonic maps. A more subtle statement is the following well-known theorem of J. Sacks and Uhlenbeck [SaU]: let 0 6= α ∈ π2 (N ), and let u i , i = 1, 2, . . . , be R a sequence of minimizers of S2 (1 + |∇u i |2 ) pi d x with [u i ] = α, and pi → 1+ as i → ∞. Then (by choosing subsequences if necessary) either {u i } converges strongly in H 1 S2 , N to a smooth harmonic map u: S2 → N such that [u] = α, or {u i } converges only weakly in H 1 S2 , N , and there is a nonconstant, smooth harmonic map v : S2 → N (obtained from a blow-up of u i ’s near a point of S2 ) such that E(v) ≤ limi E(u i ). In any case, one obtains, in particular, that the constant ε0 (M, N ) in estimate (0.1) can be replaced by two times the least possible area among all (so-called incompressible) minimal spheres in N . The Sacks-Uhlenbeck theorem has since been generalized and improved to the case of pseudoholomorphic curves (see [PW], [Y]), to the case of smooth harmonic maps from a Riemannian surface (see [J], [P]), and to DUKE MATHEMATICAL JOURNAL c 2002 Vol. 111, No. 1, Received 10 November 2000. Revision received 8 December 2000. 2000 Mathematics Subject Classification. Primary 35, 49. Lin’s work partially supported by National Science Foundation grant number DMS-9896391. 177

` LIN and RIVIERE

178

the case of heat-flow and approximate harmonic maps (see [Q], [W], [DT]). Naturally, all those works require the domain dimension to be 2. There is also a natural generalization of such statements for conformal invariant variational integrals in higher dimensions. (See [MW] and the recent interesting work of F. Duzaar and E. Kuwert [DK] in which they obtain a much more refined statement than that of [SaU] for arbitrary energy-minimizing sequences among continuous maps in given homotopy classes.) The situation becomes much more difficult to understand when the domain dimensions are larger than the conformal dimensions of variational integrals. For example, for harmonic maps the problem becomes much harder when the domain dimension n is greater than or equal to 3. The only earlier work known to the authors which might have something to do with this higher-dimensional quantization phenomenon is that of M. Giaquinta, G. Modica, and J. Sou˘cek [GMS] and of F. Bethuel, H. Brezis, and J. Coron [BBC]. In [L2], one of the authors established results similar to [SaU] for energy-minimizing sequences among continuous maps in a given homotopy class. More precisely, let {u i } ∈ C 0 (M, N ) be an energy-minimizing sequence in the homotopy class α = [u i ] , i = 1, 2, . . . . Then (by taking subsequences if needed) u i * u weakly in H 1 (M, N ), and |∇u i |2 d x * µ = |∇u|2 d x + ν as Radon measures. Here u is a weakly harmonic map from M into N that is smooth away from a closed, rectifiable set of 6 of H n−2 (6) < ∞, and here ν ≥ 0 is a Radon measure of the form 2n−2 (ν, x) H n−2 b6, with ε0 (M, N ) ≤ 2n−2 (ν, x) ≤ C(α, M, N ). The crucial fact proved in [L2] is that, for H n−2 – a.e. x ∈ 6, 2n−2 (ν, x) = inf{E(u) : u ∈ H 1 (S2 , N ), [u] = αx ∈ π2 (N )} for some nontrivial class αx ∈ π2 (N ). Moreover, `x X 2n−2 (ν, x) = E(φ j ) (0.2) j=1

for some smooth, nonconstant harmonic maps φ j : S2 → N such that φ j , j = 1, . . . , `x , is energy minimizing in the homotopy class [φ j ], and that [φ j ]’s give simply a decomposition of the class αx ∈ π2 (N ) (see [DK]). On the other hand, let {u i } be a sequence of stationary harmonic maps from M into N such that u i * u weakly in H 1 (M, N ). Then |∇u i |2 d x * |∇u|2 d x + ν as Radon measures. Here u is a weakly harmonic map from M into N which is smooth away from a closed, rectifiable set 6 ⊆ M with H n−2 (6) < ∞, and here ν = 2n−2 (ν, x) H n−2 b6

(0.3)

satisfies ε0 ≤ 2n−2 (ν, x) < ∞ (see [L1]). Naturally, one believes that 2n−2 (ν, x), for H n−2 – a.e. x, is a sum of energies of harmonic maps from S2 into N . This is the issue the present article addresses. We see that the analytic issues involved are rather subtle, and one of them is very much like our recent work concerning quantization

ENERGY QUANTIZATION FOR HARMONIC MAPS

179

phenomena for 3-dimensional Ginzburg-Landau equations (see [LR]). To simplify the presentation, we assume throughout this paper that N is the standard k-dimensional sphere (and in some of the results below, k = 2). One of the main results of the paper is the following theorem. A Let N be the standard k-dimensional sphere, k ≥ 2, and let ν = 2n−2 (ν, x) H n−2 be as in (0.3). Then for H n−2 – a.e. x ∈ 6, THEOREM

2n−2 (ν, x) =

`x X

E(φ j ),

j=1

where for j = 1, . . . , `x , φ j is a smooth, nonconstant harmonic map from S2 into Sk = N . The conclusion of Theorem A can also be viewed as a higher-dimensional version of the “energy identity” for a weakly convergent sequence of harmonic maps which was shown earlier by J. Jost [J] and T. Parker [P] (see also [DT], [Q], [W], [LW]). In the case when N is the standard sphere S2 , then we have the following corollary. COROLLARY 1 Let u : R3 → S2 be a stationary harmonic map. Then Z 1 lim |∇u|2 d x = 8πn, r →∞ r B (0) r

n = 0, 1, 2, . . . , +∞. If u is, in addition, smooth, then Z 1 lim |∇u|2 d x = 16πn, r →∞ r B (0) r

(0.4)

(0.5)

n = 0, 1, 2, . . . , +∞. The above result remains true when one studies the tangent map of u at 0 (i.e., when one studies the limit as r ↓ 0). We note that if u : R2 → S2 is a smooth harmonic map, then [SaU] says that Z |∇u|2 d x = 8πn for n = 0, 1, 2, . . . , +∞. R2

1. Proof of Corollary 1 We start with the following theorem.

` LIN and RIVIERE

180

THEOREM B Let N be a smooth, compact Riemannian manifold without boundary, and let u : R3 → N be an entire stationary harmonic map with bounded normalized energy: Z 1 sup |∇u|2 d x ≤ 3 < ∞. (1.1) r >0 r Br (0)

Then any tangent map, u 0 , at infinity of u can be described as follows: (i) there is a sequence Ri → +∞ such that u i (x) ≡ u(Ri x) * u 0 (x) ≡ 1 (R3 , N ), where u : S2 → N is a smooth harmonic u 0 (x/|x|) weakly in Hloc 0 map; (ii) µi = |∇u i |2 (x) d x * µ = |∇u 0 |2 (x) d x + ν as Radon measures on R3 , where the defect measure ν is a nonnegative Radon measure supported on finitely many rays emitted from the origin; moreover, on each ray ν is a constant multiple of the one-dimensional Hausdorff measure; (iii) there is a balancing condition on the energy density of u 0 and masses of ν on various rays: Z ` 2 X x · ∇u 0 (x) + 2(xi ) xi = 0. (1.2) S2

Here xi ∈ S2 , S =

S`

i=1

O xi , and ν =

i=1

P`

i=1 2(x i ) H

1 bO x

i.

Proof Let u be a stationary harmonic map from R3 into N such that Z 1 sup |∇u|2 d x = 3 < ∞. r Br (0) r >0 R3 :

We recall first the energy monotonicity formula (cf. [Sc]) for stationary maps in Z Z Z 1 1 1 ∂u 2 2 2 |∇u| d x − |∇u| d x = 2 (1.3) dx R B R (0) r Br (0) B R (0) ρ ∂ρ

for all 0 < r < R < ∞. In particular, Z Z 1 1 2 3 = sup |∇u| d x = lim |∇u|2 d x, r →+∞ r B (0) r >0 r Br (0) r

(1.4)

which is finite by our assumption. Next, for any sequence of λi → +∞, we may find a subsequence of {u(λi , x)} 1 R3 , N to a map u (x). By taking subsequences if that weakly converges in Hloc 0 1 (R3 , N ), and needed, we may assume that u i (x) = u(λi x), u i → u 0 weakly in Hloc


181

that |∇u i |2 d x = µi * µ = |∇u 0 |2 d x + ν as Radon measures. It follows from (1.3) and (1.4) that Z 1 ∂u 2 (1.5) d x → 0 as R → ∞. ρ ∂ρ R3 \B R (0)

We thus obtain for a.e. 0 < r < R < ∞, h1 1 1 i 1 µ B R (0) − µ Br (0) = lim µi B R (0) − µi Br (0) i→∞ R R r r Z 1 ∂u 2 = lim dx = 0 i→∞ ρ ∂ρ

(1.6)

Bλi R(0)\Bλi r (0)

and

1 µ0 B R (0) = 3. R In particular, for 0 < r < R < ∞, Z Z 1 ∂u 0 2 1 ∂u i 2 d x < lim d x = 0; ρ ∂ρ ρ ∂ρ i B R (0)\Br (0)

(1.7)

(1.8)

B R (0)\Br (0)

that is, ∂u 0 /∂ρ ≡ 0 or, equivalently, u o (x) = u 0 (x/|x|). Next, we let φ = S2 → R+ be a smooth function, and we let ψ ∈ C0∞ (0, 1) be R1 such that ψ ≥ 0 and 0 ψ(t) dt = 1. We consider, for 0 < a < ∞, 0 < ε a, the functions E(u, φ, a, ε) (1.9) Z ∞Z ∂u 2 ∂u 2 i i = (r + a)2 + (r + a, θ) φ(θ) · ψε (r ) dθ dr ; 2 ∂r ∂θ 0 S here ψε (r ) = (1/ε) ψ(r/ε). Then with a direct computation using the fact that div δkl |∇u i |2 − 2Dk u i Dl u i = 0 in the sense of distribution, (1.10) we obtain Z ∞Z ∂u 2 d d i E (u i , φ, a, ε) = 2 (r + a)2 (r + a, θ) φ(θ) ψε (r ) dθ dr 2 da da 0 ∂r S Z ∞Z ∂u 2 i +2 (r + a) (r + a θ) φ(θ) ψε (r ) dθ dr 2 ∂r 0 S Z ∞Z ∂ ∂ ∂ − ui u i (r + a θ) φ(θ) ψε (r ) dθ dr. ∂θ ∂θ 0 S2 ∂r (1.11)

` LIN and RIVIERE

182

Integrating both sides of (1.11) with respect to a ∈ (r, R), we get E(u i , φ, R, ε) − E(u i , φ, r, ε) a=R Z ∞Z 2 2 ∂u i = 2(ρ + a) (ρ + a, θ) φ(θ) ψε (ρ) dθ dρ ∂ρ 0 S2 a=r Z ∞Z RZ ∂u 2 i +2 (ρ + a) (ρ + a, θ) φ(θ) ψε (ρ) dθ da dρ ∂ρ 0 r S2 Z ∞Z RZ ∂u i ∂u i − 2 · (ρ + a, θ) φθ (θ) φε (ρ) dθ da dρ. 2 ∂ρ ∂θ 0 r S Now let ε → 0+ . We obtain for a.e. 0 < r < R < ∞, Z ∂u i 2 ∂u i 2 R2 + (R, θ) φ(θ) dθ ∂ρ ∂θ S2 Z ∂u i 2 ∂u i 2 − r 2 + (r, θ) φ(θ) dθ ∂ρ ∂θ S2 Z Z ∂u 2 ∂u 2 i i =2 R2 r 2 (R, θ) φ(θ) dθ − (r θ) φ(θ) dθ 2 2 ∂ρ ∂ρ S S Z RZ ∂ 2 + 2ρ u i (ρ, θ ) φ(θ) dθ dρ 2 ∂ρ r S Z RZ ∂ ∂ ∂φ − 2 ui u i (ρ, θ) dθ dρ. ∂θ ∂θ r S2 ∂ρ

(1.12)

(1.13)

We write |∇u i |2 d x = [ρ 2 |∂u i /∂ρ|2 + |∂u i /∂θ|2 ](ρ, θ) dθ dρ = dσi (ρ, θ) dρ; then (1.13) implies that Z Z φ(θ) dσi (R, θ ) − φ(θ) dσi (r, θ) → 0 (1.14) Sn−1

Sn−1

as i → ∞. On the other hand, µi = |∇u i |2 d x * µ as Radon measures on (0, ∞) × S2 ; that is, dσi (ρ, θ) dσ * dµ. We may then deduce, from (1.14), the translation invariance property of the weak limit of dσi (ρ, θ) in ρ to obtain dµ = dσ (θ) dρ. Note that the Radon measure dσ (θ) can be obtained also as a weak limit of measures dσi (ri , θ) for a suitable sequence in ri ’s. What we have shown is that the Radon measure µ has the property that µλ ≡ µ for any λ > 0. Here µλ (A) = µ(λA)/λ. Finally, since µ = |∇u 0 |2 d x + v and since v = 2(x) H 1 bS (by [L1]), we conclude that S consists of finitely many rays emitted from the origin, and 2(x) is a constant on each ray. In the next section we show that all such constant 2(x) must be P of the form `j=1 E(φ j ) for some smooth harmonic maps φ j , j = 1, . . . , `x , from S2 into N .


183

Finally, since each u i is a stationary harmonic map, a simple first variation gives Z Z ∂u i x |∇u i |2 (x) d x = 2 · ∇u i (x). (1.15) S2 S2 ∂r Passing to the limit i → +∞, we obtain Z S2

Here xi ∈ S2 and

S`

*

i=1 0x i

` 2 X x · ∇u 0 (x) + 2(xi ) xi = 0. i=1

= S. This completes the proof of the theorem.

The statement of Corollary 1 now simply follows from Theorems B and A. InPì deed, from Theorem A one concludes that each 2(xi ) is a sum j=1 E(φi j ) 2 k for some harmonic maps φi j : S → N = S . Hence, if N = S2 , then R P` Pì limr →∞ (1/r ) Br (0) |∇u|2 d x = E(u 0 ) + i=1 j=1 E(φi j ) = 8πn for a nonnegative integer n. Suppose the starting map u is also smooth. Then it is easy to see that deg(u 0 ) +

ì ` X X

deg(φi j ) = 0.

i=1 j=1

Here deg denotes the degree of maps from S2 into itself. Thus | deg u 0 | is an even number, and so E(u 0 ) +

ì ` X X

E(φi j ) = 16π n

P`

i=1

Pì

j=1 | deg φi j |+

for some nonnegative integer n.

i=1 j=1

The remainder of the present article is devoted to the proof of Theorem A. 2. Setup and basic estimates Let ν = 2n−1 (ν, x) H n−2 b6 be as in (0.3). Then by [L1, §2], one has for H n−2 – a.e. x0 ∈ 6 that 6 has a unique classical tangent plane P0 at x0 , and ν has the tangent measure at x0 given by 2n−2 (ν, x0 ) H n−2 bP0 . For simplicity we take one such x0 to be the origin and let P0 be the (n − 2)-dimensional plane Rn−2 × {(0, 0)} ⊆ Rn . We note that the measure 2n−2 (ν, x0 ) H n−2 bP0 can be obtained from a sequence of stationary harmonic maps in the sense that there is a weakly convergent sequence of stationary maps {u i } such that u i ’s con1 (Rn , N ) and such that |∇u |2 (x) d x * verge to a constant map weakly in Hloc i n−2 n−2 2 (ν, x0 ) H bP0 as Radon measures on Rn . In particular, if we restrict each map to Q 1 = {x ∈ Rn : |(x1 , . . . , xn−2 )| < 1, |(xn−1 , xn )| < 1}, we have u i *

` LIN and RIVIERE

184

a constant weakly in H 1 (Q 1 , N ) and |∇u i |2 d x * 2n−2 (ν, x0 ) H n−2 b60 weakly as Radon measures on Q 1 . Here 60 = P0 ∩ Q 1 . Moreover, by discussions in [L1, §2] one has Z X n−2 ∂u i 2 (2.1) (x) d x → 0 as i → +∞. ∂ xk Q1 k=1

To prove Theorem A it suffices to verify that such a 2n−2 (ν, x0 ) is a finite sum of energies of smooth harmonic maps from S2 into N . Let us recall how the first harmonic map from S2 into N is obtained from the blow-up analysis in [L1]. We let X 1 = (x1 , . . . , xn−2 ), X 2 = (xn−1 , xn ), and Z n−2 X ∂u i 2 n−2 f i (X 1 ) = (2.2) ∂ x (X 1 , X 2 ) d X 2 , X 1 ∈ B1 (0). 2 k B1 (0) k=1

Then the Fubini theorem and (2.1) imply that f i → 0 in L 1 (B1n−2 (0)). By the partial regularity theorem of Bethuel [B] for stationary harmonic maps (when N = S K , see [E]), one has that, for H n−2 – a.e. X 1 ∈ B1n−2 (0), u i (x) is smooth near all points of (X 1 , X 2 ) ∈ B1n−2 (0) × B12 (0). By the weak L 1 -estimate for Hardy-Littlewood maximal functions, one can easily find a sequence of points {X 1i }, i = 1, 2, . . . , such that u i (x) is smooth near all points (X 1i , X 2 ) ∈ B1n−2 (0) × B12 (0)

(2.3)

and such that sup r 2−n 0 0.

E Wi , Q(i, M) → 0,

(2.13)

as i → ∞ and R → ∞, for any given M > 0. 3. Proof of Theorem A We prove Theorem A by inducting on the numbers of harmonic spheres that arise in the blow-up analysis. That is the number m in the sum of the right-hand side of (2.11). We consider first the case m = 1. In that case we must show that E(Wi , Bi ) → 0 as i → ∞ and R → ∞. We claim first that, for any ε1 > 0, there are sufficiently large R and I such that, for all i ≥ I , one has Z Z t+1 Z (n−2) t |∇Wi |2 (X 1 , t, θ) dθ dt d X 1 < ε1 e (3.1) |X 1 |≤e−t

t

S1


187

for all t ∈ log 2 |log δi R| − 1 . Indeed, if the claim is false, then we may assume that, as i → ∞, there exists ti such that Z ti +1 Z Z (n−2) ti |∇Wi |2 (X 1 , t, θ) dθ dt d X 1 e |X 1 |≤e−ti

ti

S1

is equal to the maximum of the left-hand side of (3.1) for t ∈ log 2, |log δi R| − 1 , and it is larger than ε1 . In view of (2.13), we must have ti → ∞ and | log δi R| − ti → ∞. Let X 1 = e−ti Y1 , and define Vi (Y1 , t, θ) = Wi e−ti Y1 , t − ti , θ . Then we can view Vi as a map defined on B2n−2 (0) × [−Mi , Mi ] × S1 with Mi → ∞. Moreover, Z

Z

1Z S1

|Y1 |≤1 0

|∇Vi |2 (Y1 , t, θ) dθ dt dY1 ≥ ε1 .

(3.2)

Here ∇Vi = (e−t ∇Y1 Vi , ∂ Vi /∂t, ∂ Vi /∂θ). From (2.12) one also sees that Vi is a stationary harmonic map on Q i ≡ B2n−2 (0) × [−Mi , Mi ] × S1 and that Vi satisfies the equation ∂2 ∂2 V + Vi + A(Vi ) (∇Vi , ∇Vi ) = −e−2t 1Y1 Vi . i ∂t 2 ∂θ 2 Furthermore, by (2.4) we have Z Z Mi Z 2−n sup r |e−t ∇Y1 Vi |2 (Y1 , t, θ) dθ dt dY1 → 0

0 0 is arbitrary, we prove (2.11) for the case m = 1. Remark. In order to make the idea more transparent, we present here another proof, which is modified from [SaU] and [DT] for the case m = 1. However, we again need (3.10) and (3.11). We want to estimate the energy of Wi (0, t, θ) on Bi . Let n be an integer such that d = | log δi R| − log 2 /n ∈ (1/2, 1]. Let t j = log 2 + jd for j = 0, 1, . . . , n. We simply write W (t, θ) for Wi (0, t, θ) and denote P j = t j , t j+1 × S2 , and S j = {t j } × S1 . Let h(t) be an Rk+1 -valued function, linear on each interval [t j , t j+1 ] and with h(t j ) equal to the mean value of W on S j . Consider h as a map from Bi into Rk+1 . Then ∂2 ∂2 (W − h) + (W − h) ∂t 2 ∂θ 2 = −e−2t (1 X 1 W )(0, t, θ) − A(W )(∇W, ∇W )(0, t, θ)

(3.13)

=α on each (t j , t j+1 ) × S1 for j = 0, . . . , n − 1. After taking the inner product of this equation with W − h and integrating over P j , we obtain Z Z ∇(W − h) 2 dθ dt = (W − h) α dθ dt Pj

Pj

Z

Z

+

(W − h) (Wt − h t ) dθ.

− S j+1

Sj

Note that the boundary integrals of (W − h) h t vanish. So we get Z Z ∇(W − h) 2 dθ dt ≤ (W − h) α dθ dt Pj

Pj

Z

Z

+

(W − h) Wt dθ dt.

− S j+1

Sj

` LIN and RIVIERE

190

Summing the inequalities over j, we obtain Z Z Z Z ∇(W − h) 2 dθ dt ≤ (W − h) Wt − (W − h) Wt + (W − h) α dt dθ. Bi

S0

By (3.5), one has |W − h| ≤ C Z Z |α| dt dθ =

Sn

√

Z ≤ 2 (X i ) B1/2 2

(3.14)

ε1 . On the other hand,

δi R≤|X 2 −X 2i |≤1/2

Bi

Bi

1 X u i + |∇u i |2 u i (X i , X 2 ) d X 2 1 1

1 X u i (X i , X 2 ) d X 2 + 1 1

Z 2 (X i ) B1/2 2

(3.15)

|∇u i |2 X 1i , X 2 d X 2 .

Using (2.7) and (2.8), one has Z |α| dt dθ ≤ C.

(3.16)

∇(W − h) 2 dθ dt ≤ C √ε1 .

(3.17)

∂ 2 √ W dθ dt ≤ C ε1 . ∂θ

(3.18)

Bi

We thus obtain

Z Bi

In particular, Z Bi

To estimate ∂ W/∂t, we need to control the Hopf differential. For any u(X 2 ), X 2 = (y1 , y2 ), we define φ(u) to be the Hopf differential associated with u as follows: ∂u ∂u ∂u 2 ∂u 2 φ(u) = − · i. (3.19) −2 ∂ y1 ∂ y2 ∂ y1 ∂ y2 We note that if u : ⊆ R2 → N is a harmonic map, then φ(u) is holomorphic, and if in addition = R2 and E(u) < ∞, then φ(u) ≡ 0. In the case m = 1, our previous arguments imply already that Z φ(u i ) d X 2 ≤ C ε1/4 for all i ≥ I. (3.20) 1 2 (X i ) B1/2 2

Indeed, we observe first that Z Z |φ(u i )| d X 2 = (X 1i ) i R

Bδ2

B R2 (0)

|φ(Vi )| d X 2 →

Z B R2 (0)

|φ(ψ1 )|(X 2 ) d X 2

1 (R2 , Sk ) as i → ∞. The last term as i → ∞. This is valid because Vi → ψ1 in Hloc 2 k goes to zero as R → ∞ since ψ1 : R → S is a finite-energy harmonic map. Next,


191

by (3.12) we have also that Z

φ(u i ) (X 2 ) d X 2 ≤ C ε1/4 . 1

δi R≤|X 2 −X 2i |≤1/2

This proves (3.20). Therefore Z Z 2 ∂ W dθ dt ≤ Bi Bi ∂t

Z ∂ 2 φ(u i ) d X 2 W dθ dt + 2 (X i ) ∂θ B1/2 2 1/2 1/4 1/4 ≤ C ε1 + ε1 ≤ C ε1 .

(3.21)

Here we note that |φ(u i )|(X 2 ) d X 2 is conformal invariant with respect to conformal change of variables (see [H]). In any case, we have completed the proof of the case m = 1. 4. Completion of the proof of Theorem A Let us consider the case m ≥ 2. As in [DT], the essential part of the proof is identical to the proof for the case m = 1. One has to distinguish more bubble domains, where a blow-up gives rise to harmonic maps from S2 into Sk , and neck domains, where one has to show that energy can be made arbitrarily small. The general case can be done by an induction. We then simply sketch the proof for the case m = 2. For this case, m = 2, (3.1) can no longer be valid. Then from (3.2) we have 1 . Define a = two possibilities. First, the convergence of Vi to V∞ is strong in Hloc i ti − log 2, bi = | log δi R| − ti . We want to show that the energy of Vi on the cylinder [ai , bi ] × S1 is close to the energy of V∞ on R × S1 . Indeed, we have lim lim E Vi , [−M, M] × S1 = lim E V∞ , [−M, M] × S1 , M→∞ i→∞

M→∞

by the strong convergence of Vi to V∞ and by [SaU]. On the other hand, we follow the proof for the case m = 1 to verify that energies of Vi on cylinders [−ai , −M]×S1 and [M, bi ] × S1 can be made arbitrarily small if one lets i → ∞, M → ∞ (and also R → ∞). In other words, E(Wi , B) = E(φ2 ) + oi (1). Here φ2 is V∞ ◦ G and G : S2 → R × S1 is a conformal mapping with north and south poles corresponding to {+∞} × S1 and {−∞} × S1 . Note that via [SaU], φ2 : S1 → Sk is a smooth harmonic map. To show that E Vi , [M, bi ] × S1 → 0 is the same as to show that E(u i , Bri /δi R ) → 0,

as i → ∞, M → ∞, R → ∞.

(4.1)

Here ri = e−ti −M δi R. The proof of (4.1) is exactly the same as for the case m = 1 that follows from (3.12) with 1/2 replaced by ri . To show that E Vi , [−ai , M] × S1 → 0 is the same as to show that 2 E u i , B1/2 (X 2i ) \ B R2 i (X 2i ) → 0. (4.2)

` LIN and RIVIERE

192

Here Ri = e−ti +M . We note that on ∂ B Ri (X 2i ), u i satisfies 2

|∇ u i |2 (X 2 ) + |∇ u i |(X 2 ) ≤ C Ri−2 ε1 .

(4.3)

As for the m = 1, we can extend u i inside B R2 i (X 2i ) properly so that estimates similar to (3.11) and (3.12) are valid. That leads to 2 E u i , B1/2 (X 2i ) \ B R2 i (X 2i ) ≤ C ε1 . (4.4) Hence, we prove E (Wi , Bi ) = E(φ2 ) + oi (1) for the first possibility. The second possibility is that energy Vi may be concentrated near a point p ∈ (0, 1] × S1 . In this case we apply the arguments for the case m = 1 to the map Vi : B1n−2 (0) × [−1, 2] × S1 → Sk to obtain E Vi , [−1, 2] × S1 = E(φ2 ) + oi (1). Then we use (4.1) and (4.4) to show that E Vi , [−ai , −1] × S1 + E Vi , [2, bi ] × S1 ≤ C ε1 . (4.5) We conclude that E Vi , [−ai , bi ] × S1 = E (Wi , Bi ) = E(φ2 ) + oi (1). We would like to note that one can also follow the arguments in [DT] but use our estimates (3.18), (3.20), and (3.21) to prove the same conclusion. This completes the proof of Theorem A. References [B]

F. BETHUEL, On the singular set of stationary harmonic maps, Manuscripta Math. 78

[BBC]

F. BETHUEL, H. BREZIS, and J. M. CORON, “Relaxed energies for harmonic maps” in

(1993), 417–443. MR 94a:58047 184

[BMR] [DT]

[DK]

[E] [GMS]

[H]

Variational Methods (Paris, 1988), Progr. Nonlinear Differential Equations Appl. 4, Birkhäuser, Boston, 1990, 37–52. MR 94a:58046 178 ` , Quantization effects for −1u = u(1 − |u|2 ) in H. BREZIS, F. MERLE, and T. RIVIERE 2 R , Arch. Rat. Mech. Anal. 126 (1994), 35–58. MR 95d:35042 W. Y. DING and G. TIAN, Energy identity for a class of approximate harmonic maps from surfaces, Comm. Anal. Geom. 3 (1995), 543–554. MR 97e:58055 178, 179, 189, 191, 192 F. DUZAAR and E. KUWERT, Minimization of conformally invariant energies in homotopy classes, Calc. Var. Partial Differential Equations 6 (1998), 285–313. MR 99d:58045 178 L. C. EVANS, Partial regularity for stationary harmonic maps into spheres, Arch. Rational Mech. Anal. 116 (1991), 101–113. MR 93m:58026 184, 185, 188 ˘ , The Dirichlet energy of mappings with M. GIAQUINTA, G. MODICA, and J. SOUCEK values into the sphere, Manuscripta Math. 65 (1989), 489–507. MR 90i:49053 178 ´ F. HELEIN , Harmonic Maps, Conservation Laws and Moving Frames, Diderot, Paris, 1997. 185, 187, 191


193

[J]

J. JOST, Two-Dimensional Geometric Variational Problems, Pure Appl. Math., Wiley,

[L1]

F.-H. LIN, Gradient estimates and blow-up analysis for stationary harmonic maps,

Chichester, England, 1991. MR 92h:58045 177, 179

[L2] [LR] [LW]

[MW]

[P] [PW] [Q] [SaU] [Sc]

[T] [W] [Y]

Ann. of Math. (2) 149 (1999), 785–829. MR 2000j:58028 178, 182, 183, 184, 185, 187, 188 , Mapping problems, fundamental groups and defect measures, Acta Math. Sin. (Engl. Ser.) 15 (1999), 25–52. MR 2000m:58029 178 ` , A quantization property for static Ginzburg-Landau F.-H. LIN and T. RIVIERE vortices, Comm. Pure Appl. Math. 54 (2001), 206–228. MR 2001k:35097 179 F.-H. LIN and C. Y. WANG, Energy identity of harmonic map flows from surfaces at finite singular time, Cal. Var. Partial Differential Equations 6 (1998), 369–380. MR 99k:58047 179 L. B. MOU and C. Y. WANG, Bubbling phenomena of Palais-Smale-like sequences of m-harmonic type systems, Cal. Var. Partial Differential Equations 4 (1996), 341–367. MR 97e:35041 178 T. PARKER, Bubble tree convergence for harmonic maps, J. Differential Geom. 44 (1996), 545–633. MR 98k:58069 177, 179 T. PARKER and J. WOLFSON, Pseudo-holomorphic maps and bubble trees, J. Geom. Anal. 3 (1993), 63–98. MR 95c:58032 177 J. QING, On singularities of the heat flow for harmonic maps from surfaces into spheres, Comm. Anal. Geom. 3 (1995), 297–315. MR 97c:58154 178, 179 J. SACKS and K. UHLENBECK, The existence of minimal immersions of 2-spheres, Ann. of Math. (2) 113 (1981), 1–24. MR 82f:58035 177, 178, 179, 187, 189, 191 R. SCHOEN, “Analytic aspects of the harmonic map problem” in Seminar on Nonlinear Partial Differential Equations (Berkeley, 1983), Math. Sci. Res. Inst. Publ. 2, Springer, New York, 1984, 321–358. MR 86b:58032 177, 180 L. TARTAR, Imbedding theorems of Sobolev spaces into Lorentz spaces, Boll. Unione Mat. Ital. Sez. B Artic. Ric. Mat. (8) 1 (1998), 479–500. MR 99k:46060 185 C. Y. WANG, Bubble phenomena of certain Palais-Smale sequences from surfaces to general targets, Houston J. Math. 22 (1996), 559–590. MR 98h:58053 178, 179 R. YE, Gromov’s compactness theorem for pseudo holomorphic curves, Trans. Amer. Math. Soc. 342 (1994), 671–694. MR 94f:58030 177

Lin Department of Mathematics, New York University, Courant Institute of Mathematical Sciences, 251 Mercer Street, New York, New York 10012-1185, USA; [email protected] Rivière Departement Mathematik, Eidgenössische Technische Hochschule, CH-8092 Zürich, Switzerland; [email protected]


GREEN’S CONJECTURE FOR THE GENERIC r -GONAL CURVE OF GENUS g ≥ 3r − 7 MONTSERRAT TEIXIDOR I BIGAS

Abstract The syzygies of a generic canonical curve are expected to be as simple as possible for p ≤ (g − 3)/2. We prove this result here for p ≤ (g − 2)/3 only. The proof is carried out by considering infinitesimal deformations near a hyperelliptic curve. Introduction Let C be a nonsingular curve of genus g over an algebraically closed field k of characteristic zero. Let K be the canonical sheaf on C. If C is not hyperelliptic, the map associated to the complete canonical series |K |, C → Pg−1 , is an embedding and the image curve is projectively normal. If the curve is neither trigonal nor a plane quintic, the ideal of C is known to be generated by quadrics. Continuing in this vein, M. Green made the conjecture that the resolution of the ideal of C in Pg−1 should depend on the linear series that C has. To make this precise, one defines property N p . Take a minimal resolution of the ideal sheaf of C in Pg−1 . Then one says that property N0 holds if C is projectively normal, N1 means that the ideal of the curve is generated by quadrics, and N2 means that in addition the syzygies among these quadrics are generated by linear relations. In general, N p means that N p−1 holds and the pth syzygies are generated by linear relations. Define the Clifford index of C by Cliff(C) = min deg L − 2 h 0 (C, L) − 1 |L ∈ PicC, h 0 (C, L) ≥ 2, h 1 (C, L) ≥ 2 . In particular, for a curve that is generic in the sense of moduli, the Clifford index of C is given by [(g − 1)/2], while the most special curves from the point of view of the Clifford index are hyperelliptic curves that have Cliff(C) = 0. DUKE MATHEMATICAL JOURNAL c 2002 Vol. 111, No. 2, Recieved 19 November 1999. Revision received 30 November 2000. 2000 Mathematics Subject Classification. Primary 14H51, 14N05; Secondary 14D15, 14D20, 14G05, 13C99. 195

196

MONTSERRAT TEIXIDOR I BIGAS

CONJECTURE 0.1 (Green (cf. [G, Conj. 5.1])) The curve C has property N p if and only if Cliff(C) > p.

Most of the early effort on syzygies of curves seems to concentrate on the proof of this conjecture. The only if part of Conjecture 0.1 was proved by Green and R. Lazarsfeld (cf. [G, Appendix]). The conjecture has been proved for g ≤ 8 (cf. [S1]) and for p = 2 (cf. [V], [S3]). Much less publicised (maybe outside a circle of experts) is the following statement that says the generic curve satisfies Conjecture 0.1. 0.2 (Generic Green’s conjecture (cf. [G, Conj. 5.6])) The generic curve C of genus g satisfies N[(g−3)/2] . CONJECTURE

An early attempt to deal with this weaker statement was made by L. Ein [E]. Some methods that could be used to deal with the problem are suggested by D. Bayer, D. Eisenbud, and Green (cf. [BE], [EiG]). The most complete result to date seems to be in [S2], where it is proved that the generic curve satisfies N p for p bounded in the √ order of g The relevance of Conjecture 0.2 is enhanced by the following result of A. Hirschowitz and S. Ramanan (cf. [HiR, Th. 1.1]). 0.3 (Hirschowitz-Ramanan) For odd g = 2k + 1 ≥ 5, Green’s conjecture holds for the generic curve if and only if it holds for all curves of (maximal) Clifford index k. THEOREM

We have to content ourselves with the following. 0.4 If 3 p ≤ g − 2, then the generic curve of genus g satisfies N p . THEOREM

In fact, we can prove slightly more. The generic ( p + 3)-gonal curve satisfies N p . The previously known result in [S2] gave a bound g ≥ p 2 + p. Our approach is as follows: define a vector bundle E as the dual of the kernel of the evaluation map of the canonical linear series. Definition 0.5 The vector bundle E is the dual of the kernel of the following exact sequence: ∗ 0 → E ∗ → H 0 (C, K C ) ⊗ OC → K → 0.

GREEN’S CONJECTURE FOR GENERIC CURVE

197

From a result of K. Paranjape and Ramanan (cf. [PR, Rem. 2.8, p. 507]), Green’s conjecture follows from the surjectivity of the maps ∗ ∧r H 0 (C, K C ) → H 0 (∧r E), r ≤ Cliff(C). We expect this to be the case for C generic. To this end, consider a hyperelliptic curve C0 . Notice that the map ∗ H 0 (C, K C0 ) ⊗ OC0 → E → 0 identifies (H 0 (C, K C0 ))∗ to a subspace W of H 0 (C0 , E). As C0 is hyperelliptic, this subspace is proper. We start by computing W and the image W r ⊂ H 0 (C0 , ∧r E) of its exterior powers ∧r W . Every infinitesimal deformation of the curve C0 determines a unique infinitesimal deformation of E which preserves it as the dual of the kernel of the canonical evaluation map. Consider the deformation of ∧r E that this induces. We then look at the sections of H 0 (∧r E) that give rise to sections of the infinitesimal deformation of ∧r E . For a deformation corresponding to a k-gonal curve, we expect that for r ≤ k − 2 these sections are those in W r .

1. Identification of the vector bundle E and its space of sections Notation. In this section and in Section 3, C = C0 denotes a hyperelliptic curve, and L denotes the unique line bundle of degree 2 with two sections. If F is a vector bundle on a curve C, we write H i (F) for H i (C, F) if there is no danger of confusion. If several curves are involved or if we are considering sections on an open set only, we make this clear. As in the previous section, E denotes the vector bundle defined in Definition 0.5, W = W 1 is the image of H 0 (K )∗ ⊂ H 0 (E), and W r is the image of the exterior powers of W in H 0 (∧r E). 1.1 Define the map π : C0 → P1 associated to L. Then E is a direct sum of g − 1 copies of L. More concretely, let H be the space of sections of L. Then PROPOSITION

E∼ = S g−2 H ∗ ⊗ ∧2 H ∗ ⊗ L . Moreover, the map (H 0 (K ))∗ → H 0 (E) can be identified to the natural inclusion ϕ1 : S g−1 H ∗ → S g−2 H ∗ ⊗ ∧2 H ∗ ⊗ H. Proof Note that the canonical sheaf K C on C is of the form K = L ⊗g−1 = π ∗ OP1 (g − 1)

198


and that H 0 (C, K C ) = S g−1 (H ) = π ∗ H 0 (P1 , OP1 (g − 1)) . Therefore, the exact sequence defining E ∗ is the pullback of the exact sequence in P1 , 0 → Ker → H 0 P1 , OP1 (g − 1) ⊗ OP1 → OP1 (g − 1) → 0.

(*)

Denote by H¯ the space of sections H 0 (P1 , OP1 (1)). Tensoring (*) with O (1) and taking global sections, one obtains 0 → H 0 Ker ⊗O (1) → S g−1 H¯ ⊗ H¯ → S g H¯ → 0. Hence, H 0 (Ker ⊗O (1)) ≡ S g−2 H¯ ⊗ ∧2 H¯ (cf., e.g., [FH, Ex. 15.20, p. 224]). Using the exact sequence (*), one checks that Ker is a direct sum of line bundles of degree −1. Hence, Ker = S g−2 H¯ ⊗∧2 H¯ ⊗ O (−1). As E ∗ = π ∗ (Ker), H = π ∗ H¯ , the result follows. PROPOSITION 1.2 Let the notation be as above. Choose r ≤ g − 1. Then

∧r E = ∧r (S g−2 H ∗ ⊗ ∧2 H ∗ ) ⊗ L ⊗r . Moreover, the map ∧r W → H 0 (∧r E) can be identified to ϕr : ∧r (S g−1 H ∗ ) → ∧r (S g−2 H ∗ ⊗ ∧2 H ∗ ) ⊗ Sr H and is an immersion. In particular, its image W r has dimension gr . Proof From the exact sequence defining E (cf. Definition 0.5 ), we obtain the following exact sequence: 0 → ∧r −1 E ⊗ K ∗ → ∧r W ⊗ OC → ∧r E → 0. Note that the slope µ(∧r −1 E ⊗ K ∗ ) = µ(L r −1 ⊗ K ∗ ) = 2(r − 1) − 2(g − 1) < 0. Hence, by the semistability of ∧r −1 E ⊗ K ∗ , h 0 (∧r −1 E ⊗ K ∗ ) = 0. Then, taking cohomology in the sequence above, we obtain the injectivity of the map ∧r W → H 0 (∧r E). The map ϕ1 : S g−1 H ∗ → S g−2 H ∗ ⊗ ∧2 H ∗ ⊗ H


199

is obtained as follows. Consider the natural map t, t : ∧2 H ⊗ H ∗ → H, a ∧ b ⊗ c → c(a)b − c(b)a. Tensoring with S g−2 H and composing with the natural cup-product map cup, we obtain S g−2 H ⊗ ∧2 H ⊗ H ∗ → S g−2 H ⊗ H → S g−1 H. Define ϕ1 as the dual of the above composition. We give next a description of this map in coordinates. Let 1, x be a basis for H , and let 1, y be the dual basis for H ∗ . Then t (x ∧ 1 ⊗ y) = 1,

t (x ∧ 1 ⊗ 1) = −x.

For any positive integer k, we can identify S k H with the space of polynomials of degree at most k in a variable x, and S k H ∗ with the space of polynomials of degree at most k in one variable y = x ∗ . Identify ∧2 H ∗ → k by the isomorphism y ∧ 1 → 1. Then cup(Id ⊗t)(x k ⊗ x ∧ 1 ⊗ y) = cup(x k ⊗ 1) = x k , cup(Id ⊗t)(x k ⊗ x ∧ 1 ⊗ 1) = cup x k ⊗ (−x) = −x k+1 . Hence, by duality, ϕ1 acts by ϕ1 (y k ) = y k ⊗ y ∧ 1 ⊗ x − y k−1 ⊗ y ∧ 1 ⊗ 1 with the convention that on the right-hand side y g−1 = 0, y −1 = 0. With the identification of ∧2 H ∗ with k, this can be written as ϕ1 (y k ) = y k ⊗ x − y k−1 ⊗ 1. By taking wedge products of this map, one then sees that X (−1)1 +···+r y k1 −1 ∧ · · · ∧ y kr −r ⊗ x r −(1 +···+r ) , ϕr (y k1 ∧ · · · ∧ y kr ) = 0≤i ≤1

again with the convention that on the right-hand side y g−1 = 0, y −1 = 0. A basis of ∧r (S g−1 H ∗ ) consists of the elements y k1 ∧ · · · ∧ y kr , 0 ≤ k1 < · · · < kr ≤ g − 1. The images of these elements are linearly independent as their leading terms (i.e., the terms of highest degrees on x and y jointly) obviously are. This gives an alternative proof of the result.

200


2. General setup for infinitesimal deformations Recall 2.1 We recall the basic setup for deformations of a curve, a vector bundle, and its space of sections. Write k[t]/t 2 = k . By an infinitesimal deformation of the curve C we mean a curve C over Spec k with central fiber C. Similarly, by an infinitesimal deformation of a vector bundle F we mean a vector bundle over C × Spec k with central fiber F. By an infinitesimal deformation of the pair we mean a curve C and a vector bundle E over C . Recall that the set of infinitesimal deformations of the curve C can be parametrised by H 1 (C, TC ) and the set of infinitesimal deformations of the vector bundle F can be parametrised by H 1 (F ∗ ⊗ F), while the set of infinitesimal deformations of a pair consisting of a curve C and a vector bundle F on C can be parametrised by H 1 (6 F ), where 6 F denotes the sheaf of first-order differential operators acting on F. We describe next the correspondence between these objects (cf. [W, proof of Prop. 1.2 ] and also [BiR, proof of Th. 2.3]). Assume as given an element ν ∈ H 1 (TC ). We think of the sections of the sheaf TC over an open set U as the set of (k-linear)maps OU → OU satisfying ν( f g) = ν( f )g + f ν(g). Take an affine open cover C = ∪ Ui . Write Ui j for Ui ∩ U j . Represent ν by a cocycle ν = (νi j ), νi j ∈ H 0 (Ui j , TC ). We associate to ν the following deformation of C. Consider the trivial deformations of the Ui , namely, Ui × Spec k . Glue them along the intersections Ui j × Spec k using the matrices Id 0 . νi j Id The correspondence νi j → C obtained in this way is a bijection. Assume now as given an element ϕ ∈ H 1 (F ∗ ⊗ F). Represent it by a cocycle (ϕi j ) with ϕi j ∈ H 0 (Ui j , Hom(F, F)). Consider the trivial extension of F to Ui × Spec k , namely, FUi ⊕ FUi . Take gluings on Ui j given by Id 0 . ϕi j Id This gives the correspondence between H 1 (F ∗ ⊗ F) and deformations of F. Assume now that a section s of F can be extended to a section s of the deformation. Then there exist local sections si0 ∈ H 0 (Ui , F|Ui ) such that (s|Ui , si ) define a section of F . By construction of F , this means that ! s|U j Id 0 s|Ui = . s 0j ϕi j Id si0


201

This can be written as ϕi j (s) = s 0j − si0 . Equivalently, ϕi j ∈ Ker H 1 (F ∗ ⊗ F) → H 1 (F) , νi j → νi j (s). This result can be formulated using the language of Brill-Noether theory; the set of infinitesimal deformations of the vector bundle F which have sections deforming a certain subspace V ⊂ H 0 (F) consists of the orthogonal to the image of the Petri map PV : V ⊗ H 0 (K ⊗ F ∗ ) → H 0 (K ⊗ F ⊗ F ∗ ).

(2.1.1)

Assume now as given an element σ ∈ H 1 (6 F ). We think of 6 F (U ) as the set of additive morphisms σ : F(U ) → F(U ) such that, for a suitable element νσ ∈ TC , σ ( f s) = νσ ( f )s + f σ (s). Represent σ by a cocycle σ = (σi j ), σi j ∈ H 1 (Ui j , 6 F ). Consider the associated element (νi j ) ∈ H 1 (TC ) and the corresponding deformation C of C. Take then the vector bundle on C obtained by gluing the trivial extensions of F on Ui by means of the matrices Id 0 . σi j Id As in the case of deforming the vector bundle alone, deformation of sections is easy to interpret; the set of infinitesimal deformations of the pair (C, F) that have sections deforming a certain subspace V ⊂ H 0 (F) consists of the orthogonal to the image of the Petri map P¯V : V ⊗ H 0 (K ⊗ F ∗ ) → H 0 (K ⊗ 6 ∗F )

(2.1.2)

defined as the dual of the natural cup-product map H 1 (6 F ) → Hom V, H 1 (F) . Consider the exact sequence 0 → F ∗ ⊗ F → 6 F → TC → 0. The canonical map π : 6 F → TC is defined by π(σ ) = νσ . The map i : F ∗ ⊗ F → 6 F sends an element of F ∗ ⊗ F (considered as an endomorphism of F) to itself. One obtains a commutative diagram 0 → 0 →

H 1 (F ∗ ⊗ F) → H 1 (6 F ) → H 1 (TC ) 0 ∗ ∗ ↓ PV ↓ P¯V ↓ PV∗ ImP ∗ → Hom V, H 1 (F) → (KerP)∗

→ 0 → 0 (2.1.3)

202


and its dual (cf. [AC, p. 18]) H 0 (2K ) ← 0 ↑ PV0 0 ← KerP ← 0 (2.1.4) When V = H 0 (F), we write PF , PF0 instead of PV , PV0 . When V and F are clear, we suppress them from the notation. We later use the following result. Its proof appears in [T, Lem. 2.12]. 0 ←

H 0 (K ⊗ F ⊗ F ∗ ) ← ↑ ImP ←

H 0 (K ⊗ (6 F )∗ ) ← ↑ P¯V V ⊗ H 0 (K ⊗ F ∗ ) ←

LEMMA 2.2 Let M be a line bundle on a curve C with two independent sections s0 , s1 and such that |K ⊗ M −2 | has a section t. Denote by D1 the fixed part of the series determined by s0 , s1 , and denote by D2 the divisor corresponding to the section t. Denote by R the ramification divisor of the map C → P1 associated to the series hs0 , s1 i. Then 0 (s ⊗ ts − s ⊗ ts ) corresponds to the divisor 2D + D + R. In particular, it PM 0 1 1 0 1 2 is nonzero. LEMMA 2.3 n F is a direct sum of vector bundles. Then Assume that F = ⊕i=1 i

6 F = ⊕TC 6 Fi ⊕ [⊕i6= j Fi∗ ⊗ F j ]. Here ⊕TC 6 Fi denotes the fibered sum over TC of the 6 Fi . Proof Consider an open set U . Let σ : F(U ) → F(U ) be a first-order differential operator acting on F. Using the decomposition of F as a direct sum, σ admits a representation as a matrix (σi j ), where σi j : Fi → F j . Take a local section sk ∈ Fk (U ). One then checks that σ (0, . . . , 0, f sk , 0, . . . , 0) = σ1k ( f sk ), . . . , σnk ( f sk ) . Using the fact that σ (0, . . . , 0, f sk , 0, . . . , 0) = f σ (0, . . . , 0, sk , 0, . . . , 0) + ν( f )(0, . . . , 0, sk , 0, . . . 0), we find that σkk is a first-order differential operator corresponding to the same ν as σ , while σik is OC -linear if i 6= k.


203

LEMMA 2.4 n F is a direct sum of vector bundles. Denote by P , P¯ , P 0 the Assume that F = ⊕i=1 i i i i Petri maps corresponding to the Fi . Then PF0 can be obtained as the composition n Ker P → ⊕i=1 Ker Pi0 → H 0 (2K ),

where the first map is the projection and the second map is (1/n) ⊕ Pi0 . Proof Consider the right-hand square in (2.1.3) for each one of the Fi . As in Lemma 2.3, consider the fibered sum of the 6 Fi over TC . One then has a commutative square ⊕ H 1 (TC ) H 1 (6 Fi ) → H 1 (TC ) n P¯ ∗ n P 0∗ ↓ ⊕i=1 ↓ ⊕i=1 i i n Hom H 0 (F ), H 1 (F ) n (Ker P )∗ ⊕i=1 → ⊕i=1 i i i Take also the corresponding square for F, H 1 (6 F ) → H 1 (TC ) 0 ¯ ↓ P∗ ↓P∗ Hom H 0 (F), H 1 (F) → (Ker P)∗ Consider the cube that has these diagrams as back and front faces, respectively. We define four maps in the side edges. The maps ⊕ H 1 (TC ) H 1 (6 Fi ) → H 1 (6 F ) and n ⊕i=1 Hom H 0 (Fi ), H 1 (Fi ) → Hom H 0 (F), H 1 (F) are natural diagonal injections (with zeros on the terms corresponding to a pair Fi , F j , i 6= j). Notice that from Lemma 2.3 the first map is well defined. With these definitions the left-hand square commutes. The map n ⊕i=1 (Ker Pi )∗ → (Ker P)∗ is defined as the dual of the natural projection. By dualisation, one can then check that the bottom face commutes. If we take as the fourth map the homothety ×n : H 1 (TC ) → H 1 (TC ), then the top face commutes too. As H 1 (6 F ) → H 1 (TC ) is onto, this shows that the right-hand square commutes. Dualising this square, one obtains the result in Lemma 2.4.

204


Recall 2.5 We want to study next the infinitesimal deformations of a hyperelliptic curve that correspond to loci of (k = r + 2)-gonal curves. Consider a gk1 on a hyperelliptic curve C, k ≤ g − 1. Such a linear series can be written as its fixed part D plus its mobile part V , where D is a divisor of degree k − 2 j and V is a two-dimensional subspace of the space of sections of jg21 , 1 ≤ j ≤ k/2. Let i be the hyperelliptic involution. ¯ corresponding to D, i(D), respectively. Let Denote by t, t¯ sections of O (D), O ( D) 1 s0 , s1 be sections of jg2 generating V . Let PV be the Petri map corresponding to this gk1 . By the base-point free pencil trick, the kernel of PV can be identified to the space of sections of (g − 1 − k)g21 as follows: (g − 1 − k)g21 → Ker PV , s → ts0 ⊗ t¯s1 s − ts1 ⊗ t¯s0 s. The image in H 0 (2K ) of this element is D + i(D) + R V + Ds , where Ds is the divisor associated to the section s and R V denotes the ramification divisor corresponding to V . Notice that R V = R + ϕ ∗ (RP1 ), where RP1 is the ramification divisor of a g 1j in P1 , ϕ : C → P1 is the map associated to the g21 , and R is the ramification divisor of ϕ. Hence, all these linear series have R as a fixed divisor and contain an additional fixed divisor that can be written as D 0 + i(D 0 ) with D 0 of degree k − 2. The orthogonal in H 1 (TC ) to one such subspace is a vector space of dimension 2g + k − 3. As the divisors D 0 depend on k − 2 j parameters, their union fills a subvariety of dimension at most 2g + 2k − 5 . If k < (g + 2)/2, this is a proper subspace of H 1 (TC ). In particular, this gives another proof of the well-known fact that a generic curve is not k-gonal for k < (g + 2)/2. We now want to fix for every k < (g+2)/2 one particular direction corresponding to a k-gonal curve that is not (k − 1)-gonal. Giving a direction in H 1 (TC ) is equivalent to giving a hyperplane in H 0 (2K ). Also, as we are only considering linear series in H 0 (2K − R), it is enough for our purposes to give the trace of the hyperplane with H 0 (2K − R). The hyperelliptic directions are those that deform the g21 , hence, those that contain H 0 (2K − R). Any nonhyperelliptic direction gives rise to a proper hyperplane in H 0 (2K − R). Choose a basis of the space of sections of the g21 as 1, x, vanishing at infinity and zero, respectively. Then a basis for (g − 3)g21 is given by 1, x, . . . , x g−3 . We want to choose a hyperplane in (g − 3)g21 corresponding to a k-gonal curve that is not (k − 1)gonal. This is equivalent to saying that the hyperplane contains a subspace of the form Dk−2 +(g −k −1)g21 but does not contain any subspace of the form Dk−3 +(g −k)g21 . Take the hyperplane to be the subspace of codimension 1 of (g − 3)g21 with basis the monomials in x except for x k−3 . Then this hyperplane contains the linear series of divisors that vanish with multiplicity k − 2 at zero but does not contain any complete linear series with a fixed divisor of degree k − 3.


205

3. Deformations of E We apply the setup of the previous section to the hyperelliptic curve C0 and the vector bundles ∧r E. As in Section 1, L denotes the hyperelliptic line bundle (i.e., the line bundle on C of degree 2 with two sections). We assume in all that follows that r ≤ g − 1 − r. PROPOSITION 3.1 The Petri map PH 0 (∧r E) (cf. (2.1.2)) associated to the vector bundle ∧r (E) gives by restriction an isomorphism PW r : W r ⊗ H 0 K ⊗ (∧r E)∗ → H 0 K ⊗ ∧r E ⊗ (∧r E)∗ .

Proof From 1, tensoring with K ⊗ ∧r E, one obtains the following exact sequence: 0 → ∧r −1 E ⊗K ∗ ⊗K ⊗(∧r E)∗ → ∧r W ⊗K ⊗(∧r E)∗ → ∧r E ⊗K ⊗(∧r E)∗ → 0. As ∧r −1 E ⊗ K ∗ ⊗ K ⊗ (∧r E)∗ = ∧r −1 E ⊗ (∧r E)∗ has negative slope and is semistable, it has no sections. Hence, taking cohomology above, the result follows. We give next a second proof that allows us to deal with the subject in coordinates. From Proposition 1.2, ∧r E = ∧r (S g−2 H ∗ ⊗ ∧2 H ∗ ) ⊗ L r , so one has ∧r E ∗ = ∧r (S g−2 H ⊗ ∧2 H ) ⊗ L −r . Hence, H 0 (∧r E) = ∧r (S g−2 H ∗ ⊗ ∧2 H ∗ ) ⊗ Sr H and H 0 (K ⊗ ∧r E ∗ ) = ∧r (S g−2 H ⊗ ∧2 H ) ⊗ S g−1−r H. Then the Petri map PH 0 (∧r E) and its restriction PW r to W r ⊗ H 0 (K ⊗ (∧r E)∗ ) can be written as the tensor product with the vector space U = ∧r (S g−2 H ⊗ ∧2 H ) of the diagram ∧r (S g−2 H ∗ ⊗ ∧2 H ∗ ) ⊗ Sr H ⊗ S g−1−r H ↑ r W ⊗ S g−1−r H

→ ∧r (S g−2 H ∗ ⊗ ∧2 H ∗ ) ⊗ S g−1 H ↑ r g−2 ∗ → ∧ (S H ⊗ ∧2 H ∗ ) ⊗ S g−1 H

Therefore, it is enough to show that the map pW r in the lower row of this diagram is an isomorphism. Recall that ϕr identifies ∧r S g−1 H ∗ to its image W r (where ϕr is the natural immersion ∧r S g−1 H ∗ → ∧r (S g−2 H ∗ ⊗∧2 H ∗ )⊗Sr H defined in Proposition

206


1.2). The natural product map Sr H ⊗ S g−1−r H → S g−1 H can be identified to PL r . Then pW r = (Id ⊗PL r )o(ϕr ⊗Id). We show that pW r is an isomorphism by exhibiting its inverse qW r . Using the notation in Proposition 1.2 and the identification ∧2 H ∗ ∼ = k, one obtains pW r (y k1 ∧ · · · ∧ y kr ⊗ x a ) = (Id ⊗PL r )o(ϕr ⊗ Id)(y k1 ∧ · · · ∧ y kr ⊗ x a ) X = Id ⊗PL r (−1)1 +···+r y k1 −1 ∧ · · · ∧ y kr −r ⊗ x r −(1 +···+r ) ⊗ x a 0≤i ≤1

=

X

(−1)1 +···+r y k1 −1 ∧ · · · ∧ y kr −r ⊗ x a+r −(1 +···+r )

0≤i ≤1

with the convention that on the right-hand side y −1 = 0, y g−1 = 0. We describe qW r as follows. Assume as given integers 0 ≤ j1 < · · · < jr ≤ g − 2, 0 ≤ b ≤ g − 1. There is then a value l with 0 ≤ l ≤ r such that jl + 1 ≤ b ≤ jl+1 . Define then X qW r (y j1 ∧ · · · ∧ y jr ⊗ x b ) = (−1)r −l y j1 −t1 ∧ · · · ∧ y jl −tl ∧ y jl+1 +sl+1 ∧ · · · ∧ y jr +sr ⊗ x b−r −t1 −···−tl +sl+1 +···+sr , where the sumation extends to the indices 0 ≤ ti ≤ ji − ji−1 − 1,

1 ≤ si ≤ ji+1 − ji

with the conventions jr +1 = g − 1, j−1 = −1. Notice that the map qW r is well defined as 0 ≤ b − r − t1 − · · · − tl + sl+1 + · · · + sr ≤ g − 1 − r and 0 ≤ j1 − t1 < · · · < jl − tl < jl+1 + sl+1 ≤ · · · ≤ jr + sr ≤ g − 1. We check next by direct computation that the composition pW r oqW r = Id. As the two vector spaces involved have the same dimension, this suffices in order to prove the isomorphism: pW r qW r (y j1 ∧ · · · ∧ y jr ⊗ x b ) X = pW r (−1)r −l y j1 −t1 ∧ · · · ∧ y jl −tl 0≤ti ≤ ji − ji−1 −1,1≤si ≤ ji+1 − ji

∧ y jl+1 +sl+1 ∧ · · · ∧ y jr +sr ⊗ x b−r −t1 −···−tl +sl+1 +···+sr X X (−1)r −l (−1)1 +···+r y j1 −t1 −1 ∧ · · · ∧ y jl −tl −l = 0≤ti ≤ ji − ji−1 −1,1≤si ≤ ji+1 − ji

0≤i ≤1

∧ y jl+1 +sl+1 −l+1 ∧ · · · ∧ y jr +sr −r ⊗ x b−r −t1 −···−tl +sl+1 +···+sr −1 −···−r . Note that for a fixed value of t2 , . . . , tl , sl+1 , . . . , sr , 2 , . . . , r the terms corresponding to t1 and 1 = 1 and to t10 = t1 + 1 and 1 = 0 cancel each other. The


207

only terms that then remain are those with maximum t1 and 1 = 1 or with minimum t1 and 1 = 0. The former correspond to y j1 − j1 −1 = y −1 , and so they are zero by convention. The latter correspond to y j1 . Hence, any term that remains in the sum has j1 as the first exponent for y. Similarly, for one of these fixed values of t1 , 1 , and arbitrary values of t3 , . . . , tl , sl+1 , . . . , sr , 3 , . . . , r , the terms corresponding to t2 , 2 = 1 and t20 = t2 + 1, 2 = 0 cancel each other. The only terms that then remain are those with maximum t2 and 2 = 1 or with minimum t2 and 2 = 0. The former correspond to y j2 −( j2 − j1 −1)−1 = y j1 . As we know that the first exponent of y is j1 , we get y j1 ∧ y j1 ∧ · · · = 0. The case of minimum t2 , 2 = 0 corresponds to y j1 ∧ y j2 ∧ · · · . Continuing in this way, we see that we only need to consider terms of the form y j1 ∧ · · · ∧ y jl ∧ · · · . Start then at the other end. Reasoning as above, one sees that the only term that stays for the last coefficient is y jr . After this, one checks successively that the only term that remains in the (r − 1) place is y jr −1 ; in the (l + 1) place, y jl+1 . COROLLARY 3.2 For any given infinitesimal deformation ν of the curve C0 , there is a unique infinitesimal deformation σ of the pair consisting of C0 and the vector bundle ∧r E such that W r can be extended to a space of sections of the deformation.

Proof Consider the Petri map PW r : W r ⊗ H 0 (K ⊗ F ∗ ) → H 0 (K ⊗ F ⊗ F ∗ ). Consider the commutative diagram (2.1.3) in the case when F = ∧r E, V = W r : 0

→

0

→

H 1 (∧r E)∗ ⊗ ∧r E ↓ P∗ ImP ∗

→ →

H 1 (6∧r E ) ↓ P¯ ∗ Hom W r , H 1 (∧r E)

→ →

H 1 (TC ) 0 ↓ P∗ (KerP)∗

→

0

→

0

From Proposition 3.1, P ∗ is an isomorphism and Ker P = 0. Hence, every element in H 1 (TC ) has a unique inverse image in H 1 (6 F ) that belongs to the kernel of P¯ ∗ . This proves the result. COROLLARY 3.3 For any given infinitesimal deformation ν of the curve C0 , the unique infinitesimal deformation σ of the vector bundle ∧r E that preserves it as the exterior power of the dual of the kernel of the evaluation map is the σ above.

208


Proof An infinitesimal deformation of E as the dual of the kernel of the evaluation map preserves the subspace W = W 1 as a space of sections. Hence, an infinitesimal deformation of ∧r E as the exterior product of this dual preserves W r as a space of sections. By the unicity of such a deformation, the result follows. THEOREM 3.4 Take ν an infinitesimal deformation of C0 corresponding to a generic (r + 2)-gonal curve. Let σ be the deformation of ∧r E associated to ν as in Corollary 3.3. Assume 3r ≤ g + 1. Let Wˆ be a subspace of H 0 (∧r (E)) which strictly contains W r . Then Wˆ does not extend to a space of sections of the infinitesimal deformation of ∧r E corresponding to σ .

Proof It is enough to prove the result when Wˆ has dimension a = dimW r + 1. Consider the diagram (2.1.4) for the case F = ∧r E, V = Wˆ . We obtain 0

←

H 0 K ⊗ (∧r E) ⊗ (∧r E)∗ ↑

0

←

Im P

←

H 0 K ⊗ (6∧r E )∗ ↑ P¯Wˆ

←

Wˆ ⊗ H 0 K ⊗ (∧r E)∗

←

H 0 (2K ) ↑ P 0ˆ

←

0

←

Ker PWˆ

←

0

W

Define P 0 = PH0 0 (∧r (E)) . Write U = ∧r (S g−2 H ∗ ⊗ ∧2 H ∗ ). We make the identification Ker(P 0 ) ≡ U ⊗ Sr −1 H ⊗ S g−2−r H ⊗ U ∗ by using the exactness of the sequence below after tensoring with U and U ∗ : 0 → Sr −1 H ⊗ S g−2−r H → Sr H ⊗ S g−1−r H → S g−1 H → 0. Here the map i : Sr −1 H ⊗ S g−2−r H → Sr H ⊗ S g−2−r H is given in coordinates by i(x a ⊗ x b ) = x (a+1) ⊗ x b − x a ⊗ x b+1 . The map from Ker P 0 to H 0 (2K ) is now easy to describe using Lemma 2.2. Identify H 0 (2K − R) = S g−3 H . Consider the natural cup-product map Sr −1 H ⊗ S g−2−r H → S g−3 H.


209

Take the tensor product with U and U ∗ , and get p¯ : U ⊗ Sr −1 H ⊗ S g−2−r H ⊗ U ∗ → U ⊗ S g−3 H ⊗ U ∗ . Compose this with the cup product with the Id in U , πId : U ⊗ S g−3 H ⊗ U ∗ → S g−3 H and, finally, the inclusion of H 0 (2K − R) → H 0 (2K ) to obtain the desired map. As W r ⊗ S g−1−r H ⊗ U ∗ is identified with the image of the Petri map, the quotient U ⊗ Sr H ⊗ S g−1−r H ⊗ U ∗ Wr is identified to Ker P. A subspace of H 0 (∧r E) containing W r and of dimension dim W r + 1 gives rise to an element q∈

U ⊗ Sr H . Wr

Fix an (r + 2)-gonal not (r + 1)-gonal direction. We want to show that for every q the image of the subspace corresponding to q is not contained in the corresponding direction. There is a unique map ψ¯ : U ⊗ Sr H ⊗ S g−1−r H ⊗ U ∗ → U ⊗ Sr −1 H ⊗ S g−2−r H ⊗ U ∗ ¯ = Id and ψ¯ vanishes on W r ⊗ S g−1−r H ⊗ U ∗ . The map ψ¯ can be such that ψi obtained by tensoring with IdU ∗ from a map ψ : U ⊗ Sr H ⊗ S g−1−r H → U ⊗ Sr −1 H ⊗ S g−2−r H. The equations of ψ can be given in coordinates. Choose a basis element in U ⊗ Sr H ⊗ S g−1−r H . Write it as y j1 ∧· · ·∧y jr ⊗x a ⊗x b . Let k be such that jk +1 ≤ a+b ≤ jk+1 . Here we take j0 = −1, jr +1 = g − 1. Define ji0 = ji−1 ,

ji00 = ji ,

i ≤ k;

ji0 = ji ,

ji00 = ji+1 ,

i > k.

210


Also, for another set of subindices 0 ≤ t1 < · · · < tr ≤ g − 2, define λ1( ji ) (ti ) = k − o ti < ji , i ≤ k + o ti = ji+1 , k ji , i > k − o ti = ji−1 , 1 < i ≤ k . We drop the (ti ) and/or the ( ji ) from the notation when these are clear. Then, using again the convention y −1 = 0, y g−1 = 0, ψ(y j1 ∧ · · · ∧y jr ⊗ x a ⊗ x b ) X =

(−1)c−k

ji0 ≤ti ≤ ji00 , λ1( j) (t)≤c≤λ2( j) (t)

2 λ − λ1 c − λ1 P

P

× y t1 ∧ · · · ∧ y tr ⊗ x c ⊗ x b+a−c−1+ ti − ji X − y j1 ∧ · · · ∧ y jr ⊗ x c ⊗ x b+a−c−1 a≤c≤k−1

X

+

y j1 ∧ · · · ∧ y jr ⊗ x c ⊗ x b+a−c−1 .

k≤c≤a−1

Here the first summation excludes the term (t1 , . . . , tr ) = ( j1 , . . . , jr ). Let us check first that the indices that appear are in the allowable ranges, namely, X X 0 ≤ c ≤ r − 1, 0 ≤ b + a − c + ti − ji − 1 ≤ g − 2 − r. As λ1 ≤ c ≤ λ2 , the condition for c is equivalent to λ1 ≥ 0, λ2 ≤ r − 1, and these follow from the definitions of λ1 , λ2 . Now from c ≥ λ1 , b + a ≤ jk+1 , it follows that X X X b+a−c+ ti − ji − 1 ≤ jk+1 − λ1 + (ti − ji ) − 1. Using the definition of λ1 , this quantity equals X jk+1 − k + o ti < ji |i ≤ k − o ti = ji+1 |k k

As ti ≤ ji if i ≤ k, then X o ti < ji |i ≤ k + (ti − ji ) ≤ 0. i≤k

Hence, b + a − c +

P

ti −

P

ji − 1 ≤ g − 2 − r .


211

Similarly, from c ≤ λ2 , b + a ≥ jk + 1, it follows that X X X b+a−c+ ti − ji − 1 ≥ jk − λ2 + (ti − ji ). Using the definition of λ2 , this quantity equals X jk − k + 1 − o ti > ji |i > k + o ti = ji−1 |1 k, then o{ti > ji |i > k} +

X (ti − ji ) ≥ 0. i>k

P

P

Hence, b + a − c + ti − ji − 1 ≥ 0. In order to check that the definition of ψ is correct, it suffices to show the following two things. CLAIM 1 The map ψ satisfies the following property:

ψ y j1 ∧ · · · ∧ y jr ⊗ (x a+1 ⊗ x b − x a ⊗ x b+1 ) = y j1 ∧ · · · ∧ y jr ⊗ x a ⊗ x b . CLAIM 2 The map ψ satisfies the following property: X ψ (−1)1 +···+r y j1 −1 ∧ · · · ∧ y jr −r ⊗ x r −(1 +···+r ) ⊗ x b = 0. 0≤i ≤1

Proof of Claim 1 As (a + 1) + b = a + (b + 1) and the j1 , . . . , jr are fixed, the value of k in the definition of ψ is the same for y j1 ∧ · · · ∧ y jr ⊗ x a+1 ⊗ x b and y j1 ∧ · · · ∧ y jr ⊗ x a ⊗ x b+1 .

212


Hence, the first large summand in the definition of ψ is the same for both. Then ψ y j1 ∧ · · · ∧ y jr ⊗ (x a+1 ⊗ x b − x a ⊗ x b+1 ) X =− y j1 ∧ · · · ∧ y jr ⊗ x c ⊗ x b+a−c a+1≤c≤k−1

+

X

y j1 ∧ · · · ∧ y jr ⊗ x c ⊗ x b+a−c

k≤c≤a

+

X

y j1 ∧ · · · ∧ y jr ⊗ x c ⊗ x b+a−c

a≤c≤k−1

−

X

y j1 ∧ · · · ∧ y jr ⊗ x c ⊗ x b+a−c .

k≤c≤a−1

If k ≤ a, the first and third terms in the above expression do not appear, while the difference between the second and fourth terms is y j1 ∧ · · · ∧ y jr ⊗ x a ⊗ x b as needed. If k > a, the second and fourth terms in the above expression do not appear, while the difference between the third and first is the same as before. This finishes the proof of Claim 1 Proof of Claim 2 We distinguish several cases. Assume first that some ti (say, ti0 ) is different from ji , ji − 1. Consider the coefficient of (t1 , . . . , tr ) in the two sets of indices given by ( j1 − 1 , . . . , ji0 , . . . , jr − r ) and ( j1 − 1 , . . . , ji0 − 1, . . . , jr − r ) for fixed values of 1 , . . . , r . If both indices give the same value of k, then the λ’s are also equal. Hence, y t1 ∧ · · · ∧ y tr appears in both sets of indices with the same coefficients, and the corresponding expressions cancel each other. If the k is not the same for both expressions, there is an l, l 6= i 0 , such that jl − l + 1 = r + b − 1 − · · · − r . Assume, for example, i 0 < l. (The case i 0 > l would be completely analogous.) We further regroup the indices j’s by considering the following set of four: (1) = ( j1 − 1 , . . . , ji0 , . . . , jl , . . . , jr − r ), (2) = ( j1 − 1 , . . . , ji0 , . . . , jl − 1, . . . , jr − r ), (3) = ( j1 − 1 , . . . , ji0 − 1, . . . , jl , . . . , jr − r ), (4) = ( j1 − 1 , . . . , ji0 − 1, . . . , jl − 1, . . . , jr − r ).

The first two sets of indices give k = l, while the second two give k = l − 1.


213

If tl = jl , let λ1 = λ1(1) , λ2 = λ2(1) denote the lambdas corresponding to (1). Then this value of t’s does not appear in the expression of ψ for (2). In (3) the values are λ1(3) = λ1 − 1, λ2(3) = λ2 − 1, and in (4) they are λ1(4) = λ1 − 1, λ2(4) = λ2 . For a given value of c the sum of the signed coefficients is 2 2 2 λ − λ1 λ − λ1 λ − λ1 + 1 + − = 0. c − λ1 c − λ1 + 1 c − λ1 + 1 Similarly, if tl = jl − 1, the λ’s are, respectively, λ1(1) = λ1 , λ2(1) = λ2 , λ1(2) = λ2(2) = λ2 , λ1(4) = λ1 , λ2(4) = λ2 − 1, while this set of indices does not appear in (3). Again, the sum of corresponding coefficients is zero. If tl 6= jl , jl−1 , the lambdas satisfy λ1 + 1,

λ1(1) = λ1(2) ,

λ2(1) = λ2(2)

λ1(3) = λ1(4) ,

λ2(3) = λ2(4) .

Hence, the corresponding coefficients cancel each other. Assume now that each ti equals either ji or ji−1 . We use induction on r . The case r = 2 can be checked by hand. Assume first that for some i, ti = ji − 1. We can take i maximal so that ti+1 = ji+1 , . . . , tr = jr . We want to compare the coefficients corresponding to the (r − 1)index (0) = ( j1 − 1 , . . . , ji−1 − i−1 , ji+1 − i+1 , . . . , jr − r ) with those of (1) = ( j1 − 1 , . . . , ji−1 − i−1 , ji − 1, ji+1 − i+1 , . . . , jr − r ) and (2) = ( j1 − 1 , . . . , ji−1 − i−1 , ji , ji+1 − i+1 , . . . , jr − r ). P There is an l such that jl − l + 1 ≤ r − 1 − i ≤ jl+1 − l+1 . If l ≤ i, then in (0), k = l. Denote by λ1 = λ1(0) , λ2 = λ2(0) the values of the lambdas in (0). Then in (1) they are again k = l, λ1(1) = λ1 , λ2(1) = λ2 , while these values of t do not appear in the expression of ψ for (2). Hence, the coefficients in (0) and (1) for a fixed value of c are the same and (2) can be ignored. P If l > i, then in (0), k = l − 1. We distinguish two cases. If r − 1 − i < jl+1 − l+1 , then k = l for both (1) and (2), while λ1(1) = λ1 + 1, λ2(1) = λ2 + 1 and λ1(2) = λ1 , λ22 = λ2 + 1, respectively. As 2 2 2 λ + 1 − λ1 λ − λ1 λ − λ1 − = , c − λ1 c − λ1 − 1 c − λ1 the coefficients in (1) and (2) add up to the coefficient in (0).

214


P If l > i and r − 1 − i = jl+1 − l+1 , we further regroup the indices. We can assume l = 0; otherwise this value of t would not appear in any of the indices considered. In order to simplify notation, assume that l+1 = 0. Consider then, in addition to (0) and the corresponding (1) and (2), (0)0 = ( j1 − 1 , . . . , ji−1 − i−1 , ji+1 − i+1 , . . . , jl , jl+1 − 1, . . . , jr − r ), together with the corresponding (1)0 and (2)0 , where (1)0 = ( j1 − 1 , . . . , ji−1 − i−1 , ji − 1, ji+1 − i+1 , . . . , jl , jl+1 − 1, . . . , jr − r ) and (2)0 = ( j1 − 1 , . . . , ji−1 − i−1 , ji , ji+1 − i+1 , . . . , jl , jl+1 − 1, . . . , jr − r ). The value of k in (0) is k = l. Define λ1 = λ1(0) , λ2 = λ2(0) . Then in (0)0 we have k = l − 1 and λ1(0)0 = λ1 , λ2(0)0 = λ2 + 1. In (1) one has k = l, λ1(1) = λ1 + 1, λ2(1) = λ2 + 1. In (1)0 the values are k = l, λ1(1)0 = λ1 + 1, λ2(1)0 = λ2 + 2. In (2) they are l + 1, λ1(2) = λ1 + 1, λ2(2) = λ2 + 2. These values of t do not appear in (2)0 . As ! ! ! ! ! λ2 + 1 − λ1 λ2 + 1 − λ1 λ2 + 1 − λ1 λ2 − λ1 λ2 − λ1 =− + − , − c − λ1 − 1 c − λ1 − 1 c − λ1 − 1 c − λ1 c − λ1

the signed coefficients in (1), (2) , (1)0 , and (2)0 add up to the coefficient in (0) and (0)0 . Finally, consider the case ti = ji for all i. Compare the (r − 1)-index (0) = ( j2 − 2 , . . . , jr − r ) with (1) = ( j1 − 1, j2 − 2 , . . . , jr − r ) and (2) = ( j1 , j2 − P 2 , . . . , jr − r ). If jl − l + 1 ≤ r − 1 − i < jl+1 − l+1 , then in (0), k = l − 1. Denote by λ1 = λ1(0) , λ2 = λ2(0) the values of the lambdas for (0). Then in (2) they are k = l, λ1(2) = λ1 + 1, λ2(2) = λ2 + 1, while these values of t do not appear in the expression of ψ for (1). Hence, the coefficients in (2) for a fixed value of c, λ1 + 1 ≤ c ≤ λ2 + 1, can be compared to the coefficients of c − 1 in (0). P If r − 1 − i = jl+1 − l+1 (assume l+1 = 0; otherwise, just interchange (0), (0)0 below), consider, along with (0), the index (0)0 = ( j2 −2 , . . . , jl −l , jl+1 − 1, . . . , jr − r ). Then in (0), k = l − 1. Denote by λ1 = λ1(0) , λ2 = λ2(0) the values of the lambdas for (0). Then in (0)0 , k = l −1, λ1(0)0 = λ1 , λ2(0)0 = λ2 +1. In (1) they are k = l + 1, λ1 + 2, λ2 + 2; in (2)0 = ( j1 , j2 − 2 , . . . , jl − l , jl+1 − 1, . . . , jr − r ), k = l + 1, and as tl+1 = jl+1 , the index t does not appear in (2)0 . Also, the index does not appear in (1) and (1)0 when they have first index j1 − 1. Now 2 2 2 (λ + 2) − (λ1 + 2) λ − λ1 λ + 1 − λ1 =− + . c − (λ1 + 2) c − λ1 − 1 c − λ1 − 1


215

So the coefficients of (0) and (0)0 for c − 1 add up to the coefficient of (1) for c. P If j1 + 1 ≤ r − i , the coefficients in (0) and (2) are both k = 0, λ1 , λ2 , while in (1) they are 0, λ1 , λ2 + 1. Again, 2 2 2 λ + 1 − λ1 λ − λ1 λ − λ1 − = . c − λ1 c − λ1 c − λ1 − 1 Hence, the sum of coefficients in (1) and (2) adds up to the coefficient in (0). This completes the proof of Claim 2. Consider now the composition of the map ψ with the natural product map p : U ⊗ Sr −1 H ⊗ S g−2−r H → U ⊗ S g−3 H. The image by p of the first summand in the definition of ψ is zero unless λ1( ji ) (ti ) = λ2( ji ) (ti ). Notice that o{ti = ji−1 , 1 ≤ i ≤ k} ≤ o{ti < ji , 1 ji , i > k}. It follows that λ1( ji ) (ti ) = λ2( ji ) (ti ) precisely when all ti except one satisfy either ti = ji or (if 1 k P

· y s1 ∧ · · · ∧ y t ∧ · · · ∧ y sr ⊗ x b+a−1+

si +t−

P

ji

+ (a − k)y j1 ∧ · · · ∧ y jr ⊗ x b+a−1 , where in the above sum t ≤ ji , i < k and t > ji , i > k. As before, let πid be a cup product with the identity in U and p¯ = p ⊗ IdU ∗ Denote by α the composition ¯ α = πId p¯ ψ. Consider the following condition: (∗)

i 6= i0 ,

li ∈ { ji , ji−1 },

i < k,

li ∈ { ji , ji+1 },

i > k.

Then α(y j1 ∧ · · · ∧ y jr ⊗ x a ⊗ x b ⊗ y ∗l1 ∧ · · · ∧ y ∗lr ) =0

if (∗) does not hold,

= (a − k)x b+a−1 k−λ1j (li ) i

= (−1)

if l1 , . . . , lr = j1 , . . . , jr , P

x b+a+

P li − ji −1

otherwise.

216


LEMMA 3.5 Consider a subspace W of U ⊗ Sr H of dimension gr containing W r . Then W contains up to homothety a unique element of the form X caj1 ,..., jr y j1 ∧ · · · ∧ y jr ⊗ x a

such that c0j1 ,..., jr = 0,

1 c0, j2 ,..., jr = 0,

2 c0,1, j3 ,..., jr = 0,

r c0,1,...,r −1 = 0.

Conversely, any element of this form determines a unique such W . Proof Any element w in U ⊗ Sr H that does not belong to W r together with W r generates a subspace W of dimension one more than the dimension of W r and containing W r . Two such elements generate the same subspace if they differ in an element of W r . By definition, W r is generated by elements of the form X ek1 ,...,kr = (−1)1 +···+r y k1 −1 ∧ · · · ∧ y kr −r ⊗ x r −(1 +···+r ) , 0≤i ≤1

where 0 ≤ k1 < k2 < · · · < kr ≤ g − 1. Here any term containing y −1 , y g−1 or · · · ∧ y i ∧ y i ∧ · · · is zero by convention. Assume as given an element w¯ of U ⊗ Sr H − W r . This can be written in the form X w¯ = c¯aj1 ,..., jr y j1 ∧ · · · ∧ y jr ⊗ x a . 0≤ j1 0. Choose a minimal with the condition cna1 ,...,nr 6= 0, n 1 > 0. In particular, a ≥ 2. Let k be such that n k ≤ r − 1, n k+1 ≥ r . As 1 ≤ j1 < j2 < · · · < jr , k ≤ r − 1. We now look at β(q ⊗ x b ⊗ y l1 ∧ · · · ∧ y lr ) for the following values of (b; l1 , . . . , lr ): (r − a; n 1 , . . . , nr ), (r − a + 1; n 1 , . . . , n i − 1, . . . , nr ),

i = 1, . . . , k,

(r − a + 2; n 1 , . . . , n i1 − 1, . . . , n i2 − 1, . . . , nr ), .. .

i 1 < i 2 ≤ k,

(r − a + k; n 1 − 1, . . . , n k − 1, n k+1 , . . . , nr ). Note that by assumption 3r ≤ g + 1, k ≤ r − 1, a ≥ 1. It follows that r − a + k ≤ 2r − a − 1 ≤ 2r − 2 ≤ g − 1 − r . Therefore, the values of b above are allowable. The values of the li may not be allowable if there is a pair of consecutive terms satisfying n i+1 = n i + 1 as we then cannot take n i+1 − 1. We deal with this situation later. We check that β(q ⊗ x b ⊗ y l1 ∧ · · · ∧ y lr ) = 0 for all these values of (b; l1 , . . . , lr ) implies cna1 ,...,nr = 0, contradicting our choice. We continue to make the conventions cbj1 ,..., jr = 0 if b > r or ji ≤ jl , i > l or jr > g − 2. From the expression of ψ and the definition of β, we obtain the following equations: β(q ⊗ x r −a ⊗ y ∗n 1 ∧ · · · ∧ y ∗nr ) = (a − k)cna1 ,...,nr −

r −a X k X t=1 i=1

−

k−2 XX t

i=1

cna+t + 1 ,...,n i +t,...,n r

k−1 XX t

cna+t 1 ,...,n i−1 ,n i+1 ,n i +t,n i+2 ,...,n r

i=1

cna+t 1 ,...,n i−1 ,n i+1 ,n i+2 ,n i +t,n i+3 ,...,n r


+ · · · + (−1)k−1

X

219

cna+t , 2 ,...,n k−1 ,n 1 +t,n k+1 ,...,n r

t

β(q ⊗ x r −a+1 ⊗ y ∗n 1 ∧ · · · ∧ y ∗ni1 −1 ∧ · · · ∧ y ∗nr ) X = − cna1 ,...,n i −1,...,n i +1,...,n i − cna1 ,...,nr 1

i 2 ≤k

−

X X

cna+t 1 ,...,n i

1≤t i 2 ≤k

+

X X t

+

cna+t 1 ,...,n i

t

+ (−1)

k−1

X t

1

r

−1,...,n i2 +t+1,...,nr

cna+t 1 ,...,n i

i 2 ≤k−1

X

2

1

+

X

cna+t 1 ,...,n i

t

1

+t,...,nr

−1,...,n i2 −1 n i2 +1 n i2 +t+1,...,,nr

n n +t,...,nr 1 −1 i 1 +1 i 1

cna+t 2 ,...,n i

1

−1,...,n k−1 ,n 1 +t+1,n k+1 ,...,nr ,

.. . β(q ⊗ x r −a+k ⊗ y ∗n 1 −1 ∧ · · · ∧ y ∗n k −1 ∧ y ∗n k+1 · · · ∧ y ∗nr ) X =− cna+t 1 −1,...,n i +t−1,...,n k −1,n k+1 ,...,n r t

+ · · · + (−1)k cna+t . 2 −1,...,n k −1,n 1 +k+t−1,n k+1 ,...,n r Now take the alternating sum of the expressions above. Consider the terms in β(q ⊗ x r −a ⊗ y ∗n 1 ∧ · · · ∧ y ∗nr ). The first term cna1 ,...,nr appears in each of the expressions for the coefficients n 1 , . . . , n i − 1, . . . , nr , r − a + 1. Hence, the coefficient for this term is a. Each term in the second summand in the expression for n 1 , . . . , nr , r −a appears as the second summand in the second parenthesis for an expression for n 1 , . . . , n i −1, . . . , nr , r −a+1. Hence these terms cancel each other. Each term in the third summand in the expression for n 1 , . . . , nr , r −a appears as the second summand in the third parenthesis for an expression for n 1 , . . . , n i − 1, . . . , nr , r − a + 1. Hence, these terms cancel each other. Finally, the last summand in n 1 , . . . , nr , r − a appears as the last term in the expression for n 1 − 1, n 2 , . . . , nr , r − a + 1. Similarly, the first summand in each parenthesis for the expression of n 1 , . . . , n i − 1, . . . , nr , r − a + 1 cancels with a term in n 1 , . . . , n i1 −1 , . . . , n i2 −1 , . . . , nr , r − a + 2, and so on. We obtain β(q ⊗ x r −a ⊗ y ∗n 1 ∧ · · · ∧ y ∗nr ) −

k X i 1 =1

β(q ⊗ x r −a+1 ⊗ y ∗n 1 ∧ · · · ∧ y ∗n i1 −1 ∧ · · · ∧ y ∗nr )

220


+ · · · + (−1)k β(q ⊗ x r −a+k ⊗ y ∗n 1 −1 ∧ · · · ∧ y ∗n k −1 ∧ y ∗n k +1 ∧ · · · ∧ y ∗nr ) = acna1 ,...,nr . We can assume a 6= 0 as by assumption cn01 ,...,nr = 0. Hence, as we work in characteristic zero, cna1 ,...,nr = 0. Note that if n i+1 = n i +1 for some i, we cannot consider the values (b; l1 , . . . , lr ) that contain n i+1 − 1 unless they also contain n i − 1. But the formal expressions β(q ⊗ x b ⊗ y l1 ∧ · · · ∧ y lr ) are zero; each of its terms contains either the pair c...,n i ni ,... or c...,n i n i +t,... . The latter terms appear twice with different signs: the first occurrence appears when we modify the term (i + 1)th ; the second one comes from the term n i+1 − 1 = n i that slides down to the place i. Therefore, for all values of d, l1 , . . . , lr , l1 6 = 0, cld1 ,...,lr = 0. d+1 d It remains to show that c0,l = 0. Define now c¯ld1 ,...,lr −1 = c0,l . 2 ,...,lr 1 +1,...,lr −1 +1 d d Then the equations for c0,l2 ,...,lr give rise to equations for c¯l1 ,...,lr −1 that are analogous to the set of equations for r −1 and g−2. By induction on r , this finishes the proof.

4. Extending the results to the generic curve PROPOSITION 4.1 Let C be a generic curve of genus g. Let E be defined as in Definition 0.5. Denote by W the image of (H 0 (C, K C ))∗ in H 0 (C, E). Then the natural map ψC,r : ∧r W → H 0 (C, ∧r E) is injective. Note that for C nonhyperelliptic, W = H 0 (E). This follows from the projective normality of C (case p = 0 of the conjecture). Proof If, for a given curve C, ψC,r is injective, the same holds for every curve in a neighbor hood of C in Mg . As dim ∧r W = gr , Proposition 1.2 shows that ψC0 ,r is injective for C0 hyperelliptic. Hence, the result follows. Alternatively, the proof of Proposition 1.2 holds for any curve. The following proposition now concludes the proof of Theorem 0.4. PROPOSITION 4.2 Let C be a generic curve of genus g. Then h 0 (C, ∧r E) = H 0 (C, ∧r E) is an isomorphism.

g r

and ∧r W →


221

Proof From Proposition 4.1, h 0 (C, ∧r E) ≥ dim Im(∧r W → H 0 (C, ∧r E)) = dim ∧r W = g g r 0 r r . From Theorem 3.4, h (C, ∧ E) ≤ dim WC0 = r . This concludes the proof. Acknowledgments. I would like to thank all the people and institutions that contributed in different ways to the present work, in particular, C. Voisin, S. Ramanan, D. Eisenbud, M. Green, and the Pure Mathematics Department of the University of Cambridge, where this work was started. I am a member of the Vector Bundles on Algebraic Curves (VBAC) research group of European Algebraic Geometry Research Training Network (EAGER). References [AC]

E. ARBARELLO and M. CORNALBA, Su una congettura di Petri, Comment. Math.

[BE]

D. BAYER and D. EISENBUD, Ribbons and their canonical embeddings, Trans. Amer.

[BiR]

I. BISWAS and S. RAMANAN, An infinitesimal study of the moduli of Hitchin pairs, J.

[E]

L. EIN, A remark on the syzygies of the generic canonical curves, J. Differential Geom.

[EiG]

D. EISENBUD AND M. GREEN, Clifford indices of ribbons, Trans. Amer. Math. Soc.

[FH]

W. FULTON and J. HARRIS, Representation Theory: A First Course, Grad. Texts in

[G]

M. GREEN, Koszul cohomology and the geometry of projective varieties, J. Differential

[HiR]

A. HIRSCHOWITZ and S. RAMANAN, New evidence for Green’s Conjecture on

Helv. 56 (1981), 1–38. MR 82k:14029 202 Math. Soc. 347 (1995), 719–756. MR 95g:14032 196 London Math. Soc. (2) 49 (1994), 219–231. MR 94k:14006 200 26 (1987), 361–365. MR 89a:14031 196 347 (1995), 757–765. MR 95g:14033 196 Math. 129, Springer, New York, 1991. MR 93a:20069 198 Geom. 19 (1984), 125–171. MR 85e:14022 196 ´ syzygies of canonical curves, Ann. Sci. Ecole Norm. Sup. (4) 31 (1998), 145–152. MR 99a:14033 196 [HuPR] K. HULEK, K. PARANJAPE, and S. RAMANAN, On a conjecture on canonical curves, J. Algebraic Geom. 1 (1992), 335–359. MR 93c:14029 [PR] K. PARANJAPE and S. RAMANAN, “On the canonical ring of a curve” in Algebraic Geometry and Commutative Algebra, II, Kinokuniya, Tokyo, 1988, 503–516. MR 90b:14024 197 [S1] F.-O. SCHREYER, Syzygies of canonical curves and special linear series, Math. Ann. 275 (1986), 105–137. MR 87j:14052 196 [S2] , “Green’s conjecture for general p-gonal curves of large genus” in Algebraic Curves and Projective Geometry (Trento, Italy, 1988), Lecture Notes in Math. 1389, Springer, Berlin, 1989, 254–260. MR 90j:14041 196 [S3] , A standard basis approach to syzygies of canonical curves, J. Reine Angew. Math. 421 (1991), 83–123. MR 87j:14052 196

222


[T]

M. TEIXIDOR I BIGAS, Half-canonical series on algebraic curves, Trans. Amer. Math.

[V]

C. VOISIN, Courbes tétragonales et cohomologie de Koszul, J. Reine Angew. Math.

[W]

G. E. WELTERS, Polarized abelian varieties and the heat equations, Compositio Math.

Soc. 302 (1987), 99–115. MR 88e:14037 202 387 (1988), 111–121. MR 89e:14036 196 49 (1983), 173–194. MR 85f:14045 200

Department of Mathematics, Tufts University, Medford, Massachusetts 02155, USA; [email protected]


UNIFORM SEMICLASSICAL ESTIMATES FOR THE PROPAGATION OF QUANTUM OBSERVABLES A. BOUZOUINA and D. ROBERT

Abstract We prove here that the semiclassical asymptotic expansion for the propagation of quantum observables, for C ∞ -Hamiltonians growing at most quadratically at infinity, is uniformly dominated at any order by an exponential term whose argument is linear in time. In particular, we recover the Ehrenfest time for the validity of the semiclassical approximation. This extends the result proved in [BGP]. Furthermore, if the Hamiltonian and the initial observables are holomorphic in a complex neighborhood of the phase space, we prove that the quantum observable is an analytic semiclassical observable. Other results about the large time behavior of observables with emphasis on the classical dynamic are also given. In particular, precise Gevrey estimates are established for classically integrable systems. 1. Introduction and main results According to Bohr’s correspondence principle, the quantum evolution of an observable is closer and closer to its classical evolution as the Planck constant ~ becomes negligible. In mathematical literature this result is known as the Egorov theorem (see [Ho], [Ro]). But usually this result is proved for finite time and the asymptotics hold in the C ∞ - (Poincaré) sense. Here we put emphasis on large time behavior of asymptotic expansions in ~, in the C ∞ -case as well as in the analytic (or Gevrey) case. In the book [Iv, Sec. 2.3] (propagation of singularities along long bicharacteristics), one can find related results for the C ∞ -case. But our results are more explicit. Many years ago, physicists conjectured that semiclassical approximation was still valid in a large time interval with length T (~) ≈ log(~−1 ) which is called Erhenfest time (see [Ch], [Za]). That kind of result was rigorously proved for propagation of coherent states in [CR] (and in [HJ] in the analytic case). In [BGP] the authors established long time estimates for the propagation of quantum observables. The main goal of this paper is to prove more accurate estimates in ~ and time. In particular, if the data are holomorphic in a complex neighborhood of the real phase space, then the semiclassical expansion DUKE MATHEMATICAL JOURNAL c 2002 Vol. 111, No. 2, Received 2 December 1999. Revision received 9 February 2001. 2000 Mathematics Subject Classification. Primary 35Q40; Secondary 35J10, 81Q20. 223

224

BOUZOUINA and ROBERT

of the time-dependent quantum observable holds with an exponentially small remainder term, for time interval with length of order log(~−1 ). Let us denote by X = Rn the configuration space of a classical mechanical system with n degrees of freedom. The corresponding phase space Z is identified with R2n equipped with the symplectic form σ defined by σ (z; z 0 ) = hJ z, z 0 i, (1) where h , i is the Euclidean scalar product and 0 J= −1

J is the (2n × 2n)-matrix 1 . 0

(2)

A generic point in Z is denoted by z, and its coordinates are denoted by (q, p), where q, p ∈ Rn . A classical Hamiltonian is a smooth real function H : Z → R. Our basic example is H (q, p) = k pk2 /(2m) + V (q), m > 0, where k pk2 = h p, pi. We could also consider a Riemannian manifold (X, g). In what follows we place emphasis on the case X = Rn . The motion of the classical system is determined by the system of Hamilton equations ∂H dp ∂H dq = (q, p), =− (q, p). (3) dt ∂p dt ∂q The equations (3) generate a flow 8t on the phase space Z , defined by 8t (q(0), p(0)) = (q(t), p(t)); 80 = 1. 8t exists locally by the Cauchy-Lipschitz theorem for ordinary differential equations (ODE). But we need more assumptions on H to define 8t globally on Z . 8t defines a symplectic diffeomorphism (canonical transformation) group of transformations on Z . Let us consider a classical observable A, that is, A a smooth complex-valued function defined on phase space Z . The time evolution of A can be easily computed: d A 8t (z) = {H, A} 8t (z) , z = (q, p), (4) dt where {H, A} is the Poisson bracket defined by {H, A} = ∂q H · ∂ p A − ∂ p H · ∂q A.

(5)

Here we have used the notation ∂q = ∂/∂q. Now let us assume that H, A are quantiˆ zable. That means that we can associate to them the quantum observables Hˆ and A, that is, self-adjoint operators in L 2 (X ). By solving formally the Schrödinger equation, i~∂t ψt = Hˆ ψt , we can define the one-parameter group of unitary operators ˆ U (t) = exp(−(it/~) Hˆ ). The quantum time evolution of Aˆ is then given by A(t) = ˆ U (−t) AU (t), which satisfies the Heisenberg–von Neumann equation ˆ d A(t) i ˆ = [ Hˆ , A], dt ~

(6)

THE LONG TIME SEMICLASSICAL EGOROV THEOREM

225

where [K , B] = K B − B K is the commutator of K , B. Here we use the ~-Weyl quantization defined for A ∈ S (Z ) (the space of Schwartz functions) by the following formula, with ψ ∈ S (X ): Z Z x + y −1 ˆ , p ei~ hx−y, pi ψ(y) dy dp. (7) Aψ(x) = (2π~)−n A 2 Z Let us now introduce a more general set of classical observables for which the ~-Weyl quantization is well defined and has nice properties (see [Ho], [Ro]). Definition 1.1 A

(i) A ∈ O (m), m ∈ R, if and only if Z → C is C ∞ in Z and for every multi-index γ ∈ N2n there exists C > 0 such that γ ∂z A(z) ≤ Chzim , ∀z ∈ Z . (ii) We say that A is a C ∞ -semiclassical observable of weight m if there exist ~0 > 0 and a sequence A j ∈ O (m), j ∈ N, so that A is a map from ]0, ~0 ] into O (m) satisfying the following asymptotic condition: for every N ∈ N and every γ ∈ N2n , there exists C N > 0 such that for all ~ ∈]0, ~0 [ we have X γ suphzi−m ∂z A(~, z) − ~ j A j (z) ≤ C N ~ N +1 . (8) Z

0≤ j≤N

ˆ The A0 is called the principal symbol, and A1 is called the subprincipal symbol of A. set of semiclassical observables of weight m is denoted by Osc (m). By the ~-Weyl bsc (m). quantization, its range in L (S (X )) is denoted by O Notation. For any A’s and A j ’s satisfying (8), we write the following: A(~) P j j≥0 ~ A j in Osc (m). Let us now recall the statement of the propagation theorem that is improved in this paper. The microlocal version of the result is due to Yu. Egorov [Eg]. R. Beals [Be] found a nice simple proof that is reproduced in [Ro]. THEOREM 1.2 Let us consider a Hamiltonian H and a real observable A satisfying γ ∂z H j (z) ≤ Cγ j for |γ | + j ≥ 2,

~

−2

(H − H0 − ~H1 ) ∈ Osc (0), γ ∂z A(z) ≤ Cγ for |γ | ≥ 1.

(9) (10) (11)

226


Then we have the following properties. (a) For ~ small enough, Hˆ , Aˆ are essentially self-adjoint operators in L 2 (X ) with core S (X ); hence the quantum evolution U (t) = exp(−(it/~) Hˆ ) is well defined for all t ∈ R. bsc (1). Its symbol has an asymptotic ˆ = U (−t) AU ˆ (t) ∈ O (b) For each t ∈ R, A(t) P t j expansion such that A(t) − A ◦ 8 j≥1 ~ A j (t) holds in Osc (0), uniformly in t, for t in a bounded interval. Moreover, A j (t) can be computed by the formulas A0 (t, z) = A 8t (z) , (12) Z t A1 (t, z) = A(8τ ), H1 8t−τ (z) dτ, (13) 0

and for j ≥ 2, by induction, Z t X α β A j (t, z) = 0(α, β) (∂ p ∂q Hk )(∂qα ∂ βp A` )(τ ) 8t−τ (z) dτ (14) |(α,β)|+k+`= j+1 0≤`≤ j−1

0

with

(−1)|β| − (−1)|α| −1−|(α,β)| i , α!β!2|α|+|β| where 8t is the classical flow defined by the principal term H0 . 0(α, β) =

Since our aim is to improve Theorem 1.2, let us recall here briefly the method used to ˆ Hˆ are essentially self-adjoint (for a proof, see [Ro]). prove it. We admit here that A, Let us remark that, under the assumption on H0 in Theorem 1.2, the classical flow 8t exists globally in Z . Indeed, the Hamiltonian vector field (∂ξ H0 , −∂x H0 ) has at most a linear growth at infinity, and hence no classical trajectory can blow up in a finite time. Moreover, using methods usual in nonlinear ODE (variation equation), we can γ prove that for every γ ∈ N2n , |γ | ≥ 1 ∂z A(8t ) ∈ O (0) is uniformly bounded for z ∈ Z and t-bounded. Now from the Heisenberg equation and the classical equation of motion, we get ni o d \ U (−s) c A0 (t − s)U (s) = U (−s) Hˆ , c A0 (t − s) − { H, A0 }(8t−s ) U (s), (15) ds ~ where A0 (t) = A(8t ). But, from the product rule formula (see the appendix), the b0 (t − s)] − { H, \ principal symbol of (i/~)[ Hˆ , A A0 }(8t−s ) vanishes. So, in the first step, we get the error term Z t i ˆ c \ U (−t) AU (t) − A0 (t) = U (−s) Hˆ , c A0 (t − s) − { H, A0 }(8t−s ) U (s) ds. ~ 0 (16) Now it is not difficult to obtain, by induction on j, the full asymptotics in ~ (see [Ro] for details). 2


227

Remark 1.3 If H = H0 is a polynomial function of degree less than or equal to 2 on the phase space Z , then the propagation theorem has a very simple form, A(t) = A ◦ 8t , and the remainder term is null. (U (t) is a metaplectic transformation.) In [PU1] a semiclassical version of the Egorov theorem on compact manifolds is given. Our first result is an improvement of Theorem 1.2 by giving estimates for large time, in the C ∞ -case. THEOREM 1.4 Let us assume that the Hamiltonian H and the observable A satisfy the assumptions of Theorem 1.2. Let us introduce an upper bound of the stability exponents of the classical system 0 := sup J ∇z(2) H0 (z) , z∈Z

(2) ∇z

where for any observable f , f is the corresponding Hessian matrix.∗ Then, for every j ∈ N and every multi-index γ such that j + |γ | ≥ 1, there exists C j,γ > 0 such that, for every z ∈ Z and every t ∈ R, we have γ ∂z A j (t, z) ≤ C j,γ exp 0(2 j − 1 + |γ |)|t| . (17) Furthermore, we have the following estimates in the L 2 –operator norm of the remainder term. For every N ∈ N, there exists C N such that for every t ∈ R we have

X

ˆ

~ j Aˆ j (t)

A(t) − 0≤ j≤N

L2

≤ C N ~ N +1 1 + |t|

N +δn

exp 0(2N + δn )|t| ,

(18)

where δn is a universal constant (δn ≤ 5n + 3). This result entails the following corollary about the Ehrenfest time for the validity of the semiclassical approximation. COROLLARY 1.5 Under the assumptions of Theorem 1.4, for every N ≥ 1 there exists C N > 0 such that, for every ε > 0 and for |t| ≤ ((1 − ε)/(20)) log(~−1 ), we have

X

ˆ

(19) ~ j Aˆ j (t) 2 ≤ C N ~εN +1 ~((ε−1)/2)(5n+3) .

A(t) − 0≤ j≤N

L

In particular, the semiclassical asymptotic expansion is valid under the above condition on t. ∗ Here

the norm of a symmetric matrix M is defined by |||M||| = supkxk≤1 |hM x, xi|.

228


Remarks in the case of “classical” energy observables We see from the proof of Theorem 1.4 that if the expansion of H in ~ is even (in particular, if H is “classical,” that is, if H = H0 ), then the ~-expansion of A(t) is even and the exponential term in (17) becomes exp 0(3 j/2 − 1 + |γ |)|t| . In the remainder estimate (18), the 2N also becomes 3N /2, so that in this case the Ehrenfest time is not smaller than (2/(30)) log ~−1 . More precisely, for every ε > 0 and for |t| ≤ ((2 − ε)/(30)) log(~−1 ), we have

X

ˆ

(20) ~ j Aˆ j (t) 2 ≤ C N ~(ε/2)N +1 ~((ε−2)/3)(5n+3) .

A(t) − 0≤ j≤N

L

We remember that, some time ago, S. De Bièvre suggested that the Ehrenfest time could be greater than (1/(20)) log ~−1 . The above results confirm this guess. In Section 5 we make more comments about this. Remark 1.6 In many cases the assumptions of Theorem 1.2 on the Hamiltonian H are not satisfied. In particular, for a Schrödinger Hamiltonian with nonconstant magnetic field with vector-potential a(q), H (q, p) = k p−a(q)k2 +V (q) or for a quartic electric potential H (q, p) = k pk2 + kqk4 , the second derivatives are not always bounded. But if the ¯ of the phase space, then observable A is supported in a flow-invariant compact set using the functional calculus on ~-pseudodifferential operators (see [Ro]), we can get the same conclusion as in Theorem 1.4 with a constant 0 defined as follows: 0 := sup J ∇z(2) H0 (z) ,

where A is such that supp(A) ⊂ . A more precise statement is given at the end of Section 2 (see Proposition 2.7). As we can see from the proofs, the use of the numerical constant 0 is to control the exponential growth of the flow in time. Of course, it is also possible to make directly some growth assumptions on the flow or to make some geometrical assumptions on the classical Hamiltonian. This is done in Section 4 with periodic systems (see Proposition 2.8) and stated in this section with integrable systems (see Theorem 1.13). Some extensions to manifolds are also clearly possible. But in general (compact) manifolds no exact global quantization procedure exists, so it seems difficult to compute Aˆ j (t) for j ≥ 2 for nonclassical symbols. As expected, the dependence in j, γ , N of the constants in Theorem 1.4 can be specified under analyticity assumptions on A and H . Let us first recall the following definition (see [BK], [Sj]). Definition 1.7 Let A(~, z) be a C ∞ -semiclassical observable, A ∈ Osc (0). We say that A(~, z) is a


229

semiclassical observable with a Gevrey index σ ≥ 1, if the following conditions are satisfied, for some constant C > 0, γ ∀γ ∈ N2n , ∀ j ∈ N, ∀z ∈ Z , we have ∂z A j (z) ≤ C |γ |+ j+1 |γ | + j !σ ; (21) X γ ~ j A j (z) ≤ ~ N +1 C |γ |+N +2 |γ | + N + 1 !σ . (22) ∂z A(~, z) − 0≤ j≤N

As usual, a semiclassical observable with a Gevrey index equal to 1 is said to be analytic. THEOREM 1.8 Let us assume that the energy Hamiltonian H has an expansion in ~ such that ~−2 (H − H0 − ~H1 ) is an analytic semiclassical observable and that there exists some δ > 0 such that H0 , H1 , and the initial observable A are holomorphic in the (2) complex domain D(δ) := {z ∈ C2n , k=zk < δ}. Suppose, moreover, that ∇z H0 , ∇z H1 , ∇z A are bounded and continuous in D(δ). Then if we define 0 := sup J ∇z(2) H0 (z) , z∈D(δ)

we have the following improvement of Theorem 1.4, saying that ~−1 (A(t, z) − A ◦ 8t ) is an analytic semiclassical observable, with control in time. More explicitly, there exist constants K 0 > 0, K 1 > 0 such that, for every t ∈ R, j ∈ N, z ∈ Z , N ≥ 1, and ~ ∈]0, 1], γ ∂z A j (t, z) ≤ K 0 j + |γ | !K j+|γ | 1 + |t| j exp 0(2 j − 1 + |γ |)|t| , (23) 1 and we have the following estimate in the L 2 –operator norm of the remainder term,

X

ˆ

~ j Aˆ j (t)

A(t) − 0≤ j≤N

L2

≤ ~ N +1 K 0 (N + 1)!K 1N +1 |t| 1 + |t|

N +δn

exp 0(2N + δn )|t| , (24)

where δn ≤ 5n + 3 is a universal constant. Remark 1.9 As in the C ∞ -case, if the expansion of H in ~ is even, then in estimates (23) and (24), in the exponential term, the 2N becomes 3N /2. This remark is an extension of the remark concerning the “classical” observables. We can deduce from Theorem 1.8 a semiclassical approximation of A(t) with an exponentially small error in ~ for long time intervals.

230


COROLLARY 1.10 Under the assumptions of Theorem 1.8, for every ε > 0, C > 0 such that ε + 2C0 < 1, there exist constants K > 0, a > 0, such that for |t| ≤ C log(~−1 ) and N = [a/~ε ] we have

a X

ˆ

~ j Aˆ j (t) 2 ≤ K exp − ε . (25)

A(t) − L ~ 0≤ j≤N

In applications it can be useful to consider observables with support outside some given nonempty open set. Such observables cannot be analytic but can be in some Gevrey class s > 1. Let us introduce the following technical assumptions. (C.0) H is analytic in D(δ), and there exist C > 0, M ≥ 0 such that H (z) ≤ C 1 + |z| M , ∀z ∈ D(δ). (26) Moreover, Hˆ is supposed to be self-adjoint in L 2 (Rn ). ¯ is 8t -invariant for every (C.1) Let be a bounded open subset of Z such that t ∈ R, and let A be a smooth observable such that supp(A) ⊂ . (C.2) A is Gevrey s > 1. Let us define (2) J ∇ H (z) . 0δ := sup z Re z∈;=z 0, C > 0, such mation of A(t) that ε(3s − 1)/2 + 3C0/2 < 1, there exist constants K > 0, a > 0, such that for |t| ≤ C log(~−1 ), N = [a/~ε ], and ~ ∈]0, 1], we have

a X

ˆ

(29) ~ j Aˆ j (t) 2 ≤ K exp − ε .

A(t) − L ~ 0≤ j≤N


231

Remark 1.12 ¯ is 8t -invariant for every t ≥ 0 (resp., t ≤ 0), then all the If, in condition (C.1), conclusions of Theorem 1.11 hold for t ≥ 0 (resp., t ≤ 0). The above results are general and do not depend on specific properties of the classical dynamic. Nevertheless, some important systems in classical and quantum mechanics are integrable and for them we can prove much better results: the semiclassical expansions are valid for time less than some algebraic power ~−1 instead of O(log(~−1 )) (which is optimal for unstable systems). Let us first introduce the following analyticintegrability condition. (C.3) There exists a symplectic map χ from into A × Tn , where A is an open set in Rn and where Tn is an n-dimensional torus such that χ 8t (z) = I (z), ϕ(z) + tω(I (z)) , ∀z ∈ , ∀t ∈ R, (30) where χ (z) = (I (z), ϕ(z)). Moreover, there exist complex open neighbor˜ A˜, T˜ n of , A , Tn such that χ is an analytic diffeomorphism from hoods , ˜ onto A˜ × T˜ n . THEOREM 1.13 Let us assume conditions (C.0), (C.1), (C.2), and (C.3). Then we have the following Gevrey-type estimates, for some constants K 0 , K 1 , γ ∂z A j (t, z) ≤ K 0 3 j/2 + |γ | !s ( j/2)!−1 K j+|γ | 1 + |t| |γ |+2 j , (31) 1

and we have the following estimate in the L 2 –operator norm of the remainder term,

X 2N +δn

ˆ

~ j Aˆ j (t) 2 ≤ ~ N +1 K 0 (N +1)!(3s−1)/2 K 1N +1 |t| 1+|t| , (32)

A(t)− 0≤ j≤N

L

where δn ≤ 5n + 3 is a universal constant. Moreover, as above, we have a long time ˆ with exponentially small error estimate: for every ε > 0, there approximation of A(t) exist K > 0, a > 0, such that, for |t| ≤ C~−1/2+ε and N = [a/~3ε/(2s−1) ], we have

X a

ˆ

(33) ~ j Aˆ j (t) 2 ≤ K exp − 3ε/(2s−1) .

A(t) − L ~ 0≤ j≤N

Remark 1.14 Many integrable systems satisfy condition (C.3). Let us assume that the system is analytic-integrable in the Liouville sense. That means that there exist n real-analytic Hamiltonians F1 , . . . , Fn in with F1 = H , Poisson commuting, {F j , Fk } = 0. ¯ into Rn and that for every Assume that F := (F1 , . . . , Fn ) is a regular map from

232


¯ is a compact and connected component of F −1 (λ). Then λ ∈ F(), F −1 (λ) ∩ revisiting the proof of the Liouville theorem (see V. Arnold [Ar]), we can prove that (C.3) is satisfied. Remark 1.15 Let us recall that only even terms A j appear in Theorem 1.13. In the more general case with nonnull subprincipal terms (H = H0 + H1 ), where H1 is analytic and H0 analytically integrable in , odd terms contribute and in the above estimates we have to replace (3s − 1)/2 by 2s − 1. In particular, the exponential error estimate holds for a smaller time |t| ≤ ~−1/3+ε . Remark 1.16 Estimates (18) and (24) improve results given in [Iv] and [BGP] in different ways. In the C ∞ -case we get results similar to those of [BGP] without analyticity assumptions and with a better control for large time. In the analytic and Gevrey cases, we get better estimates with respect to N and t (see (24)). Note that in [BGP] the A j (t)’s are not defined through algorithm (14). Similar exponential estimates were recently proved in [HJ] for propagation of Gaussian coherent states. Remark 1.17 The above results still hold true if the Hamiltonian H has singularities (like Coulomb potentials) outside of . This is commented on in Section 4. Example 1.18 Our results apply, in particular, to the following examples. (1) H (q, p) = k pk2 + V (q), A(q, p) = q1 with V holomorphic in {q ∈ Cn }, {k=qk < δ} for some δ > 0 and such that (2) ∇ V (q) < +∞. sup q {q∈Cn , k=qk 0 such that

∀t ∈ R, ∀z ∈ Z ,

γ k∂z 8t zk ≤ Cγ exp |γ ||t|0 .

(36)

Proof We proceed by induction on |γ |. We start with the Jacobi stability equation d ∇z 8t (z) = J ∇z(2) H0 (8t z)∇z 8t z. dt

(37)

234


Using the definition of 0 and the Gronwall inequality, we get (36) for |γ | = 1 with Cγ = 1. For |γ | = k ≥ 2, let us assume that (36) holds for |γ | < k. Computing derivatives in z of (37) and applying the Faà de Bruno formula, we get X d γ t γ ∂z (8 z) = γ ! ∂ α (J ∇z H0 )(8t z)(∂z 8t z)α + Y (t). (38) dt |α|=1

Using the induction assumption, there exists C > 0 such that

∀t ∈ R, Y (t) ≤ C exp 0|γ ||t| .

(39)

To complete the proof of the lemma, we need some standard properties of linear differential equations which are recalled in the following lemma. LEMMA 2.3 Let us consider the linear differential equation

d X (t) = M(t)X (t) + Y (t), dt

(40)

where M(t) is a smooth family of (d × d)-matrices defined on R such that 0 = supt∈R |||M(t)||| < +∞. Let R(t, s) be the resolvent of the homogeneous system, that is, such that ∂ R(t, s) = M(t)R(t, s), R(s, s) = 1. (41) ∂t Then we have Z t X (t) = R(t, 0)X (0) + R(t, s)Y (s) ds (42) 0

and

R(t, s) ≤ exp 0|t − s| , In particular, if kY (t)k ≤

∀t, s ∈ R.

with K ≥ 0, then there exists

X (t) ≤ C 0 e K |t| , ∀t ∈ R.

Ce K |t|

(43) C0

> 0 such that (44)

Thus Lemma 2.2 is proved. Using inequality (36) and the Faà de Bruno formula again, we easily get the next lemma. 2.4 For every multi-index γ 6= 0, there exists C0,γ > 0 such that γ ∀t ∈ R, ∀z ∈ Z , ∂z A0 (t, z) ≤ C0,γ exp 0|γ ||t| . LEMMA

(45)


235

Now using induction formula (14), the Leibniz formula, and again the Faà de Bruno formula, we get estimates for A j (t). LEMMA 2.5 For every j ≥ 1 and every multi-index γ , there exists C j,γ > 0 such that, for every z ∈ Z and every t ∈ R, we have γ ∂z A j (t, z) ≤ C j,γ exp 0(2 j − 1 + |γ |)|t| . (46)

Now we want to estimate the error term in the propagation of observables. Let us define X A(N ) (t) := ~ j A j (t). (47) 0≤ j≤N

Recall that the Moyal bracket, {K , of two observables K , B, is defined as the ˆ which admits the following formal ~-expansion: Weyl symbol of (i/~)[ Kˆ , B] X {K , B}∗ ~ j {K , B} j+1 . B}∗

j≥0

Using the rule product (see the appendix), we can expand {H, B}∗ in a power series in ~, {H, B}∗ = {H0 , B}1 + ~ {H0 , B}2 + {H1 , B}1 + · · · + ~k−1 {H0 , B}k + {H1 , B}k−1 + ~k (δ kH0 ,B + δ k−1 H1 ,B ) + · · · , (48) where {H0 , B}1 is the usual Poisson bracket (see (5)). The remainder term is given by the remainder term in the product rule, with notation defined in the appendix, δ kH,B = i~−k−1 Rk (H, B) − Rk (B, H ) . (49) The algorithm used to construct the A j ’s is such that we have ∗ d (N ) A (t) = H, A(N ) (t) + ~ N +1 R (N +1) (t), dt

(50)

N +1 N R (N +1) (t) = δ H + · · · + δ 1H0 ,A N + δ H + · · · + δ 0H1 ,A N . 1 ,A0 0 ,A0

(51)

where

The following lemma gives the error in the L 2 –operator norm. 2.6 For every N ∈ N and every t ∈ R, we have

A(t) ˆ − Aˆ (N ) (t) 2 ≤ ~ N +1 |t| sup Rˆ (N +1) (s) 2 . L L LEMMA

|s|≤|t|

(52)

236


Proof ˆ − Aˆ (N ) (t) and compute (15): Let us denote E(t) = A(t) ni o d d U (−s)E(t − s)U (s) = U (−s) Hˆ , E(t − s) − E(t − s) U (s). ds ~ dt

(53)

After integration in s and using E(0) = 0, we get the lemma. End of proof of Theorem 1.4 N +1− j Using the appendix, we can estimate L ∞ -norms of the derivatives of δ Hi ,A j (t) and then conclude using the Calderon-Vaillancourt theorem, with the recent improvement by A. Boulkhemair [Bo]. The statement is the following. There exists γn such that, for all B ∈ O (0), we have α,β ∂ B(z) . ˆ L 2 ≤ γn k Bk sup (54) z |α|,|β|≤[n/2]+1 z∈Z

By revisiting the proof, we can easily see that the conclusions of Theorem 1.4 are still valid if the following conditions on the Hamiltonian H and the observable A are fulfilled. For simplicity, we assume that H is classical. We relax growth assumptions on H , but we reinforce assumptions on the observable A. (C.4) There exists M > 0 such that for every γ ∈ N2n there exists Cγ such that γ ∂z H (z) ≤ Cγ 1 + |z| M ,

∀z ∈ R2n .

(55)

¯ is a compact, (C.5) Let A be a smooth observable such that supp(A) ⊂ , where t 8 -invariant set of Z , ∀t ≥ 0. Let us introduce the real number 0 := sup J ∇z(2) H (z) . z∈

PROPOSITION 2.7 Let us assume that H and A satisfy conditions (C.4) and (C.5). Then the following estimate holds for every t ≥ 0 and j ≥ 0: γ ∂z A j (t, z) ≤ C j,γ exp 0(3 j/2 + |γ |)|t| . (56)

Furthermore, we have the following estimates in the L 2 –operator norm of the remainder term. For every N ∈ N, there exists C N such that we have

X

ˆ

~ j Aˆ j (t) 2 ≤ C N ~ N +1 exp 0(3N /2 + δn )|t| . (57)

A(t) − 0≤ j≤N

L


237

As we see later in more detail for integrable systems, when the classical system is stable, it is expected that the quantum motion stays localized in the phase space for a much larger time interval. This can be easily proved for periodic systems, as we now see. Let us consider a classical Hamiltonian H and an observable A satisfy¯ a connected component of the compact set ing assumptions (C.4) and (C.5) with −1 H ([E 1 , E 2 ]), where E 1 < E 2 . Let us introduce the following periodicity property. ¯ is a compact connected component (Per) (i) For every λ ∈ [E 1 , E 2 ], H −1 (λ) ∩ ¯ (iii) The Hamiltonian flow 8t of H −1 (λ). (ii) ∇ H (z) 6 = 0 for every z ∈ . H defined by H is periodic with period T (z) in where T is a smooth function in . PROPOSITION 2.8 Let us assume that A and H satisfy the above assumptions, in particular, (Per). Then for any N there exists C N > 0 such that, for any time t ∈ R,

X 2N +δn

ˆ

~ j Aˆ j (t) 2 ≤ C N ~ N +1 |t| 1 + |t| , (58)

A(t) − L

0≤ j≤N

where δn is a universal constant. In particular, the semiclassical regime is still valid in time interval [−~−1/2 , ~−1/2 ] for every > 0, and the quantum particle stays localized close to the classical trajectory in this time interval. Proof We have to revisit the proof of Theorem 1.4 in this particular case. The proof of Proposition 2.8 is clear if we can choose the period T (z) constant in . Indeed, under this stronger assumption, for any multi-index γ ∈ N2n , there exists Cγ > 0 such that, γ for all z ∈ and all t ∈ R, we have the uniform control in time: |∂z A(8t z)| ≤ Cγ . This is easily proved using conservation of the energy and the periodicity of the flow. Using induction formula (14), the derivatives of the A j ’s are also uniformly controlled in time for the same reasons. To get the conclusion, we again follow the same method as in the C ∞ -case (see proof of Theorem 1.4). When the period is not constant but is smoothly varying, we can replace the e = f (H ), such that the Hamiltonian flow for H e has Hamiltonian H by a new one, H a constant period in . For a detailed construction of f , we refer, for example, to [DS]. In particular, we have f 0 (E) 6= 0 for E ∈ [E 1 , E 2 ] and t/ f 0 (H (z)) (z), H˜

8tH (z) = 8 γ

∀z ∈ .

We get from that |∂z A(8t z)| ≤ Cγ (1 + |t|)|γ | . We then finish the proof using the same method as for Theorem 1.4.

238


3. The analytic case To prove Theorem 1.8, we use the same strategy as for Theorem 1.4. For simplicity, we assume that H = H0 + ~H1 . The general case is not more difficult. Let us point out that we use in this section the same notation to define the Hermitian norm on Cn and on C2n , that is, k · k. First of all, we estimate the classical flow in complex domains (similar estimates are considered in [BGP]). That is, under the assumptions of Theorem 1.8, we have the following lemma. LEMMA 3.1 8t is holomorphic

in (δe−|t| 0 ), and the following estimate holds: γ k∂z 8t zk ≤ γ !δ 1−|γ | exp |γ ||t| 0 , ∀γ ∈ N2n , |γ | ≥ 1, ∀z ∈ Z .

Moreover, if 0 ≤ s ≤ t or if t ≤ s ≤ 0, for τ ∈ [0, 1], we have

k=zk ≤ τ δe−|t|0 ⇒ =(8t−s z) ≤ τ δe−|s|0 .

(59)

(60)

Proof We can assume that t > 0. Let us consider the differential equation d t 8 z = F(8t z), dt

80 z = z,

(61)

where F(z) := J ∇z H0 (z). F(·) is then holomorphic in (δ). Using Lemma 2.3 on linear differential equation (61), we easily get k=8s zk < δ, ∀s ∈ [0, t 0 ] ⇒ k∇z 8s zk < es0 , ∀s ∈ [0, t 0 ].

(62)

We also have

k=8s zk ≤ 8s z − 8s (Re z) ≤ k=zk

sup ∇z 8s (Re z + i y) .

(63)

kyk≤k=zk

Let us now assume that k=zk < δe−t0 , and let us define t 0 = sup t 00 > 0, k=8s ζ k < δ, ∀ζ, k=ζ k ≤ k=zk, ∀s ∈ [0, t 00 ] . Using (62) and (63), we get k=8t ζ k ≤ e(t −t)0 , 0

0

0

∀ζ, k=ζ k ≤ k=zk,

(64)

so that for ε > 0, we have k=8t +ε ζ k < δ, which contradicts the assumption on t 0 and proves that t 0 = t. This proves (60) for s = 0. To prove it for all s ∈ [0, t], we use the same method but take τ δe−s0 as a new δ and t − s as a new t. Finally, using smoothness for solutions of differential equations, we get that 8t is holomorphic in (δe−t0 ). Then, using Cauchy inequalities, we get (59).


239

From Lemma 3.1 and Cauchy inequalities, we also get the following estimates. 3.2 There exists a constant M > 0 such that, for every γ ∈ N2n , |γ | ≥ 1, z ∈ (δe−0|t| ), we have γ ∂z [A(8t (z)] ≤ Mγ !δ 1−|γ | e0|γ ||t| . (65) LEMMA

To prove Theorem 1.8, the analyticity assumptions on H and A are also used through the following lemma (see also [Tr]). 3.3 Let f : (δ) → C be a holomorphic function, where (δ) = {z ∈ Cd , k=zk < δ}. For τ ∈]0, 1[ , let us define k f kτ = sup f (z) , LEMMA

z∈(τ δ)

and let us assume that there exist M > 0, a ≥ 0, such that e a k f kτ ≤ M , ∀τ ∈]0, 1[. 1−τ

(66)

Then ∀γ ∈ Nd , ∀τ ∈]0, 1[ , we have γ

k∂z f kτ ≤ Mδ −|γ | (a + 1) · · · a + |γ |

e a+|γ | . 1−τ

(67)

Proof Let us start with |γ | = 1. Using the Cauchy inequality, we have ∀τ 0 ∈]0, τ [ ,

∂f δ −1

k f kτ .

0≤ ∂z j τ τ − τ0

(68)

Now using (66) with τ = τ 0 + (1 − τ 0 )/(1 + a) and the elementary inequality (1 + 1/a)a ≤ e, we get

∂f e a+1

.

0 ≤ Mδ −1 (a + 1)

∂z j τ 1 − τ0

(69)

Then, coming back to the notation τ 0 = τ , we easily prove the lemma by induction on |γ |. The proof of Theorem 1.8 is based on the following estimate.

240


PROPOSITION 3.4 Let A and H satisfy the assumptions of Theorem 1.8. Then there exist two constants K 0 and K 1 such that for all j ≥ 1, t ∈ R, τ ∈]0, 1[ , we have A j (t, z) sup z∈(τ δe−|t|0 )

≤ K0

n

e j−1 (2 j−1)|t|0 o X ( j + m)! m j − 1 j+m e |t| K 1 , (70) m−1 1−τ m! 1≤m≤ j

where

( j − 1)! j −1 = . m−1 (m − 1)!( j − m)!

Proof We prove (70) by induction on j using Lemmas 3.1–3.3. The estimate is clearly satisfied for j = 1. Indeed, according to Lemma 3.1 and formula (13), A1 (t, ·) is holomorphic in the domain (τ δe−|t|0 ). And then, from Lemma 3.2, there exists K 0 > 0 such that for any z ∈ (τ δe−|t|0 ), e 0 A1 (t, z) ≤ K 0 e|t|0 |t|. (71) 1−τ Note that by application of Lemma 3.3, e |γ | γ ∂z A1 (t, z) ≤ K 0 γ ! sup e(1+|γ |)|t|0 |t|. (72) 1−τ z∈(τ δe−|t|0 ) Now assume that (70) is satisfied for 0 ≤ ` ≤ j − 1, so that, according to Lemmas 3.1 and 3.3, for k=zk ≤ τ δe−|t|0 , γ ∂z A` (t, z) e `−1+|γ | X (` + m)! (2`−1+|γ |)|t|0 −|γ | m `−1 ≤M e δ γ! |t| K `+m . m−1 1−τ m! 1≤m≤`

(73) We now apply (14) and (73) to estimate each term in the sum, given the A j ’s. In particular, we use the identity X k − 1 j −1 = (74) m m−1 m≤k≤ j−1

and the elementary inequality γ ! ≤ |γ |! ≤ 2|γ | γ !. So we can easily prove, using standard estimates, that (70) is satisfied.

(75)


241

We then deduce the following result on the A j ’s. 3.5 There exist K 1 > 0 and K 2 > 0 such that, for every γ ∈ N2n , j ∈ N, satisfying |γ | + j ≥ 1, t ∈ R, we have γ j+|γ | j sup ∂z A j (t, z) ≤ K 1 j + |γ | !K 2 1 + |t| e(2 j−1+|γ |)|t|0 . (76) COROLLARY

z∈Z

Proof According to Lemma 3.3 and Proposition 3.4, formula (73) is valid for any j ≥ 1 in the domain (τ δe−|t|0 ), where τ is arbitrarily small and δ > 1. Note first that using (75), for 0 ≤ m ≤ j, ( j + m)! (2 j)! ≤ ≤ 4 j j!. m! j! Thus, for k=zk ≤ τ δe−|t|0 , γ ∂z A j (t, z) e j−1+|γ | X j m e(2 j−1+|γ |)|t|0 j + |γ | !(4C1 ) j |t|C1 ≤ C0 m 1−τ 0≤m≤ j e j−1+|γ | j ≤ C0 e(2 j−1+|γ |)|t|0 j + |γ | !(4K ) j 1 + C1 |t| . 1−τ The result follows by restriction to the real Euclidean space Z (taking τ small enough). End of proof of Theorem 1.8 We follow the same method as in the C ∞ -case, with the above estimates on A j , by using the appendix and Lemma 2.6. We leave the details to the reader. 4. The Gevrey case The proofs of the Gevrey estimates in Theorems 1.11 and 1.13 use the following more or less well-known lemma. 4.1 Let us consider a real-analytic map f from 1 to 2 , where j is an open set in Rn j and u is a Gevrey s-function from 2 into the complex plane C. Then u ◦ f is Gevrey s in 1 . More precisely, there exist universal positive constants δ, δ1 such that if there exist constants C0 , C1 , K 0 , K 1 and β α ∂ u(z) ≤ C0 C |β| β!s , ∂ f (x) ≤ α!K 0 K |α| , (77) x z 1 1 LEMMA

242


then we have, for every real number r such that 0 < r ≤ 1/(2K 1 ) and r ≤ 1/(2δ K 0 K 1 C1 ), α ∂ (u ◦ f )(x) ≤ δ1 C0r −|α| α!s . (78) x Proof As in L. Hörmander [Ho, Vol. 1], we fix x ∈ Rn 1 and consider the Taylor expansion for u in f (x), at order k = |α|, X ∂zβ u f (x) β F(y) = f (y) − f (x) . (79) β! |β|≤k

∂xα u

∂ yα F(y)| y=x .

Note that ◦ f (x) = Cauchy integral formula to F.

The lemma is then proved by applying the

Theorem 1.11 is proved by induction on j using Lemma 4.1 several times. The main step to prove Theorem 1.13 is the following result. 4.2 There exist K 0 > 0, K 1 > 0, such that, for all t ∈ R, j ∈ N, z ∈ Z , γ ∈ N2n , we have γ ∂z A j (t, z) ≤ K 0 K |γ |+3 j/2 1 + |t| |γ |+2 j |γ | + 3 j/2 !s ( j/2)!−1 . (80) 1 PROPOSITION

Let us recall that A j = 0 for j odd because here H has no lower order terms. It is convenient to prove estimates first in action-angle variables and afterwards to use Lemma 4.1 to get estimates in position-momentum coordinates. So let us consider the new classical observables A˜ j (t, I, ϕ) = A j (t, χ(I, ϕ + tω(I )). 4.3 There exist K 0 > 0, K 1 > 0, K 2 > 0 such that, for all t ∈ R, j ∈ N, z ∈ Z , γ ∈ N2n , we have α β ∂ ∂ A˜ 0 (t, I, ϕ) ≤ K 0 K |α|+|β| |α| + |β| !s 1 + |t| |α| (81) I ϕ 1 LEMMA

and for j ≥ 2, α β ∂ ∂ A˜ j (t, I, ϕ) ≤ K 0 K |α|+|β|+3 j/2 |α| + |β| + 3 j/2 !s−1 1 + |t| |α|+3 j/2 I ϕ 1 X |t|m j/2 − 1 · |α| + |β| + j + m ! K 2m . (82) m−1 m! 1≤m≤ j/2

Proof ˜ ϕ + tω(I )), where A˜ := A ◦ χ −1 . We know For j = 0, we have f A0 (t, I, ϕ) = A(I, that A˜ is Gevrey s and that ω is analytic, so we easily get (81) from Lemma 4.1.


243

For j ≥ 2, with some tedious but not very difficult computations, we can prove (82) by induction on j, as we have done in the analytic case (see Section 3). End of proof of Proposition 4.2 It is easily proved using estimates (82) and the systematic application of Lemma 4.1.

End of proof of Theorem 1.13 We get control of the remainder terms by the same method as in Section 2, using Lemma 2.6, the appendix, and the Calderon-Vaillancourt theorem. Remarks concerning Hamiltonians with singularities For the statement of Theorems 1.11 and 1.13, smoothness of the Hamiltonian H is required only on a flow-invariant domain of Z containing the support of the observable A. These results can then be extended, as explained briefly below, in the case of local Hamiltonians of the form X H (x, ξ ) = aα (x)ξ α |α|≤m

satisfying the following for every α, |α| ≤ m. β (H.1) ∂x aα ∈ L 1òc (Rn ), ∀ |β| ≤ |α|. (H.2) There exist a compact K ⊂ Rn and ρ > 0 such that aα is analytic in I K := {k=xk < ρ, Re x ∈ / K } and satisfies in I K , for some C > 0 and M > 0, aα (x) ≤ C 1 + |x| M . Moreover, Hˆ is supposed to be self-adjoint in L 2 (Rn ). As an example of such H , we can consider the following Coulomb-like Hamiltonian on R3 : L X cj H (x, ξ ) = kξ k2 + + W (x), kx − x ( j) k j=1

where c j ∈ R, x ( j) ∈ R3 , and W satisfies (H.2) with K = ∅. Of course, one needs to adapt to this case the previous assumptions (C.1) and (C.2) concerning the observable A. ¯ is 8t -invariant for every (C.10 ) Let be a bounded open subset of Z such that 2n n ¯ ⊂ R \ (K × R ). The observable A is such that supp(A) ⊂ . t ∈ R and (C.20 ) A is Gevrey s > 1. Under assumptions (H.1), (H.2), (C.10 ), and (C.20 ), we can prove that the singular region contributes in the Egorov theorem by an error term that is of the same order

244


as in the smooth case. This fact can be seen by standard commutator arguments and integrations by parts. We give here the main steps of the proof without going into detail. Let us introduce two C ∞ -cutoff functions χ and ζ , where ζ is supported in a small neighborood of K such that ζ = 1 on K and χ = 1 on supp(ζ ). The distance between the supports of ζ and 1 − χ is then positive. Write the commutator (N ) (t)] as follows: [ Hˆ , A\ (N ) (t) = H (N ) (t) χ + H (N ) (t)(1 − χ) + H (N ) (t) (1 − χ ). ˆ , A\ ˆ ζ A\ ˆ (1 − ζ ), A\ Hˆ , A\ (83) In the last term of the right-hand side of (83), each operator has a smooth symbol so that we can use the same computations as in the C ∞ -case with ~-pseudodifferential operators. We now explain how the first two terms give negligible contributions. We suppose here, to fix ideas, that the system is integrable in so that the Gevrey estimates on A j (t) established in Theorem 1.13 are still valid under the above assumptions. Now, using integration by parts and taking into account the support of the in[ volved symbols, we can estimate the integral kernel of operators [ Hˆ , A j (t)]χ and 2 [ ˆ H ζ A j (t)(1 − χ) to get the following L -estimate:

[ Hˆ , B]χ ˆ 2 + Hˆ ζ B(1 ˆ − χ) 2 ≤ K 0 K N N !(3s−1)/2 1 + |t| 2N ~ N . (84) 1 L L (N ) (t) d − Ad By combining (83) and (84), we then obtain the same L 2 -estimate for A(t) as in Theorem 1.13.

5. Propagation of coherent states and Ehrenfest time Let us consider a usual Gaussian coherent state φz 0 concentrated around the point z 0 = (q0 , p0 ) of the phase space, defined as follows: φz 0 := W (z 0 )η, where η(x) = (π~)−n/4 e−|x| /(2~) is the ground state of the n-dimensional harmonic oscillator K 0 = (1/2)(P 2 + Q 2 ) and where W (z 0 ) = e−(i/~)(q0 P− p0 Q) is the unitary translation operator in the phase space. Here P = −i~∂x and Qψ(x) = xψ(x). Note that W (0, 0) = Id, so that η is concentrated at the origin (see [CR] for a more complete description of the Gaussian coherent states and [PU2] for a more general ˆ class). Here we study the semiclassical limit of a quantum observable A(t) along ˆ the coherent state φz 0 through the quantity h A(t)φz 0 , φz 0 i. In other words, we study the average on A of the quantum evolution U (t)φz 0 of a well-localized wave packet φz 0 . It is well known that U (t)φz 0 stays concentrated around the classical trajectory 8t (z 0 ) (see Remark 5.2). Here we give an estimate of the duration T~ such that for t ˆ |t| ≤ T~ , h A(t)φ z 0 , φz 0 i goes to its classical limit A(8 (z 0 )) when ~ goes to zero. The following result can be interpreted as a weak version of Theorem 1.4. 2


245

PROPOSITION 5.1 Under the hypotheses of Theorem 1.4, for any > 0, we have the following uniform (in time) limit: t ˆ lim h A(t)φ (85) z 0 , φz 0 i − A(8 (z 0 )) = 0. ~&0 |t|≤((1−)/(20)) ln(1/~)

Proof ˆ Using the Wigner function A~ (t, z) of φz 0 and the Weyl ~-symbol A~ (t, z) of A(t), we get the following well-known integral: Z

2 −n ˆ A(t)φ , φ = (π~) A~ (t, z) e−|z−z 0 | /~ dz. (86) z0 z0 R2n

Then, since we are interested in the limit as ~ goes to zero, according to Theorem 1.4 ˆ in (86). So we now have to estimate the following one can substitute c A0 (t) for A(t) expression: Z 2 −n ρ(t, ~) = (π~) A 8t (z) e−|z−z 0 | /~ dz − A 8t (z 0 ) 2n ZR 2 −n = (π~) A(8t (z)) − A(8t (z 0 )) e−|z−z 0 | /~ dz. R2n

Now, using the estimate |∇z A(8t (z))| ≤ Ce0|t| for any t ∈ R (see Lemma 2.4) and the Taylor formula, we get Z 2 ρ(t, ~) ≤ C 0 ~−n |z − z 0 | e−|z−z 0 | /~ dz e0|t| R2n 00 1/2 0|t|

≤C ~

e

,

where C 0 , C 00 are constants independent on t and ~. Hence, for any t satisfying |t| ≤ ((1 − )/(20)) ln(1/~) with > 0, we have |ρ(t, ~)| ≤ C 00 ~ . Remark 5.2 (i) Since for any A ∈ O (0) we have k Aˆ − Aˆ aw k L 2 = O (~), where Aˆ aw is the anti-Wick quantization of A, we can replace Aˆ by Aˆ aw in (85) (see [HMR] for the definition and properties of anti-Wick quantization) and consider the semiclassical Husimi measures

µt,~,z 0 : A 7 −→ Aˆ aw U (t)φz 0 , U (t)φz 0 .

246


Then if we denote by δζ the Dirac measure in ζ ∈ Z , Proposition 5.1 means that we have the following uniform (in time) weak limit of measures. For any > 0, we have lim ~&0 |t|≤((1−)/(20)) ln(1/~)

(µt,~,z 0 − δ8t (z 0 ) ) = 0.

(87)

(ii) A result similar to (87) has already been obtained for some specific quantized hyperbolic maps on the torus (hyperbolic automorphisms and Baker map) (see [BDB]). (iii) In [CR] a result similar to Proposition 5.1 and (87) was obtained as a consequence of propagation results for coherent states but for smaller times: |t| ≤ ((1 − )/(60)) ln(1/~). So by the method of this paper, we have improved 1/6 by 1/2. (iv) Now a natural question is, What happens for times |t| ≥ (1/(20)) ln(1/~)? An answer is given in [BDB] for quantized hyperbolic maps on the torus. After the critical time T~ := (1/(20)) ln(1/~), there appears a new regime, called a “mixing regime,” where the Dirac measure at 8t (z 0 ) is replaced by some absolutely continuous measure on some flow-invariant set (delocalization phenomenon). For Schrödinger operators, and more generally for classical Hamiltonians H , we ˆ have already seen that the semiclassical expansion for the quantum observable A(t) is valid for |t| ≤ ((2 − )/(30)) ln(1/~) for every 0 < . This result suggests that the mixing regime is also accessible for Schrödinger operators, at least for some examples. In [DBR] we shall consider this problem, in particular, when the starting point z 0 is a hyperbolic fixed point of the flow. 6. Examples The goal of this section is to check, through two examples, that the error bounds in Theorems 1.4 and 1.8 are in some sense optimal. To this end, we establish, for these examples, exponential lower bounds of the remainder term. We have already seen in previous sections that, for some particular (but important) dynamical systems, this phenomenon does not hold. We suppose in our examples that t > 0. For the complementary case t < 0, we have the same results. Example 6.1 Let us consider the semiclassical Hamiltonian H = H0 + ~H1 , where H0 , H1 : R2 −→ R are defined as follows: H0 (q, p) =

1 2 ( p − q 2 ), 2

H1 (q, p) = εq.


247

The linear Hamiltonian flow 8t : R2 −→ R2 associated to H0 is given by 8t (q, p) = (q cosh t + p sinh t, q sinh t + p cosh t). If we consider the observable A(q, p) = q, according to Theorem 1.4 we have for any coherent states φz concentrated around the point z = (q, p),

ˆ ˆ 0 (t)φz , φz A(t)φ z , φz − A lim = A1 (t, z). ~→0 ~ Using formula (14), for any z ∈ R2 we have A1 (t, z) = ε(cosh t − 1). So

A(t) ˆ − Aˆ 0 (t) 2 ˆ ˆ 0 (t)φz , φz A(t)φ z , φz − A L ≥ lim = ε(cosh t − 1). lim inf ~→0 ~→0 ~ ~ (88) Then, according to (88), we can assert that the remainder in the Egorov theorem is exponentially large in time. Indeed, suppose that H and A are as above. More precisely, there exists a function M : R∗ −→ R such that

A(t) ˆ − Aˆ 0 (t) 2 ≤ ~M(t), ∀~ ∈]0, 1], ∀t > 0; (89) L then there exists Cε > 0 such that M(t) ≥ Cε et , ∀t > 0. Example 6.2 (Nonsymmetric double well: Local phenomenon around an unstable fixed point) Let us consider the classical Hamiltonian H defined on R2 as follows: H (q, p) =

p2 + V (q), 2

where V 0 (0) = 0, V 00 (0) = −1, and V (3) (0) 6= 0. The condition on the third derivative of V implies that the double well is nonsymmetric. It ensures that the coefficient A2 (t, 0) in the formal expansion of the Heisenberg observable is not zero. Consider, for example, the following potential: 1 q3 1 2 (q − 1)2 − + χ(q), where χ ∈ C0∞ (R) and χ(0) 6= 0. 4 4 3 Note that the associated linearized flow at the equilibrium point is cosh t sinh t t ∇z 8 (0, 0) = . sinh t cosh t V (q) =

According to Theorem 1.4, if φ0 is a coherent state concentrated around (0, 0), we have, as in the previous example,

ˆ ˆ 0 (t)φ0 , φ0 A(t)φ 0 , φ0 − A lim = A2 (t, 0). (90) ~→0 ~2

248


Suppose that A is a classical observable satisfying A(q, p) := f (q) with f 0 (0) = f 00 (0) = 0 and f (3) (0) 6= 0. According to formula (14), Z t (3) (3) A2 (t, 0) = 0(0, 3)V (0) f (0) (sinh s)3 ds. 0

Then, as in the previous example, we get the following inequality, for some C > 0:

A(t) ˆ − Aˆ 0 (t) 2 L ≥ Ce3t , ∀t > 0. lim inf ~→0 ~2 Therefore, one concludes that if M : R∗ −→ R is a function satisfying

ˆ − Aˆ 0 (t) 2 ≤ ~2 M(t), ∀t ≥ 0, ∀~ ∈]0, 1], A(t) L

(91)

then M(t) ≥ Ce3t , ∀t > 0. Appendix. Product of observables Let us first recall the formal product rule for quantum observables with Weyl quantizaˆ Bˆ = C. ˆ tion. Let A, B ∈ S (Z ). We look for a semiclassical observable C such that A· Some computations with the Fourier transform give the following formula (see [Ho]): i~ C(q, p) = exp σ (Dq , D p ; Dq 0 , D p0 ) A(q, p)B(q 0 , p 0 )|(q, p)=(q 0 , p0 ) , 2

(92)

where σ is the symplectic bilinear form (1) and D = i −1 ∇. By expanding the exponential term, we get C(q, p) =

X ~j i j≥0

j! 2

j σ (Dq , D p ; Dq 0 , D p0 ) A(q, p)B(q 0 , p 0 )|(q, p)=(q 0 , p0 ) ,

(93)

so that C(q, p) is a formal power series in ~ with coefficients given by C j (q, p) =

1 2j

X |α+β|= j

(−1)|β| β α (Dq ∂ p A) · (Dqα ∂ βp B)(q, p). α!β!

(94)

It is well known that if A, B are observables with polynomial growth, then C is a C ∞ -semiclassical observable (see, e.g., [Ho], [Ro]). Here we need more accurate remainder estimates. Let us denote A#B = C, z = (q, p) ∈ Z , and for every N ≥ 1, X A#B(z) − ~ j C j (z) =: R N (A, B; z; ~). (95) 0≤ j≤N

The main result of this appendix is the following theorem.


249

THEOREM A.1 There exists K n > 0. For every m ∈ N, m ≥ 4n, and every s > 4n, there exists a constant ρn,m,s such that for every A, B ∈ S (Z ), every N ≥ 1, and every multi-index γ , the following estimate holds for every z ∈ Z : X γ N +|γ | ~ j C j (z) ≤ ~ N +1 ρn,m,s K n (N !)−1 ∂z A#B(z) − 0≤ j≤N

· sup (1 + u 2 + v 2 )(s−m)/2 |∂u(α,β)+µ A(u + z)||∂v(β,α)+ν B(v + z)| , (∗)

(96)

where sup(∗) means that the supremum holds under the conditions u, v ∈ Z , |µ| + |ν| ≤ m + |γ |, |α| + |β| = N + 1 (µ, ν ∈ N2n , α, β ∈ Nn ). Proof We follow [DR]. By Fourier transform computations and the Taylor formula, we get the following formula: R N (A, B; z; ~) =

1 i~ N +1 N! 2

1

Z 0

(1 − t) N R N ,t (z; ~) dt,

where R N ,t (z; ~) = (2π~t)

−2n

Z Z

i exp − σ (u, v) σ N +1 (Du , Dv )A(u + z)B(v + z) du dv. 2t~ Z ×Z (97)

We use the following lemma to estimate R N ,t (z; ~). LEMMA A.2 Let us consider F ∈ S (Z × Z ) and the integral Z Z I (λ) = λ2n exp −iλσ (u, v) F(u, v) du dv.

(98)

Z ×Z

Then, for every real number s > 4n and every integer m ≥ 4n, there exists κ(n, s, m) > 0 depending only on n, s, m (but independent of F) such that the following estimate holds: I (λ) ≤ κ(n, s, m) sup (1 + u 2 + v 2 )s−m/2 ∂ µ ∂ ν F(u, v) . (99) u v u,v∈Z |µ|+|ν|≤m

250


Proof This lemma is more or less standard (see [Ho]). For completeness, we give here a direct proof. Let us introduce a cutoff χ0 , C ∞ on R, χ0 (x) = 1 for |x| ≤ 1/2, and χ0 (x) = 0 for |x| ≥ 1. We split I (λ) into three pieces: Z Z I0 (λ) = λ2n exp −iλσ (u, v) , χ0 λ(u 2 + v 2 ) F(u, v) du dv, (100) Z ×Z Z Z I1 (λ) = λ2n exp −iλσ (u, v) Z ×Z · 1 − χ0 (u 2 + v 2 ) χ0 λ(u 2 + v 2 ) F(u, v) du dv, (101) Z Z I2 (λ) = λ2n exp −iλσ (u, v) 1 − χ0 (u 2 + v 2 ) F(u, v) du dv. (102) Z ×Z

For I0 (λ), we easily have I0 (λ) ≤ ω4n

sup F(u, v) ,

(103)

u 2 +v 2 ≤1

where ω4n is the volume of the unit ball in Z 2 . For I1 (λ) and I2 (λ), we integrate by parts with the differential operator L=

∂ ∂ i − J v , J u ∂v ∂u u 2 + v2

(104)

where J is the matrix associated to the symplectic form (σ (u, v) = hJ u, vi). Performing 4n integrations by parts, we can see that there exists a constant cn such that I1 (λ) ≤ cn

sup u 2 +v 2 ≤1 |µ|+|ν|≤4n

µ ν ∂ ∂ F(u, v) . u v

(105)

Similarly, performing m integrations by parts, we get for a constant c(n, s, m), I2 (λ) ≤ c(n,s,m) sup (1 + u 2 + v 2 )(s−m)/2 ∂ µ ∂ ν F(u, v) . (106) u v u,v∈Z |µ|+|ν|≤m

Now we can complete the proof of the theorem by using Lemma 6, the Leibniz formula, and the following elementary estimate, using in Z the coordinates u = (x, ξ ), v = (y, η): N σ (∂x , ∂ξ ; ∂ y , ∂η )F(x, ξ )G(y, η) β ≤ (2n) N sup ∂xα ∂ξ A(x, ξ )∂ yβ ∂ηα B(y, η) . (107) |α|+|β|=N


251

Remark A.3 We can easily extend estimate (96) for observables A, B with polynomial growth at infinity by choosing m large enough to get a finite right-hand side. Let us assume that A ∈ O (µ A ), B ∈ O (µ B ), where µ A , µ B ∈ R. Then we can apply (96) to 2 2 Aε (u) = e−εu A(u) and Bε (v) = e−εv B(v) for ε > 0 and pass to the limit ε → 0 with m − s ≥ µ A + µ B . Acknowledgments. The authors are grateful to the referees for their stimulating questions, which have helped to improve this paper. References [Ar]

V. I. ARNOLD, Mathematical Methods of Classical Mechanics, Grad. Texts in Math.

[BGP]

D. BAMBUSI, S. GRAFFI, and T. PAUL, Long time semiclassical approximation of

60, Springer, New York, 1978. MR 57:14033b 232

[Be]

[BDB]

[Bo] [BK] [Ce]

[Ch] [CR]

[Co] [DR]

[DBR]

quantum flows: A proof of the Ehrenfest time, Asymptot. Anal. 21 (1999), 149–160. MR 2001i:81084 223, 232, 238 R. BEALS, “Propagation de singularités pour les opérateurs du type Dt2 − b ” in ´ Journées “Equations aux dérivées partielles” (Saint-Jean-de-Monts, France, 1980), Soc. Math. France, Montrouge, 1980, conf. no. 19. MR 81j:35002 225 ` F. BONECHI and S. DE BIEVRE , Exponential mixing and | ln ~| time scales in quantized hyperbolic maps on the torus, Comm. Math. Phys. 211 (2000), 659–686. MR 2001h:81075 246 A. BOULKHEMAIR, L 2 estimates for Weyl quantization, J. Funct. Anal. 165 (1999), 173–204. MR 2001f:35454 236 ´ , Pseudo-differential operators and Gevrey L. BOUTET DE MONVEL and P. KREE classes, Ann. Inst. Fourier (Grenoble) 17 (1967), 295–323. MR 37:1760 228 L. CESARI, Asymptotic Behavior and Stability Problems in Ordinary Differential Equations, 2d ed., Ergeb. Math. Grenzgeb. (2) 16, Springer, Berlin, 1963. MR 27:1661 B. V. CHIRIKOV, A universal instability of many-dimensional oscillator systems, Phys. Rep. 52 (1979), 264–379. MR 80h:70022 223 M. COMBESCURE and D. ROBERT, Semiclassical spreading of quantum wave packets and applications near unstable fixed points of the classical flow, Asymptot. Anal. 14 (1997), 377–404. MR 98g:81040 223, 244, 246 L. COMTET, Analyse combinatoire, Vol. 1, Collection SUP: “Le Mathématicien” 4, Presses Univ. France, Paris, 1970. MR 41:6697 233 M. DAUGE and D. ROBERT, “Weyl’s formula for a class of pseudodifferential operators with negative order on L 2 (Rn )” in Pseudodifferential Operators (Oberwolfach, Germany, 1986), Lecture Notes in Math. 1256, Springer, Berlin, 1987, 91–122. MR 88k:35150 249 ` S. DE BIEVRE and D. ROBERT, Semiclassical propagation and the log(~−1 ) time-barrier, work in preparation. 246

252

[DS]

[Eg] [HJ] [HMR] [HR] [Ho]

[Iv] [PU1] [PU2] [Ro] [Sj] [Tr]

[Za]

BOUZOUINA and ROBERT ¨ M. DIMASSI and J. SJOSTRAND , Spectral Asymptotics in the Semi-Classical Limit,

London Math. Soc. Lecture Note Ser. 268, Cambridge Univ. Press, Cambridge, England, 1999. MR 2001b:35237 237 YU. V. EGOROV, On canonical transformations of pseudodifferential operators (in Russian), Uspekhi Mat. Nauk 24, no. 5 (1969), 235–236. MR 42:657 225 G. A. HAGEDORN and A. JOYE, Semiclassical dynamics with exponentially small error estimates, Comm. Math. Phys. 207 (1999), 439–465. MR 2000j:81062 223, 232 B. HELFFER, A. MARTINEZ, and D. ROBERT, Ergodicité et limite semi-classique, Comm. Math. Phys. 109 (1987), 313–326. MR 88e:81029 245 B. HELFFER and D. ROBERT, Calcul fonctionnel par la transformation de Mellin et opérateurs admissibles, J. Funct. Anal. 53 (1983), 246–268. MR 85i:47052 ¨ L. HORMANDER , The Analysis of Linear Partial Differential Operators, I, II; III, IV, Grundlehren Math. Wiss. 256, 257; 274, 275, Springer, Berlin, 1983; 1985. MR 85g:35002a, MR 85g:35002b; MR 87d:35002a, MR 87d:35002b 223, 225, 242, 248, 250 V. IVRII, Microlocal Analysis and Precise Spectral Asymptotics, Springer Monogr. Math., Springer, Berlin, 1998. MR 99e:58193 223, 232 T. PAUL and A. URIBE, The semi-classical trace formula and propagation of wave packets, J. Funct. Anal. 132 (1995), 192–249. MR 97c:58160 227 , On the pointwise behavior of semi-classical measures, Comm. Math. Phys. 175 (1996), 229–258. MR 97a:58189 244 D. ROBERT, Autour de l’approximation semi-classique, Progr. Math. 68, Birkhäuser, Boston, 1987. MR 89g:81016 223, 225, 226, 228, 248 ¨ J. SJOSTRAND , Singularités analytiques microlocales, Astérique 95, Soc. Math. France, Montrouge, 1982. MR 84m:58151 228 ` F. TREVES , Introduction to Pseudodifferential and Fourier Integral Operators, Vol. 2: Fourier Integral Operators, Univ. Ser. Math., Plenum, New York, 1980. MR 82i:58068 239 G. M. ZASLAVSKY, Stochasticity in quantum systems, Phys. Rep. 80 (1981), 157–250. MR 83f:81032 223

Bouzouina Département de Mathématiques, Université de Reims, BP 1039, F-51687 Reims CEDEX 2, France; [email protected] Robert Centre National de la Recherche Scientifique, Unité Mixte de Recherche 6629, Département de Mathématiques, Université de Nantes, 2 rue de la Houssinière, 44322 Nantes CEDEX 03, France; [email protected]


THE SIGNATURE OF A TORIC VARIETY NAICHUNG CONAN LEUNG and VICTOR REINER

Abstract We identify a combinatorial quantity (the alternating sum of the h-vector) defined for any simple polytope as the signature of a toric variety. This quantity was introduced by R. Charney and M. Davis in their work, which in particular showed that its nonnegativity is closely related to a conjecture of H. Hopf on the Euler characteristic of a nonpositively curved manifold. We prove positive (or nonnegative) lower bounds for this quantity under geometric hypotheses on the polytope and, in particular, resolve a special case of their conjecture. These hypotheses lead to ampleness (or weaker conditions) for certain line bundles on toric divisors, and then the lower bounds follow from calculations using the Hirzebruch signature formula. Moreover, we show that under these hypotheses on the polytope, the ith L-class of the corresponding toric variety is (−1)i times an effective class for any i. 1. Introduction Much attention in combinatorial geometry has centered on the problem of characterizing which nonnegative integer sequences ( f 0 , f 1 , . . . , f d ) can be the f -vector f (P) of a d-dimensional convex polytope P; that is, f i is the number of i-dimensional faces of P (see [3] for a nice survey). For the class of simple polytopes, this problem was completely solved by the combined work of L. Billera and C. Lee [4] and of R. Stanley [33]. A simple ddimensional polytope is one in which every vertex lies on exactly d edges. P. McMullen’s g-conjecture (now the g-theorem) gives necessary (see [33]) and sufficient (see [4]) conditions for ( f 0 , f 1 , . . . , f d ) to be the f -vector of a simple d-dimensional polytope. Stanley’s proof of the necessity of these conditions showed that they have a very natural phrasing in terms of the cohomology of the toric variety X 1 associated to the (inner) normal fan 1 of P, and then the hard Lefschetz theorem for X 1 played a crucial role. This construction of X 1 from 1 requires that P be rational, that is, DUKE MATHEMATICAL JOURNAL c 2002 Vol. 111, No. 2, Received 19 January 2000. Revision received 4 December 2000. 2000 Mathematics Subject Classification. Primary 52B05, 14M25, 57R20. Leung’s work partially supported by National Science Foundation grant number DMS-9803616. Reiner’s work partially supported by National Science Foundation grant number DMS-9877047. 253

254

LEUNG and REINER

that its vertices all have rational coordinates with respect to some lattice, which can be achieved by a small perturbation that does not affect f (P). Later, McMullen [29] demonstrated that one can construct a ring 5(P), isomorphic (with a doubling of the grading) to the cohomology ring of X 1 if P is rational, and proved that 5(P) formally satisfies the hard Lefschetz theorem, using only tools from convex geometry. In particular, he recovered the necessity of the conditions of the g-theorem in this way. This paper shares a similar spirit with Stanley’s proof. We attempt to use further facts about the geometry of X 1 to deduce information about the f -vector f (P) under certain hypotheses on P. The starting point of our investigation is an interpretation of the alternating sum of the h-vector which follows from the hard Lefschetz theorem. Recall that for a simple polytope P, the h-vector is the sequence Pd h(P) = (h 0 , h 1 , . . . , h d ) defined as follows. If we let f (P, t) := i=0 f i (P)t i , then h(P, t) :=

d X

h i (P)t i = f (P, t − 1).

i=0

The h-vector has a topological interpretation: h i is the 2ith Betti number for X 1 , or the dimension of the ith-graded component in McMullen’s ring 5(P). Part of the conditions of the g-theorem are the Dehn-Sommerville equations h i = h d−i , which reflect Poincaré duality for X 1 . Define the alternating sum σ (P) :=

d X

(−1)i h i (P)

i=0

= h(P, −1) = f (P, −2) =

d X

f i (P)(−2)i ,

i=0

a quantity that is (essentially) equivalent to one arising in a conjecture of Charney and Davis [6], related to a conjecture of Hopf (see Section 5). Note that when d is odd, σ (P) vanishes by the Dehn-Sommerville equations. When d is even, we have the following result (see Section 2). THEOREM 1.1 Let P be a simple d-dimensional polytope, with d even. Then σ (P) is the signature of the quadratic form Q(x) = x 2 , defined on the d/2 th-graded component of McMullen’s ring 5(P). In particular, when P has rational vertices, σ (P) is the signature or index σ (X 1 ) of the associated toric variety X 1 .

An important special case of the previously mentioned Charney-Davis conjecture asserts that a certain combinatorial condition on P (namely, that of 1 being a flag

THE SIGNATURE OF A TORIC VARIETY

255

complex; see Section 5) implies (−1)d/2 σ (P) ≥ 0. In this paper we prove this conjecture when P satisfies certain stronger geometric conditions. We also give further conditions that give lower bounds on (−1)d/2 σ (P). In order to state these results, we give rough definitions of some of these conditions here (see Section 3 for the actual definitions). Say that the fan 1 is locally convex (resp., locally pointed convex, locally strongly convex) if every 1-dimensional cone in 1 has the property that the union of all cones of 1 containing it is convex (resp., pointed convex, strongly convex). For example (see Propositions 4.1 and 4.10), if each angle in every 2-dimensional face of P is nonacute (resp., obtuse), then 1 is locally convex (resp., locally strongly convex). It turns out that 1 being locally convex implies that it is flag (Proposition 5.3). For a simple polytope P with rational vertices, we define an integer m(P) which measures how singular X 1 is. To be precise, let P in Rd be rational with respect to some lattice M, and then m(P) is defined to be the least common multiple over all d-dimensional cones σ of the normal fan 1 of the index [N : Nσ ], where N is the lattice dual to M and Nσ is the sublattice spanned by the lattice vectors on the extremal rays of σ . Note that the condition m(P) = 1 is equivalent to the smoothness of the toric variety X 1 , and such polytopes P are called Delzant in the symplectic geometry literature (e.g., [21]). Now we can state the following theorem. 1.2 Let P be a rational simple d-dimensional polytope with d even, and let 1 be its normal fan. (i) If 1 is locally convex, then THEOREM

(−1)d/2 σ (P) ≥ 0. (ii)

If 1 is locally pointed convex, then (−1)d/2 σ (P) ≥

(iii)

f d−1 (P) . 3m(P)d−1

If 1 is locally strongly convex, then (−1)d/2 σ (P) ≥ coefficient of x d in

h

i td −1 f (P, t ) . t7 →1−x/ tan(x) m(P)d−1

We defer a discussion of the relation between Theorem 1.2(i) and the Charney-Davis conjecture to Section 5. It is amusing to see what Theorem 1.2 says beyond the gtheorem, in the special case where d = 2, that is, when P is a (rational) polygon. The g-theorem says exactly that f 1 = f 0 ≥ 3,

256

LEUNG and REINER

or in other words, every polygon has the same number of edges as vertices, and this number is at least 3. Since (−1)d/2 σ (P) = f 0 (P) − 4, Theorem 1.2(i) tells us that when 1 is locally convex, we must have f 0 ≥ 4. In other words, triangles cannot have a normal fan 1 which is locally convex, as one can easily check. For d = 2, the conditions that 1 is locally pointed convex or locally strongly convex coincide, and Theorem 1.2(ii) and 1.2(iii) both assert that under these hypotheses a (rational) polygon P must have f 0 (P) − 4 ≥

f 1 (P) f 0 (P) = , 3m(P) 3m(P)

or, after a little algebra, f 0 (P) ≥

12 . 3 − 1/m(P)

(1)

Since the right-hand side is strictly greater than 4, we conclude that a quadrilateral P cannot have 1 locally pointed convex nor locally strongly convex. This agrees with an easily checked fact: a quadrilateral P satisfies the weaker condition of having 1 locally convex if and only if P is a rectangle, and rectangles fail to have 1 locally pointed convex. On the other hand, inequality (1) also implies a not-quite-obvious fact: even though a (rational) pentagon can easily have 1 locally strongly convex, this is impossible if m(P) = 1; that is, there are no Delzant pentagons with this property. It is a fun exercise to show directly that no such pentagon exists and to construct a Delzant hexagon with this property. In fact, in the context of algebraic geometry, the proof of Theorem 1.2 gives the following stronger assertion, valid for rational simple polytopes of any dimension d (not necessarily even) about the expansion of the total L-class L(X ) = L 0 (X ) + L 1 (X ) + · · · + L d/2 (X ), where L i (X ) is a cycle in C H i (X )Q , the Chow ring of X . 1.3 Let X = X 1 be a complete toric variety X associated to a simplicial fan 1. If 1 is locally strongly convex (resp., locally convex), then for each i we have that (−1)i L i (X ) is effective (resp., either effective or zero). THEOREM

For instance, when i = 1, this implies that if 1 is locally convex, then Z c12 (X ) − 2c2 (X ) · H1 · . . . · Hd−2 ≤ 0, X


257

where {Hi } are any ample divisor classes. This is reminiscent of the Chern number inequality for the complex spinor bundle of X when this bundle is stable with respect to all polarizations (see, e.g., [27]). Notice that if 1 is not locally convex, (−1)i L i (X ) need not be effective. For example, if 1 is the normal fan of the standard 2-dimensional simplex having vertices at (0, 0), (1, 0), (0, 1), then X is the complex projective plane, and −L 1 (X ) is represented by the negative of the Poincaré dual of a point. 2. The alternating sum as signature We wish to prove Theorem 1.1, which we repeat here. THEOREM 1.1 Let P be a simple d-dimensional polytope, with d even. Then σ (P) is the signature of the quadratic form Q(x) = x 2 , defined on the d/2 th-graded component of McMullen’s ring 5(P). In particular, when P has rational vertices, σ (P) is the signature or index σ (X 1 ) of the associated toric variety X 1 .

Proof Taking r = d/2 in a result of McMullen [29, Theorem 8.6], we find that the quadratic form (−1)d/2 Q(x) on the d/2 th-graded component of 5(P) has d/2 X

(−1)i h d/2−i (P) positive eigenvalues

(2)

i=0

and d/2−1 X

(−1)i h d/2−i−1 (P) negative eigenvalues.

(3)

i=0

Consequently, the signature σ (Q) of Q is σ (Q) = (−1)

d/2

d/2 d/2−1 hX i X i (−1) h d/2−i (P) − (−1)i h d/2−i−1 (P) i=0

=

d X

i=0

(−1)i h i (P),

i=0

where the second equality uses the Dehn-Sommerville equations (see [29, Section 4]): h i (P) = h d−i (P).

258

LEUNG and REINER

The second assertion of the theorem follows immediately from McMullen’s identification of the ring 5(P) with the quotient of the Stanley-Reisner ring of 1 by a certain linear system of parameters (see [29, Section 14]), which is known to be isomorphic (after a doubling of the grading) with the cohomology of X 1 (see [18, Section 5.2]). Remark 2.1 Starting from any complete rational simplicial fan 1, one can construct a toric variety X 1 that is complete, but not necessarily projective, and satisfies Poincaré duality. The h-vector for 1 can still be defined and again has an interpretation as the even-dimensional Betti numbers of X 1 (see [18, Section 5.2]). We suspect that the alternating sum of the h-vector is still the signature of this complete toric variety. Generalizing in a different direction, to any polytope P that is not necessarily simple, one can associate the normal fan 1 and a projective toric variety X 1 . Although the (singular) cohomology of X 1 does not satisfy Poincaré duality, its intersection cohomology (in middle perversity) I H · (X 1 ) does. There is a combinatorially defined generalized h-vector that computes these I H · Betti numbers (see [34]). Moreover, using the hard Lefschetz theorem for intersection cohomology and the fact that X 1 is a finite union of affine subvarieties, the alternating sum of the generalized h-vector equals the signature of the quadratic form on I H · (X 1 ) defined by the intersection product. Remark 2.2 The special case of the second assertion in Theorem 1.1 is known when X 1 is smooth (i.e., when P is a Delzant polytope) (see [31, Theorem 3.12(3)]). 3. Lower bounds derived from the signature theorem The goal of this section is to explain the various notions used in Theorem 1.2 and to prove this theorem. We begin with a d-dimensional lattice M ∼ = Zd and its associated real vector space MR = M ⊗Z R. A polytope P in MR is the convex hull of a finite set of points in MR . We say that P is rational if these points can be chosen to be rational with respect to the lattice M. The dimension of P is the dimension of the smallest affine subspace containing it. A face of P is the intersection of P with one of its supporting hyperplanes, and a face is always a polytope in its own right. Vertices and edges of P are zero-dimensional and 1-dimensional faces, respectively. Every vertex of a ddimensional polytope lies on at least d edges, and P is called simple if every vertex lies on exactly d edges. Let N = Hom(M, Z) be the dual lattice to M, and let NR = N ⊗Z R be the dual


259

vector space to MR , with the natural pairing MR ⊗ NR → R denoted by h·, ·i. For a polytope P in MR , the normal fan 1 is the following collection of polyhedral cones in NR : 1 = {σ F : F a face of P}, where σ F := v ∈ NR : hu, vi ≤ hu 0 , vi for all u ∈ F, u 0 ∈ P . Note that • the normal fan 1 is a complete fan; that is, it exhausts NR ; • 1 is a rational fan, in the sense that its rays all have rational slopes, if P is rational; • if P is d-dimensional, then every cone σ F in 1 is pointed; that is, it contains no proper subspaces of NR ; • P is a simple polytope if and only if 1 is a simplicial fan; that is, every cone σ in 1 is simplicial in the sense that its extremal rays are linearly independent. We next define several affinely invariant conditions on a complete simplicial fan 1 in NR (and hence on simple polytopes P in MR ) which appear in Theorem 1.2. For any collection of polyhedral cones 1 in NR , let |1| denote the support of 1, that is, the union of all of its cones as a point set. Define the star and link of one of the cones σ in 1 similarly to the analogous notions in simplicial complexes: star1 (σ ) is the subfan consisting of those cones τ in 1 such that σ, τ lie in some common cone of 1, while link1 (σ ) is the subfan of star1 (σ ) consisting of those cones that intersect σ only at the origin. For a ray (i.e., a 1-dimensional cone) ρ of 1, say that the fan star1 (ρ) is • convex if its support |star1 (ρ)| is a convex set in the usual sense, • pointed convex if |star1 (ρ)| is convex and contains no proper subspace of NR , • strongly convex if furthermore for every cone σ in link1 (ρ), there exists a linear hyperplane H in NR which supports star1 (ρ) and whose intersection with star1 (ρ) is exactly σ . Say that 1 is locally convex (resp., locally pointed convex, locally strongly convex) if every ray ρ of 1 has star1 (ρ) convex (resp., pointed convex, strongly convex). One has the easy implications locally strongly convex ⇒ locally pointed convex ⇒ locally convex. We recall here that the affine-lattice invariant m(P) for a rational polytope P was defined (in the introduction) to be the least common multiple of the positive integers [N : Nσ ] as σ runs over all d-dimensional cones in 1. Here Nσ is the d-dimensional sublattice of N generated by the lattice vectors on the d extremal rays of σ . In a sense m(P) measures how singular X 1 is (see [18, Section 2.6]), with m(P) = 1 if and only if X 1 is smooth, in which case we say that P is Delzant.

260

LEUNG and REINER

We can now recall the statements of Theorems 1.2 and 1.3. 1.2 Let P be a simple d-dimensional polytope in MR , which is rational with respect to M, and let 1 be its normal fan in NR . Assume d is even. (i) If 1 is locally convex, then THEOREM

(−1)d/2 σ (P) ≥ 0. (ii)

If 1 is locally pointed convex, then (−1)d/2 σ (P) ≥

(iii)

f d−1 (P) . 3m(P)d−1

If 1 is locally strongly convex, then (−1)d/2 σ (P) ≥ coefficient of x d in

h

i td −1 f (P, t ) . t7→1−x/ tan(x) m(P)d−1

THEOREM 1.3 Let X = X 1 be a complete toric variety X associated to a simplicial fan 1, and let the expansion of the total L-class be

L(X ) = L 0 (X ) + L 1 (X ) + · · · + L d/2 (X ), where L i (X ) is a cycle in C H i (X )Q , the Chow ring of X . If 1 is locally strongly convex (resp., locally convex), then for each i we have that (−1)i L i (X ) is effective (resp., either effective or zero). The remainder of this section is devoted to the proof of these theorems. We begin by recalling some toric geometry. As a general reference for toric varieties, we rely on W. Fulton [18], although many of the facts we use can also be found in T. Oda’s book [31] or V. Danilov’s survey article [9]. Let X denote the toric variety X 1 . Simpleness of P implies that X is an orbifold (see [18, Section 2.2]). Recall that irreducible toric divisors∗ on X correspond in a one-to-one fashion with the codimension 1 faces of P, or to 1-dimensional rays in the normal fan 1. Number these toric divisors on X as D1 , . . . , Dm . Intersection theory for these Di ’s is studied in [18, Chapter 5]. Every Di is a toric variety in its own Sm right with at worst orbifold singularities. Moreover D = i=1 Di is a simple normal crossing divisor on X (see [18, Section 4.3]). Next we want to express the signature of X in terms of these Di ’s. When X is a smooth variety, a consequence of the hard Lefschetz theorem is that its signature ∗ Actually,

these are Q-Cartier divisors on the orbifold X .


261

σ (X ) can be expressed in terms of the Hodge numbers of X as follows (see [23, Theorem 15.8.2]): d X σ (X ) = (−1)q h p,q (X ). p,q=0 p

By the Dolbeault theorem, (X ) = dim H q (X, X ), and hence the signature can be expressed in terms of twisted holomorphic Euler characteristics h p,q

σ (X ) =

d X

p

χ(X, X ),

p=0

where χ (X, E) := we can write

Pd

q q=0 (−1)

dim H q (X, E). Using the Riemann-Roch formula, Z p p χ(X, X ) = ch( X ) Td X , X

where ch is the Chern character and Td X is the Todd class of X . Therefore σ (X ) =

Z X d X p=0

p

ch( X ) Td X .

P p When X is smooth, dp=0 ch( X ) Td X equals the Hirzebruch L-class L (X ) of X (see [23, Theorem 15.8.2, p. 16], for example), and we recover the Hirzebruch signature formula Z σ (X ) =

L (X ) .

(4)

X

If X is a projective variety with at worst orbifold singularities, the hard Lefschetz, Dolbeault, and Riemann-Roch theorems continue to hold, and we can take the sum Pd p p=0 ch( X ) Td X as a definition of L(X ). Since we can express L (X ) in terms of Chern roots of the orbibundle 1X and Chern classes for orbibundles satisfy the same functorial properties as for the usual Chern classes, the same holds true for L (X ). For example, we use the splitting principle in the proof of the next lemma, where we write L (X ) in terms of toric data.∗

∗

For a general orbifold X , not necessarily an algebraic variety, T. Kawasaki [26] expressed the signature of X in terms of the integral of certain curvature forms, thus generalizing the Hirzebruch signature formula in a different way.

262

LEUNG and REINER

LEMMA

3.1

We have (−1)d/2 σ (X ) = (−1)d/2

Z

L (X ) X

=

d/2 X

2n

1 bn 1 · · · bn p (−1) p Di2n · . . . · Di p p , 1

X

p=1 n 1 +···+n p =d/2 n i >0;i 1 0;i 1 0;i 1 0, we have hπ(m j ), π(n)i > 0 by virtue of the obtuseness of P, and hπ(n), π(n)i is always nonnegative. To prove (i), we assume that P is nonacute and that there is some choice of a dependence as in (7) such that β > 0. Then similar considerations in (8) imply that we must have hπ(n), π(n)i = 0, that is, π(n) = 0. This would imply hn i , ni = 0 for

272

LEUNG and REINER

each i in F. To reach a contradiction from this, take the inner product with n on both sides of (7) to obtain X 0 = β hn, ni + β j hm j , ni. j∈G

Nonacuteness (even in codimension 1) of P implies hm j , ni ≥ 0, and hn, ni is always positive, so this last equation is a contradiction of β > 0. One source of nonacute simple polytopes is finite Coxeter groups (see [24, Chapter 1] for background). Recall that a finite Coxeter group is a finite group W, acting on a Euclidean space and generated by reflections. Given a finite Coxeter group W, there is a set of (normalized) roots 8 associated by taking all the unit normals of reflecting hyperplanes. Let Z be the zonotope (see [36, Section 7.3]) associated with 8, that is, nX o cα α : 0 ≤ cα ≤ 1 . Z= α∈8

4.5 The zonotope Z associated to any finite Coxeter group W is nonacute and simple. Furthermore, Z is obtuse in codimension 1 if W is irreducible. PROPOSITION

Proof We refer to [24] for all facts about Coxeter groups used in this proof. By general facts about zonotopes (see [36, Section 7.3]), the normal fan 1 of Z is the complete fan cut out by the hyperplanes associated with reflections in W. The maximal cones in this fan are the Weyl chambers of W , which are all simplicial cones. Hence Z is a simple polytope. To show that Z is nonacute, we must show that each of its faces is nonacute in codimension 1. However, these faces are always affine translations of Coxeter zonotopes corresponding to standard parabolic subgroups of W . So we only need to show that Z itself is nonacute in codimension 1. This is equivalent to showing that every pair of rays in 1 which spans a 2-dimensional cone forms a nonobtuse angle. Because W acts transitively on the Weyl chambers in 1, we may assume that this pair of rays lies in the fundamental Weyl chamber; that is, we may assume that these rays come from the dual basis to some choice of simple roots α1 , . . . , αd . Since every choice of simple roots has the property that hαi , α j i ≤ 0 for all i 6= j, the first assertion follows from the first part of Lemma 4.6. The second assertion follows from Lemma 4.6(ii). This is because the obtuseness graph for any choice of simple roots associated with a Coxeter group W is isomorphic to the (unlabeled) Coxeter graph, and the Coxeter graph is connected exactly when W is irreducible.


273

The following lemma was used in the preceding proof. LEMMA

4.6

d Let {αi }i=1 be a basis for Rd with hαi , α j i ≤ 0 for all i 6 = j. Then the dual basis d ∨ {αi }i=1 satisfies (i) hαi∨ , α ∨j i ≥ 0 for all i 6 = j, and (ii) hαi∨ , α ∨j i > 0 for all i 6= j if the “obtuseness graph” on {1, 2, . . . , d}, having

an edge {i, j} whenever hαi , α j i < 0, is connected.

Proof We prove assertion (i) by induction on d, with the cases d = 1, 2 being trivial. In the inductive step, assume that d ≥ 3. Without loss of generality, we must show that hα1 , α2 i ≥ 0. Let π : Rd → αd⊥ be an orthogonal projection. Write αi = π(αi ) + ci αd for each i ≤ d − 1. Our first claim is that ci ≤ 0 for each i ≤ d − 1. To see this, note that 0 ≥ hαi , αd i

= π(αi ), αd + ci hαd , αd i = ci hαd , αd i. Our second claim is that hπ(αi ), π(α j )i ≤ 0 for 1 ≤ i 6= j ≤ d − 1. To see this, note that 0 ≥ hαi , α j i

= π(αi ), π(α j ) + c j αd , π(α j ) + ci π(αi ), αd + ci c j hαd , αd i

= π(αi ), π(α j ) + ci c j hαd , αd i, and the last term in the last sum is nonnegative by our first claim. d−1 d−1 are dual bases inside αd⊥ . To see and {αi∨ }i=1 Our third claim is that {π(αi )}i=1 this, note that δi j = hαi , α ∨j i

= π(αi ), α ∨j + ci hαd , α ∨j i

= π(αi ), α ∨j . From the second and third claims, we can apply induction to conclude that hαi∨ , α ∨j i ≥ 0 for 1 ≤ i 6= j ≤ d − 1, and in particular, this holds for i = 1, j = 2, as desired.

274

LEUNG and REINER

To prove assertion (ii), we use the same induction on d, with the cases d = 1, 2 d still being trivial. We must in addition show that if {αi }i=1 have a connected obtused−1 ness graph, then there is a reindexing (that is, a choice of αd ) such that {π(αi )}i=1 also satisfies this hypothesis. To achieve this, let αd correspond to a node d in the obtuseness graph whose removal does not disconnect it; for example, choose d to be a leaf in some spanning tree for the graph. Then for i 6 = j with i, j ≤ d − 1, we have

π(αi ), π(α j ) = hαi , α j i − ci c j hαd , αd i. This implies π(αi ), π(α j ) were obtuse whenever αi , α j were, so the obtuseness graph remains connected. Remark 4.7 If the finite Coxeter group W is crystallographic (or a Weyl group) then a crystallographic root system associated with W gives a more natural choice of hyperplane normals to use than the unit normals in defining the Coxeter zonotope Z . With this choice, not only is the normal fan 1 rational with respect to the weight lattice N but also m(Z ) = 1 with respect to the dual lattice M. Hence Z is Delzant, so that the toric variety X 1 is smooth. For the classical Weyl groups W of types A, B(= C), D, there are known generating functions for the h-vectors of the associated Coxeter zonotopes Z , which specialize to give explicit generating functions for the signature σ (Z ). The h-vector in this case turns out to give the distribution of the elements of the Weyl group W according to their descents, that is, the number of simple roots which they send to negative roots (see [5]). Generating functions for the descent distribution of all classical Weyl groups may be found in [32]. For example, it follows from these that if Z An−1 is the Coxeter zonotope of type An−1 , then we have the formula X n≥0

σ (Z An−1 )

xn = tanh(x), n!

which was computed in [17, Example, p. 52] for somewhat different reasons. The fact that Coxeter zonotopes have locally convex normal fans also follows because these normal fans come from simplicial hyperplane arrangements (we thank M. Davis for suggesting this). Say that an arrangement of linear hyperplanes in Rd is simplicial if it decomposes Rd into a simplicial fan. PROPOSITION 4.8 The fan 1 associated to a simplicial hyperplane arrangement is locally convex.


275

Proof For each ray ρ of 1, we express str1 (ρ) as an intersection of closed half-spaces defined by a subset of the hyperplanes of A , thereby showing that it is convex. To describe this intersection, note that since 1 is simplicial, given any chamber (ddimensional cone) σ of 1 that contains ρ, there is a unique hyperplane Hσ bounding σ which does not contain ρ. Choose a linear functional u σ that vanishes on Hσ and is positive on ρ; then we claim that \ str1 (ρ) = {u σ ≥ 0}. chambers σ ⊃ρ

To see that the left-hand side is contained in the right, note that for any chamber σ containing ρ and any hyperplane H in A not containing ρ, we must have σ and ρ on the same side of H . Consequently, for every pair of chambers σ, σ 0 containing ρ, we have u σ ≥ 0 on σ 0 (and symmetrically u σ 0 ≥ 0 on σ ). This implies the desired inclusion. To show that the right-hand side is contained in the left, since both sets are closed and d-dimensional, it suffices to show that every chamber in the left is contained in the right, or contrapositively, that every chamber not contained in the right is not in the left. Given a chamber σ not in the right, consider the unique chamber σ 0 containing ρ which is “perturbed in the direction of σ .” In other words, σ 0 is chosen so that it contains a vector v + w, where v is any nonzero vector in ρ, w is any vector in the interior of σ , and is a very small positive number. Since σ does not contain ρ, we know σ 6= σ 0 , and hence there is at least one hyperplane of A separating them. Since 1 is simplicial, every bounding hyperplane of σ 0 except for Hσ 0 contains r and hence has σ and σ 0 on the same side (by construction of σ 0 ). This means Hσ 0 must separate σ and σ 0 , so u σ 0 < 0 on σ , implying σ is not in the left-hand side. The Coxeter zonotopes of type A are related to another infinite family of simple polytopes, the associahedra, which turn out to have locally convex normal fans. Recall (see [28]) that the associahedron An is an (n − 3)-dimensional polytope whose vertices correspond to all possible parenthesizations of a product a1 a2 · · · an−1 , and having an edge between two parenthesizations if they differ by a single “rebracketing.” Equivalently, vertices of An correspond to triangulations of a convex n-gon, and there is an edge between two triangulations if they differ only by a “diagonal flip” within a single quadrilateral. 4.9 The associahedron An has a realization as a simple convex polytope whose normal fan 1n is locally convex. PROPOSITION

276

LEUNG and REINER

Proof In [28, Section 3] the normal fan 1n is thought of as a simplicial complex, and more precisely as the boundary of a simplicial polytope Q n having the origin in its interior. There Q n is constructed by a sequence of stellar subdivisions of certain faces of an (n − 3)-simplex having vertices labelled 1, 2, . . . , n − 2. Since the normal fan 1n is simplicial, the associahedron is simple (as is well known). Our strategy for showing 1n is locally convex is to relate it to the Weyl chambers of type An−3 . If we assume that the (n − 3)-simplex above is regular and take its barycenter as the origin in Rn−3 , then the barycentric subdivision of its boundary is a simplicial polytope isomorphic to the Coxeter complex for type An−3 . Hence the normal fan 1n of An refines the fan of Weyl chambers for type An−3 . Note that an alternate description of this Weyl chamber fan is that it is the set of all chambers cut out by the hyperplanes xi = x j ; that is, its (open) chambers are defined by inequalities of the form xπ1 > xπ2 > · · · > xπn−2 for permutations π of {1, 2, . . . , n − 2}. To show that 1n is locally convex, we must first identify the rays ρ of 1n and then show that str1n (ρ) is a pointed convex cone. According to the construction of [28, Section 3], a ray ρi j of 1n corresponds to the barycenter of a face of the (n − 3)-simplex which is spanned by a set of vertices labeled by a contiguous sequence i, i + 1, . . . , j − 1, j with 1 ≤ i ≤ j ≤ n − 2, with (i, j) 6 = (1, n − 2). It is then not hard to check from the construction that str1 (ρij ) consists of the union of all (closed) chambers for type An−3 which satisfy the inequalities xi , xi+1 , . . . , x j−1 , x j ≥ xi−1 , x j+1 . (Here we omit the inequalities involving xi−1 if i = 1, and similarly for x j+1 if j = n + 2.) It is clear that these inequalities describe a convex cone, and hence 1n is locally convex. It follows then from Proposition 4.9 and Theorem 1.2(ii) that (−1)(n−3)/2 σ (An ) ≥ 0 for n odd (and of course, σ (An ) = 0 for n even). However, as in the case of Coxeter zonotopes of type A, we can compute σ (An ) explicitly using the formulas for the f vector or h-vector of An given in [28, Theorem 3]. Specifically, these formulas imply that for n ≥ 3 we have n−3 X 1 n−3 n−1 σ (A n ) = (−1)i n−1 i i +1 i=0 3 − n 2 − n = 2 F1 −1 2 ( (−1)(n−3)/2 C(n−1)/2 if n is odd, = 0 if n is even,


277

where Cn denotes the Catalan number (1/n) 2n−2 n−1 . Here the 2 F1 is hypergeometric series notation, and the last equality uses Kummer’s summation of a well-poised 2 F1 at −1 (see, e.g., [2, p. 9]). Returning to the discussion of nonacute and obtuse polytopes, it is worth noting the following facts, pointed out to us by M. Davis. Recall that a simplicial complex K is called flag if every set of vertices v1 , . . . , vr which pairwise spans edges of K also jointly spans an (r − 1)-simplex of K . 4.10 A polytope P is nonacute (resp., obtuse) if and only if each of its 2-dimensional faces is nonacute (resp., obtuse). Furthermore, nonacuteness of any polytope P implies that P is simple. PROPOSITION

Proof The first assertion for nonacute polytopes follows from an easy lemma due to G. Moussong (see [6, Lemma 2.4.1]). In the notation of [6], saying that every 2dimensional face of P is nonacute (in codimension 1) is equivalent to saying that for every vertex v of P, the spherical simplex σ = Lk(v, P) has size greater than or equal to π/2. Then [6, Lemma 2.4.1] asserts that every link Lk(τ, σ ) of a face of this spherical simplex also has size greater than or equal to π/2. But a face F of P containing v has Lk(v, F) of the form Lk(τ, σ ) for some τ , and hence F is nonacute in codimension 1 when considered as a polytope in its own right. That is, P is nonacute. An easy adaptation of this argument to the obtuse case proves the first assertion of the proposition for obtuse polytopes. The fact that nonacuteness implies simplicity again comes from considering the spherical simplex σ = Lk(v, P) for any vertex v, which has size greater than or equal to π/2. Then its polar dual spherical convex polytope σ ∗ has all of its dihedral angles less than or equal to π/2. This forces σ ∗ to be a spherical simplex, by [35, p. 44], and hence σ itself must be a spherical simplex. This implies that v has exactly d neighbors, so P is simple. Obtuse polytopes turn out to be relatively scarce in comparison with nonacute polytopes. For example, it is easily seen that Coxeter zonotopes, although always nonacute by Proposition 4.5, are not in general obtuse in dimensions 3 and higher. It is easy to find obtuse polytopes in dimensions up to 4: • in dimension 2, the regular n-gons for n ≥ 5, • in dimension 3, the dodecahedron, • in dimension 4, the “120-cell” (see [8, pp. 292–293]). However, M. Davis has pointed out to us that in dimensions 5 and higher, there

278

LEUNG and REINER

are no obtuse polytopes, due to a result of G. Kalai (see [25, Theorem 1]; see also [35, p. 68] for the case of simple polytopes): every d-dimensional convex polytope for d ≥ 5 contains either a triangular or quadrangular 2-dimensional face. 5. Relation to conjectures of Hopf and of Charney and Davis In this section we discuss the relation of Theorem 1.2(i) to the conjectures of Hopf and of Charney and Davis mentioned in the introduction. Let M d be a compact d-dimensional closed Riemannian manifold. When d is odd, Poincaré duality implies that the Euler characteristic χ (M) vanishes. When d is even, a conjecture of H. Hopf (see, e.g., [6]) asserts that if M d has nonpositive sectional curvature, the Euler characteristic χ(M d ) satisfies (−1)d/2 χ(M d ) ≥ 0. This result is known for d = 2, 4 by Chern’s Gauss-Bonnet formula, but open for general d; see [6, Section 0] for some history. Charney and Davis [6] explored a combinatorial analogue of this conjecture, and we refer the reader to their paper for terms that are not defined precisely here. Let M d be a compact d-dimensional closed manifold that has the structure of a (locally finite) Euclidean cell complex; that is, it is formed by gluing together convex polytopes via isometries of their faces. One can endow such a cell complex with a metric space structure that is Euclidean within each polytopal cell, making it a geodesic space. Gromov has defined a notion of when a geodesic space is nonpositively curved, and Charney and Davis made the following conjecture. CONJECTURE 5.1 ([6, Conjecture A]) If M d is a nonpositively curved, piecewise Euclidean, closed manifold with d even, then (−1)d/2 χ(M d ) ≥ 0.

For piecewise Euclidean cell complexes, nonpositive curvature turns out to be equivalent to a local condition at each vertex. Specifically, at each vertex v of M d , one has a piecewise spherical cell complex Lk(v, M d ), called the link of v in M d , which is homeomorphic to a generalized homology (d − 1)-sphere and inherits its own geodesic space structure. Nonpositive curvature of M d turns out to be a metric condition on each of these complexes Lk(v, M d ). Charney and Davis show in [6, (3.4.3)] that the Euler characteristic χ(M d ) can be written as the sum of certain local quantities κ(Lk(v, M d )) defined in terms of the metric structure on Lk(v, M d ): χ(M d ) =

X v

κ Lk(v, M d ) .

(9)


279

In the special case where the polytopes in the Euclidean cell decomposition of M d are all right-angled cubes, the links Lk(v, M d ) are all simplicial complexes, and the quantity κ(Lk(v, M d )) has a simple combinatorial expression purely in terms of the numbers of simplices of each dimension in these complexes (that is, independent of their metric structure). Furthermore, in this case, nonpositive curvature corresponds to the combinatorial condition that each link is a flag complex; that is, the minimal subsets of vertices in Lk(v, M d ) which do not span a simplex always have cardinality two. Charney and Davis then noted that in this special case Conjecture 5.1 would follow via (9) from the following conjecture. 5.2 ([6, Conjecture D]) If 1 is a flag simplicial complex triangulating a generalized homology (d − 1)-sphere with d even, then (−1)d/2 κ(1) ≥ 0. CONJECTURE

This Charney-Davis conjecture is trivial for d = 2, has recently been proven by Davis and B. Okun [15] for d = 4 using L 2 -homology of Coxeter groups, and is also known (by an observation of Babson and a result of Stanley; see [6, Section 7]) for the special class of flag simplicial complexes that are barycentric subdivisions of boundaries of convex polytopes. Local convexity of simplicial fans turns out to be stronger than flagness. PROPOSITION 5.3 A locally convex complete simplicial fan 1 in Rd is flag when considered as a simplicial complex triangulating a (d − 1)-sphere.

Proof Assume that 1 is not flag, so that there exist rays ρ1 , . . . , ρk whose convex hull σ := conv(ρ1 , . . . , ρk ) is not a cone of 1, but conv(ρi , ρ j ) is a cone of 1 for each i, j. Choose such a collection of rays of minimum cardinality k, so that conv(ρ1 , . . . , ρî , . . . , ρk ) is a cone of 1 for each i (in other words, the boundary complex ∂σ is a subcomplex of 1). We wish to show that str1 (ρ1 ) is not convex. To see this, consider σ ∩ str1 (ρ1 ), that is, the collection of cones 0 σ ∩ σ : σ 0 ∈ str1 (ρ1 ) . Since σ is not a cone of 1 but |1| = Rd , this collection must contain at least one 2-dimensional cone of the form σ 0 ∩ σ = conv(ρ1 , ρ), where ρ is a ray of σ but ρ 6∈ {ρ2 , . . . , ρk }. Since ρ lies inside σ and ∂σ is a subcomplex of 1, ρ cannot lie

280

LEUNG and REINER

in ∂σ (otherwise, some cone of ∂1 would be further subdivided and not be a cone of 1). Consequently, ρ lies in the interior of σ . Then the ray ρ 0 := ρ − ρ1 for very small > 0 has the following properties: • ρ 0 lies in σ because ρ was in the interior of σ ; • ρ 0 therefore lies in the convex hull of str1 (ρ1 ) since σ does (as its extreme rays ρ1 , . . . , ρk of σ are all in str1 (ρ1 )); • ρ 0 does not lie in str1 (ρ1 ) because then it would lie in a cone σ 0 of 1 containing ρ1 , and σ 0 would contain ρ in the relative interior of one of its faces, a contradiction. Therefore str1 (ρ1 ) is not convex. In light of the preceding proposition, one might ask if every flag simplicial sphere has a realization as a locally convex complete simplicial fan. We thank X. Dong for the following argument showing that an even weaker statement is false. One can show that complete simplicial fans always give rise to P L-spheres. Therefore if one takes the barycentric subdivision of any regular cellular sphere that is not P L (such as the double suspension of Poincaré’s famous homology sphere), this gives a flag simplicial sphere that is not P L and therefore has no realization as a complete simplicial fan (let alone one that is locally convex). Our results were motivated by the Charney-Davis conjecture and the following fact: when P is a simple d-polytope and 1 is its normal fan considered as a (d − 1)dimensional simplicial complex, one can check that σ (P) = 2d κ(1).

(10)

As a consequence, we deduce the following from Proposition 5.3 and Theorem 1.2(i). COROLLARY 5.4 Let P be a rational simple polytope, and let 1 be its normal fan. If 1 is locally convex, then it is flag and satisfies the Charney-Davis conjecture. In particular, by Corollary 4.2, if P is a nonacute simple rational polytope, then its normal fan 1 is flag and satisfies the Charney-Davis conjecture.

It is worth mentioning that the special case of Conjecture 5.1 considered in [6] where M d is decomposed into right-angled cubes is “polar dual” to another special case that fits nicely with our results. Say that M d has a corner decomposition if the local structure at every vertex in the decomposition is combinatorially isomorphic to the coordinate orthants in Rd ; that is, each link Lk(v, M d ) has the combinatorial structure of the boundary complex of a d-dimensional cross-polytope or hyperoctahedron. (Note


281

that this condition immediately implies that each of the d-dimensional polytopes in the decomposition must be simple.) A straightforward counting argument (essentially equivalent to the calculation proving [6, (3.5.2)]) shows that for a manifold M d with corner decomposition into simple polytopes P1 , . . . , PN one has χ(M d ) =

N 1 X σ (Pi ). 2d

(11)

i=1

The following corollary is then immediate from this relation and Theorem 1.2. COROLLARY 5.5 Let M d be a d-dimensional manifold with d even, having a corner decomposition. If each of the simple d-polytopes in the corner decomposition is rational and has a normal fan that is locally convex, then

(−1)d/2 χ(M d ) ≥ 0. In particular, this holds if each of the simple d-polytopes is nonacute. Several interesting examples of manifolds with corner decompositions into simple polytopes that are either Coxeter zonotopes (hence nonacute) or associahedra (hence locally convex) may be found in [14]. There is also an important general construction of such manifolds called mirroring, which we now discuss. This construction (or its polar dual) appears repeatedly in the work of Davis [10], [11], [13], [14], and it was used in [6, Section 6] to show that the case of their Conjecture 5.1 for manifolds decomposed into right-angled cubes is equivalent to their Conjecture 5.2. In a special case this construction begins with a generalized homology (d − 1)-sphere L with n vertices and produces a cubical orientable generalized homology d-manifold M L having 2n vertices, with the link at each of these vertices isomorphic to L. Hence we have χ(M L) = 2n · κ(L). We wish to make use of the polar dual of this construction, which applies to an arbitrary simple d-dimensional polytope P, yielding an orientable d-manifold M(P) with a corner decomposition having every d-dimensional cell isometric to P. The construction is as follows: denote the (d − 1)-dimensional faces of P by F1 , F2 , . . . , Fn , and let M(P) be the quotient of 2n disjoint copies {P }∈{+,−}n of P, in which two copies P , P 0 are identified along their face Fi whenever , 0 differ in the i th coordinate and nowhere else. As a consequence of (11) we have χ M(P) = 2n−d · σ (P), (12)

282

LEUNG and REINER

which shows that the “nonacute” assertions in Corollaries 5.5 and 5.4 are equivalent. We can now use the mirror construction to complete the proof of an assertion from the previous section. We are indebted to M. Davis for the statement and proof of this assertion. Proof of Corollary 4.2(i) without assuming rationality of P (as referred to in Remark 4.4) Assume that P is a simple nonacute d-dimensional polytope with d even. We wish to show that (−1)d/2 σ (P) ≥ 0. Construct M(P) as above, a manifold with corner decomposition into nonacute simple polytopes having χ(M(P)) = 2n−d σ (P) if P has n codimension 1 faces. In the notation of [6], this means that all the links Lk(v, M(P)) have size greater than or equal to π/2 and are combinatorially isomorphic to boundaries of cross-polytopes. This implies that the underlying simplicial complexes of these links are flag complexes satisfying [6, Conjecture D0 ], and then [6, Proposition 5.7] implies that each of these links Lk(v, M(P)) satisfies [6, Conjecture C0 ]. This implies that (−1)d/2 κ(Lk(v, M(P))) ≥ 0. Combining this with (9), we conclude that (−1)d/2 χ (M(P)) ≥ 0, and finally via (12), that (−1)d/2 σ (P) ≥ 0. We note that a similar argument (involving an adaptation of [6, Lemma 2.4.1]) proves (−1)d/2 σ (P) > 0 when P is obtuse but does not yield in any obvious way the stronger assertion of Corollary 4.2(iii). Appendix. The conormal bundle of a toric divisor In this appendix we describe the conormal bundle of a toric divisor on X = X 1 when the fan 1 is complete and simplicial. Denote the collection of toric divisors on X by D1 , . . . , Dm . As mentioned in Section 3, the conormal bundle of a divisor, say, D1 , can be identified as the restriction of O X (−D1 ) to D1 , which we renamed O D1 (−D1 ). It corresponds to a continuous piecewise linear function: D1 9−D : NR /ρ1 → R 1

as in the discussion of Section 3. Here ρ1 is the ray in the fan 1 corresponding to the D1 divisor D1 . We wish to identify the graph of 9−D with link1 (ρ1 ), which we recall 1 is the boundary of star1 (ρ1 ), the latter being the union of all cones of 1 containing ρ1 . PROPOSITION A.1 Let D1 be a toric divisor of a toric variety X = X 1 with 1 simplicial. Then the graph of the piecewise linear function for O D1 (D1 ) is affinely equivalent to the boundary link1 (ρ1 ) of star1 (ρ1 ), where ρ1 is the ray corresponding to D1 .


283

Proof We can index the toric divisors D1 , . . . , Dm of X in such a way that D = D1 and D2 , . . . , Dl are those that are adjacent to D1 . Let n i be the first nonzero lattice point along the ray ρi corresponding to Di . We choose a decomposition of N into a direct sum of Zn 1 with another lattice N 0 that is isomorphic to N /ρ1 . (Here we are abusing notation by referring to the quotient lattice N /Zn 1 as N /ρ1 .) Then we can write n i = bi n 1 + ci n i0 , where n i0 ∈ N 0 ∼ = N /ρ1 is indecomposable (i.e., not of the form k n i00 for some integer k with |k| ≥ 2 and n i00 ∈ N 0 ), and ci is some nonnegative integer. Now we choose the linear functional u on N such that hu, n 1 i = 1 and its restriction to N 0 is zero. Then in the Chow group of X we have the following relation (see [18, p. 106]): m X hu, n i i Di = 0. i=1

When we restrict this relation to the toric subvariety D1 , then those terms involving Di with i > l disappear because they are disjoint from D1 , and using the formula on [18, p. 108], we have l hu, n i O i O D1 Di = O D1 . ci i=1

Or equivalently, since hu, n 1 i = 1 and hu i , n 1 i = bi , we have O D1 (−D1 ) =

l O i=2

O D1

b

i

ci

Di .

Now under the identification N 0 ∼ = N /ρ1 , the restriction of the divisor Di to D1 corresponds to the ray in N /ρ1 spanned by n i0 when 2 ≤ i ≤ l. Therefore the piece D1 D1 wise linear function 9−D : NR /ρ1 → R is determined by 9−D n i0 = bi /ci . This 1 1 implies the assertion of the proposition. Acknowledgments. The authors are grateful to Michael Davis for several very helpful comments, proofs, references, and the permission to include them here. We thank Hugh Thomas for pointing out an error in an earlier version, and we also thank Xun Dong, Paul Edelman, Kefeng Liu, William Messing, and Dennis Stanton for helpful conversations. References [1]

E. BABSON, L. BILLERA, AND C. CHAN, Neighborly cubical spheres and a cubical

lower bound conjecture, Israel J. Math. 102 (1997), 297–315. MR 99a:52011

284

LEUNG and REINER

[2]

W. N. BAILEY, Generalized Hypergeometric Series, Cambridge Tracts in Math. and

[3]

L. J. BILLERA AND A. BJÖRNER, “Face numbers of polytopes and complexes” in

Math. Phys. 32, Stechert-Hafner, New York, 1964. MR 32:2625 277

[4]

[5] [6]

[7] [8] [9]

[10] [11] [12] [13] [14] [15]

[16] [17] [18] [19]

Handbook of Discrete and Computational Geometry, ed. J. E. Goodman and J. O’Rourke, CRC Press Ser. Discrete Math. Appl., CRC Press, Boca Raton, Fla., 1997, 291–310. MR CMP 1 730 171 253 L. J. BILLERA AND C. W. LEE, A proof of the sufficiency of McMullen’s conditions for f -vectors of simplicial convex polytopes, J. Combin. Theory Ser. A 31 (1981), 237–255. MR 82m:52006 253 A. BJÖRNER, Some combinatorial and algebraic properties of Coxeter complexes and Tits buildings, Adv. Math. 52 (1984), 173–212. MR 85m:52003 274 R. CHARNEY AND M. DAVIS, The Euler characteristic of a nonpositively curved, piecewise Euclidean manifold, Pacific J. Math. 171 (1995), 117–137. MR 96k:53066 254, 270, 277, 278, 279, 280, 281, 282 D. A. COX AND S. KATZ, Mirror Symmetry and Algebraic Geometry, Math. Surveys Monogr. 68, Amer. Math. Soc., Providence, 1999. MR 2000d:14048 262 H. S. M. COXETER, Regular Polytopes, 3d ed., Dover, New York, 1973. MR 51:6554 277 V. I. DANILOV, The geometry of toric varieties (in Russian), Uspekhi Mat. Nauk 33, no. 2 (1978), 85–134, 247; English translation in Russian Math. Surveys 33, no. 2 (1978), 97–154. MR 80g:14001 260 M. W. DAVIS, Groups generated by reflections and aspherical manifolds not covered by Euclidean space, Ann. of Math. (2) 117 (1983), 293–324. MR 86d:57025 281 , Some aspherical manifolds, Duke Math. J. 55 (1987), 105–139. MR 88j:57044 281 , “Nonpositive curvature and reflection groups” to appear in Handbook of Geometric Topology, ed. R. Daverman and R. Sher, Elsevier. M. W. DAVIS AND T. JANUSZKIEWICZ, Convex polytopes, Coxeter orbifolds and torus actions, Duke Math. J. 62 (1991), 417–451. MR 92i:52012 281 M. W. DAVIS, T. JANUSZKIEWICZ, AND R. SCOTT, Nonpositive curvature of blow-ups, Selecta Math. (N.S.) 4 (1998), 491–547. MR 2001f:53078 281 M. W. DAVIS AND B. OKUN, Vanishing theorems and conjectures for the L 2 -homology of right-angled Coxeter groups, Geom. Topol. 5 (2001), 7–74, http://maths.warwick.ac.uk/gt/GTVol5/paper2.abs.html MR CMP 1 812 434 279 J. A. DE LOERA, Triangulations of polytopes and computational algebra, Ph.D. dissertation, Cornell Univ., Ithaca, N.Y., 1995. 271 P. H. EDELMAN AND V. REINER, H -shellings and h-complexes, Adv. Math. 106 (1994), 36–64. MR 95h:52017 274 W. FULTON, Introduction to Toric Varieties, Ann. of Math. Stud. 131, Princeton Univ. Press, Princeton, 1993. MR 94g:14028 258, 259, 260, 263, 265, 267, 268, 283 , Intersection Theory, 2d ed., Ergeb. Math. Grenzgeb. (3) 2, Springer, Berlin, 1998. MR 99d:14003 264


285

[20]

B. GRÜNBAUM, Convex Polytopes, Pure Appl. Math. 16, Wiley, New York, 1967.

[21]

V. GUILLEMIN, Moment Maps and Combinatorial Invariants of Hamiltonian T n -Spaces, Progr. Math. 122, Birkhäuser, Boston, 1994. MR 96e:58064 255

[22]

R. HARTSHORNE, Algebraic Geometry, Grad. Texts in Math. 52, Springer, New York,

[23]

F. HIRZEBRUCH, Topological Methods in Algebraic Geometry, Classics Math.,

[24]

J. E. HUMPHREYS, Reflection Groups and Coxeter Groups, Cambridge Stud. Adv.

MR 37:2085

1977. MR 57:3116 Springer, Berlin, 1995. MR 96c:57002 261

[25] [26] [27]

[28] [29] [30] [31]

[32]

[33] [34]

[35]

[36]

Math. 29, Cambridge Univ. Press, Cambridge, England, 1990. MR 92h:20002 272 G. KALAI, On low-dimensional faces that high-dimensional polytopes must have, Combinatorica 10 (1990), 271–280. MR 92d:52023 278 T. KAWASAKI, The signature theorem for V -manifolds, Topology 17 (1978), 75–83. MR 57:14072 261 S. KOBAYASHI, Differential Geometry of Complex Vector Bundles, Publ. Math. Soc. Japan 15, Princeton Univ. Press, Princeton; Kanô Memorial Lectures 5, Iwanami Shoten, Tokyo, 1987. MR 89e:53100 257 C. LEE, The associahedron and triangulations of the n-gon, European J. Combin. 10 (1989), 551–560. MR 90i:52010 275, 276 P. MCMULLEN, On simple polytopes, Invent. Math. 113 (1993), 419–444. MR 94d:52015 254, 257, 258, 269 J. W. MILNOR AND J. D. STASHEFF, Characteristic Classes, Ann. of Math. Stud. 76, Princeton Univ. Press, Princeton, 1974. MR 55:13428 262 T. ODA, Convex Bodies and Algebraic Geometry: An Introduction to the Theory of Toric Varieties, Ergeb. Math. Grenzgeb. (3) 15, Springer, Berlin, 1988. MR 88m:14038 258, 260 V. REINER, The distribution of descents and length in a Coxeter group, Electron. J. Combin. 2 (1995), R25, http://combinatorics.org/Volume_2/volume2.html#R25 MR 96k:05208 274 R. P. STANLEY, The number of faces of a simplicial convex polytope, Adv. Math. 35 (1980), 236–238. MR 81f:52014 253 , “Generalized H -vectors, intersection cohomology of toric varieties, and related results” in Commutative Algebra and Combinatorics (Kyoto, Japan, 1985), Adv. Stud. Pure Math. 11, North-Holland, Amsterdam, 1987, 187–213. MR 89f:52016 258 È. B. VINBERG, Hyperbolic groups of reflections (in Russian), Uspekhi Mat. Nauk. 40, no. 1 (1985), 29–66, 255; English translation in Russian Math. Surveys 40, no. 1 (1985), 31–75. MR 86m:53059 277, 278 G. M. ZIEGLER, Lectures on Polytopes, Grad. Texts in Math. 152, Springer, New York, 1995. MR 96a:52011 272

286

LEUNG and REINER

Leung School of Mathematics, University of Minnesota, Minneapolis, Minnesota 55455, USA; [email protected] Reiner School of Mathematics, University of Minnesota, Minneapolis, Minnesota 55455, USA; [email protected]


SYZYGIES OF ORIENTED MATROIDS ISABELLA NOVIK, ALEXANDER POSTNIKOV, and BERND STURMFELS

Abstract We construct minimal cellular resolutions of squarefree monomial ideals arising from hyperplane arrangements, matroids, and oriented matroids. These are StanleyReisner ideals of complexes of independent sets and of triangulations of Lawrence matroid polytopes. Our resolution provides a cellular realization of R. Stanley’s formula for their Betti numbers. For unimodular matroids our resolutions are related to hyperplane arrangements on tori, and we recover the resolutions constructed by D. Bayer, S. Popescu, and B. Sturmfels [3]. We resolve the combinatorial problems posed in [3] by computing Möbius invariants of graphic and cographic arrangements in terms of Hermite polynomials. 1. Cellular resolutions from hyperplane arrangements A basic problem of combinatorial commutative algebra is to find the syzygies of a monomial ideal M = hm 1 , . . . , m r i in the polynomial ring k[x] = k[x1 , . . . , xn ] over a field k. One approach involves constructing cellular resolutions, where the ith syzygies of M are indexed by the i-dimensional faces of a CW-complex on r vertices. After reviewing the general construction of cellular resolutions from [4], we define the monomial ideals and resolutions studied in this paper. Let 1 be a CW-complex (see [12, §38]) with r vertices v1 , . . . , vr , which are labeled by the monomials m 1 , . . . , m r . We write c ≥ c0 whenever a cell c0 belongs to the closure of another cell c of 1. This defines the face poset of 1. We label each cell c of 1 with the monomial m c = lcm{m i | vi ≤ c}, the least common multiple of the monomials labeling the vertices of c. Also, set m ∅ = 1 for the empty cell of 1. Clearly, m c0 divides m c whenever c0 ≤ c. The principal ideal hm c i is identified with the free Nn -graded k[x]-module of rank 1 with generator in degree deg m c . For 0 a pair of cells c ≥ c0 , let pcc : hm c i → hm c0 i be the inclusion map of ideals. It is a degree-preserving homomorphism of Nn -graded modules. DUKE MATHEMATICAL JOURNAL c 2002 Vol. 111, No. 2, Received 16 June 2000. Revision received 6 February 2001. 2000 Mathematics Subject Classification. Primary 05B35, 13D02; Secondary 05E99, 13F55. Postnikov’s work partially supported by National Science Foundation grant number DMS-9840383. Sturmfels’s work partially supported by National Science Foundation grant number DMS-9970254.

287

288

NOVIK, POSTNIKOV, and STURMFELS

Fix an orientation of each cell in 1, and define the cellular complex C• (1, M), ∂3

∂2

∂1

∂0

· · · −→ C2 −→ C1 −→ C0 −→ C−1 = k[x], as follows. The Nn -graded k[x]-module of i-chains is M hm c i, Ci = c : dim c=i

where the direct sum is over all i-dimensional cells c of 1. The differential ∂i : Ci → 0 Ci−1 is defined on the component hm c i as the weighted sum of the maps pcc : X 0 ∂i = [c : c0 ] pcc , c0 ≤c, dim c0 =i−1

where [c : c0 ] ∈ Z is the incidence coefficient of oriented cells c and c0 in the usual topological sense. For a regular CW-complex, the incidence coefficient [c : c0 ] is +1 or −1, depending on the orientation of cell c0 in the boundary of c. The differential ∂i preserves the Nn -grading of k[x]-modules. Note that if m 1 = · · · = m r = 1, then C• (1, M) is the usual chain complex of 1 over k[x]. For any monomial m ∈ k[x], we define 1≤m to be the subcomplex of 1 consisting of all cells c whose label m c divides m. We call any such 1≤m an M-essential subcomplex of 1. 1.1 ([4, Prop. 1.2]) The cellular complex C• (1, M) is exact if and only if every M-essential subcomplex 1≤m of 1 is acyclic over k. Moreover, if m c 6 = m c0 for any c > c0 , then C• (1, M) gives a minimal free resolution of M. PROPOSITION

Proposition 1.1 is derived from the observation that, for a monomial m, the (deg m)graded component of C• (1, M) equals the chain complex of 1≤m over k. If both of the hypotheses in Proposition 1.1 are met, then we say that 1 is an M-complex, and we call C• (1, M) a minimal cellular resolution of M. Thus each M-complex 1 produces a minimal free resolution of the ideal M. In particular, for an M-complex 1, the number f i (1) of i-dimensional cells of 1 is exactly the ith Betti number of M, that is, the rank of the ith free module in a minimal free resolution. Thus, for fixed M, all M-complexes have the same f -vector. Examples of M-complexes appearing in the literature include planar maps (see [11]), Scarf complexes (see [2]), and hull complexes (see [4]). A general construction of M-complexes using discrete Morse theory was proposed by E. Batzies and V. Welker [1]. We next introduce a family of M-complexes which generalizes those in [3, Th. 4.4]. Let A = {H1 , H2 , . . . , Hn } be an arrangement of n affine hyperplanes in Rd , Hi = v ∈ Rd | h i (v) = ci , i = 1, . . . , n, (1)

SYZYGIES OF ORIENTED MATROIDS

289

where c1 , . . . , cn ∈ R and h 1 , . . . , h n are nonzero linear forms that span (Rd )∗ . We fix two sets of variables x1 , . . . , xn and y1 , . . . , yn , and we associate with the arrangement A two functions m x and m x y from Rd to sets of monomials: Y Y Y xi xi · yj . m x : v 7−→ and m x y : v 7 −→ i : h i (v)6 =ci

i : h i (v)>ci

j : v j (v) c0 , there is a hyperplane Hi ∈ A that contains c0 but does not contain c, in which case m x (c) is divisible by xi and m x (c0 ) is not divisible by xi . The same is true for the oriented matroid ideal OA . The essence of Theorem 1.3 is the acyclicity

290


MA = hx4 , x1 x2 , x2 x3 , x1 x3 i

@ @ @

OA = hy4 , x1 x2 , y2 x3 , x1 x3 i

@ @u x1 x2 @ @ @ x1 x2 x3

H1

@ x1 x2 x3 y4 @ @u x 1 x 3 x1 x2 y4 @ @ @ x1 x3 y4 x1 y2 x3 x1 y2 x3 y4 @ @ u @u y2 x3 y4 y4 y2 x3@ @

6

@ I -

@ @@

H2

H4 H3 Figure 1. The bounded complex BA with monomial labels

condition, which states that all MA -essential and OA -essential subcomplexes of BA are acyclic. For the whole bounded complex, the following proposition is known. PROPOSITION 1.4 (Björner and Ziegler (see [6, Th. 4.5.7])) The complex BA of bounded cells of a hyperplane arrangement A is contractible.

The acyclicity of all MA -essential subcomplexes of BA is an easy consequence of Proposition 1.4: each MA -essential subcomplex is a bounded complex of a hyperplane arrangement induced by A in one of the flats of A . The acyclicity of all OA -essential subcomplexes follows from a generalization of Proposition 1.4 stated in Proposition 2.4. We give more details in Section 2, where Theorem 1.3 is restated and proved in the more general setting of oriented matroids. The first main result in this paper is the construction of the minimal free resolution of an arbitrary matroid ideal (see Theorems 3.3 and 3.9) and an arbitrary


291

oriented matroid ideal (see Theorem 2.2). A numerical consequence of this result is a refinement of Stanley’s formula, given in [16, Th. 9], for their Betti numbers (see Corollaries 2.3 and 3.4; see also the last paragraph of Section 3). The simplicial complexes corresponding to matroid ideals and oriented matroid ideals are the complexes of independent sets in matroids (see Remark 3.1) and the triangulations of Lawrence matroid polytopes (see Theorem 2.9), respectively. In the unimodular case, oriented matroid ideals arise as initial ideals of toric varieties in P1 × P1 × · · · × P1 , by work of Bayer, Popescu, and Sturmfels [3, §4], and their Betti numbers can be interpreted as face numbers of hyperplane arrangements on a torus (see Theorem 4.1). Every ideal considered in this paper is Cohen-Macaulay; its Cohen-Macaulay type (highest Betti number) is the Möbius invariant of the underlying matroid, and all other Betti numbers are sums of Möbius invariants of matroid minors (see Section 4 and (8)). Our second main result concerns the minimal free resolutions for graphic and cographic matroid ideals. In Section 5 we resolve the enumerative problems that were left open in [3, §5]. Propositions 5.3 and 5.7 give combinatorial expressions for the Möbius invariant of any graph. More precise and explicit formulas, in terms of Hermite polynomials, are established for the Möbius coinvariants of complete graphs (see Theorem 5.8) and of complete bipartite graphs (see Theorem 5.14). 2. Oriented matroid ideals In this section we establish a link between oriented matroids and commutative algebra. In the resulting combinatorial context, the algebraists’ classic question, “What makes a complex exact?” (see [7]), receives a surprising answer: it is the topological representation theorem of J. Folkman and J. Lawrence (see [6, Chap. 5]). We start by briefly reviewing one of the axiom systems for oriented matroids (see [6]). Fix a finite set E. A sign vector X is an element of {+, −, 0} E . The positive part of X is denoted X + = {i ∈ E : X i = +}, and X − and X 0 are defined similarly. The support of X is X = {i ∈ E : X i 6 = 0}. The opposite −X of a vector X is given by (−X )i = −X i . The composition X ◦ Y of two vectors X and Y is the sign vector defined by X i if X i 6 = 0, (X ◦ Y )i = Yi if X i = 0. The separation set of sign vectors X and Y is S(X, Y ) = {i ∈ E | X i = −Yi 6= 0}. A set L ⊆ {+, −, 0} E is the set of covectors of an oriented matroid on E if and only if it satisfies the following four axioms (see [6, § 4.1.1]): (1) the zero sign vector zero is in L ; (2) if X ∈ L , then −X ∈ L (symmetry); (3) if X, Y ∈ L , then X ◦ Y ∈ L (composition);

292


if X, Y ∈ L and i ∈ S(X, Y ), then there exists Z ∈ L such that Z i = 0 and Z j = (X ◦ Y ) j = (Y ◦ X ) j for all j 6 ∈ S(X, Y ) (elimination). Somewhat informally, we say that such a pair (E, L ) is an oriented matroid. An affine oriented matroid (see [6, §10.1]), denoted M = (E, L , g), is an oriented matroid with a distinguished element g ∈ E such that g is not a loop; that is, X g 6= 0 for at least one covector X ∈ L . The positive part of L is L + = {X ∈ L : X g = +}. The set {+, −, 0} E is partially ordered by the product of partial orders (4)

0 0, 0 Ri = for every i ∈ E 0 . − if bi > 0, We apply Proposition 2.4 with this R 0 to (E\E 00 , L /E 00 , g). Then B 0 is the face poset of (BM )≤xa yb , which is therefore contractible. The oriented matroid ideal O is squarefree and hence is the Stanley-Reisner ideal of a simplicial complex 1M on 2n vertices {1, . . . , n, 10 , . . . , n 0 }, whose faces correspond


295

to squarefree monomials of k[x, y] which do not belong to O; that is, {i 1 , . . . , i k , j10 , . . . , jm0 } ∈ 1M

if and only if xi1 · · · xik y j1 · · · y jm ∈ / O.

In what follows we give a geometric description of that simplicial complex. LEMMA 2.5 We have F ∩ {i, i 0 } 6= ∅ for any facet F of 1M and i ∈ {1, . . . , n}.

Proof Let F be a face of 1M such that F ∩ {i, i 0 } = ∅. Suppose that neither F 0 = F ∪ {i} nor F 00 = F ∪ {i 0 } is a face of 1M . Then there exist cocircuits Z 0 , Z 00 ∈ BM such that Z i0 = +,

(Z 0 )+ \ {i} ⊆ {1 ≤ j ≤ n : j ∈ F} ∪ {g}, (Z 0 )− ⊆ {1 ≤ j ≤ n : j 0 ∈ F},

Z i00 = −,

(Z 00 )+ ⊆ {1 ≤ j ≤ n : j ∈ F} ∪ {g}, (Z 00 )− \ {i} ⊆ {1 ≤ j ≤ n : j 0 ∈ F}.

By the strong elimination axiom applied to (Z 0 , Z 00 , i, g), there is a cocircuit Z such that Z i = 0, Z g = +, Z + ⊆ (Z 0 )+ ∪ (Z 00 )+ , Z − ⊆ (Z 0 )− ∪ (Z 00 )− . Thus Z ∈ BM , and the monomial m x y (F) is divisible by m x y (Z ) ∈ O. This contradicts F ∈ 1M . Suppose now that the affine oriented matroid M = (E, L , g) is a single-element extension of the matroid M \g = (E\g, L \g) by an element g in general position, in the sense of [6, Prop. 7.2.2]. For the affine arrangement A in Section 1 or Example 2.1, this means that A has no vertices at infinity. In such a case, Theorem 2.2 implies the following properties of O. We denote by r the rank of M \g. 2.6 The ring k[1M ] = k[x, y]/O is a Cohen-Macaulay ring of dimension 2n − r . COROLLARY

Proof Since rk(M \g) = r , every (n − r + 1)-element subset {i 1 , . . . , i n−r +1 } of {1, . . . , n} contains the support of a (signed) cocircuit. This implies that every monomial of the form xi1 · · · xin−r +1 yi1 · · · yin−r +1 belongs to O. The variety defined by these monomials is a subspace arrangement of codimension r . Hence O has codimension greater than or equal to r , which means that the ring k[1M ] = k[x, y]/O has Krull dimension less than or equal to 2n −r . By Theorem 2.2, the bounded complex BM supports

296


a minimal free resolution of O, and therefore depth k[1M ] = 2n − (the length of this resolution) = 2n − r. Hence depth k[1M ] = dim k[1M ] = 2n−r , and k[1M ] is Cohen-Macaulay. The result in Corollary 2.6 can be strengthened to the statement that the simplicial complex 1 M is shellable. This follows from Theorem 2.9. COROLLARY 2.7 The set {x1 − y1 , . . . , xn − yn } is a regular sequence on k[1M ] = k[x, y]/O.

Proof Since k[1M ] is Cohen-Macaulay, it suffices to show that {x1 − y1 , . . . , xn − yn } is a part of a linear system of parameters (l.s.o.p.). This follows from Lemma 2.5 and the l.s.o.p. criterion due to B. Kind and P. Kleinschmidt [19, Lem. III.2.4]. Consider any signed circuit C = (C + , C − ) of our oriented matroid such that g lies in C − . By the general position assumption on g, the complement of g in that circuit is a basis of the underlying matroid. We write PC for the ideal generated by the variables xi for each i ∈ C + and by the variables y j for each j ∈ C − \{g}. PROPOSITION 2.8 T The minimal prime decomposition of the oriented matroid ideal equals O = C PC , where the intersection is over all circuits C such that g ∈ C − .

Proof The right-hand side is easily seen to contain the left-hand side. For the converse it suffices to divide by the regular sequence x1 − y1 , . . . , xn − yn and note that the resulting decomposition for the matroid ideal M is easy (see Remark 3.1). Our final result relates the ideal O to matroid polytopes and their triangulations. The monograph of F. Santos [15] provides an excellent state-of-the-art introduction. We refer in particular to [15, §4], where Santos introduces triangulations of Lawrence (matroid) polytopes, and he shows that these are in bijection with one-element liftings of the underlying matroid. Under matroid duality, one-element liftings correspond to one-element extensions. In our context these extensions correspond to adding the special element g, which plays the role of the pseudohyperplane at infinity. From Santos’s result we infer the following theorem.


297

THEOREM 2.9 The oriented matroid ideal O is the Stanley-Reisner ideal of the triangulation of the Lawrence matroid polytope induced by the lifting dual to the extension by g. In particular, O is the Stanley-Reisner ideal of a triangulated ball.

The second assertion holds because lifting triangulations of matroid polytopes are triangulated balls and, by Santos’s work, every triangulation of a Lawrence matroid polytope is a lifting triangulation. We remark that it is unknown whether arbitrary triangulations of matroid polytopes are topological balls (see [15, p. 7]). 3. Matroid ideals Let M be an (unoriented) matroid on the set {1, . . . , n}, and let L be its lattice of flats. We encode M by the matroid ideal M generated by the monomials Q m x (F) = i:i ∈F / x i for every proper flat F ∈ L. The minimal generators of M are the squarefree monomials representing cocircuits of M , that is, the monomials m x (H ), where H runs over all hyperplanes of M . Equivalently, M is the StanleyReisner ideal of the simplicial complex of independent sets of the dual matroid M ∗ . The following explains what happens when we substitute yi 7→ xi in Proposition 2.8. Remark 3.1 The matroid ideal M has the minimal prime decomposition \

M= xi | i ∈ B . B basis of M

The following characterization of our ideals can serve as a definition of the word matroid. It is a translation of the (co)circuit axiom into commutative algebra. Remark 3.2 A proper squarefree monomial ideal M of k[x] is a matroid ideal if and only if, for every pair of monomials m 1 , m 2 ∈ M and any i ∈ {1, . . . , n} such that xi divides both m 1 and m 2 , the monomial lcm (m 1 , m 2 )/xi is in M as well. Matroid ideals have been studied since the earliest days of combinatorial commutative algebra as a paradigm for shellability and Cohen-Macaulayness. Stanley computed their Betti numbers in [16, Th. 9]. The purpose of this section is to construct an explicit minimal k[x]-free resolution for any matroid ideal M. We note that in recent work of V. Reiner and Welker [14] the term “matroid ideal” is used for the squarefree monomial ideals that are Alexander dual to our matroid ideals.

298


We first consider the case where M is an orientable matroid. This means that there exists an oriented matroid M whose underlying matroid is M . Let L be the set of covectors of a single element extension of M by an element g in general position f = (E, L , g), where (see [6, Prop. 7.2.2]). Consider the affine oriented matroid M E = {1, . . . , n} ∪ {g}, and its bounded complex BMf. Note that, for each sign vector Z in BMf, the zero set Z 0 is a flat in L. Moreover, by the genericity hypothesis on g, all flats arise in this way. We label each cell Z of the bounded complex BMf by the Q monomial m x (Z ) = {xi : 1 ≤ i ≤ n and Z i 6 = 0}. THEOREM 3.3 Let M be the matroid ideal of an orientable matroid. Then the bounded complex BMf of any corresponding affine oriented matroid is an M-complex, and its cellular complex C• (BMf, M) gives a minimal free resolution of M over k[x].

Proof Let a = (a1 , . . . , an ) ∈ Nn , and consider the M-essential subcomplex (BMf)≤xa . This complex (if not empty) is the bounded complex of the contraction of (E, L , g) by {1 ≤ i ≤ n : ai = 0} and hence is acyclic by Proposition 2.4. Since m x (Z 0 ) is a proper divisor of m x (Z ) whenever Z 0 < Z and Z 0 , Z ∈ BMf, it follows that BMf is an M-complex. We remark that C• (BMf, M) is obtained from the complex C• (BMf, O), where O is f= (E, L , g), by specializing yi to xi for all i. Hence the oriented matroid ideal of M Theorem 2.2 and Corollary 2.7 give a second proof of Theorem 3.3. COROLLARY 3.4 The Nn -graded Hilbert series of any matroid ideal M equals

X F∈L

ˆ · µ L (F, 1)

n Y Y {x j : j ∈ / F} / (1 − xi ),

(3)

i=1

where L is the lattice of flats of M , and µ L is its Möbius function. There are several ways of deriving this corollary. First, it follows from [16, Th. 9]. A second possibility is to observe that the geometric lattice L coincides with the lcm lattice (in the sense of [8]) of the ideal M, and then [8, Th. 2.1] implies the claim. Finally, in the orientable case, Corollary 3.4 follows from Theorem 3.3 and the oriented matroid version of T. Zaslavksy’s face-count formula.


299

PROPOSITION 3.5 (Zaslavsky’s formula (see [22], [6, Th. 4.6.5])) f= (E, L , g) The number of bounded regions of a rank r affine oriented matroid M r ˆ 1). ˆ equals (−1) µ L (0,

We next treat the case of nonorientable matroids. It would be desirable to construct an M-complex for an arbitrary matroid ideal M and to explore the “space” of all possible M-complexes. Currently we do not know how to construct them. Therefore we introduce a different technique for resolving M minimally. Let P be any graded poset that has a unique minimal element 0ˆ and a unique ˆ (Later on, we take P to be the order dual of our geometric lattice maximal element 1. L.) Let 1(P) denote the order complex of P, that is, the simplicial complex whose ˆ simplices [F0 , F1 , . . . , Fi ] are decreasing chains 1ˆ > F0 > F1 > · · · > Fi > 0. ˆ For F ∈ P, denote by 1(F) the order complex of the lower interval [0, F]. Note that dim 1(F) = rk(F) − 2. Let Ci (1(F)) be the k-vector space of i-dimensional chains of 1(F), and let ∂2 0 −→ Crk(F)−2 1(F) −→ · · · −→ C1 1(F) ∂0 ∂1 −→ C0 1(F) −→ C−1 1(F) −→ 0 be the usual (augmented) chain complex; that is, the differential is given by ∂i [F0 , F1 , . . . , Fi ] =

i X (−1) j [F0 , . . . , c F j , . . . , Fi ]

for i > 0 and ∂0 [F0 ] = 0.

j=0

ei (1(F)) the ith (reDenote by Z i (1(F)) = ker(∂i ) the space of i-cycles, and by H duced) homology of 1(F). (For relevant background on poset homology, see [5].) For each pair F, F 0 ∈ P such that rk(F) − rk(F 0 ) = 1, we define a map φ : Ci 1(F) −→ Ci−1 1(F 0 ) by [F0 , F1 , . . . , Fi ] 7 →

0 if F0 6= F 0 , [F1 , . . . , Fi ] if F0 = F 0 .

The map φ is zero unless F 0 l F (in other words, F covers F 0 ). Note that ∂ ◦ φ = −φ ◦ ∂, and hence the restriction of φ to cycles gives a map φ : Z i (1(F)) −→ Z i−1 (1(F 0 )). Combining these maps, we obtain a complex of k-vector spaces: M φ Z (P) : 0 −→ Z r −2 1(P) −→ Z r −3 1(F) rk(F)=r −1 φ

φ

−→ · · · −→

M rk(F)=2

M φ Z 0 1(F) −→ Z −1 1(F) −→ k. rk(F)=1

300


ˆ and thus the first map φ is well defined.) The com(Here 1(P) is regarded as 1(1), 2 plex property φ = 0 is verified by direct calculation using equation (4). Let P( j) denote the poset obtained from P by removing all rank levels greater than or equal to ˆ j, and let 1(P( j) ) be the order complex of P( j) ∪ {1}. 3.6 ei (1(P(i+3) )) = 0 for all i ≤ r − 3. The complex Z (P) is exact if H PROPOSITION

L To prove Proposition 3.6 we need some notation. If x ∈ rk(F)=i Z i−2 (1(F)), we denote its F-component by x F . For a simplex σ = [F0 , F1 , . . . , Fi ], we also write σ = F0 ∗ [F1 , . . . , Fi ], and the operation “∗” extends to k-linear combinations. Remark 3.7 Suppose that z ∈ Ci (1(P(i+2) )). Then z can be expressed as X X X z= F 0 ∗ yF 0 = F 0 ∗ F 00 ∗ x F 0 , F 00 , rk(F 0 )=i+1

rk(F 0 )=i+1 F 00 lF 0

where y F 0 ∈ Ci−1 (1(F 0 )) and x F 0 , F 00 ∈ Ci−2 (1(F 00 )). Its boundary equals X X ∂(z) = F 00 ∗ x F 0 , F 00 rk(F 00 )=i

X

−

rk(F 0 )=i+1

F 0 mF 00

X

F0 ∗

F 00 lF 0

x F 0 , F 00 +

X

F 0 ∗ F 00 ∗ ∂(x F 0 , F 00 ).

F 0 , F 00

We conclude that z is a cycle if and only if the following conditions are satisfied: X x F 0 , F 00 = 0 for all F 00 with rk(F 00 ) = i; (4) F 0 mF 00

X

x F 0 , F 00 = 0

for all F 0 with rk(F 0 ) = i + 1;

(5)

for all F 0 , F 00 such that F 00 l F 0 .

(6)

F 00 lF 0

∂(x F 0 , F 00 ) = 0

Proof of Proposition 3.6 L To show that Z (P) is exact, consider y = (y F 0 ) ∈ rk(F 0 )=i+1 Z i−1 (1(F 0 )) such that φ(y) = 0. There are several cases. If i = r − 1, then y = y1ˆ can be expressed as P rk(F)=r −2 F ∗ x F , where x F ∈ Cr −3 (1(F)). Then 0 = φ(y) F = x F , and therefore y = 0. Hence the leftmost map φ is an inclusion. P 0 Let 0 0, then y F 0 can be


expressed as

P

F 00 lF 0

301

F 00 ∗ x F 0 , F 00 , where x F 0 , F 00 ∈ Ci−2 (1(F 00 )). Hence

X φ(y) F 00 = x F 0 , F 00 ,

∀F 00 with rk(F 00 ) = i,

F 0 mF 00

and ∂(y F 0 ) =

X F 00 lF 0

x F 0 , F 00 −

X

F 00 ∗ ∂(x F 0 , F 00 ),

∀F 0 with rk(F 0 ) = i + 1.

F 00 lF 0

Since φ(y) = 0 and ∂(y F 0 ) = 0 for any F 0 of rank i + 1, we infer that z satisfies conditions (4) – (6) in Remark 3.7 and therefore is a cycle. In the case i = 0, the proof is very similar. Now if i = r − 2, then z ∈ Z r −2 (1(P)), and φ(z) = P φ( F 0 ∗ y F 0 ) = (y F 0 ) = y. Hence we are done in this case. If i < r − 2, then, since ei (1(P(i+3) )) = 0, it follows that there exists Z i (1(P(i+2) )) ⊆ Z i (1(P(i+3) )) and H P w ∈ Ci+1 (1(P(i+3) )) such that ∂(w) = z. Express w as rk(F)=i+2 F ∗ v F , where P P v F ∈ Ci (1(F)). Since z = ∂(w) = rk(F)=i+2 v F − rk(F)=i+2 F ∗ ∂(v F ), we P P conclude that ∂(v F ) = 0 for all F of rank i + 2 and that F v F = z = F 0 F 0 ∗ y F 0 . L Thus v = (v F ) ∈ rk(F)=i+2 Z i (1(F)), and φ(v) = y. 3.8 If P is a Cohen-Macaulay poset, then Z (P) is exact. COROLLARY

Proof If 1(P) is Cohen-Macaulay, then 1(P(i) ) is Cohen-Macaulay for every i (see [17, Th. 4.3]). This means that all homologies of 1(P(i) ) vanish, except possibly the top one. Thus the conditions of Proposition 3.6 are satisfied. Suppose now that every atom A of P is labeled by a monomial m A ∈ k[x]. The poset ideal I P is the ideal generated by these monomials. Associate with every element F of P a monomial m F as follows: m F := lcm m A : rk(A) = 1, A ≤ F if F 6 = 0ˆ and m 0ˆ := 1. We say that the labeled poset P is complete if all monomials m F are distinct, and for every a ∈ Nn the set {F ∈ P : deg(m F ) ≤ a} has a unique maximal element. We identify the principal ideal hm F i with the free Nn -graded k[x]-module of rank 1 with generator in degree deg m F . If F, G ∈ P and F < G, then m F is a divisor of m G . Thus there is an inclusion of the corresponding ideals i : hm G i −→ hm F i. Recall that there is a complex Z (P) of k-vector spaces associated with P. Tensoring summands of this complex with the ideals {hm F i : F ∈ P}, we obtain a complex of

302


Nn -graded free k[x]-modules: M C (P) = Z rk(F)−2 1(F) ⊗k hm F i

with differential ∂ = φ ⊗ i.

(7)

F∈P

3.9 ei (1(F(i+3) )) Suppose that the labeled poset P is complete and that the homology H vanishes for any 0 ≤ i ≤ r − 3 and any F ∈ P of rank ≥ i + 3; then (C (P), ∂) is a minimal Nn -graded free k[x]-resolution of the poset ideal I P . THEOREM

Proof (C (P), ∂) is a complex of Nn -graded free k[x]-modules. To show that it is a resolution, we have to check that, for any a ∈ Nn , the ath graded component (C (P), ∂)a is an exact complex of k-vector spaces. Let a ∈ Nn , and let F ∈ P be the maximal element among all elements G ∈ P such that deg(m G ) ≤ a. Such an element F exists since the labeled poset P is complete. Then (C (P), ∂)a is isomorphic to the complex ˆ F]) of the poset [0, ˆ F] and hence is exact over k (by Proposition 3.6). Thus Z ([0, (C (P), ∂) is exact over k[x]. Finally, since m F and m G are distinct monomials for any pair F l G, the resolution (C (P), ∂) is minimal. From Corollary 3.8 we obtain the following corollary. 3.10 If P is a complete labeled poset such that every lower interval of P is CohenMacaulay, then (C (P), ∂) is a minimal Nn -graded free resolution of I P . COROLLARY

Returning to our matroid M , let P be a lattice of flats ordered by reverse inclusion. Hence P is the order dual of the geometric lattice L above. In particular, 0ˆ corresponds to the set {1, 2, . . . , n}, and 1ˆ corresponds to the empty set. Label each atom H of P (i.e., hyperplane of M ) by the monomial m x (H ), as in the beginning of Section 3. Identifying the variables xi with the coatoms of P, we see that m x (H ) is the product over all coatoms not above H . Then P is a complete labeled poset and its poset ideal I P is precisely the matroid ideal M. Moreover, all lower intervals of the poset P are Cohen-Macaulay (see [16, §8]). From Corollary 3.10 we obtain the following alternative to Theorem 3.3. 3.11 Let M be any matroid. Then the complex (C (P), ∂) is a minimal Nn -graded free k[x]-resolution of the matroid ideal M. THEOREM


303

The two resolutions presented in this section provide a syzygetic realization of Stanley’s formula [16, Th. 9] for the Betti numbers of matroid ideals. That formula states that the number of minimal ith syzygies of k[x]/M is equal to X µ L (F, 1) ˆ , βi (M) = F

where the sum is over all flats F of corank i in M . The generating function ψM (q) =

rk(M) X

βi (M) · q i =

i=0

X

µ L (F, 1) ˆ · q corank(F)

(8)

F flat of M

for the Betti numbers of M is called the cocharacteristic polynomial of M . In the next two sections we examine this polynomial for some special matroids. 4. Unimodular toric arrangements A toric arrangement is a hyperplane arrangement that lives on a torus Td rather than in Rd . One construction of such arrangements appears in recent work of Bayer, Popescu, and Sturmfels [3]. Experts on geometric combinatorics might appreciate the following description: fix a unimodular matroid M , form the associated tiling of Euclidean space by zonotopes (see [21, Prop. 3.3.4]), dualize to get an infinite arrangement of hyperplanes, and divide out by the group of lattice translations. Here is the same construction again, but now in slow motion. Fix a central hyperplane arrangement C = {H1 , . . . , Hn } in Rd , where Hi = {v ∈ Rd : h i · v = 0} for some h i ∈ Zd . Let L denote the intersection lattice of C ordered by reverse inclusion. We assume that C is unimodular, which means that the (d × n)-matrix (h 1 , . . . , h n ) has rank d, and all its (d ×d)-minors lie in the set {0, 1, −1}. We retain this hypothesis throughout this section. (See [21] and [3, Th. 1.2] for details on unimodularity.) The set of all integral translates of hyperplanes of C , Hi j = {v ∈ Rd : h i · v = j}

for i ∈ {1, . . . , n} and j ∈ Z,

forms an infinite arrangement Ce in Rd . The unimodularity hypothesis is equivalent to saying that the set of vertices of Ce is precisely the lattice Zd ; that is, no new vertices can be formed by intersecting the hyperplanes Hi j . Define the unimodular toric arrangement Ce/Zd to be the set of images of the Hi j in the torus Td = Rd /Zd . Slightly abusing notation, we refer to these images as hyperplanes on the torus. The images of cells of Ce in Td are called cells of Ce/Zd . These cells form a cellular decomposition of Td . Denote by f i = f i (Ce/Zd ) the number of i-dimensional cells in this decomposition. The next result concerns the f -vector ( f 0 , f 1 , . . . , f d ) of Ce/Zd .

304


THEOREM 4.1 If Ce/Zd is a unimodular toric arrangement, then d X

f i (Ce/Zd ) · q i = ψC (q),

where ψC (q) =

X

ˆ · (−q)dim F µ L (F, 1)

F∈L

i=0

is the cocharacteristic polynomial of the underlying hyperplane arrangement C . Proof Choose a vector w ∈ Rd which is not perpendicular to any 1-dimensional cell of the arrangement C . Consider the affine hyperplane { v ∈ Rd : w · v = 1}. Let A = C ∩ H be a restriction of C to H . Then A is an affine arrangement in H . For any i ≥ 0, there is a one-to-one correspondence between the (i − 1)-dimensional bounded cells of A and the i-dimensional cells of toric arrangement Ce/Zd . To see this, consider the cells in the infinite arrangement Cewhose minimum with respect to the linear functional v 7→ w · v is attained at the origin. These cells form a system of representatives modulo the Zd -action. But they are also in bijection with the bounded cells of A . Using Proposition 3.5 (see also Example 2.1), we conclude X ˆ f i (Ce/Zd ) = f i−1 (BA ) = (−1)i · µ L (F, 1), dim(F)=i

where the sum is over elements of L of corank i. This completes the proof. Theorem 4.1 was found independently by V. Reiner, who suggested that we include the following alternative proof. His proof has the advantage that it does not rely on Zaslavsky’s formula. Second proof of Theorem 4.1 Starting with the unimodular toric arrangement C˜/Zd , for each intersection subspace F in the intersection lattice L, let TF denote the subtorus obtained by restricting C˜/Zd to F. So T0 is just C˜/Zd itself, and T1 is not actually a torus but rather a point. Our assertion is equivalent to µ(F, 1) = (−1)dim F · #{max cells in TF }.

(9)

Let µ0 (F) denote the right-hand side above. By the definition of the Möbius function of a poset, equation (9) is equivalent to X µ0 (G) = δ F,1 (Kronecker delta). F≤G≤1

The left-hand side of this equation is the (nonreduced) Euler characteristic of TF . This is zero since TF is a torus, unless F = 1 so that TF is a point, and then it is 1.


305

We remark that Theorem 4.1 can be generalized to arbitrary toric arrangements Ce/Zd without the unimodularity hypothesis. The face count formula is a sum of local Möbius function values over all (now more than one) vertices of Ce/Zd . That generalization has interesting applications to hypergeometric functions, and it will be studied in [13]. We know of no natural syzygetic interpretation of the complexes Ce/Zd when C is not unimodular. The enumerative applications in Section 5 all involve unimodular arrangements, so we restrict ourselves to this case. We need the following recursion for computing cocharacteristic polynomials. PROPOSITION 4.2 Let H be a hyperplane of the arrangement C . Then X ψC (q) = ψC ∩H (q) + q · ψC /c (q), c

where the sum is over all lines c of the arrangement C that are not contained in H . The lines c of the arrangement C are the coatoms of the intersection lattice L. The arrangement C /c is the hyperplane arrangement { Hi /c : c ∈ Hi } in the (d − 1)dimensional vectorspace Rd /c. Note that if c is a simple intersection, that is, if c lies on only d −1 hyperplanes Hi , then ψC /c (q) = (1+q)d−1 . Note that Proposition 4.2, together with the condition ψC (q) = 1 for the zero-dimensional arrangement C , uniquely defines the cocharacteristic polynomial. Proof The intersection lattice L of any central hyperplane arrangement C is semimodular; that is, if both F and G cover F ∧ G, then F ∨ G covers both F and G (see [18, §3.3.2]). The assertion follows from the relation [18, §3.10, (27)] for the Möbius functions of any semimodular lattice. In the remainder of this section we review the algebraic context in which unimodular toric arrangements arise in [3]. This provides a Gröbner basis interpretation for our proof of Theorem 4.1, and it motivates our enumerative results in Section 5. Denote by B the (n × d)-matrix whose rows are h 1 , . . . , h n . All (d × d)-minors of B are −1, 0, or +1. The unimodular Lawrence ideal of C is the binomial prime ideal

JC := xa yb − ya xb | a, b ∈ Nn , a − b ∈ Image(B) in k[x1 , . . . , xn , y1 , . . . , yn ]. The main result of [3] states that the toric arrangement C /Zd supports a cellular resolution of JC . In particular, the Betti numbers of the unimodular Lawrence ideal JC are precisely the coefficients of the cocharacteristic polynomial ψC (q).

306


The construction in the proof of Theorem 4.1 has a Gröbner basis interpretation. Indeed, the generic vector w ∈ Rd defines a term order for the ideal JC as follows: xa yb ya xb

if a − b = B · u for some u ∈ Rd with w · u > 0.

It is shown in [3, §4] that the initial monomial ideal in (JC ) of JC with respect to these weights is the oriented matroid ideal associated with the restriction of the central arrangement C to the affine hyperplane { v ∈ Rd : w · v = 1}. In symbols, in (JC ) = OA . In fact, in the unimodular case, Theorem 1.3(b) is precisely [3, Th. 4.4]. COROLLARY 4.3 The Betti numbers of the unimodular Lawrence ideal JC , and all its initial ideals in (JC ), are the coefficients of the cocharacteristic polynomial ψC .

We close this section with a nontrivial example. Let n = 9, d = 4, and consider x11 

1  0 BT =   0 0

x12

x13

x21

x22

x23

x31

x32

−1 1 0 0

0 −1 0 0

−1 0 1 0

1 −1 −1 1

0 1 0 −1

0 0 −1 0

0 0 1 −1

x33  0 0  . 0  1

All nonzero (4×4)-minors of this matrix are −1 or +1, and hence we get a unimodular central arrangement C of nine hyperplanes in R4 . This is the cographic arrangement associated with the complete bipartite graph K 3,3 . The nine hyperplane variables xi j represent edges in K 3,3 . The associated Lawrence ideal can be computed by saturation (e.g., in Macaulay 2) from (binomials representing) the four rows of B T : J B = h x11 x22 y12 y21 − x12 x21 y11 y22 , x12 x23 y13 y22 − x13 x22 y12 y23 , Y ∞ x21 x32 y22 y31 − x22 x31 y21 y32 , x22 x33 y23 y32 − x23 x32 y22 y33 i : xi j yi j . 1≤i, j≤3

This ideal has 15 minimal generators, corresponding to the 15 circuits in the directed graph K 3,3 . A typical initial monomial ideal in≺ (J B ) = OA looks like this:

x11 x22 y12 y21 , x11 x23 y13 y21 , x11 x32 y12 y31 , x11 x33 y13 y31 , x12 x23 y13 y22 , x12 x33 y13 y32 , x21 x32 y22 y31 , x21 x33 y23 y31 , x22 x33 y23 y32 , x11 x22 x33 y13 y21 y32 , x11 x22 x33 y12 y23 y31 , x11 x23 x32 y13 y22 y31 , x12 x21 x33 y11 y23 y32 , x12 x21 x33 y13 y22 y31 , x13 x21 x32 y12 y23 y31 .


307

This is the oriented matroid ideal of the 3-dimensional affine arrangement A gotten from C by taking a vector w ∈ R4 with strictly positive coordinates. This ideal is the intersection of 81 monomial primes, one for each spanning tree of K 3,3 . By Theorem 2.9, they form a triangulation of a 13-dimensional Lawrence polytope, which is given by its centrally symmetric Gale diagram (B T , −B T ), as in [6, Prop. 9.3.2(b)]. Resolving this ideal (e.g., in Macaulay 2), we obtain the cocharacteristic polynomial ψC (q) = 1 + 15q + 48q 2 + 54q 3 + 20q 4 .

(10)

It was asked in [3, §5] what such Betti numbers arising from graphic and cographic ideals are in general. This question is answered in the following section. 5. Graphic and cographic matroids Among all matroids the unimodular ones play a special role; among unimodular matroids those that arise from graphs play a special role; among all graphs the complete graph plays a special role. Our aim in this section is to compute the cocharacteristic polynomial of graphic and cographic arrangements, with an emphasis on complete graphs. The material in this section is purely combinatorial and can be read independently from the commutative algebra seen earlier. However, the motivation that led us to prove Theorems 5.8 and 5.14 arose from the desire to count minimal syzygies. The results in this section provide answers to questions posed in [3, §4]. We start out by discussing graphic arrangements. Cographic arrangements are more challenging and are discussed further below. Fix a connected graph G with vertices [d] = {1, . . . , d} and edges E ⊂ [d]×[d]. Let V = {(v1 , . . . , vd ) ∈ Rd : v1 + · · · + vd = 0} ' Rd−1 . The graphic arrangement CG is the arrangement in V given by the hyperplanes vi = v j for (i, j) ∈ E. It is unimodular (see [21]). For each subset S ⊂ [d], we get an induced subgraph G| S = (S, E ∩ (S × S)). For a partition π of [d], we denote by G/π the graph obtained from G by contracting all edges whose vertices lie in the same part of π . The intersection lattice L G of the graphic arrangement CG has the following well-known description in terms of the partition lattice 5d (see, e.g., [22] for proofs and references). PROPOSITION 5.1 The intersection lattice L G is isomorphic to the sublattice of the partition lattice 5d consisting of partitions π such that, for each part S of π, the subgraph G| S is connected. The element Vπ of L G corresponding to π ∈ 5d is the intersection of the hyperplanes {vi = v j } for pairs i, j in the same part of π . The dimension of Vπ is equal to the number of parts of π minus 1. The interval [Vπ , 1ˆ ] of the intersection lattice L G is isomorphic to the intersection lattice L G/π .

ˆ 1)| ˆ for the Möbius invariant of the intersection lattice. We write µ(G) = |µ L G (0,

308


Thus µ(G) equals the Cohen-Macaulay type (top Betti number) of the matroid ideal \

MG = xi j : (i, j) ∈ F | F ⊆ E is a spanning tree of G . From Proposition 5.1 and (8), we conclude that all the lower Betti numbers can be expressed in terms of the Möbius invariants of the contractions G/π of G. 5.2 The cocharacteristic polynomial of the graphic arrangement CG is X ψCG (q) = µ(G/π) · q |π |−1 . COROLLARY

π ∈L G

This reduces our problem to computing the Möbius invariant µ(G) of a graph G. C. Greene and Zaslavsky [10] found the following combinatorial formula. An orientation of the graph G is a choice, for each edge (i, j) of G, of one of the two possible directions: i → j or j → i. An orientation is acyclic if there is no directed cycle. PROPOSITION 5.3 Fix a vertex i of G. Then µ(G) equals the number of acyclic orientations of G such that, for any vertex j, there is a directed path from i to j.

Proof The regions of the graphic arrangement CG are in one-to-one correspondence with the acyclic orientations of G: the region corresponding to an acyclic orientation o is given by the inequalities xi > x j for any directed edge i → j in o. The linear functional w : (u 1 , . . . , u d ) 7 → u i is generic for the arrangement CG . The Möbius invariant µ(G) equals the number of regions of CG which are bounded below with respect to w. We claim that the acyclic orientations corresponding to the w-bounded regions are precisely the ones given in our assertion. Suppose that, for any vertex j in G, there is a directed path i → · · · → j. For any point (u 1 , . . . , u d ) of the corresponding region, this path implies u i > · · · > u j . The condition u 1 + · · · + u m = 0 forces w(u) = u i > 0. This implies that the region is w-positive. Conversely, consider any acyclic orientation that does not satisfy the condition in Proposition 5.3. Then there exists a vertex j 6= i which is a source of o. Then the vector v = (−1, . . . , −1, d − 1, −1, . . . , −1), where d − 1 is in the jth coordinate, belongs to the closure of the region associated with o. But w(v) = −1. Hence the region is not w-positive. The above discussion can be translated into a combinatorial recipe for writing the minimal free resolution of graphic ideals MG , where each syzygy is indexed by a


309

certain acyclic orientation of a graph G/π. For the case of the complete graph G = K d , we recover the resolution in [3, Th. 5.3]. Note that the intersection lattice L K d is isomorphic to the partition lattice 5d . For any partition π of {1, . . . , d} with i + 1 parts, K d /π is isomorphic to K i+1 . The number of such partitions equals S(d, i + 1), the Stirling number of the second kind. The number of acyclic orientations of K i+1 with a unique fixed source equals i!. We deduce the following corollary. COROLLARY 5.4 The number of minimal ith syzygies of M K d equals i! S(d, i + 1).

Remark 5.5 Reiner suggested to us the following combinatorial interpretation of µ(G). It can be derived from Proposition 5.3. For any graph G, the Möbius invariant µ(G) counts the number of equivalence classes of linear orderings of the vertices of G, under the equivalence relation generated by the following operations: • commuting two adjacent vertices v, v 0 in the ordering if {v, v 0 } is not an edge of G, • cyclically shifting the entire order, that is, v1 v2 · · · vn ↔ v2 · · · vn v1 . Invariance under the second operation makes this interpretation convenient for writing down the minimal free resolution of the graphic Lawrence ideals in [3, §5]. Another application arises when (W, S) is a Coxeter system and G its Coxeter graph (considered without its edge labels). Suppose S = {s1 , . . . , sn }. Then µ(G) counts the number of Coxeter elements si1 · · · sin of G up to the equivalence relation si1 si2 · · · sin ↔ si2 · · · sin si1 . We now come to the cographic arrangement CG⊥ , whose matroid is dual to that of CG . Fix a directed graph G on [d] with edges E, where G is allowed to have loops and multiple edges. We associate with G the multiset of vectors {ve ∈ Zd : e ∈ E}, where, for an edge e = (i → j), the ith coordinate of ve is 1, the jth coordinate is −1, and all other coordinates are zero. Set ve = 0 for a loop e = (i → i) of G. Let P VG = {λ : E → R | e∈E λ(e)ve = 0}. Note that VG is a vector space of dimension #{edges} − #{vertices} + #{connected components}. The cographic arrangement CG⊥ is the arrangement in VG given by hyperplanes He = {λ ∈ VG : λ(e) = 0} for ˆ 1)| ˆ for the Möbius e ∈ E. It is unimodular (see [21]). We write µ⊥ (G) = |µ L ⊥ (0, G

⊥ invariant of the intersection lattice L ⊥ G of CG , and we refer to this number as the ⊥ Möbius coinvariant of G. Thus µ (G) is the Cohen-Macaulay type of the cographic ideal JC ⊥ in [3, §5]. G

310


Remark 5.6 The characteristic polynomial of a matroid can be expressed via the Tutte dichromatic polynomial (see [20]). Thus the Möbius invariant and coinvariant of a graph G are certain values of the Tutte polynomial: µ(G) = TG (1, 0) and µ⊥ (G) = TG (0, 1). We do not know, however, how to express the cocharacteristic polynomial ψ(q) in terms of the Tutte polynomial. A formula for the Tutte polynomial due to I. Gessel and B. Sagan [9, Th. 2.1] implies the following proposition. 5.7 P d−|F|−1 , where the sum is The Möbius coinvariant of G is µ⊥ (G) = F⊆G (−1) over all forests in G and |F| denotes the number of edges in F. PROPOSITION

We derive explicit formulas for the Möbius coinvariant of complete and complete bipartite graphs. A subgraph M of a graph G is called a partial matching if it is a collection of pairwise disjoint edges of the graph. For a partial matching M, let a(M) be the number of vertices of G that have degree zero in M. The Hermite polynomial Hn (x), n ≥ 0, is the generating function of partial matchings in the complete graph Kn : X Hn (x) = x a(M) , M

where the sum is over all partial matchings in K n . In particular, H0 (x) = 1. Set also H−1 (x) = 0. The main result of this section is the following formula. THEOREM 5.8 The Möbius coinvariant of the complete graph K m equals

µ⊥ (K m ) = (m − 2) Hm−3 (m − 1),

m ≥ 2.

(11)

A few initial numbers µ⊥ (K m ) are given below: m

2 3 4

5

6

7

8

9

10

µ⊥ (K m )

0 1 6 51 560 7575 122052 2285353 48803904

..., ....

The proof of Theorem 5.8 relies on several auxiliary results and is given below. The next proposition summarizes well-known properties of Hermite polynomials.


311

PROPOSITION 5.9 The Hermite polynomial Hn (x) satisfies the recurrence

H−1 (x) = 0,

H0 (x) = 1,

Hn+1 (x) = x Hn (x) + n Hn−1 (x),

n ≥ 0.

(12)

It is given explicitly by the formula Hn (x) = x + n

[n/2] X k≥1

n (2k − 1)!! x n−2k , 2k

where (2k − 1)!! = (2k − 1)(2k − 3)(2k − 5) · · · 3 · 1. Proof In a partial matching the first vertex has either degree 0 or 1. This gives two terms in the right-hand side of the recurrence (12). The formula for Hn (x) follows from the fact that there are (2k − 1)!! matchings with k edges on 2k vertices. Returning to general cographic arrangements, recall that an edge e of the graph G is called an isthmus if G\e has more connected components than G; a graph is called isthmus-free if no edge of G is an isthmus. The minimal nonempty isthmus-free subgraphs of G are the cycles of G. For a subgraph H of G, denote by G/H the graph obtained from G by contracting the edges of H . Note that G/H may have loops and multiple edges even if G does not. The following result appears in [10]. 5.10 The intersection lattice L ⊥ G of the cographic arrangement is isomorphic to the lattice of isthmus-free subgraphs of G ordered by reverse inclusion. The element of the intersection lattice that corresponds to an isthmus-free subgraph H is V H ⊂ VG . The coatoms of the lattice L ⊥ G are the cycles of G. For two isthmus-free subgraphs H ⊃ K of G, the interval [VH , VK ] of the intersection lattice L ⊥ G is isomorphic to the interval ˆ 1] ˆ of the intersection lattice L ⊥ . [0, H/K PROPOSITION

Proposition 4.2 implies the following recurrence for the cocharacteristic polynomial ψC ⊥ (q) of the cographic arrangement CG⊥ . G

5.11 Let e be an edge of the graph G. Then COROLLARY

ψC ⊥ (q) = ψC ⊥ (q) + q G

X

G\e

ψC ⊥ (q), G/C

C

where the sum is over all cycles C of G that contain e.

(13)

312


Considering terms of the highest degree in (13), we obtain the following corollary. 5.12 If e is any edge of G that is not an isthmus, then X µ⊥ (G) = µ⊥ (G/C), COROLLARY

(14)

C

where the sum is over all cycles C of G that contain e. e obtained from G Note that µ⊥ (G) is equal to the Möbius coinvariant of the graph G by removing all loops and isthmuses. Thus, when we use relation (14) to calculate µ⊥ (G), we may remove all new loops obtained after contracting the cycle C. (k) We are ready to prove Theorem 5.8. For n ≥ 0 and k ≥ 1, define K n to be the complete graph K n on the vertices 1, . . . , n, together with one additional vertex n + 1 (k) (k) (root) connected to each vertex 1, . . . , n by k edges. Let µn = µ⊥ (K n ) be the (k) (1) (1) ⊥ Möbius coinvariant of the graph K n . Note that K m = K m−1 and µ (K m ) = µm−1 . Theorem 5.8 can be extended as follows. 5.13 (k) We have the following formula: µn = Hn (n + k − 1) − n Hn−1 (n + k − 1) for n, k ≥ 1. PROPOSITION

Proof (k) We utilize Corollary 5.12. Select an edge e = (n, n + 1) of the graph K n . There are k − 1 choices for a cycle C of length 2 that contains the edge e, and the graph (k) (k+1) K n /C, after removing loops, is isomorphic to K n−1 . There are (n−1) k choices for (k)

a cycle C of length 3 that contains the edge e, and the graph K n /C, after removing (k+2) loops, is isomorphic to K n−2 . In general, for cycles of length l ≥ 3, there are k (n − (k+l−1)

1)(n −2) · · · (n −l +2) choices, and we obtain a graph that is isomorphic to K n−l+1 . (k)

Equation (14) implies the following recurrence for µn : (k+1)

(k+2)

µ(k) n = (k − 1) µn−1 + k(n − 1) µn−2 (k+3)

(k+4)

+ k(n − 1)(n − 2) µn−3 + k (n − 1)(n − 2)(n − 3) µn−4 + · · · , (k)

(15)

(k)

which, together with the initial condition µ0 = 1, defines the numbers µn uniquely. Set (k+1)

(k+2)

(k+n)

bn(k) = µ(k) n + n µn−1 + n(n − 1) µn−2 + · · · + n(n − 1) · · · 1 µ0 (k)

(k)

(k+1)

Then µn = bn − nbn−1 and the relation (15) can be rewritten as (k+1) (k+1) (k+2) (k+2) bn(k) − n bn−1 = (k − 1) bn−1 − (n − 1)bn−2 + k (n − 1) bn−2

.


313

or, simplifying, as (k+1)

(k+2)

bn(k) = (n + k − 1)bn−1 + (n − 1) bn−2 . (k)

(16)

(k)

We claim that bn = Hn (n + k − 1). Indeed, b0 = 1, b1 (k) = k, and equation (16) (k) is equivalent to the defining relation (12) for the Hermite polynomials. Hence µn = (k) (k+1) bn − nbn−1 = Hn (n + k − 1) − n Hn−1 (n + k − 1). Proof of Theorem 5.8 By Proposition 5.13 and equation (12), (1)

µ⊥ (K m ) = µm−1 = Hm−1 (m−1)−(m−1) Hm−2 (m−1) = (m−2) Hm−3 (m−1). We now discuss a bipartite analog of Hermite polynomials. For a partial matching M in the complete bipartite graph K m,n , denote by a(M) the number of vertices in the first part that have degree zero in M, and by b(M) the number of vertices in the second part that have degree zero. Define X Hm,n (x, y) = x a(M) y b(M) , M

where the sum is over all partial matchings in K m,n . In particular, Hm,0 = x m and H0,n = y n . Set also Hm,−1 = H−1,n = 0. The following statement is a bipartite analogue of Theorem 5.8. THEOREM 5.14 The Möbius coinvariant of the complete bipartite graph K m,n equals

µ⊥ (K m,n ) = (m − 1)(n − 1)Hm−2, n−2 (n − 1, m − 1),

m, n ≥ 1.

The following proposition is analogous to Proposition 5.9. PROPOSITION 5.15 The polynomial Hm,n (x, y) is given by

Hm,n (x, y) =

min(m,n) X k=0

m k

n k! x m−k y n−k . k

It satisfies the following recurrence relations: Hm,n (x, y) = x Hm−1,n (x, y) + n Hm−1,n−1 (x, y), Hm,n (x, y) = y Hm,n−1 (x, y) + m Hm−1,n−1 (x, y), Hm,0 = x m ,

H0,n = y n .

(17)

314


Proof The first formula is obtained by counting the partial matchings in K m,n . The recurrence relations (17) are obtained by distinguishing two cases when the first vertex in the first (second) part of K m,n has degree 0 or 1 in a partial matching. (k,l)

Let us define the graph K m,n as the complete bipartite graph K m,n with an additional vertex v such that v is connected by k edges with each vertex in the first part and by (k,l) (k,l) l edges with each vertex in the second part. Let µm,n = µ⊥ (K m,n ) be the Möbius (1,0) (1,0) coinvariant of this graph. Note that K m,n = K m,n−1 and, thus, µ⊥ (K m,n ) = µm,n−1 . Theorem 5.14 can be extended as follows. PROPOSITION

5.16

We have l) µ(k, m,n = Hm,n (n + k − 1, m + l − 1) − mn Hm−1, n−1 (n + k − 1, m + l − 1).

Proof Our proof is similar to that of Proposition 5.13. We utilize Corollary 5.12. Select an (k,l) edge e of the graph K m,n that joins the additional vertex v with a vertex from the first part. There are k − 1 choices for a cycle C of length 2 that contains the edge e, (k,l) (k,l+1) and the graph K m,n /C, after removing loops, is isomorphic to K m−1,n . There are (k,l)

n l choices for a cycle C of length 3 that contains the edge e, and the graph K m,n /C, (k+1,l+1) after removing loops, is isomorphic to K m−1,n−1 . For cycles of length 4, we have (k+1,l+2)

n(m −1) k choices and obtain a graph isomorphic to K m−2,n−1 , and so on. In general, for cycles of odd length 2r + 1 ≥ 3, we have l n(m − 1)(n − 1)(m − 2) · · · (m − r + (k+r, l+r ) 1)(n − r + 1) choices, and we obtain a graph isomorphic to K m−r, n−r . For cycles of even length 2r + 2 ≥ 4, we have k n(m − 1)(n − 1)(m − 2) · · · (n − r + 1)(m − r ) (k+r, l+r +1) choices, and we obtain a graph isomorphic to K m−r −1, n−r . Equation (14) implies the (k, l)

following recurrence for µm,n : (k, l+1)

(k+1, l+1)

(k+1, l+2)

l) µ(k, m,n = (k − 1) µm−1,n + lnµm−1,n−1 + kn(m − 1) µm−2,n−1 (k+2, l+2)

+ l n(m − 1)(n − 1) µm−2,n−2

(k+2, l+3)

+ k n(m − 1)(n − 1)(m − 2) µm−3,n−2 + · · · , (k, l)

(18) (k, l)

which, together with the initial conditions µ0,n = (l − 1)n and µm,0 = (k − 1)m , (k, l)

unambiguously defines the numbers µm,n . Let us fix the numbers p = k + n − 1 and ( p−n+1, q−m+1) q = l + m − 1 and write µm,n for µm,n . Set bm,n = µm,n + n m µm−1, n−1 + n(n − 1) m(m − 1) µm−2, n−2 + · · · .


315

Then µm,n = bm,n − m nbm−1, n−1 and the relation (18) can be rewritten as bm,n − m n bm−1, n−1 = − bm−1,n − (m − 1) n bm−2, n−1 + ( p − n + 1) bm−1, n + (q − m + 1) n bm−1, n−1 or, simplifying, as bm,n = ( p − n) bm−1, n + (q + 1)n bm−1, n−1 + (m − 1)n bm−2, n−1 .

(19)

This relation, together with the initial conditions b0,n = q n , bm,0 = p m , b−1,n = bm,−1 = 0, uniquely determines the numbers bm,n . We claim that bm,n = Hm,n ( p, q). Indeed, the above initial conditions are satisfied by Hm,n ( p, q), and (19) follows from the defining relations (17) for the bipartite Hermite polynomials. In order to see this, we write by (17), Hm,n ( p, q) = p Hm−1, n ( p, q) + n Hm−1, n−1 ( p, q), n Hm−1, n ( p, q) = n q Hm−1, n−1 ( p, q) + n(m − 1) Hm−2, n−1 ( p, q). (k l)

The sum of these two equations is equivalent to equation (19). Hence µm,n = bm,n − m n bm−1,n−1 = Hm,n ( p, q) − m n Hm−1, n−1 ( p, q). (k l)

An alternative expression for µm,n can be deduced from Proposition 5.16: l) µ(k m,n =

min(m,n) X r =0

(1 − r )

m n r ! (n + k − 1)m−r (m + l − 1)n−r . r r

(20)

Proof of Theorem 5.14 By Proposition 5.16 and the recurrence relations (17), µ⊥ (K m,n ) (1,0)

= µm,n−1 = Hm,n−1 (n − 1, m − 1) − m(n − 1) Hm−1, n−2 (n − 1, m − 1) = (n − 1) Hm−1, n−1 (n − 1, m − 1) − (m − 1)(n − 1) Hm−1, n−2 (n − 1, m − 1) = (m − 1)(n − 1) Hm−2, n−2 (n − 1, m − 1). For a commutative algebra example illustrating Theorem 5.14, consider the Lawrence ideal J B ⊂ k[x11 , . . . , x33 , y11 , . . . , y33 ] associated with the bipartite graph K 3,3 . This is the Lawrence lifting of the ideal of (2 × 2)-minors of a generic (3 × 3)-matrix. It is discussed in the end of Section 4. Its Cohen-Macaulay type is µ⊥ (K 3,3 ) = (3 − 1) · (3 − 1) · H1,1 (2, 2) = 2 · 2 · 5 = 20. This is the leading coefficient of the cocharacteristic polynomial in equation (10).

316


Acknowledgments. We wish to thank Vic Reiner and Günter Ziegler for valuable communications. Their ideas and suggestions have been incorporated in Proposition 2.4, Theorem 4.1, and Remark 5.5. References [1]

[2] [3]

[4] [5]

[6]

[7] [8] [9]

[10]

[11]

[12] [13] [14]

E. BATZIES and V. WELKER, Discrete Morse theory for cellular resolutions, preprint,

2001, http://mathematik.uni-marburg.de/˜welker/sections/preprints.html, to appear in J. Reine Angew. Math. 288 D. BAYER, I. PEEVA, and B. STURMFELS, Monomial resolutions, Math. Res. Lett. 5 (1998), 31–46. MR 99c:13029 288 D. BAYER, S. POPESCU, and B. STURMFELS, Syzygies of unimodular Lawrence ideals, J. Reine Angew. Math. 534 (2001), 169–186. MR CMP 1 831 636 287, 288, 291, 293, 303, 305, 306, 307, 309 D. BAYER and B. STURMFELS, Cellular resolutions of monomial modules, J. Reine Angew. Math. 502 (1998), 123–140. MR 99g:13018 287, 288 A. BJÖRNER, “The homology and shellability of matroids and geometric lattices” in Matroid Applications, ed. N. White, Encyclopedia Math. Appl. 40, Cambridge Univ. Press, Cambridge, 1992, 226–283. MR 94a:52030 299 A. BJÖRNER, M. LAS VERGNAS, B. STURMFELS, N. WHITE, and G. ZIEGLER, Oriented Matroids, 2d ed., Encyclopedia Math. Appl. 46, Cambridge Univ. Press, Cambridge, 1999. MR 2000j:52016 290, 291, 292, 293, 294, 295, 298, 299, 307 D. BUCHSBAUM and D. EISENBUD, What makes a complex exact?, J. Algebra 25 (1973), 259–268. MR 47:3369 291 V. GASHAROV, I. PEEVA, and V. WELKER, The lcm-lattice in monomial resolutions, Math. Res. Lett. 6 (1999), 521–532. MR 2001e:13018 298 I. M. GESSEL and B. E. SAGAN, The Tutte polynomial of a graph, depth-first search, and simplicial complex partitions, Electron. J. Combin. 3, no. 2 (1996), research paper 9, http://combinatorics.org/Volume_3/volume3_2.html#R9 MR 97d:05149 310 C. GREENE and T. ZASLAVSKY, On the interpretation of Whitney numbers through arrangements of hyperplanes, zonotopes, non-Radon partitions, and orientations of graphs, Trans. Amer. Math. Soc. 280 (1983), 97–126. MR 84k:05032 308, 311 E. MILLER and B. STURMFELS, “Monomial ideals and planar graphs” in Applied Algebra, Algebraic Algorithms and Error-Correcting Codes (Honolulu, 1999), ed. M. Fossorier, H. Imai, S. Lin, and A. Poli, Lecture Notes in Comput. Sci. 1719, Springer, Berlin, 1999, 19–28. MR CMP 1 846 480 288 J. R. MUNKRES, Elements of Algebraic Topology, Addison-Wesley, Menlo Park, Calif., 1984. MR 85m:55001 287 A. POSTNIKOV and B. STURMFELS, Toric arrangements and hypergeometric functions, in preparation. 305 V. REINER and V. WELKER, Linear syzygies of Stanley-Reisner ideals, Math. Scand. 89 (2001), 117–132. MR CMP 1 856 984 297


317

[15]

F. SANTOS, Triangulations of oriented matroids, to appear in Mem. Amer. Math. Soc.

[16]

R. P. STANLEY, “Cohen-Macaulay complexes” in Higher Combinatorics (Berlin,

296, 297

[17] [18]

[19] [20] [21]

[22]

1976), ed. M. Aigner, NATO Adv. Study Inst. Ser., Ser. C: Math. and Phys. Sci. 31, Reidel, Dordrecht, 1977, 51–62. MR 58:28010 291, 297, 298, 302, 303 , Balanced Cohen-Macaulay complexes, Trans. Amer. Math. Soc. 249 (1979), 139–157. MR 81c:05012 301 , Enumerative Combinatorics, I, Wadsworth Brooks/Cole Math. Ser., Wadsworth & Brooks/Cole Adv. Books Software, Monterey, Calif., 1986. MR 87j:05003 305 , Combinatorics and Commutative Algebra, 2d ed., Progr. Math. 41, Birkhäuser, Boston, 1996. MR 98h:05001 296 W. T. TUTTE, On dichromatic polynomials, J. Combin. Theory 2 (1967), 301–320. MR 36:6320 310 N. WHITE, “Unimodular matroids” in Combinatorial Geometries, ed. N. White, Encyclopedia Math. Appl. 29, Cambridge Univ. Press, Cambridge, 1987, 40–52. MR CMP 921 067 303, 307, 309 T. ZASLAVSKY, Facing up to arrangements: Face-count formulas for partitions of space by hyperplanes, Mem. Amer. Math. Soc. 1 (1975), no. 154. MR 50:9603 299, 307

Novik Department of Mathematics, University of Washington, Seattle, Washington 98125-3810, USA; [email protected] Postnikov Department of Mathematics, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139-4307, USA; [email protected] Sturmfels Department of Mathematics, University of California, Berkeley, California 94720-3840, USA; [email protected]


ARAKELOV INTERSECTION INDICES OF LINEAR CYCLES AND THE GEOMETRY OF BUILDINGS AND SYMMETRIC SPACES ANNETTE WERNER

Abstract This paper generalizes Yu. Manin’s approach toward a geometrical interpretation of Arakelov theory at infinity to linear cycles in projective spaces. We show how to interpret certain non-Archimedean Arakelov intersection numbers of linear cycles on Pn−1 with the combinatorial geometry of the Bruhat-Tits building associated to PGL(n). This geometric setting has an Archimedean analogue, namely, the Riemannian symmetric space associated to SL(n, C), which we use to interpret analogous Archimedean intersection numbers of linear cycles in a similar way. 1. Introduction In this paper we provide a geometrical interpretation of certain local Arakelov intersection indices of linear cycles on projective spaces. In the non-Archimedean case the corresponding geometric framework is the combinatorial geometry of the Bruhat-Tits building for PGL. In the Archimedean case we use the Riemannian geometry of the symmetric space corresponding to SL(n, C). Our motivation is the desire to generalize the results of Manin’s paper [Ma] to higher dimensions. In [Ma], Manin’s goal is to enrich the picture of Arakelov theory at the infinite places by constructing a differential-geometric object playing the role of a “model at infinity.” He suggests such an object, a certain hyperbolic 3-manifold, in the case of curves, and he corroborates his suggestion by interpreting various Arakelov intersection numbers in terms of geodesic configurations on this space. It is certainly desirable to find such a differential-geometric object for higherdimensional varieties also, but up to now there have been no results in this direction. One of the goals of this paper is to present a candidate in the case of projective spaces of arbitrary dimension. A good strategy for finding such a “space at infinity” is to look for a geometric DUKE MATHEMATICAL JOURNAL c 2002 Vol. 111, No. 2, Received 7 August 2000. Revision received 9 January 2001. 2000 Mathematics Subject Classification. Primary 14G40; Secondary 14M15, 53C35, 51E24. Author’s work supported in part by Max-Planck-Institut für Mathematik, Bonn. 319

320

ANNETTE WERNER

object at the non-Archimedean places which is closely related to a non-Archimedean model and which has an Archimedean analogue. The idea here is to consider the Bruhat-Tits building X for the group G = PGL(V ), where V is an n-dimensional vector space over a non-Archimedean local field K of characteristic zero. The vertices in X correspond to the homothety classes {M} of R-lattices M in V , where R is the ring of integers in K . The boundary X ∞ of X in the Borel-Serre compactification can be identified with the Tits building of G, which is just the flag complex in V . Thereby the vertices in X ∞ correspond to the nontrivial subspaces of V . We write P(W ) for the irreducible subscheme of the projective space P(V ) induced by a linear subspace W ⊂ V , and we show that for any fixed vertex v = {M} in X the half-geodesic [v, W ] connecting v with the boundary point induced by W governs the reduction of P(W ) in the model P(M) of P(V ) in the following way: [v, W ] and [v, W 0 ] share the first m + 1 vertices if and only if the reductions of the closures of P(W ) and P(W 0 ) in P(M) modulo π m (where π is a prime element in R) coincide. Then we show how to express a certain intersection index of homologically trivial linear cycles with the combinatorial geometry of X . Consider subspaces A and B of dimension p and subspaces C and D of dimension q = n − p of V , such that the cycles P(A) − P(B) and P(C) − P(D) on P(V ) have disjoint supports. Then the local Arakelov intersection number of the closures of these cycles in P(M) is defined and independent of the choice of a lattice M in V . We denote it by hP(A) − P(B), P(C) − P(D)i. Under certain conditions (which are, e.g., fulfilled if p = 1 and the intersection is nontrivial), we define explicitly an oriented geodesic γ such that

P(A) − P(B), P(C) − P(D) = p distorγ (A ∗ γ , B ∗ γ ), where A ∗ γ is the point on γ closest to the boundary point induced by A in a suitable sense, and where distor means oriented distance along the oriented geodesic γ . In [We], we show another result in this direction. Namely, we give a geometrical interpretation of the intersection index of several arbitrary linear cycles meeting properly on some model P(M). So in fact, the geometrical interpretation of nonArchimedean intersections can be pushed quite far. The building X has an Archimedean analogue, namely, the symmetric space Z corresponding to SL(n, C). We also have a compactification of Z sharing many features with the Borel-Serre compactification of X . It can be obtained by attaching to Z the set Z (∞) of geodesic rays in Z emanating at a fixed point z (see, e.g., [BGS]). Similar to the Borel-Serre boundary in the non-Archimedean case, Z (∞) has a decomposition into faces, so that the partially ordered set of faces can be identified with the partially ordered set of proper parabolic subgroups of SL(n, C). Thereby maximal parabolics correspond to minimal faces, which are points. In this way every nontrivial subspace W of Cn gives rise to a point in Z (∞).

ARAKELOV INTERSECTION OF LINEAR CYCLES

321

We prove that geodesics in Z connecting two boundary points corresponding to subspaces W and W 0 of Cn have features similar to those of geodesics in the building X connecting two boundary points given by subspaces of V . More precisely, there exists a geodesic in X (resp., Z ) connecting the points corresponding to the subspaces W and W 0 if and only if these are complementary in V (resp., Cn ). Moreover, the set of geodesics between these points is in bijection with the set of pairs of homothety classes of lattices (resp., hermitian metrics) on W and W 0 . This fits very nicely into P. Deligne’s picture of analogies, where lattices on the non-Archimedean side correspond to hermitian metrics on the Archimedean side (see [De]). We conclude this paper by interpreting certain Archimedean intersection indices with geodesic configurations in Z . Consider p-dimensional subspaces A and B and q = (n − p)-dimensional subspaces C and D of Cn such that the cycles P(A) − P(B) and P(C) − P(D) on Pn−1 (C) have disjoint supports. Then we use the Levine currents for these linear cycles to define their local Archimedean intersection number hP(A) − P(B), P(C) − P(D)i. Under certain conditions (which are, e.g., fulfilled if p = 1 and the intersection is nontrivial), we define explicitly an oriented geodesic γ such that √

p P(A) − P(B), P(C) − P(D) = √ distorγ (A ∗ γ , B ∗ γ ), q where A ∗ γ is again the point on γ “closest” to the boundary point induced by A, namely, the orthogonal projection of this point to γ . (This formula specializes to a formula in [Ma] if n = 2 and p = q = 1.) Hence we get completely parallel formulas in the Archimedean and the nonArchimedean picture. This result and the similar behaviour of geodesics in both cases may suggest that we can regard Z as some kind of “model at infinity” for the projective space Pn−1 C , and the set of half-geodesics in Z leading to vertices in Z (∞) as “∞-adic reductions” of linear cycles. 2. The building and its compactification Throughout this paper we denote by K a finite extension of Q p , by R its valuation ring, and by k the residue class field. Besides, v is the valuation map, normalized so that it maps a prime element to 1. We write q for the cardinality of the residue class field, and we normalize the absolute value on K so that |x| = q −v(x) . Let V be an n-dimensional vector space over K . Let us briefly recall the definition of the Bruhat-Tits building X for G = PGL(V ) (see [BrTi] and [La]). We fix a maximal K -split torus T, and we let N = N G T be its normalizer. Note that T is equal to its centralizer in G. We write G = G(K ), T = T(K ), and N = N(K ) for the groups of rational points. By X ∗ (T) (resp., X ∗ (T)), we denote the

322

ANNETTE WERNER

cocharacter (resp., the character) group of T. We have a natural perfect pairing h , i : X ∗ (T) × X ∗ (T) −→ Z, (λ, χ) 7 −→ hλ, χi, where hλ, χi is the integer such that χ ◦ λ(t) = t hλ,χi for all t ∈ Gm . Let 3 be the Rvector space 3 = X ∗ (T) ⊗Z R. We can identify the dual space 3∗ with X ∗ (T) ⊗Z R, and we can extend h , i to a pairing h , i : 3 × 3∗ −→ R. Since h , i is perfect, there exists a unique homomorphism ν : T → 3 such that

ν(z), χ = −v χ(z) for all z ∈ T and χ ∈ X ∗ (T). We fix a basis v1 , . . . , vn of V such that T is induced by the group of diagonal matrices in GL(V ) with respect to v1 , . . . , vn . The group W = N /T is the Weyl group of the corresponding root system; hence it acts as a group of reflections on 3, and we have a natural homomorphism W −→ GL(3). We can embed W in N as the group of permutation matrices with respect to v1 , . . . , vn . (A permutation matrix is a matrix that has exactly one entry 1 in every line and column and that is zero otherwise.) Thereby N is the semidirect product of T and W . Since Aff(3) = 3 o GL(3), we can extend ν to a map ν : N = T o W −→ 3 o GL(3) = Aff(3). The pair (3, ν) is called the empty apartment given by T (see [La, Def. 1.9]), and whenever we think of it as an apartment, we write A = 3. One can define a collection of affine hyperplanes in A decomposing A into infinitely many faces, which are topological simplices (see [La, §11]). Besides, one defines for every x ∈ A a certain subgroup Px ⊂ G (the would-be stabilizer of x) (see [La, §12]). Then the building X is given as X = G × A/ ∼, where the equivalence relation ∼ is defined as follows: (g, x) ∼ (h, y) if and only if there exists an element n ∈ N such that ν(n)x = y and g −1 hn ∈ Px . We have a natural action of G on X via left multiplication on the first factor, and we can embed the apartment A in X , mapping a ∈ A to the class of (1, a). This is injective (see [La, Lem. 13.2]). For x ∈ A, the group Px is the stabilizer of x. A


323

subset of X of the form g A for some g ∈ G is called an apartment in X . Similarly, we define the faces in g A as the subsets g F, where F is a face in A. Then two points (and even two faces) in X are always contained in a common apartment (see [La, Prop. 13.12] and [BrTi, Th. 7.4.18]). Any apartment that contains a point of a face contains the whole face and even its closure (see [La, Prop. 13.10, Cor. 13.11] and [BrTi, Prop. 7.4.13, Cor. 7.4.14]). We fix once and for all a W -invariant scalar product on 3 which exists by [Bou, Chap. VI, §1]. This induces a metric on A. Using the Gaction, it can be continued to a metric d on the whole of X (see [La, Rem. 13.14] and [BrTi, Prop. 7.4.20]). We denote by X 0 the set of vertices (i.e., zero-dimensional faces) in X . We define a simplex in X 0 to be a subset {x1 , . . . , xk } of X 0 such that x1 , . . . , xk are the vertices of a face in X . Let ηi : Gm → T be the cocharacter induced by mapping x to the diagonal matrix with diagonal entries d1 , . . . , dn such that dk = 1 for k 6 = i and di = x. Then Ln−1 η1 , . . . , ηn−1 is an R-basis of 3, and the set of vertices in A is equal to i=1 Zηi . Let L be the set of all homothety classes of R-lattices of full rank in V . We write {M} for the class of a lattice M. Two different lattice classes {M 0 } and {N 0 } are called adjacent if there are representatives M and N of {M 0 } and {N 0 } such that π N ⊂ M ⊂ N. This relation defines a flag complex, namely, the simplicial complex with vertex set L such that the simplices are the sets of pairwise adjacent lattice classes. We have a natural G-action on L preserving the simplicial structure. Moreover, there is a G-equivariant bijection ϕ : L −→ X 0 preserving the simplicial structures. If {N } ∈ L can be written as {N } = g{M} for some g ∈ G and M = π k1 Rv1 + · · · + π kn Rvn , then ϕ({N }) is given by the pair (g, ϕ{M}) ∈ G × A, where n−1 X ϕ {M} = (kn − ki )ηi i=1

is a vertex in A. From now on we identify the vertices in X with L without explicitly mentioning the map ϕ. Definition 2.1 The combinatorial distance dist(x, y) between two points x and y in X 0 is defined as dist(x, y) = min{k : there are vertices x = x0 , x1 , . . . , xk = y, so that xi and xi+1 are adjacent for all i = 0, . . . , k − 1}.

324

ANNETTE WERNER

Hence dist is the minimal number of 1-simplices forming a path between x and y. Note that dist is, in general, not proportional to the metric d on X . If x = {M} and y = {L} are two vertices in X , we have dist(x, y) = s − r , where s = min{k : π k L ⊂ M} and r = max{k : M ⊂ π k L} (see [We, Lem. 4.2]). A. Borel and J.-P. Serre have defined a compactification of X by attaching the Tits building for G at infinity (see [BoSe]), which we now briefly describe. First we compactify the apartment A. For any half-line c in A and any point a ∈ A, there exists a unique half-line starting in a which is parallel to c. We denote it by [a, c]. Now fix a point a ∈ A, and let A∞ be the set of half-lines in A starting in a. Then we define A = A ∪ A∞ . For all x ∈ A, all c ∈ A∞ , and all > 0, we define the cone C x (c, ) as C x (c, ) = z ∈ A : z 6 = x and ≺x ([x, c], [x, z]) < . Here [x, z] is the line from x to z if z ∈ A, and it is the half-line defined above otherwise. The angle ≺x ([x, c], [x, z]) is defined as the angle between y1 − x and z 1 − x in the Euclidean space 3, where y1 ∈ [x, c] and z 1 ∈ [x, z] are arbitrary points different from x. We endow A with the topology generated by the open sets in A and by all of these cones. Then A is homeomorphic to the ball A1 = {x ∈ A : d(a, x) ≤ 1} in A. Namely, we can embed A in A1 as j : A −→ A1 , ( a+ x 7 −→ a

1−e−d(a,x) d(a,x) (x

− a) if x 6= a, if x = a,

and we map a half-line c ∈ A∞ to the point x ∈ c with d(a, x) = 1 (cf. [SchSt, Chap. IV, §2]). Note that A is independent of the choice of a. For every n ∈ N , the affine bijection ν(n) can be continued to a homeomorphism ν(n) : A → A since d is ν(n)-invariant. This yields a continuous action of N on A. We use a description of the Borel-Serre compactification due to P. Schneider and U. Stuhler (in [SchSt, Chap. IV, §2]) which is formally similar to the definition of X . Let 8 = 8(T, G) be the root system corresponding to T. It consists of finitely many elements in X ∗ (T) ⊂ 3∗ . For all a ∈ 8, there exists a unique closed, connected, unipotent subgroup Ua of G which is normalized by T and has Lie algebra ga = {X ∈ g : Ad(t)X = a(t)X for all t ∈ T } (see [Bo, Prop. 21.9]). We denote the K -rational points of Ua by Ua .


325

Now we define for a boundary point c ∈ A∞ , Pc = subgroup generated by T and the groups Ua for all a ∈ 8 such that c ∈ {x ∈ A : a(x) ≥ 0}. Note that if y ∈ A1 is the point corresponding to c ∈ A∞ via the map j defined with 0 ∈ A, then c ∈ {x ∈ A : a(x) ≥ 0} if and only if a(y) ≥ 0. Now we define an equivalence relation on G × A by (g, x) ∼ (h, y) if and only if there exists an n ∈ N such that nx = y and g −1 hn ∈ Px (using the old groups Px for points x ∈ A). Let X be the quotient X = G × A/ ∼ . Then G acts on X via left multiplication on the first factor. The compactified apartment A can be embedded as x 7→ (1, x). Besides, Px is the stabilizer of x ∈ A, and we have a natural G-equivariant embedding X ,→ X . Let X ∞ = X \X be the boundary of X . Then X ∞ is the Tits building corresponding to G. To be more precise, let 1 be the simplicial complex whose simplices are the parabolic subgroups of G with the face relation P ≤ Q if and only if Q ⊂ P. Therefore vertices in 1 correspond to maximal parabolic subgroups P ⊂ G with P 6= G. Let |1| be the geometric realization. Then we have a G-equivariant bijection τ : |1| → X ∞ such that for any b ∈ |1| the stabilizer of τ (b) is the parabolic subgroup corresponding to the simplex of 1 containing b in its interior (see [CLT, Prop. 6.1]). There is a natural bijection between parabolic subgroups of G and flags in V , associating to a flag in V its stabilizer in G. Here maximal proper parabolics correspond to minimal nontrivial flags and hence to nontrivial subspaces W of V . For any subspace W of V , we denote by yW the vertex in X ∞ corresponding to W . We now investigate geodesics in X , that is, maps c : R −→ X such that d(c(t1 ), c(t2 )) = |t1 − t2 | for all t1 , t2 ∈ R. A map c : R≥0 → X with the same isometry property is called a half-geodesic. Note that for any x0 ∈ X and for any point y ∈ X ∞ , there is a unique halfgeodesic γ in X , starting at x0 and converging to y (see [SchSt, Chap. IV, §2]). We write γ = [x0 , y]. If x0 is also a vertex, we can describe γ as follows.

326

ANNETTE WERNER

LEMMA 2.2 Fix a vertex x0 = {M} in X . Let W be a nontrivial subspace of V , and let yW be the corresponding vertex in X ∞ . Let w1 , . . . , wn be a base of V such that M = Pn i=1 Rwi and such that W is generated by w1 , . . . , wr . Then the vertices in [x 0 , yW ] are exactly the lattice classes

{Rw1 + · · · + Rwr + π k Rwr +1 + · · · + π k Rwn } for all integers k ≥ 0. Proof Note that there exists a basis w1 , . . . , wn as in our claim. (By the invariant factor theorem, we find an R-basis w1 , . . . , wn of M such that α1 w1 , . . . , αr wr is an Rbasis of M ∩ W for some αi ∈ K × .) After applying a suitable element g ∈ PGL(V ), we can assume that M = Pn Pr i=1 Rvi and W = i=1 K vi for our fixed basis v1 , . . . , vn . Hence {M} = 0 ∈ 3. Let γ be the half-line X γ (t) = c t ηi for all t ≥ 0, i≤r

P where c > 0 is a constant so that d(0, c i≤r ηi ) = 1. We denote the associated point in A∞ by z. Now we want to determine Pz . We denote by χi : T → Gm the character induced by mapping a diagonal matrix to its ith entry. Then 1 = {ai,i+1 = χi − χi+1 : i = 1, . . . , n − 1} is a base of the root system 8. By 8+ , we denote the set of positive P roots. If a = i n i ai,i+1 is an arbitrary root, then z ∈ x ∈ A : a(x) ≥ 0 iff nr ≥ 0. P Hence Pz is generated by T and all groups Ua for all a = n i ai,i+1 with nr ≥ 0. The set of roots fulfilling this condition is equal to 8+ ∪ [I ], where I = 1\{ar,r +1 } and where [I ] denotes the set of roots that are linear combinations of elements in I . Hence Pz is the standard parabolic subgroup corresponding to I (see [BoTi, §4.2]). Therefore it fixes the flag 0 ⊂ W ⊂ V . Hence z = yW ; that is, γ = [0, yW ]. The P vertices on γ are the points k i≤r ηi for all integers k ≥ 0; hence they correspond to the module classes {π −k Rv1 + · · · + π −k Rvr + Rvr +1 + · · · + Rvn } = {Rv1 + · · · + Rvr + π k Rvr +1 + · · · + π k Rvn }, as desired.


327

LEMMA 2.3 Let W and W 0 be nontrivial subspaces of V . Then the vertices yW and yW 0 in X ∞ can be connected by a geodesic in X if and only if W ⊕ W 0 = V .

Proof Assume that yW and yW 0 can be connected by a geodesic γ , that is, that γ (t) → yW as t → ∞ and that γ (t) → yW 0 as t → −∞. By [Br, Th. 2, p. 166], γ lies in an apartment. Since our claim is G-invariant, we can assume that γ lies in our standard apartment A. After replacing γ by a parallel geodesic in A, we can assume that γ contains the vertex x0 = 0. Furthermore, after reparametrization we have γ (0) = x0 . So the restriction of γ to R≥0 is equal to [x0 , yW ]. Let 8+ be the set of positive roots corresponding to the base 1 = {a1,2 , a2,3 , . . . , an−1,n } of 8. Let D be the sector D = x ∈ A : a(x) ≥ 0 for any a ∈ 1 in A. Since D is a fundamental domain for the operation of the Weyl group, we can furthermore assume that [x0 , yW ] ⊂ D. Hence yW is contained in the boundary of D. For every point z in the boundary of D, let I ⊂ 1 be the set of all ai,i+1 such that [0, z] ⊂ {x ∈ A : ai,i+1 (x) = 0}. Then Pz is generated by T and all Ua for a ∈ 8+ ∪ [I ], where [I ] is the set of roots that are linear combinations of elements in I . Hence Pz is the standard parabolic corresponding to I . Now PyW is a maximal proper parabolic; hence for z = yW the set I is just 1\{ar,r +1 } for some r ≤ n − 1. Therefore W is generated by v1 , . . . , vr . By the proof of Lemma 2.2, we know that [x0 , yW ] is the half-geodesic γ1 (t) = P c1 t i≤r ηi for t ≥ 0, where c1 > 0 is a suitable constant. This half-geodesic can be uniquely continued to a geodesic in A by letting t run over the whole of R. Since P γ lies in A, we find that γ (t) = c1 t i≤r ηi for all t ∈ R. The proof of Lemma 2.2 P also shows that the half-geodesic γ2 (t) = c2 t i≤n−r ηi for t ≥ 0 (and some c2 > 0) connects x0 with the vertex in A∞ corresponding to the vector space K v1 + · · · + K vn−r . Let p ∈ N be the permutation matrix with pv1 = vr +1 , . . . , pvn−r = vn , pvn−r +1 = v1 , . . . , pvn = vr . Then the half-geodesic pγ2 connects x0 with the point in A∞ corresponding to the P P vector space K vr +1 + · · · + K vn . Since p maps i≤n−r ηi to i≤r (−ηi ), we have P pγ2 (t) = −c2 t i≤r ηi , so that W 0 = K vr +1 + · · · + K vn , which implies that W and W 0 are indeed complementary. Now assume that V = W ⊕ W 0 . Since our claim is G-equivariant, we can assume that W = K v1 + · · · + K vr and that W 0 = K vr +1 + · · · + K vn for our standard basis v1 , . . . , vn and for some r between 1 and n − 1. Then

328

ANNETTE WERNER

γ (t) = c1 t

X

ηi

i≤r

is a geodesic in A connecting yW and yW 0 . Note that this result shows that two vertices in X ∞ can be connected by a geodesic if and only if the corresponding parabolic subgroups are opposite in the sense of [Bo, §14.20]. We call a geodesic in X combinatorial if it consists of 1-simplices and their vertices. The proof of Lemma 2.3 shows that any geodesic in X connecting two vertices in X ∞ which contains a vertex in X is already combinatorial. We now describe the combinatorial geodesics connecting two fixed vertices on the boundary of X . 2.4 Let W and be nontrivial complementary subspaces of V ; that is, W ⊕ W 0 = V . Let M and M 0 be lattices of full rank in W , respectively, W 0 . Then the vertices {M + π k M 0 } in X for all k ∈ Z define a combinatorial geodesic connecting W and W 0 . In fact, this induces a bijection between the set of pairs of lattice classes ({M}, {M 0 }) with M ⊂ W and M 0 ⊂ W 0 and the set of combinatorial geodesics connecting W and W 0 (up to reparametrization). PROPOSITION

W0

Proof We first show that the vertices {M +π k M 0 } indeed form the vertices of a combinatorial P geodesic. After applying a suitable g ∈ G, we can assume that M = i≤r Rvi and P that M 0 = Rv for our standard basis v1 , . . . , vn and r = dim W . Then P i≥r +1 i γ (t) = c1 t i≤r ηi is a combinatorial geodesic in A containing exactly the vertices {M + π k M 0 }. (Here c1 is again a constant so that γ is an isometry.) It is clear that up to reparametrization γ is the unique geodesic in X containing all those vertices. If M is equivalent to N and if M 0 is equivalent to N 0 , that is, if M = π a N and 0 M = π b N 0 for some a, b ∈ Z, then M + π k M 0 is equivalent to N + π k+b−a N 0 . Hence the geodesic defined by M and M 0 coincides with the one defined by N and N 0 up to reparametrization. Assume that γ is a combinatorial geodesic connecting W and W 0 , and assume that γ (t)→yW as t → ∞. As in the proof of Lemma 2.3, there is an element g ∈ G such that gγ is contained in our standard apartment A. Since γ is combinatorial, it contains a vertex that we can move to 0 ∈ A by applying some t ∈ T . After reparametrization, we can therefore assume that gγ (0) = 0. The proof of Lemma 2.3 shows furthermore that after composing g with some element of the Weyl group, we can assume that gγ |R≥0 is contained in the sector D, which implies that gγ (t) = P P Pn c1 t i≤r ηi for r = dim W . For M = ri=1 Rvi and M 0 = i=r +1 Rvi , this is the


329

geodesic determined by the vertices {M + π k M 0 } for all k ∈ Z. Hence γ is given by the pair (g −1 {M}, g −1 {M 0 }). Now suppose that ({M}, {M 0 }) and ({N }, {N 0 }) yield the same geodesic γ . Then, after taking suitable representatives of our module classes, there exists a k0 ∈ Z such that M + M 0 = N + π k0 N 0 . Put r = dim W , and fix a basis w1 , . . . , wr of W such Pn P that M = ri=1 Rwi and a basis wr +1 , . . . , wn of W 0 with M 0 = i=r +1 Rwi . Let A ∈ GL(r, K ) and B ∈ GL(n − r, K ) be matrices with AM = N and B M 0 = N 0 . Then A 0 ∈ GL(n, R); 0 π k0 B hence A is contained in GL(r, R) and π k0 B is contained in GL(n−r, R), which means that M = N and M 0 = π k0 N 0 , that is, that {M} = {N } and {M 0 } = {N 0 }. Let P(V ) = ProjSym V ∗ be the projective space corresponding to our n-dimensional vector space V , where V ∗ is the linear dual of V . Every nonzero linear subspace W of V defines an integral (i.e., irreducible and reduced) closed subscheme P(W ) = ProjSym W ∗ ,→ P(V ) of codimension n − dim W . These cycles given by subspaces of V are called linear. Every lattice M (of full rank) in V defines a model P(M) = ProjSym R (M ∗ ) of P(V ) over R, where M ∗ is the R-linear dual of M. If the lattices M and N differ by ∼ multiplication by some λ ∈ K × , then the corresponding isomorphism P(M)−→P(N ) induces the identity on the generic fibre. We call a nontrivial submodule N of M split if the exact sequence 0 → N → M → M/N → 0 is split, that is, if M/N is free (or, equivalently, torsion free). Every split R-submodule N of M defines a closed subscheme P(N ) = ProjSym N ∗ ,→ P(M). It is integral and has codimension n − rk N (see [We, Lem. 3.1]). These cycles in P(M) induced by split submodules are also called linear. Let y = yW be a vertex in X ∞ corresponding to the subspace W of V , and let x = {M} be a vertex in X . Then the half-line [x, yW ] connecting x and yW is combinatorial. We now show that [x, yW ] governs the reduction of the linear cycle P(W ) induced by W on the model P(M). 2.5 Let M be a lattice in V , and let x = {M} be the corresponding vertex in X . For all vertices y in X ∞ , let [x, y]m denote the initial segment of the combinatorial halfgeodesic [x, y] consisting of the first m + 1 vertices. If y = yW , we write Z y for the PROPOSITION

330

ANNETTE WERNER

linear cycle on P(V ) defined by W . Then we have a bijection [x, y]m : y vertex in X ∞ −→ linear cycles in P(M) ⊗ R R/π m , [x, y]m 7 −→ Z y ⊗ R R/π m , where Z y denotes the closure of Z y in P(M). Hence the initial segments [x, y1 ]m and [x, y2 ]m coincide if and only if the reductions of Z y1 and Z y2 in P(M) ⊗ R R/π m coincide. Proof Fix a vertex y = yW in X ∞ . We first determine the closure Z y in P(M). Put L = W ∩ M. Then L is a free (since torsion free) R-module of rank r = dim W . It is easy to see that the quotient of L ,→ M is a free R-module. Hence L is a split submodule of M, so that P(L) is an integral closed subscheme of P(M). Obviously, the generic fibre of P(L) is equal to P(W ) = Z y . Hence Z y = P(L). We define linear cycles on P(M) ⊗ R R/π m as cycles P(N ) ,→ P(M) ⊗ R R/π m for split R/π m -submodules N ,→ M ⊗ R R/π m . For such a split submodule N , let N 0 be its preimage in M. Note that N 0 contains π m M, so that it has rank n. By the invariant factor theorem, we find an R-basis x1 , . . . , xn of M and nonnegative integers a1 , . . . , an such that π a1 x1 , . . . , π an xn is a basis of N 0 . Since N = N 0 /π m M is a split submodule of M/π m M, all ai must be equal to zero or m. We can assume that a1 = · · · = ar = 0 and that ar +1 = · · · = an = m. Then N 00 = Rx1 + · · · + Rxr is a split submodule of M with reduction N ; hence P(N ) is equal to P(N 00 )⊗ R R/π m . Since P(N 00 ) is equal to the closure of Z yW for W = K x1 + · · · + K xr , we see that our map is surjective. We now show that for split submodules L 1 and L 2 of M, ProjSym L ∗1 ⊗ R R/π m = ProjSym L ∗2 ⊗ R R/π m

iff L 1 + π m M = L 2 + π m M.

First suppose that L 1 +π m M = L 2 +π m M. Then L 1 ⊗ R R/π m = L 2 ⊗ R R/π m ; hence by dualizing we find that L ∗1 ⊗ R R/π m = L ∗2 ⊗ R R/π m . So ProjSym(L ∗1 ⊗ R R/π m ) and ProjSym(L ∗2 ⊗ R R/π m ) coincide as subschemes of P(M) ⊗ R R/π m , which gives one direction of our claim. Now assume that ProjSym L ∗1 ⊗ R R/π m = ProjSym L ∗2 ⊗ R R/π m . Let I1 and I2 be the corresponding quasi-coherent ideal sheaves on P(M) ⊗ R R/π m . We denote the quotients of L j ,→ M by Q j . Then Q ∗j is a free R-module; hence Q ∗j ⊗ R R/π m is free over R/π m . Let I j be the ideal in Sym(M ∗ ⊗ R R/π m ) generated by a basis of Q ∗j ⊗ R R/π m . Then I j coincides with the kernel of the natural map Sym(M ∗ ⊗ R R/π m ) → Sym(L ∗j ⊗ R R/π m ). By [EGAII, Prop. 2.9.2], we find that I j = I ∼ j .


331

Therefore I1∼ = I2∼ . Now fix a basis x1 , . . . , xn of M ∗ ⊗ R R/π m . For every homogeneous ideal I in S = Sym(M ∗ ⊗ R R/π m ), let Sat(I ) be the ideal in S defined as Sat(I ) = {s ∈ S : for all i = 1, . . . , n there exists a k ≥ 0 such that sxik ∈ I }. Then it is easy to check that I ∼ = J ∼ if and only if Sat I = Sat J (cf. [Ha, Chap. II, Exer. 5.10, p. 125]). Hence we find that Sat I1 = Sat I2 . Since we can choose x1 , . . . , xn so that for some r the subset x1 , . . . , xr is a basis of Q ∗1 ⊗ R R/π m , it is easy to see that Sat I1 = I1 . Similarly, Sat I2 = I2 . Hence I1 = I2 , which implies that L ∗1 ⊗ R R/π m and L ∗2 ⊗ R R/π m are isomorphic as quotient modules of M ∗ ⊗ R R/π m . Since all three are free over R/π m , we find that L 1 ⊗ R R/π m and L 2 ⊗ R R/π m are equal as submodules of M ⊗ R R/π m , which implies our claim. Now we show that [x, yW1 ]m = [x, yW2 ]m

iff (W1 ∩ M) + π m M = (W2 ∩ M) + π m M.

Let w1 , . . . , wn be an R-basis of M so that W1 is generated by w1 , . . . , wr . Then by Lemma 2.2, the vertices on [x, yW1 ] are given by the module classes {Mk } for k ≥ 0, where Mk = Rw1 + · · · + Rwr + π k Rwr +1 + · · · + π k Rwn . Now Mk = (W1 ∩ M) + π k M. Hence we find (W1 ∩ M) + π m M = (W2 ∩ M) + π m M iff (m + 1)th vertex on [x, yW1 ] = (m + 1)th vertex on [x, yW2 ] iff [x, yW1 ]m = [x, yW2 ]m , as claimed. This result justifies our regarding X as a kind of graph of P(M): its combinatorial geometry keeps track of the reduction of linear cycles. 3. Non-Archimedean intersections Let us fix a lattice M in P(V ). This defines a smooth, projective model = P(M) of P(V ) over R. By Z p (), we denote the codimension p cycles on , that is, the free abelian group on the set of integral (i.e., irreducible and reduced) closed subschemes p of codimension p. If T ⊂ is a closed subset, we write C HT () for the Chow group of codimension p cycles supported on T (see [GiSo1, §4.1]). For irreducible closed subschemes Y and Z of codimension p, respectively, q, in , we can define p+q an intersection class Y · Z ∈ C HY ∩Z () (see [Fu, §20.2] and [GiSo1, Th. 4.5.1]). We denote by deg the degree map for zero-cycles in the special fibre of ; that is, for all

332

ANNETTE WERNER

P P z= n P P ∈ Z d (k ) we put deg z = n P [k(P) : k], where k(P) is the residue field of P. Let Y ∈ Z p () and Z ∈ Z q () be two irreducible closed subschemes such that p + q = n which intersect properly on the generic fibre of . (Recall that n is the dimension of V .) This means that their generic fibres are disjoint, so that Y ∩ Z is contained in the special fibre k of . Hence we can define a local intersection number hY, Z i = deg(Y · Z ), where we take the degree of the image of Y · Z ∈ C HYn∩Z () in C H n−1 (k ). We now fix linear subspaces A, B, C, and D of V such that A and B have dimension p and such that C and D have dimension q for some p, q ≥ 1 with p + q = n. We always assume that q ≥ p. Besides, we assume that the intersections A ∩ C, A ∩ D, B ∩ C, and B ∩ D are all zero. This implies that the intersection number hP(A)−P(B), P(C)−P(D)i is defined, where, as before, P(A) denotes the closure of the linear cycle P(A) in the model P(M). From [We, Th. 3.4] we can deduce that

P(A), P(C) = v det( f j (ai ))i, j=1,..., p , where a1 , . . . , a p is an R-basis of A ∩ M and where f 1 , . . . , f p is an R-basis of the free R-module (M/C ∩ M)∗ . (In other words, f 1 , . . . , f p are elements in M ∗ generating the ideal corresponding to the linear cycle P(C).) Hence we find that det( f (a )) det(g (b ))

j i j i P(A) − P(B), P(C) − P(D) = v det( f j (bi )) det(g j (ai )) for certain K -bases a1 , . . . , a p of A, b1 , . . . , b p of B and f 1 , . . . , f p of (V /C)∗ , g1 , . . . , g p of (V /D)∗ . Since the right-hand side is invariant under arbitrary base changes of these vector spaces, the intersection number on the left-hand side is independent of the choice of a lattice M in V . Hence we also write hP(A) − P(B), P(C) − P(D)i for this intersection number. Now we can prove a formula for such a local intersection number in terms of the combinatorial geometry of the Bruhat-Tits building X . We call two lattices equivalent if they define the same lattice class, that is, if they differ by a factor in K × . THEOREM 3.1 Let A, B, C, and D be as above, and assume additionally that C + D = V . Besides, we assume that there are complementary subspaces C 0 (resp., D 0 ) of C ∩ D in C (resp., D), and full rank lattices L A in A and L B in B such that the following two conditions hold.


333

First, the vector space hA, Bi generated by A and B is contained in C 0 ⊕ D 0 . Second, the lattice pC 0 (L A ) is equivalent to pC 0 (L B ), and the lattice p D 0 (L A ) is equivalent to p D 0 (L B ), where pC 0 and p D 0 denote the projections with respect to the decomposition V = (C ∩ D) ⊕ C 0 ⊕ D 0 . Choose a lattice M0 in C ∩ D, and put MC 0 = pC 0 (L A ) and M D 0 = p D 0 (L A ). By Proposition 2.4, there is a geodesic γ corresponding to M0 ⊕ MC 0 and M D 0 which connects C and D 0 . We orient γ from C to D 0 . Let A ∗ γ be the vertex on γ closest to the boundary point y A ∈ X ∞ in the following sense: A ∗ γ is the first vertex x on γ such that the half-geodesic [x, y A ] intersects γ only in x. Then

P(A) − P(B), P(C) − P(D) = p distorγ (A ∗ γ , B ∗ γ ), where distorγ means oriented distance along the oriented geodesic γ . It would be desirable to rephrase the conditions on A, B, C, and D which we impose in Theorem 3.1 in terms of the geometry of the boundary X ∞ . Before we prove Theorem 3.1, let us formulate a corollary in the case p = 1, where our conditions are rather mild. Note that in this case the intersection pairing we are considering coincides with A. Néron’s local height pairing (cf. [GiSo1, §4.3.8]). 3.2 Let a and b be different points in P(V )(K ), and let HC , H D be two different hyperplanes in P(V ) such that the cycles a − b and HC − H D in P(V ) have disjoint supports. Denote by A and B the lines in V corresponding to a and b, and by C and D the codimension 1 subspaces in V corresponding to HC = P(C) and H D = P(D). If hA, Bi ∩ C = hA, Bi ∩ D, then ha − b, HC − H D i = 0. Otherwise, choose lattices N in C ∩ D, M1 in hA, Bi ∩ C, and M2 in hA, Bi ∩ D. Then N ⊕ M1 is a lattice in C, and by Proposition 2.4 there is a geodesic γ corresponding to N ⊕ M1 and M2 which connects C and hA, Bi ∩ D. We orient γ from C to hA, Bi ∩ D. Let a ∗ γ be the vertex on γ closest to the boundary point y A . Then ha − b, HC − H D i = distorγ (a ∗ γ , b ∗ γ ). COROLLARY

Proof If the one-dimensional vector spaces hA, Bi ∩ C and hA, Bi ∩ D are equal and, say, generated by w, we take generators wa of A and wb of B, and we write w = λwa + µwb with nonzero coefficients λ and µ. If f is a homogeneous equation for C and if g is a homogeneous equation for D, we have 0 = f (w) = λ f (wa ) + µf (wb ) and similarly 0 = λg(wa ) + µg(wb ). Hence ha − b, HC − H D i = v( f (wa )/ f (wb )) − v(g(wa )/g(wb )) is indeed zero. If hA, Bi ∩ C and hA, Bi ∩ D are not equal, we can apply Theorem 3.1.

334

ANNETTE WERNER

Proof of Theorem 3.1 Note that our conditions imply that C ∩ D has dimension 2q − n = q − p, so that (C ∩ D) ⊕ C 0 ⊕ D 0 = V . Since A and D have trivial intersection, pC 0 induces an isomorphism pC 0 | A : A → C 0 , so that MC 0 is indeed a lattice of full rank in C 0 . Fix an R-basis w2 p+1 , . . . , wn of the lattice M0 in C ∩ D. Let a1 , . . . , a p be an R-basis of the lattice L A . If we denote the projection of ai to C 0 by w∼ p+i , and the pro∼ ∼ ∼ 0 ∼ jection to D by wi , we get a basis w1 , . . . , w p of M D 0 and a basis w∼ p+1 , . . . , w2 p ∼ ∼ 0 0 of MC 0 . Since A is contained in C ⊕ D , we have ai = wi + w p+i . Since MC 0 = pC 0 (L A ) is equivalent to pC 0 (L B ), we can find a constant α ∈ K × such that pC 0 (L B ) = α MC 0 . Similarly, we find some β ∈ K × , such that p D 0 (L B ) = β M D 0 . We can write an R-basis b1 , . . . , b p of L B as b1 = βw1 + αw p+1 , . . . , b p = βw p + αw2 p for some R-bases w1 , . . . , w p of M D 0 and w p+1 , . . . , w2 p of MC 0 . Hence we can calculate the intersection number α

. P(A) − P(B), P(C) − P(D) = p v β Pp Pn The vertices on γ are the lattice classes π k i=1 Rwi + i= p+1 Rwi for k ∈ Z. Note that orienting γ from C to D 0 means following these lattice classes in the direction of decreasing k. We now determine B ∗ γ . Let x = {M} be a vertex on γ , where M = π k M D 0 + Pp Pn (MC 0 + M0 ) = i=1 π k Rwi + i= p+1 Rwi for some integer k. Let us first assume that k > v(β/α), and let us put k0 = k + v(α) − v(β) > 0. Then π k0 α −1 b1 , . . . , π k0 α −1 b p , w p+1 , . . . , wn is an R-basis of M. Hence by Lemma 2.2 the vertices in [x, y B ] correspond to the Pp Pn k0 −1 l module classes i= p+1 Rwi for all l ≥ 0. The vertex next i=1 Rπ α bi + π to x on this half-geodesic is p nX i=1

R

p n n o nX o X X π k0 π k0 −1 R bi + π Rwi = bi + Rwi , α α i= p+1

i=1

i= p+1

which lies on γ since k0 − 1 ≥ 0. Hence [x, y B ] does not meet γ only in x. Now assume that k ≤ v(β/α). Then α −1 b1 , . . . , α −1 b p , π k w1 , . . . π k w p , w2 p+1 , . . . , wn is an R-basis of M. Hence by Lemma 2.2, the vertices in [x, y B ] cor Pp Pp Pn −1 k+l l respond to the module classes Rw i=1 Rα bi +π i=1 Rwi +π P p i=2 p+1−1 i for all l ≥ 0. Therefore the vertex next to x on this geodesic is i=1 Rα bi + Pp Pn k+1 π i=2 p+1 Rwi . Let us assume for the moment that this vertex is i=1 Rwi +π


335

Pp Pp Pn contained in γ . Then the module i=1 Rα −1 bi +π k+1 i=1 Rwi +π i=2 p+1 Rwi Pp P n Rw for some l ∈ Z, which implies that is equivalent to i=1 Rwi + π −l i= i p+1 there exists an integer l0 such that the matrix   β −l0 π I p π k+1−l0 I p 0 α  l−l0  Ip 0 0  π  0 0 π 1+l−l0 Iq− p is in GL(n, R). Here I p denotes the ( p × p)-unit matrix. Now we have to distinguish the cases p 6 = q and p = q (where the last block of rows is nonexistent). Let us first assume that p 6= q. Since all entries of this matrix are in R and the determinant is a unit in R, we get 1 + l − l0 = 0 and l − l0 = 0, which is a contradiction. Therefore, in the case p 6 = q for k ≤ v(β/α), the half-geodesic [x, y B ] meets γ only in x. If p = q and if k = v(β/α), we find in a similar way that l0 = k + 1 = v(β/α) + 1 and v(β/α) − l0 ≥ 0, which is a contradiction. Hence for this vertex x the half-geodesic [x, y B ] meets γ only in x. Pp Pp However, if k < v(β/α), we find that i=1 Rα −1 bi + π k+1 i=1 Rwi = P P p n π k+1 i=1 Rwi + i= p+1 Rwi , which implies that for all these x the half-line [x, y B ] meets γ not only in x. If p = q, the vertex x corresponding to k = v(β/α) is therefore the only vertex on γ such that [x, y B ] meets γ exclusively in x. In any case, we have shown that in our orientation of γ the vertex x corresponding to M B := (β/α)M D 0 + MC 0 + M0 is the first vertex on γ such that [x, y B ] meets γ only in x. Hence B ∗ γ is well defined and given by the module M B . In a similar way we can show that A ∗ γ is induced by the class of the module M A := M D 0 + MC 0 + M0 . Hence we can calculate distor(A ∗ γ , B ∗ γ ) = distor {M A }, {M B } ( |v(β) − v(α)| if 0 ≥ v(β) − v(α), = −|v(β) − v(α)| if 0 < v(β) − v(α) = v(α) − v(β), which implies our claim. The proof of Theorem 3.1 also proves the following corollary, which generalizes Manin’s formula on P1 to higher dimensions (see [Ma, §3.2]). COROLLARY 3.3 Let n = 2 p, and let A, B, C, and D be vector spaces in V of dimension p, such that A ⊕ C = A ⊕ D = B ⊕ C = B ⊕ D = C ⊕ D = V . Assume that there are lattices L A

336

ANNETTE WERNER

in A and L B in B such that pC (L A ) is equivalent to pC (L B ), and the lattice p D (L A ) is equivalent to p D (L B ), where pC and p D denote the projections with respect to the decomposition V = C ⊕ D. Put MC = pC (L A ) and M D = p D (L A ). By Proposition 2.4, there is a geodesic γ corresponding to MC and M D which connects C and D. We orient γ from C to D, and we denote by A ∗ γ the unique vertex x on γ such that the half-geodesic [x, y A ] intersects γ only in x. Then

P(A) − P(B), P(C) − P(D) = p distorγ (A ∗ γ , B ∗ γ ). We think of the point A ∗ γ on γ as the gate to γ when entering γ from the point y A at infinity. In fact, if n = 2 p, then passing from y A to a vertex x on γ means going first to A ∗ γ and then tracking along γ until x is reached. If n 6= 2 p, then this holds for all the vertices x on γ before A ∗ γ . 4. The symmetric space and its compactification Let Z be the symmetric space G/K corresponding to the real Lie group G = SL(n, C) and its maximal compact subgroup K = SU(n, C) for some n ≥ 2. By g = sl(n, C) and k = su(n, C), we denote the corresponding Lie algebras. Let σ : G → G be the involution A 7 → (t A)−1 , and put p = {X ∈ g : dσ X = −X }. Note that dσ X = −t X and k = {X ∈ g : dσ X = X }. We denote by AdG : G → GL(g) the adjoint representation. Then p is invariant under AdG K and g = p + k. Let τ : G → G/K = Z be the projection map, mapping 1 to u ∈ Z . The homomorphism dτ : g → Tu Z induces an isomorphism dτ : p ' Tu Z with dτ (Ad(k)X ) = dλ(k)dτ (X ) for all k ∈ K and X ∈ p, where λ(g) denotes the left action of g ∈ G on G/K (see [He, Chap. IV, §3]). Let B : g × g → R, defined by B(X, Y ) = Tr(ad X ad Y ) = 4n Re Tr(X Y ), be the Killing form on g (see [He, Chap. III, Lem. 6.1, and Chap. III, §8]). B is positive definite on p × p and induces a scalar product h , i on Tu Z via the isomorphism dτ : p ' Tu Z . Shifting this product around with the G-action, we get a G-invariant metric on Z . We write dist(x, y) for the corresponding distance between two points in Z . For any X ∈ p, the geodesic in u with tangent vector dτ X is given by γ (t) = (exp t X )u, where exp : g → G is the exponential map, induced by the matrix exponential function (see [He, Chap. IV, Th. 3.3]). The geodesic connecting two points in Z can be described as follows.


337

LEMMA 4.1 Let z 1 and z 2 be two points in Z . Then there exists an element f ∈ G such that z 1 = f u and z 2 = f du, where d is a diagonal matrix of determinant one with positive real entries d1 , . . . , dn . Put ai = log di ∈ R, and let X be the diagonal matrix with entries a1 , . . . , an . The geodesic connecting z 1 and z 2 is

γ (t) = f exp(t X )u √ qP 2 ai . and we have dist(z 1 , z 2 ) = 2 n

for t ∈ [0, 1],

Proof This is a straightforward calculation. Note that the map Z 3 z = gK 7 −→ t g −1 g −1 provides a bijection between points in Z and positive definite hermitian matrices in SL(n, C), or, what amounts to the same, equivalence classes {h} of hermitian metrics, that is, positive definite hermitian forms on Cn , with respect to the relation h ∼ h 0 , if h is a positive real multiple of h 0 . We now describe the differential geometric compactification of Z using halfgeodesics (see, e.g., [BGS, §3]). A ray emanating at z ∈ Z is a unit speed (half-) geodesic γ : R≥0 → Z with γ (0) = z. Two rays γ1 and γ2 are called asymptotic if dist(γ1 (t), γ2 (t)) is bounded in t ∈ R≥0 . The set of equivalence classes of rays with respect to this relation is denoted by Z (∞), and we put Z = Z ∪ Z (∞). 4.2 For any z ∈ Z and any c ∈ Z (∞), there is a unique ray γ starting in z whose class is c. We refer to γ as the geodesic connecting z and c. LEMMA

Proof See [Jo, Lem. 6.5.2, p. 255]. This fact implies that for each z ∈ Z we can identify Z (∞) with the unit sphere Sz Z = {X ∈ Tz Z : ||X || = 1} in Tz Z by associating to an element X ∈ Sz Z the · equivalence class of the ray γ with γ (0) = z and γ (0) = X . For z ∈ Z and c1 , c2 ∈ Z ∪ Z (∞) such that z 6 = c1 and z 6= c2 , we define · · ≺z (c1 , c2 ) as the angle between γ1 (0) and γ2 (0) in Tz Z , where γ1 and γ2 are the geodesics from z to c1 and c2 , respectively. For z ∈ Z , c ∈ Z (∞), and > 0, let C z (c, ) be the cone C z (c, ) = y ∈ Z : y 6 = z and ≺z (c, y) < .

338

ANNETTE WERNER

The cone topology on Z is the topology generated by the open sets in Z and these cones (see [BGS, §3.2, p. 22]). The bijection Sz Z → Z (∞) is then a homeomorphism. Since every g ∈ G acts by isometries on Z , it acts in a natural way on Z (∞), and the corresponding action on Z is continuous (see [BGS, §3.2, p. 22]). A flat E in Z is a complete totally geodesic Euclidean submanifold of maximal dimension (which is by definition the rank of Z ). Let E be a flat in Z with u ∈ E. Then Tu E is a maximal abelian subalgebra in Tu Z ' p and E = τ (exp a). In fact, the flats in Z containing u correspond bijectively to the maximal abelian subspaces of p (see [Jo, Cor. 6.4.2, p. 248]). Let a be a maximal abelian subspace of p. It is easy to see that the bilinear form on g defined as   if X, Y ∈ p,   B(X, Y ) hX, Y ig = −B(X, Y ) if X, Y ∈ k,   0 if X ∈ p, Y ∈ k or X ∈ k, Y ∈ p is positive definite and that {ad H : H ∈ a} is a commuting family of self-adjoint endomorphisms with respect to h , ig (cf. [Jo, §6.4]). Hence g can be decomposed as an orthogonal sum of common eigenspaces of the ad H : M g = g0 ⊕ gλ , λ∈3

where gλ = X ∈ g : ad(H )(X ) = λ(H )X for all H ∈ a , g0 = X ∈ g : ad(H )(X ) = 0 for all H ∈ a , and where 3 is the set of all λ 6= 0 in HomR (a, R) such that gλ 6= 0. Note that 3 is a root system (see [Kn, Cor. 6.53]). Let E be a flat containing u. A geodesic γ : R → E with γ (0) = u is called singular if it is also contained in other flats besides E; if not, it is called regular. Tangent vectors to regular (singular) geodesics are also called regular (resp., singular). A vector H ∈ a is singular if and only if there is an element Y ∈ g\g0 such that [H, Y ] = 0, and hence if and only if there exists a λ ∈ 3 with λ(H ) = 0 (see [Jo, Lem. 6.4.7, p. 251]). We denote by asing the subset of singular elements in a, that is, asing = {H ∈ a : there exists some λ ∈ 3 with λ(H ) = 0}, and by areg the complement areg = {H ∈ a : for all λ ∈ 3 : λ(H ) 6= 0}. The singular hyperplanes


339

Hλ = {H ∈ a : λ(H ) = 0} divide areg into finitely many components, the Weyl chambers. More generally, a face of a with respect to the Hλ is defined as an equivalence class of points in a with respect to the following equivalence relation: x ∼ y if and only if for each Hλ , x and y are either both contained in Hλ or lie on the same side of Hλ . The faces not contained in any hyperplane Hλ are called chambers (see [Bou, p. 60]). The set of chambers in a corresponds to the set of bases of the root system 3 in the following way. If B is a basis of 3, then the corresponding chamber C = C(B) can be described as

C = X ∈ a : λ(X ) > 0 for all λ ∈ B . Besides, the faces contained in C correspond bijectively to the subsets of B if we associate to I ⊂ B the set C I = X ∈ a : λ(X ) = 0 for all λ ∈ I and λ(X ) > 0 for all λ ∈ B\I (see [Bou, Chap. V, §1, and Chap. VI, §1, Th. 2]). Note that every face in a is contained in the closure of a chamber (see [Bou, Prop. 6, p. 61]). We consider only the faces of dimension bigger than zero; that is, we always assume that I 6 = B. We can now transfer the faces in p to the boundary. For every maximal abelian subspace a ⊂ p and every face F ⊂ a, we denote by F(∞) the so-called face at ∞, F(∞) = [γ ] : γ (t) = exp(t H )u for some H ∈ F of norm 1 ⊂ Z (∞), where [γ ] is the equivalence class of the ray γ . We now investigate these faces at infinity. Note that G can be identified with the set of R-rational points of a semisimple linear algebraic group G over R (which can be defined as a suitable subgroup of the algebraic group SL(2n, R)). For every closed algebraic subgroup of G, the group of R-rational points is a Lie subgroup of G since it is R-closed (see [Wa, Th. 3.42]). LEMMA 4.3 All maximal abelian subspaces of p are Lie algebras of maximal R-split tori in G. If a and a 0 are maximal abelian subspaces of p, then there exists an element k ∈ K with Ad(k)a = a 0 .

Proof Any abelian subspace a of p consists of pairwise commuting hermitian matrices over C; hence there exists an orthonormal basis of Cn of common eigenvectors of the elements in a. Therefore for some k ∈ K the maximal abelian subspace kak −1 is

340

ANNETTE WERNER

contained in (and hence equal to) the abelian subspace d ⊂ p of real diagonal matrices with trace zero. Since d = Lie T , where T = T(R) for the maximal R-split torus T in G of real diagonal matrices with determinant 1, our claim follows. Now let T be a maximal R-split torus in G such that a = Lie T (for T = T(R)) L is a maximal abelian subspace of p, and let g = g T ⊕ α∈8(T,G) gα be the root decomposition corresponding to T (see [Bo, §§8.17 and 21.1]). Here g T = {X ∈ g : Ad(t)X = X for all t ∈ T }, gα = {X ∈ g : Ad(t)X = α(t)X for all t ∈ T }, and 8(T, G) is the set of roots, that is, the set of nontrivial characters of T such that gα 6 = {0}. This is the same decomposition as the one we defined previously for the maximal abelian subspace a of p. Namely, by choosing a basis of g T and all gα , we find a basis of g such that Ad(t) ∈ GL(g) is given by a diagonal matrix for all t ∈ T . Passing to the Lie algebras, we find that g T = g0 and gα = gλ for λ = dα ∈ a ∗ . In particular, we have an additive bijection 8(T, G) −→ 3, which we use from now on to identify 8(T, G) and 3. Let B be a base of the root system 8(T, G), and let I ⊂ B be a proper subset. By [I ] we denote the set of roots that are linear combinations of elements in I , and we set ψ(I ) = 8+ \[I ], where 8+ denotes the set of positive roots with respect to B. Let Uψ(I ) be the unique closed connected unipotent subgroup of G, normalized by Z (T), L with Lie algebra (see [Bo, Prop. 21.9]), and let T I be the connected α∈ψ(I ) gα T component of the intersection α∈I ker α. Then the standard parabolic subgroup P I corresponding to I is defined as the semidirect product P I = Z (T I )Uψ(I ) . Now L L T Lie(Z (T I )) = g T ⊕ α∈[I ] gα . Hence Lie P I = g ⊕ α∈[I ]∪ψ(I ) gα . Note that we exclude the trivial parabolic G here since we do not allow I = B. Every proper parabolic subgroup of G is conjugate to a uniquely determined standard parabolic by an element in G = G(R) (see [Bo, Prop. 21.12]). Let us put P = P∅ . Then P is a minimal parabolic in G. Let N be the normalizer of T in G, and put N = N(R), P = P(R). Then (G, P, N ) gives rise to a Tits system by [Bo, Th. 21.15]; that is, it fulfills the conditions in [Bo, §14.15]. Hence (P, N ) is a B N -pair for G in the terminology of [Br, Chap. V, §2]. A subgroup Q of the group G is called parabolic if Q contains a conjugate of P. As this terminology suggests, we have a bijection Q 7→ Q = Q(R) between the set of parabolic subgroups of G and the set of parabolic subgroups of G (see [Bo, §21.16]). The Tits building Y corresponding to the B N -pair (P, N ) is


341

defined as the partially ordered set (poset) of proper parabolic subgroups of G with the relation Q 1 ≤ Q 2 if Q 2 ⊂ Q 1 . This poset is, in fact, the poset of simplices of a simplicial complex (see [Br, Chap. V, §3]). There is a natural G-equivariant bijection between proper parabolic subgroups of G and nontrivial flags in Cn , associating to a flag its stabilizer in G, so that we can identify Y with the poset of flags in Cn . We can now describe the stabilizers of points in Z (∞) as follows. 4.4 Let a be a maximal abelian subspace of p, let C be a chamber in a, and let B be the associated base of the root system 3. Let X be a vector in the face C I ⊂ C for some I ⊂ B satisfying ||X || = 1. Then we denote by γ (t) = exp(t X )u the ray in u defined by X , and by z the corresponding point in Z (∞). Let G z be the stabilizer of z. Then G z is the standard parabolic subgroup PI = P I (R) corresponding to I . PROPOSITION

Proof L As in [Jo, Th. 6.2.3], one can show that Lie G z = g0 ⊕ λ(X )≥0 gλ . Since X is in C I , we have λ(X ) = 0 for all λ ∈ I and λ(X ) > 0 for all λ ∈ B\I . We denote by 8+ (resp., 8− ) the positive (resp., negative) roots with respect to B. Then λ(X ) ≥ 0 for all λ ∈ 8+ . Besides, λ(X ) ≥ 0 for some λ ∈ 8− if and only if λ ∈ [I ]. L Hence Lie G z = g T ⊕ α∈8+ ∪[I ] gα = Lie PI . Note that the minimal parabolic P∅ stabilizes z, so that it is contained in G z . Hence G z is a standard parabolic, and we must have G z = PI . The following corollary shows that the poset of faces at infinity is isomorphic to the Tits building Y of G = SL(n, C). 4.5 (cf. [BGS, p. 248 f]) The association F 7−→ G z for any z ∈ F(∞) defines an order preserving bijection between the set of faces (of dimension greater than zero) in all the maximal abelian subspaces of p (ordered by the relation F 0 < F if and only if F 0 ⊂ F) and the set of proper parabolic subgroups of G ordered by the relation P 0 < P if and only if P ⊂ P 0. COROLLARY

Proof Let us first assume that two faces F and F 0 are mapped to the same parabolic subgroup. Hence there are elements X ∈ F and X 0 ∈ F 0 of norm 1 such that the points z and z 0 in Z (∞) corresponding to the geodesics (exp t X )u and (exp t X 0 )u, respectively, satisfy G z = G z 0 . Choose maximal abelian subspaces a and a 0 of p containing

342

ANNETTE WERNER

F, respectively, F 0 , with corresponding root systems 3 and 30 . There is a base B of 3 and a subset I ⊂ B such that F = C I for the chamber C in a given by B, and, similarly, there is a base B 0 of 30 and a subset I 0 ⊂ B 0 such that F 0 = C I0 0 , where C 0 is the chamber in a 0 induced by B 0 . By Lemma 4.3, there exists an element k ∈ K such that Ad(k)a 0 = a. It is easy to see that the map ϕ

a 0 ∗ −→ a ∗ , λ0 7 −→ λ0 ◦ Ad(k −1 ) induces a bijection beween 30 and 3 so that Ad(k)gλ0 = gϕ(λ0 ) . Hence the homomorphism Ad(k) : a 0 → a maps hyperplanes to hyperplanes and thus faces to faces. So Ad(k)C I0 0 is a face in a, and it is therefore contained in the closure of a chamber in a. Let N K (a) = {k ∈ K : Ad(k)a = a} and Z K (a) = {k ∈ K : Ad(k)H = H for all H ∈ a} be the normalizer and the centralizer, respectively, of a in K . Then W = N K (a)/Z K (a) is the Weyl group of 3 (see [Kn, Th. 6.57]); hence it acts transitively on the set of chambers. So we may choose k so that Ad(k)C I0 0 is a face in the closure of C, that is, so that Ad(k)C I0 0 = C J for some J ⊂ B. Now we have for all t ∈ R, exp t Ad(k)X 0 u = k exp(t X 0 )u; hence kz 0 is the class of the geodesic (exp t Ad(k)X 0 )u. Applying Proposition 4.4, we find that PJ = G kz 0 = kG z 0 k −1 = kG z k −1 = k PI k −1 , which implies that I = J and that k ∈ PI = G z = G z 0 . Hence we have kz 0 = z 0 . Since exp(t X 0 )u and exp(t Ad(k)X 0 )u are unit speed geodesics connecting u with z 0 = kz 0 , we have X 0 = Ad(k)X 0 by Lemma 4.2. Therefore X 0 must be in C J = C I , so that X and X 0 both lie in the face C I . Now let Y be an arbitrary element of C I0 0 . Then Y0 = Y/||Y || is also contained in C I0 0 and induces a point z 0 in Z (∞). Since z 0 lies in the same face at infinity as z 0 , we have G z 0 = G z 0 . Hence the same reasoning as above implies that Y0 , and hence Y , lies in C I . Altogether we find that C I0 0 ⊂ C I , that is, that F 0 ⊂ F. Reversing the roles of F and F 0 , we also have the opposite inclusion, so that F = F 0 , which proves injectivity. This implies, in particular, that two faces in p are either disjoint or equal, and also that two faces at infinity are either disjoint or equal.


343

Let us now show surjectivity. Fix a chamber C in a maximal abelian subspace a of p. A proper parabolic subgroup P ⊂ G is conjugate to a standard parabolic subgroup associated to C; hence, using Proposition 4.4, it is the stabilizer of some z ∈ Z (∞). Let X ∈ p be the unit vector such that z is the class of (exp t X )u, and let a 0 be any maximal abelian subspace containing X . Then X lies in some face F in a 0 which is mapped to G z = P. Now assume that F and F 0 are two faces in p satisfying F 0 ⊂ F. Let a be a maximal abelian subspace of p containing F. Then there exists a chamber C in a such that F = C I for some subset I of the base corresponding to C. Since F 0 is contained in C, it meets a face C J for some J ⊂ B. Since we have already seen that faces are disjoint or equal, we find that F 0 = C J , so that I ⊂ J . Hence PI ⊂ PJ , which by Proposition 4.4 implies G z ⊂ G z 0 for any two points z ∈ F(∞) and z 0 ∈ F 0 (∞). On the other hand, assume that G z ⊂ G z 0 for two points z and z 0 in Z (∞) such that z is the class of (exp t X )u and such that z 0 is the class of (exp t X 0 )u for unit vectors X , X 0 in p. Take a maximal abelian subspace a and a chamber C such that X ∈ C I for some subset I of the base corresponding to C. Then G z = PI by Proposition 4.4. Therefore G z 0 is a standard parabolic; hence we find a set J ⊃ I with G z 0 = PJ . By injectivity, X 0 lies in C J , so that I ⊂ J implies C J ⊂ C I . Hence our map is order preserving in both directions. The minimal faces of positive dimension in some maximal abelian subspace a ⊂ p are the faces F = H ∈ a : λ0 (H ) > 0 and λ(H ) = 0 for all H ∈ B\{λ0 } , where B is a base of 3 and λ0 is an element of B. The corresponding face at infinity consists of one point. According to Corollary 4.5, the minimal faces correspond to the maximal proper parabolic subgroups of G, which in turn correspond to minimal flags in G, that is, to nontrivial subspaces W ⊂ Cn . Hence we find that the set of all nontrivial subspaces of Cn can be regarded as a subset of Z (∞). We denote the point in Z (∞) corresponding to a subspace W ⊂ Cn by z W . We endow Cn with the canonical scalar product with respect to the standard basis e1 , . . . , en . LEMMA 4.6 Let W be an r -dimensional subspace of Cn with 0 < r < n. Choose an orthonormal basis w1 , . . . , wr of W , and complete it to an orthonormal basis w1 , . . . , wn of Cn so that the matrix g mapping ei to wi for all i = 1, . . . , n is contained in K . Then z W

344

ANNETTE WERNER

is the class of the following ray emanating at u:  ρ  ..  .   ρ  {g exp t   σ   .. . 



σ

      u : t ≥ 0},    

1/2 where ρ = (n − r )/(4r n 2 ) and σ = −(r/(n − r ))ρ, and where ρ appears r times. Proof Let T be the maximal split torus consisting of all real diagonal matrices of determinant 1 in G with respect to e1 , . . . , en , and put T = T(R). The corresponding root system in a = Lie T is 3 = {λi j : i 6 = j}, where λi j = λi − λ j , and λi ∈ a ∗ maps a diagonal matrix to its ith entry. The subset B = {λ12 , . . . , λn−1n } is a base of 3. Let C be the corresponding chamber. The vector space W ∼ = Ce1 ⊕ · · · ⊕ Cer corresponds to the standard parabolic ∗ ∗ }r P= ⊂G 0 ∗ 1/2 given by B\{λrr +1 }. Now put ρ = (n −r )/(4r n 2 ) and σ = − r/(n −r ) ρ. Then the diagonal matrix X with entries (ρ, . . . , ρ, σ, . . . , σ ) has norm 1 and is contained in C B\{λrr +1 } . Hence z W ∼ is given by the ray {exp(t X )u : t ≥ 0}. Applying g, our claim follows. From now on, we write 

d1

 diag(d1 , . . . , dn ) =  0

0 ..

  .

. dn

Lemma 4.6 says that γ (t) = g exp t diag(ρ, . . . , ρ, σ, . . . , σ )u is the ray connecting u and z W . We write [u, z W ] for this ray.


345

COROLLARY 4.7 Let x ∈ Z be an arbitrary point, and let W be an r -dimensional subspace of Cn with 0 < r < n. Put W ∼ = Ce1 ⊕ · · · ⊕ Cer . Then there exists an element g ∈ G such that gu = x and gW ∼ = W . For any such g, let

γ (t) = g exp t diag(ρ, . . . , ρ , σ, . . . , σ ) · u | {z }

for t ≥ 0,

r

1/2 where ρ = (n − r )/(4r n 2 ) and σ = − r/(n − r ) ρ. Then γ = [x, z W ]. Proof The last assertion is an immediate consequence of Lemma 4.6. Note that an element g ∈ G exists, as in our claim. Namely, let f ∈ G be any element satisfying u = f x, and choose an orthonormal basis w1 , . . . , wr of f (W ). Then we can complete it to an orthonormal basis w1 , . . . , wn of Cn such that the element k mapping ei to wi is contained in K . Hence g = f −1 k maps u to x and W ∼ to W . Note that if h x denotes the hermitian metric corresponding to x, then all points on the geodesic γ = [x, z W ] correspond to metrics h such that h|W is equivalent to h x |W on W and h|W ⊥ is equivalent to h x |W ⊥ on W ⊥ , the orthogonal complement of W with respect to h x . We now investigate full geodesics in Z connecting two zero-simplices on the boundary Z (∞). The following result is the Archimedean analogue of Lemma 2.3. LEMMA 4.8 Let W and W 0 be two nontrivial subspaces of Cn . Then there exists a geodesic joining z W and z W 0 if and only if W ⊕ W 0 = Cn .

Proof Assume first that W ⊕ W 0 = Cn and that dim W = r . Applying a suitable g ∈ G, we can assume that W = Ce1 + · · · + Cer and W 0 = Cer +1 + · · · + Cen . Put again 1/2 ρ = (n − r )/(4r n 2 ) and σ = − r/(n − r ) ρ, and let γ (t) = exp t diag(ρ, . . . , ρ , σ, . . . , σ )u | {z } r

for all t ∈ R. For t ≥ 0, this is equal to [u, z W ] by Lemma 4.6. Let N be the permutation matrix mapping (e1 , . . . , en ) to (er +1 , . . . , en , e1 , . . . , er ). Then we have for all

346

ANNETTE WERNER

t ≥ 0, γ (−t) = N exp t (−σ, . . . , −σ , −ρ, . . . , −ρ)N −1 u | {z } n−r

= N exp t (−σ, . . . , −σ, −ρ, . . . , −ρ)u. 1/2 and −ρ = −((n − r )/r )(−σ ), we can apply Since −σ = (r/(4(n − r )n 2 ) Lemma 4.6 and find that γ (−t) is the ray [u, z W 0 ]. Hence γ is a geodesic in Z connecting W and W 0 . Now suppose that there exists a geodesic γ joining W and W 0 , and let x = γ (0). For t ≥ 0, the half-geodesic γ (t) connects x and one of the vector spaces, say, W . Hence by Corollary 4.7 (after a reparametrization of γ so that it has unit speed), γ (t) = g exp t diag(ρ, . . . , ρ, σ, . . . , σ )u

for t ≥ 0,

where g maps Ce1 + · · · + Cer to W and u to x. Then this equality holds for all t ∈ R. We have already seen in the other half of our proof that exp(−t) diag(ρ, . . . , ρ, σ, . . . , σ )u connects u with Cer +1 +· · ·+Cen , so that γ (−t) for t ≥ 0 connects x with g(Cer +1 + · · · + Cen ). Hence W 0 = g(Cer +1 + · · · + Cen ), which implies that W ⊕ W 0 = V . Note that this result—similar to its non-Archimedean counterpart Lemma 2.3—shows that two minimal faces in Z (∞) can be connected by a geodesic if and only if the corresponding parabolic subgroups are opposite. We can now prove an Archimedean analogue of Proposition 2.4. PROPOSITION 4.9 Let W and W 0 be complementary subspaces of Cn , that is, W ⊕ W 0 = Cn , and put r = dim W . Let {h} and {h 0 } be equivalence classes of hermitian metrics on W , respectively, W 0 , and let g be an element in SL(n, C) such that ge1 , . . . , ger is an orthonormal basis of αh and ger +1 , . . . , gen is an orthonormal basis of α 0 h 0 for some representatives αh of {h} and α 0 h 0 of {h 0 }. Then

g exp t diag(ρ, . . . , ρ , σ, . . . , σ )u | {z } r

1/2 is a geodesic connecting W and W 0 , where ρ = (n − r )/(4r n 2 ) and σ = − r/(n − r ) ρ. In fact, this association defines a bijection between the set of pairs ({h}, {h 0 }) of metric classes on W and W 0 and the set of geodesics (up to reparametrization) connecting W and W 0 .


347

Proof Obviously, given {h} and {h 0 }, we can always find an element g, as in our claim. It is clear that g exp t diag(ρ, . . . , ρ, σ, . . . , σ )u is a geodesic connecting W and W 0 . Let f ∈ SL(n, C) be another element such that f e1 , . . . , f er is an orthonormal basis of βh and such that f er +1 , . . . , f en is an orthonormal basis of β 0 h 0 for some positive real numbers β and β 0 . Then there are positive real numbers δ and δ 0 with δr δ 0n−r = 1 such that diag(δ −1 , . . . , δ −1 , δ 0−1 , . . . , δ 0−1 )g −1 f is in K . Besides, g −1 f is of the form ∗ 0 g −1 f = }r . 0 ∗ Hence f exp t diag(ρ, . . . , ρ, σ, . . . , σ )u = g exp t diag(ρ, . . . , ρ, σ, . . . , σ )(g −1 f ) u = g exp t diag(ρ, . . . , ρ, σ, . . . , σ ) diag(δ, . . . , δ, δ 0 , . . . , δ 0 )u = g exp(t + t0 ) diag(ρ, . . . , ρ, σ, . . . , σ )u, where t0 satisfies ρt0 = log δ (hence σ t0 = −(r/(n − r ))ρt0 = log δ 0 ). This shows that up to reparametrization our geodesic is independent of the choice of g. Now let γ be a geodesic connecting W and W 0 . We have seen in the proof of Lemma 4.8 that up to reparametrization γ (t) = g exp t diag(ρ, . . . , ρ, σ, . . . , σ )u, where g maps Ce1 ⊕ · · · ⊕ Cer to W and Cer +1 ⊕ · · · ⊕ Cen to W 0 . Let wi = g(ei ) for all i = 1, . . . , n, and let h (resp., h 0 ) be the metric on W (resp., W 0 ) with orthonormal basis w1 , . . . , wr (resp., wr +1 , . . . , wn ). Then γ is induced by the pair ({h}, {h 0 }). Now suppose that ({h 1 }, {h 01 }) and ({h 2 }, {h 02 }) are pairs of metric classes leading to the same geodesic. Let g1 and g2 be elements in SL(n, C) such that for i = 1 or 2, gi e1 , . . . , gi er is an orthonormal basis of αi h i and gi er +1 , . . . , gi en is an orthonormal basis of αi0 h i0 , where αi , αi0 are positive real numbers. Then there exists some t0 ∈ R such that g1 u = g2 exp t0 diag(ρ, . . . , ρ, σ, . . . , σ )u. Put d = exp t0 diag(ρ, . . . , ρ, σ, . . . , σ ). Then g1−1 g2 d is contained in K . Let us denote by λ0 the canonical scalar product on Cn , and let λi be the metric on Cn with orthonormal basis gi e1 , . . . , gi en for i = 1 or 2. Then λi is the orthogonal sum of αi h i and αi0 h i0 . By definition, we have for all v, w ∈ Cn , λ0 (v, w) = λ1 (g1 v, g1 w)

and

λ0 (v, w) = λ2 (g2 v, g2 w).

Since g1−1 g2 d is in K , we find that λ2 (g2 v, g2 w) = λ0 (v, w) = λ0 (g1−1 g2 dv, g1−1 g2 dw).

348

ANNETTE WERNER

If v and w are in Ce1 + · · · + Cer , then dv = δv and dw = δw for the positive real number δ = exp(t0 ρ). Hence α2 h 2 (g2 v, g2 w) = λ2 (g2 v, g2 w) = |δ|2 λ0 (g1−1 g2 v, g1−1 g2 w) = |δ|2 λ1 (g2 v, g2 w) = |δ|2 α1 h 1 (g2 v, g2 w), which implies that h 1 is equivalent to h 2 on W . Similarly, looking at vectors in Cer +1 + · · · + Cen , we find that h 01 is equivalent to h 02 . Note that we can use the identification of Z with classes of hermitian metrics on Cn to describe the geodesic γ corresponding to ({h}, {h 0 }) as follows: γ consists of all hermitian metrics {h 0 } such that W is orthogonal to W 0 under h 0 and such that h 0 |W is equivalent to h on W and h 0 |W 0 is equivalent to h 0 on W 0 . 5. Archimedean intersections In this section we prove an Archimedean analogue of Theorem 3.1. We first define the local Archimedean intersection number of linear cycles in Pn−1 C . Fix a hermitian metric h on V = Cn . Then we can also define a metric h ∗ on the dual vector space V ∗ . Let W be a linear subspace of V of codimension p, and let z 1 , . . . , z n be an Tp orthonormal basis of V ∗ such that W is the intersection W = i=1 ker(z i ). (Then the linear cycle P(W ) ⊂ P(V ) = Pn−1 is given by the homogeneous ideal generated by C z 1 , . . . , z p .) On P(V )\P(W ) we define τ = log |z 1 |2 + · · · + |z n |2 and σ = log |z 1 |2 + · · · + |z p |2 . Besides, we define (1, 1)-forms α = dd c τ and β = dd c σ on P(V )\P(W ), where dd c = (i/(2π ))∂∂. Put 3W = (τ − σ )

p−1 X

α ν β p−1−ν ,

ν=0

which is the Levine form for the linear cycle P(W ) (see [GiSo1, Exam. 1.4] and [GiSo2]). Then 3W induces a Green current for P(W ) with associated form α p (see [GiSo2, §5.1]). From now on we fix linear subspaces A, B, C, and D of V = Cn , such that A and B have dimension p, and C and D have dimension q for some p, q ≥ 1 with p + q = n. We always assume that q ≥ p. Besides, we assume that the intersections A ∩ C, A ∩ D, B ∩ C, and B ∩ D are all zero. In this case the cycles P(A) − P(B) and P(C) − P(D) meet properly on P(V ). We define their Archimedean intersection


349

number as

P(A) − P(B), P(C) − P(D) =

Z

(3C − 3 D ) − P(A)

Z

(3C − 3 D ), P(B)

(cf. [GiSo1, §4.3.8]). Using the commutativity of the ∗-product for Green currents (see [GiSo1, Cor. 2.2.9]), we find that this is independent of the choice of a hermitian metric h on V . Now we can prove a formula for such a local intersection number in terms of the geometry of the symmetric space Z . Taking into account the correspondence between lattices on the non-Archimedean side and hermitian metrics on the Archimedean side (cf. [De]), the following result is the Archimedean counterpart of our nonArchimedean Theorem 3.1. 5.1 Let A, B, C, and D be as above, and assume additionally that C + D = V . Besides, we assume that there are complements C 0 (resp., D 0 ) of C ∩ D in C (resp., D) and hermitian metrics h A on A and h B on B such that the following two conditions hold. First, the vector space hA, Bi generated by A and B is contained in C 0 ⊕ D 0 . Second, the metric pC 0 ∗ (h A ) is equivalent to pC 0 ∗ (h B ), and the metric p D 0 ∗ (h A ) is equivalent to p D 0 ∗ (h B ), where pC 0 and p D 0 denote the projections with respect to the decomposition V = (C ∩ D) ⊕ C 0 ⊕ D 0 . Choose a metric h 0 in C ∩ D, and put h C 0 = pC 0 ∗ (h A ) and h D 0 = p D 0 ∗ (h A ). By Proposition 4.9, there is a geodesic γ corresponding to the orthogonal sum h 0 ⊕ h C 0 on C and the metric h D 0 on D 0 which connects C and D 0 . We orient γ from C to D 0 . Let A ∗ γ be the unique point z on γ such that the ray [z, z A ] meets γ at a right angle. Then √

p P(A) − P(B), P(C) − P(D) = √ distorγ (A ∗ γ , B ∗ γ ), q THEOREM

where distorγ means oriented distance along the oriented geodesic γ . Before we prove Theorem 5.1, let us formulate a corollary in the case p = 1, where our conditions are rather mild. Note that in this case the intersection pairing we are considering coincides with Néron’s local height pairing (cf. [GiSo1, §4.3.8]). The following result generalizes Manin’s formula for P1 to higher dimensions (see [Ma, Th. 2.3]). COROLLARY 5.2 Let a and b be different points in Pn−1 C , and let HC and H D be different hyperplanes in Pn−1 such that the cycles a − b and HC − H D have disjoint supports. Denote by C

350

ANNETTE WERNER

A and B the lines in Cn corresponding to a and b, and by C and D the codimension 1 subspaces in Cn corresponding to HC and H D . If hA, Bi ∩ C = hA, Bi ∩ D, then ha − b, HC − H D i = 0. If this is not the case, choose hermitian metrics h 0 on C ∩ D, h 1 on hA, Bi ∩ C, and h 2 on hA, Bi ∩ D. Then h 0 ⊕ h 1 is a metric on C, and by Proposition 4.9 there exists a geodesic γ connecting C and hA, Bi ∩ D associated to the pair ({h 0 ⊕ h 1 }, {h 2 }). We orient γ from C to hA, Bi ∩ D. Then 1 distorγ (a ∗ γ , b ∗ γ ). ha − b, HC − H D i = √ n−1 Proof If the one-dimensional vector spaces hA, Bi ∩ C and hA, Bi ∩ D are equal, reasoning similar to that in the proof of Corollary 3.2 shows that ha − b, HC − H D i is indeed zero. If they are not equal, we can apply Theorem 5.1. Proof of Theorem 5.1 Our conditions imply that C ∩ D has dimension 2q − n = q − p, so that indeed V = (C ∩ D) ⊕ C 0 ⊕ D 0 . Since A and D have trivial intersection, the projection pC 0 : A → C 0 is a linear isomorphism, so that we can define pC 0 ∗ h A as h A ◦ pC−10 . Hence the orthogonal sum h 0 ⊕ h C 0 is indeed a hermitian metric on C. Now let w2 p+1 , . . . , wn be an orthonormal basis of h 0 in C ∩ D. Let a1 , . . . , a p be an orthonormal basis of 0 h A in A. If we denote the projection of ai to C 0 by w∼ p+i , and the projection to D ∼ ∼ ∼ by wi , we get an orthonormal basis w1 , . . . , w p of h D 0 , and an orthonormal basis ∼ ∼ ∼ 0 0 0 w∼ p+1 , . . . , w2 p of h C . Since A is contained in C ⊕ D , we have ai = wi + w p+i . Since h C 0 = pC 0 ∗ (h A ) is equivalent to pC 0 ∗ (h B ), we can find a constant α ∈ R>0 so that α 2 pC 0 ∗ (h B ) = h C 0 . Similarly, we find some β ∈ R>0 such that β 2 p D 0 ∗ (h B ) = h D 0 . Hence there is an orthonormal basis b1 , . . . , b p of h B in B such that b1 = βw1 + αw p+1 , . . . , b p = βw p + αw2 p for some orthonormal bases w1 , . . . , w p of h D 0 and w p+1 , . . . , w2 p of h C 0 . We define a metric h on V as the orthogonal sum of h 0 , h C 0 , and h D 0 . If 3C and 3 D are the Levine currents with respect to h, we can calculate Z Z p−1 3C − 3 D = 2 p log(α/β) dd c log(|u 1 |2 + · · · + |u p |2 ) , P(B)

P(B)

where u 1 , . . . , u p are projective coordinates on P(B). Since (i/2)∂∂ log(|u 1 |2 + · · · + |u p |2 ) is the (1, 1)-form with respect to the Fubini-Study metric on P(B), we have Z i p−1 ∂∂ log |u 1 |2 + · · · + |u p |2 = ( p − 1)! vol P(B) P(B) 2


351

by Wirtinger’s theorem (see [GrHa, p. 31]). Therefore Z i p−1 1 ( p − 1)! 2 2 ∂∂ log |u | + · · · + |u | = vol P(B) = 1 p 1 p−1 p−1 π π P(B) 2 since the volume of P(B) with respect to the Fubini-Study metric is π p−1 /( p − 1)! R (see, e.g., [BGM, p. 18]). A similar calculation gives P(A) 3C − 3 D = 0, so that our intersection number is

β P(A) − P(B), P(C) − P(D) = 2 p log . α For some complex number δ, the element g mapping e1 , . . . , en to δ −1 w1 , . . . , δ −1 wn is in SL(n, C). Then by Proposition 4.9, putting ρ = 1/2 q/(4 pn 2 ) and σ = −( p/q)ρ, γ (t) = g exp t diag(ρ, . . . , ρ , σ, . . . , σ )u | {z } | {z } p

q

is the geodesic connecting C and D 0 corresponding to {h 0 ⊕h C 0 } and {h D 0 }. Orienting γ from C to D 0 means following the direction of increasing t. We now determine B ∗ γ . Let z = g exp t diag(ρ, . . . , ρ, σ, . . . , σ )u be a point on γ . We put g ∼ = g exp t diag(ρ, . . . , ρ, σ, . . . , σ ), so that g ∼ u = z. We want to describe [z, z B ]. Let c be the positive real number 1/2 c = β 2 exp(−2tρ) + α 2 exp(−2tσ ) , and let k ∈ SL(n, C) be the matrix  β exp(−tρ)I p  αc k =  c exp(−tσ )I p 0

− αc exp(−tσ )I p β c exp(−tρ)I p 0

0 0

  ,

Iq− p

where I p denotes the ( p × p)-unit matrix. Obviously, k is an element in K such that g ∼ k = g exp t diag(ρ, . . . , ρ, σ, . . . , σ )k maps the vector space generated by e1 , . . . , e p to B. Using Corollary 4.7, we find that [z, z B ] is given by g ∼ k exp s diag(ρ, . . . , ρ, σ, . . . , σ ) u for s ≥ 0. On the other hand, [z, z D 0 ] is the ray g ∼ exp s diag(ρ, . . . , ρ, σ, . . . , σ ) u

for s ≥ 0.

Now the angle between [z, z A ] and [z, z D 0 ] in z = g ∼ u is equal to the angle between γ1 (s) = k exp s diag(ρ, . . . , ρ, σ, . . . , σ ) u

352

ANNETTE WERNER

and γ2 (s) = exp s diag(ρ, . . . , ρ, σ, . . . , σ ) u in u. Recall that τ : G → Z is the projection map, and recall that λ(g) denotes the left action of g ∈ G on Z . Then we have ·

γ 2 (0) = dτ diag(ρ, . . . , ρ, σ, . . . , σ ) and ·

γ 1 (0) = dλ(k)dτ diag(ρ, . . . , ρ, σ, . . . , σ ) = dτ Ad(k) diag(ρ, . . . , ρ, σ, . . . , σ )

(see Sec. 4). Recall that h , i is the scalar product on Tu Z induced by the Killing form on p. We have

· · γ 1 (0), γ 2 (0) = 4n Re Tr k diag(ρ, . . . , ρ, σ, . . . , σ )k −1 diag(ρ, . . . , ρ, σ, . . . , σ ) . Calculating this matrix, we find that Tr k diag(ρ, . . . , ρ, σ, . . . , σ )k −1 diag(ρ, . . . , σ, . . . , σ ) i h ρ2β 2 α 2 ρσ σ 2β 2 exp(−2tρ) + 2 exp(−2tσ ) + exp(−2tρ) =p c2 c2 c2 2 + (q − p)σ ρ2 2 2 2 2 2 2 2 p( p + q )β exp(−2tρ) − 2 p qα exp(−2tσ ) + (q − p) p c c2 q 2 ρ2 = 2 2 pqnβ 2 exp(−2tρ) − p 2 nα 2 exp(−2tσ ) . c q

=

Therefore

· · γ 1 (0) γ 2 (0) = 0

p α2 , q β2 q β2 n hence iff 2tρ = log . q p α2

iff exp(−2tρ + 2tσ ) =

Thus [z, z B ] meets [z, z D 0 ] (and hence γ ) at a right angle if and only if t =


353

(q/(2nρ)) log(qβ 2 / pα 2 ). So B ∗ γ is well defined and equal to the point q β 2 q/(2n) q β 2 − p/(2n) q β 2 q/(2n) B ∗ γ = g diag , . . . , , , p α2 p α2 p α2 q β 2 − p/(2n) u. ..., p α2 An analogous calculation gives q q/(2n) q − p/(2n) q − p/(2n) q q/(2n) A ∗ γ = g diag ,..., , ,..., u. p p p p Now we can calculate dist(A ∗ γ , B ∗ γ ) β q/n β − p/n β − p/n β q/n = dist u, diag ,..., , ,..., u α α α α β √ = 2 pq log , α by Lemma 4.1. Now we have 1 ≤ β/α if and only if A ∗γ appears before B ∗γ in our orientation of γ . Hence ( √ 2 pq log βα if α ≤ β, distorγ (A ∗ γ , B ∗ γ ) = √ β −2 pq log if α > β β = 2 pq log , α

α

√

which implies that

√ p β P(A) − P(B), P(C) − P(D) = 2 p log = √ distorγ (A ∗ γ , B ∗ γ ), α q

whence our claim. Acknowledgments. I would like to thank S. Bloch, L. Bröcker, A. Deitmar, Ch. Deninger, G. Kings, K. Künnemann, E. Landvogt, Yu. I. Manin, P. Schneider, K. Stramm, M. Strauch, and E. de Shalit for useful and inspiring discussions concerning this paper. I am also grateful to the Max-Planck-Institut für Mathematik in Bonn for the stimulating atmosphere during the early stages of this work.

354

ANNETTE WERNER

References [BGS]

W. BALLMANN, M. GROMOV, and V. SCHROEDER, Manifolds of Nonpositive

Curvature, Progr. Math. 61, Birkhäuser, Boston, 1985. MR 87h:53050 320, 337, 338, 341 [BGM] M. BERGER, P. GAUDUCHON, and E. MAZET, Le spectre d’une variété riemannienne, Lecture Notes in Math. 194, Springer, Berlin, 1971. MR 43:8025 351 [Bo] A. BOREL, Linear Algebraic Groups, 2d ed., Grad. Texts in Math. 126, Springer, New York, 1991. MR 92d:20001 324, 328, 340 [BoSe] A. BOREL and J.-P. SERRE, Cohomologie d’immeubles et de groupes S-arithmétiques, Topology 15 (1976), 211–232. MR 56:5786 324 ´ [BoTi] A. BOREL and J. TITS, Groupes réductifs, Inst. Hautes Etudes Sci. Publ. Math. 27 (1965), 55–150. MR 34:7527 326 ´ ements de mathématique, fasc. 34: Groupes et algèbres de Lie, [Bou] N. BOURBAKI, El´ chapitres 4–6, Actualités Sci. Indust. 1337, Hermann, Paris, 1968. MR 39:1590 323, 339 [Br] K. S. BROWN, Buildings, Springer, New York, 1989. MR 90e:20001 327, 340, 341 [BrTi] F. BRUHAT and J. TITS, Groupes réductifs sur un corps local, I: Données radicielles ´ valuées, Inst. Hautes Etudes Sci. Publ. Math. 41 (1972), 5–251. MR 48:6265 321, 323 [CLT] C. W. CURTIS, G. I. LEHRER, and J. TITS, Spherical buildings and the character of the Steinberg representation, Invent. Math. 58 (1980), 201–210. MR 81f:20060 325 [De] P. DELIGNE, “Le déterminant de la cohomologie” in Current Trends in Arithmetical Algebraic Geometry (Arcata, Calif., 1985), Contemp. Math. 67, Amer. Math. Soc., Providence, 1987, 93–177. MR 89b:32038 321, 349 [Fu] W. FULTON, Intersection Theory, 2d ed., Ergeb. Math. Grenzgeb. (3) 2, Springer, Berlin, 1998. MR 99d:14003 331 ´ [GiSo1] H. GILLET and C. SOULE´ , Arithmetic Intersection Theory, Inst. Hautes Etudes Sci. Publ. Math. 72 (1990), 93–174. MR 92d:14016 331, 333, 348, 349 [GiSo2] , Characteristic classes for algebraic vector bundles with Hermitian metric, II, Ann. of Math. (2) 131 (1990), 205–238. MR 91m:14032b 348 [GrHa] P. GRIFFITHS and J. HARRIS, Principles of Algebraic Geometry, Pure Appl. Math., Wiley, New York, 1978. MR 80b:14001 351 ´ ements de géométrie algébrique, II: Etude ´ [EGAII] A. GROTHENDIECK and J. DIEUDONNE´ , El´ ´ globale e´ lémentaire de quelques classes de morphismes, Inst. Hautes Etudes Sci. Publ. Math. 8 (1961). MR 36:177b 330 [Ha] R. HARTSHORNE, Algebraic Geometry, Grad. Texts in Math. 52, Springer, New York, 1977. MR 57:3116 331 [He] S. HELGASON, Differential Geometry, Lie Groups and Symmetric Spaces, Pure Appl. Math. 80, Academic Press, New York, 1978. MR 80k:53081 336 [Jo] J. JOST, Riemannian Geometry and Geometric Analysis, Universitext, Springer, Berlin, 1995. MR 96g:53049 337, 338, 341 [Kn] A. W. KNAPP, Lie Groups beyond an Introduction, Progr. Math. 140, Birkhäuser, Boston, 1996. MR 98b:22002 338, 342


355

[La]

E. LANDVOGT, A Compactification of the Bruhat-Tits Building, Lecture Notes in

[Ma]

YU. I. MANIN, Three-dimensional hyperbolic geometry as ∞-adic Arakelov geometry,

[Ne]

´ A. NERON , Quasi-fonctions et hauteurs sur les variétés abéliennes, Ann. of Math. (2)

[SchSt]

P. SCHNEIDER and U. STUHLER, Representation theory and sheaves on the

Math. 1619, Springer, Berlin, 1996. MR 98h:20081 321, 322, 323 Invent. Math. 104 (1991), 223–243. MR 92f:14019 319, 321, 335, 349 82 (1965), 249–331. MR 31:3424

[Wa] [We]

´ Bruhat-Tits building, Inst. Hautes Etudes Sci. Publ. Math. 85 (1997), 97–191. MR 98m:22023 324, 325 F. W. WARNER, Foundations of Differentiable Manifolds and Lie Groups, Grad. Texts in Math. 94, Springer, New York, 1983. MR 84k:58001 339 A. WERNER, Non-archimedean intersection indices on projective spaces and the Bruhat-Tits building for PGL, Ann. Inst. Fourier (Grenoble) 51 (2001), 1483–1505. 320, 324, 329, 332

Mathematisches Institut, Universität Münster, Einsteinstrasse 62, D-48149 Münster, Germany; [email protected]


UNIQUENESS OF CONTINUOUS SOLUTIONS FOR BV VECTOR FIELDS FERRUCCIO COLOMBINI and NICOLAS LERNER

Abstract We consider a vector field whose coefficients are functions of bounded variation, with a bounded divergence. We prove the uniqueness of continuous solutions for the Cauchy problem. Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. Statement of the results . . . . . . . . . . . . . . . . . . . . . . . . 2.1. A local result for bounded vector fields . . . . . . . . . . . . . 2.2. A global result for transport equations . . . . . . . . . . . . . 3. Lemmas on nonnegative solutions . . . . . . . . . . . . . . . . . . 3.1. Local results for bounded vector fields . . . . . . . . . . . . . 3.2. Global results for transport equations with integrable coefficients 4. Commutation lemmas . . . . . . . . . . . . . . . . . . . . . . . . 4.1. The DiPerna-Lions commutation argument . . . . . . . . . . . 4.2. Commutation for a BV vector field . . . . . . . . . . . . . . . 5. Uniqueness results . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1. Local results . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2. Global results for transport equations . . . . . . . . . . . . . . Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.1. On the divergence of a vector field . . . . . . . . . . . . . . . A.2. A Log-Lipschitz function is not in W 1,1 . . . . . . . . . . . . A.3. A weakened version of condition (2.11) . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

357 360 360 361 364 364 367 370 370 372 374 374 375 379 379 380 382 383

1. Introduction The study of transport equations with irregular coefficients has been flourishing in the last decade, mainly following the paper by R. DiPerna and P. Lions [DL]. The DUKE MATHEMATICAL JOURNAL c 2002 Vol. 111, No. 2, Received 30 August 2000. Revision received 27 January 2001. 2000 Mathematics Subject Classification. Primary 35F05, 34A12, 26A45.

357

358

COLOMBINI and LERNER

Eulerian approach, in which the partial differential equation (PDE) X ∂t + a j (t, x)∂ j

(1.1)

1≤ j≤d

is under consideration, was developed in [DL], under the assumption of W 1,1 -regularity for the a j (t, ·) (and bounded divergence), yielding uniqueness for L ∞ -solutions. 0 In fact, L p -solutions are proven unique for a j in W 1, p (and bounded divergence) for p ∈ [1, ∞]. It should be pointed out here that this result was new even for Lipschitz coefficients for which the classical method of Carleman estimates (see, e.g., [H1, Chapter 28]) would give uniqueness only for L 2loc -solutions, whereas [DL] gives uniqueness for L 1loc -solutions as well. Other important results with applications to fluid mechanics were recently given by B. Desjardins [De1], [De2], [De3], F. Bouchut and L. Desvillettes [BD], and Bouchut and F. James [BJ]. The papers by G. Petrova and B. Popov [PP] and by F. Poupaud and M. Rascle [PR], as well as the recent note by Lions [Li], raise the question of uniqueness for BV vector fields, that is, for those whose coefficients are L 1 with derivative measure. In the present paper, we give an affirmative answer to the question of uniqueness for continuous solutions of BV transport equations with bounded divergence. In particular, we prove the following local theorem. THEOREM 1.1 Let X be a vector field with coefficients in BV ∩ L ∞ , and let c be a Radon measure. Let S be a C 1 -oriented hypersurface, noncharacteristic for X . Let u be a continuous function such that X u = cu, supp u ⊂ S+ ,

where S+ is the half-space above the oriented S. Then if the positive part of the divergence (div X )+ belongs to L ∞ as well as c+ , the function u vanishes in a neighborhood of S. This theorem is a consequence of Theorem 2.1. We also provide in Section 2 a specific statement for transport equations with BV coefficients and nonfinite speed of propagation. Our proof is divided into two steps. The first step, described in Section 3, is a proof of uniqueness for nonnegative solutions. It turns out that it is quite easy to prove that nonnegative solutions are unique. In fact, if X is a bounded vector field, with L 1loc -divergence and (div X )+ ∈ L ∞ , the assumptions 0 ≤ w ∈ L ∞,

Xw = 0

are sufficient to imply uniqueness through a noncharacteristic hypersurface. (In fact, the above X w = 0 can be weakened to X w ≤ 0.) Of course, the strong assumption

VECTOR FIELDS OF BOUNDED VARIATION

359

here is that w is supposed to be nonnegative. However, almost nothing is then required from the vector field, which should only be bounded, with some natural requirements on its divergence. The second step, developed in Section 5, is devoted to proving that Xu = 0

implies

X (u 2 ) = 0.

To get this we use an approximation argument; in Section 4 we review the DiPernaLions commutation argument. Of course, we do not prove convergence in L 1 of the key commutator. We point out that the weak convergence (vague topology of measures) is enough for our argument to work. Once this is proved, we need only apply the first step on nonnegative solutions. In fact, more generally we show that, assuming only some boundedness or integrability property of the coefficients and the positive part of the divergence, it is easy to prove uniqueness for nonnegative solutions, provided the vector field is noncharacteristic with respect to the initial hypersurface. Then if we know that u satisfies X u = 0 and that u vanishes on the initial surface, we pick up a nonnegative function α(u), for example, u 2 if u is bounded (or u 2 /1 + u 2 if u is in L p for a finite p), and we try to prove that X (α(u)) is zero as well. For this we need some approximation argument. It would also be interesting to study the links between the Eulerian and the Lagrangian approach, the latter being concerned with the properties of the ordinary differential equation (ODE) x(t) ˙ = a t, x(t) , x(0) = x0 . (1.2) This problem has a long history, going back to the mid-nineteenth century. The main widely known result is the Cauchy-Lipschitz theorem, providing existence, uniqueness, and stability for (1.2) under an assumption of Lipschitz continuity for a with respect to the variable x (and integrability in the time variable t). In [ChL], the authors proved that a hypothesis somewhat weaker than Lipschitz continuity could replace the classical assumptions without altering the main conclusions. The requirements in that paper were the existence of a positive nondecreasing continuous modulus of continuity ω and R0 > 0 such that, for all R ∈ (0, R0 ), Z R dr = +∞, 0 ω(r ) and the condition on a, for kx1 − x2 k < R0 , that

a(t, x1 ) − a(t, x2 ) ≤ α(t)ω kx1 − x2 k , Let t 7 → ψ(t, x) stand for a continuous solution of Z t ψ(t, x) = x + a s, ψ(s, x) ds. 0

α ∈ L 1loc .

(1.3)

360


Rr Setting ν(r ) = R0 ds/(ω(s)), we see that the function ν is of class C 1 , negative for 0 < r < R0 , and such that ν 0 (r ) = 1/ω(r ) > 0. One can prove, for kx1 − x2 k small enough, that Z t

ψ(t, x1 ) − ψ(t, x2 ) ≤ ν −1 ν kx1 − x2 k + α(s) ds . (1.4) 0

In the classical Lipschitz case, ω(r ) = r , ν(r ) = ln r , so that we get the familiar Rt

ψ(t, x1 ) − ψ(t, x2 ) ≤ kx1 − x2 k e 0 α(s) ds .

In the so-called Log-Lipschitz case, we have ω(r ) = r ln

1 r

,

1 ν(r ) = − ln ln ; r

we get R

− t α(s) ds

ψ(t, x1 ) − ψ(t, x2 ) ≤ kx1 − x2 ke 0 .

The paper [ChL] provides as well the existence of solutions under assumption (1.3). These results are true in infinite dimension, that is, for a valued in a Banach space. However, in finite dimension, with the help of Peano’s existence theorem, this result goes back to Osgood in 1904 [Fl]. The paper by H. Bahouri and J.-Y. Chemin [BC] studies for the transport equation the particular case of (1.3) in which ω(r ) = r ln(1/r ); at any rate, their paper is an interesting excursion away from W 1,1 territory† . On the other hand, the existence of a flow for the ODE (1.2), guaranteed by assumption (1.3), does not seem to imply trivially a uniqueness result for the associated PDE. Although the generalization of the results of [BC] seems very likely, with (1.3) replacing the L L regularity assumption, it would probably require some significant effort. This may be related to [CoL], where a wave equation with L L coefficients was studied and in which the energy method had to be substantially modified.

2. Statement of the results 2.1. A local result for bounded vector fields Let be an open set of Rn . Let X=

X 1≤ j≤n

† See

a j (x)

∂ , ∂x j

a j ∈ L∞ loc () ∩ BVloc (),

the appendix for an example of a Log-Lipschitz function which is not in W 1,1 .

(2.1)


361

be a real-valued‡ vector field. Note that this implies div X ∈ M () = D 0(0) (), the Radon measures on . For u ∈ C 0 (), we define Xu =

X 1≤ j≤n

∂ (a j u) − u div X. ∂x j

(2.2)

The product u div X makes sense since u is continuous and div X is a Radon measure. P We note that for u ∈ C 1 () we have the usual X u = 1≤ j≤n a j ∂ j u. Let S be a C 1 oriented hypersurface of . For all x0 ∈ S, there exists a neighborhood V0 of x0 such that S ∩ V0 = {x ∈ V0 , ϕ(x) = 0} with ϕ ∈ C 1 (V0 ), such that dϕ 6= 0 on V0 ; moreover, S+ ∩ V0 is defined as {x ∈ V0 , ϕ(x) ≥ 0}. (We say then that ϕ is a defining function for S.) We assume that S is noncharacteristic for X ; that is, there exists a neighborhood K 0 of x0 such that, with an essential infimum, inf X (ϕ) > 0.

(2.3)

K0

Note that this property does not depend on the choice of the defining function for the oriented hypersurface S. In fact, if ϕ˜ is another C 1 -defining function for S, we have ϕ˜ = eϕ with e ∈ C 0 and positive, so that d ϕ˜ − edϕ is continuous and vanishes at x0 , since =0

z }| { ϕ(x ˜ 0 + h) − ϕ(x ˜ 0 ) = ϕ(x0 + h)e(x0 + h) − ϕ(x0 ) e(x0 ) = ϕ 0 (x0 )h + o(h) e(x0 + h) = e(x0 )ϕ 0 (x0 )h + o(h). Consequently, if K is a neighborhood of x0 , K ⊂ K 0 , where K 0 is given in (2.3), inf X (ϕ) ˜ ≥ (inf e) inf X (ϕ) − kX k L ∞ (K 0 ) kd ϕ˜ − edϕk L ∞ (K ) > 0 K

K0

K0

since one may shrink K around x0 ∈ S ad libitum. It is important also to notice that assumption (2.3) is stable under C 1 -perturbation of ϕ. Let us assume that (2.3) is satisfied near x0 ∈ S, and let φ be a C 1 -function defined on V0 . We get from (2.3)

inf X (φ) ≥ inf X (ϕ) − kX k L ∞ (K 0 ) d(ϕ − φ) L ∞ (K ) > 0 K0

K0

0

for kd(ϕ − φ)k L ∞ (K 0 ) small enough. ‡ A distribution a is said to be real valued whenever a ¯ = a, where the complex conjugate a¯ is defined

by duality: a, ¯ ψ = a, ψ¯ . Complex vector fields are very complicated objects whose study is beyond the scope of this paper. Even with polynomial coefficients, a nonzero complex vector field could fail to be locally solvable, as shown by the Hans Lewy counterexample. On the other hand, the uniqueness for the noncharacteristic Cauchy problem could fail for first-order operators with C ∞ complex-valued coefficients.

362


THEOREM 2.1 Let be an open set in Rn , and let X be a bounded real-valued vector field satisfying (2.1). Let S be a C 1 -oriented hypersurface, noncharacteristic for X as in (2.3). Let u be a continuous function such that, for some Radon measure c,

supp u ⊂ S+ .

X u = cu,

Then if the positive part (2c + div X )+ ∈ L ∞ loc , the function u vanishes in a neighborhood of S. Remark. It would be natural to consider real vector fields of type (2.1) but satisfying the extra condition div X ∈ L 1 (and (2c + div X )+ ∈ L ∞ ). For u ∈ L ∞ , we can P define X u = 1≤ j≤n ∂(a j u)/∂ x j − u div X. For an L 1 -function c, the uniqueness through a noncharacteristic hypersurface of L ∞ -solutions of X u = cu is certainly a natural question that we are unfortunately unable to answer. 2.2. A global result for transport equations Let us first introduce some notation. We use the familiar D 0(m) (Rd ) to denote the distributions of order m on Rd , the dual space of Ccm (Rd ). Below we denote by M = D 0(0) (Rd ) the Radon measure on Rd . Its subspace Mb denotes the Banach space of 0 (Rd ), the continuous measures with finite total mass; this is the dual space of C(0) functions tending to zero at infinity. For a ∈ M , with K a compact subset of Rd , we define ha, ϕi N K (a) = sup (2.4) M ,Cc0 / sup |ϕ|, 06 ≡ϕ∈Cc0 (K )

where Cc0 (K ) stands for the functions of Cc0 (Rd ) supported in K . We note that, for a ∈ Mb , and all compact K , we have N K (a) ≤ kakMb . The Banach space BV(Rd ) is defined as BV(Rd ) = u ∈ L 1 (Rd ), ∇u ∈ Mb . A modern treatment of most of the main properties of these functions can be found in the book by W. Ziemer [Z, Chapter 5] (see also the classic book by Federer [Fe]). We define also BVloc (Rd ) = u ∈ L 1loc (Rd ), ∇u ∈ M . The set of continuous bounded functions on Rd is denoted by Cb0 (Rd ). Let us consider a vector field X X = ∂t + a j (t, x)∂ j (2.5) 1≤ j≤d


363

defined on (0, T0 ) × Rd where T0 > 0. We assume that the coefficients a j are real valued and such that a j ∈ L 1 (0, T0 ); BVloc (Rd ) ; (2.6) that is, for all ψ ∈ Cc∞ (Rd ), ψa j ∈ L 1 ((0, T0 ); BV(Rd )). This implies of course that P the divergence div X = 1≤ j≤d ∂ j (a j ) is real valued and such that div X ∈ L 1 (0, T0 ); M ; that is, ∀ψ ∈ Cc∞ (Rd ),

ψ div X ∈ L 1 (0, T0 ); Mb .

Let us consider a real-valued measure c such that c ∈ L 1 (0, T0 ); M . We examine weak solutions of ( P ∂t u + 1≤ j≤d a j ∂ j u = cu + f

(2.7)

in (0, T0 ) × Rd ,

(2.8)

in Rd .

u |t=0 = u 0

This means that for all ϕ ∈ Cc∞ ([0, T0 ) × Rd ), using∗ the notation ϕ0 (x) = ϕ(0, x), T0

Z

hZ Rd

0

i

u −X (ϕ) − ϕ div X − cϕ d x dt =

T0

Z 0

hZ

i

Rd

f ϕ d x dt +

Z Rd

u 0 ϕ0 d x.

(2.9) In particular, (2.9) makes sense when c(t, ·) and (div X )(t, ·) are Radon measures and u(t, ·) is a continuous function. The integrals on Rd in the formula (2.9) should then be written as brackets of duality; more precisely, for u ∈ L ∞ ((0, T ); C 0 (Rd )) the left-hand side of (2.9) is defined as T0

Z 0

Z Rd

X u(t, x) −∂t ϕ(t, x) − a j (t, x)∂ j ϕ(t, x) d x dt 1≤ j≤d

Z − 0

T0

(div X + c)(t, ·), ϕ(t, ·)u(t, ·) M ,C 0 dt. c

Let us check that the last term makes sense in the above expression. Since ϕ ∈ Cc∞ ([0, T0 ) × Rd ), there exists K 0 , a compact subset of Rd such that, for all t, is of course important to notice that ϕ does not necessarily vanish at t = 0. In fact, (2.9) is a compliP cated but standard way of writing ∂t u˜ + j ∂ j (a˜ j u) ˜ − udiv ˜ X = c˜u˜ + f˜ + δ ⊗ u 0 , where v˜ stands for the extension of v by 0 to {t < 0}. ∗ It

364


supp ϕ(t, ·) ⊂ K 0 . Let χ0 ∈ Cc∞ (Rd ), identically 1 on K 0 . Then we have

(div X +c)(t, ·), ϕ(t, ·)u(t, ·) 0 M ,Cc0 = χ0 (div X +c)(t, ·), ϕ(t, ·)u(t, ·) Mb ,C(0)

≤ χ0 (div X + c)(t, ·) M sup (ϕu)(t, ·) L ∞ (Rd ) ∈ L 1 (0, T0 ). b

t∈(0,T0 )

THEOREM 2.2 Let X be the vector field (2.5), and let assumptions (2.6) and (2.7) be satisfied. Moreover, we assume that (2c + div X )+ ∈ L 1 (0, T0 ); L ∞ (Rd ) . (2.10)

We also assume∗ that Z

T0

lim

λ→+∞ 0

λ

−1

Z λ≤|x|≤2λ

a j (t, x) d x dt = 0.

(2.11)

Let u be a function in L ∞ ((0, T0 ); Cb0 (Rd )) such that, in the sense of (2.9), ( P ∂t u + 1≤ j≤d a j ∂ j u = cu in (0, T0 ) × Rd , in Rd .

u |t=0 = 0

(2.12)

Then u vanishes in (0, T0 ) × Rd .

3. Lemmas on nonnegative solutions 3.1. Local results for bounded vector fields Let be an open set of Rn , and let p ∈ [1, +∞]. We denote by p 0 the conjugate exponent of p such that 1 = 1/ p + 1/ p 0 . Let X ∂ p , a j ∈ L∞ (3.1) X= a j (x) loc (), div X ∈ L loc , ∂x j 1≤ j≤n

p

p0

be an L ∞ loc real-valued vector field with L loc -divergence. For u ∈ L loc (), we define Xu =

X 1≤ j≤n

∂ (a j u) − u div X. ∂x j

(3.2)

note that this assumption is satisfied when a j (t, x)/(1 + |x|) ∈ L 1 ((0, T0 ) × Rd ). We give in the appendix a weaker condition than (2.11) which is satisfied in particular whenever a j (t, x) ∈ L 1 (0, T0 ) × Rd . (1 + |x|) ln(1 + |x|) ∗ We


365

Note that assumptions (3.1) are invariant through a C 2 -diffeomorphism.† Let S be a C 1 -oriented hypersurface of (see Section 2 for a precise definition). LEMMA 3.1 Let be an open set in Rn , let p ∈ [1, +∞], and let X be a bounded real-valued p vector field with divergence in L loc . Let S be a C 1 -oriented hypersurface, noncharp0

acteristic for X as in (2.3). Let u be a L loc -function such that, for some function p c ∈ L loc , X u ≤ cu, supp u ⊂ S+ , and u ≥ 0. (3.3) Then if (c + div X )+ ∈ L ∞ loc ,

(3.4)

the function u vanishes in a neighborhood of S. Proof Let us consider a point x0 ∈ S and ϕ a defining function for S in a neighborhood of x0 . 0 We know that, on an open neighborhood V0 of x0 , with u 0 = u |V0 ≥ 0, u 0 ∈ L p (V0 ), we have X u 0 ≤ cu 0 , supp u 0 ⊂ {ϕ ≥ 0}, X ϕ ≥ ρ0 > 0. (3.5) Let us consider the following C 1 -function defined on V0 : 1 2 2 ψ(x) = ϕ(x) + |x − x0 |2 , θ ψ(x) = α − ψ(x) + , 2 ¯ 0 , α) with center x0 and where α is a positive parameter such that the closed ball B(x radius α is included in V0 . We have supp θ(ψ) ⊂ {ψ ≤ α 2 } and supp u 0 θ(ψ) ⊂ {ϕ ≥ 0} ∩ {ψ ≤ α 2 } = K α 3 x0 , ¯ 0 , α)). Let χ ∈ Cc∞ (V0 ; which is a compact subset of V0 (as a closed subset of B(x [0, 1]), χ = 1 on a neighborhood of K α . Since ψ and θ (ψ) are C 1 -functions, and X (χ ) = 0 on a neighborhood of supp u 0 θ(ψ), we have =0

X

}| { z a j u 0 ∂ j θ(ψ)χ = u 0 χθ (ψ)X (ψ) + u 0 θ(ψ)X (χ) .

0

1≤ j≤n p

q

assumption div X ∈ L loc , div X ∈ L loc is invariant by a C 2 -change of coordinates. Also, this assumption is indeed local since with 8 ∈ Cc∞ () the vector field 8X still satisfies the same assumption. This is not the case for the assumption X ∈ W 1,1 and div X ∈ L ∞ , which is not local. † The

366


We calculate, with dm standing for the Lebesgue measure, Z

cu 0 θ (ψ)χ dm ≥ X u 0 , θ(ψ)χ D 0(1) (V ),C 1 (V ) 0 c 0 | {z } ≥0

=−

X Z

a j u 0 ∂ j θ(ψ)χ dm −

Z

u 0 θ (ψ)χ div X dm

1≤ j≤n

=

X Z

χ u 0 a j ∂ j (ψ)(α 2 − ψ)+ dm −

Z

u 0 θ(ψ)χ div X dm.

1≤ j≤n

We obtain Z 0≥

h i 1 χ u 0 (α 2 − ψ)+ X (ψ) − (α 2 − ψ)+ (div X + c) dm. 2

(3.6)

Now on the set x ∈ V0 , ϕ(x) + |x − x0 |2 = ψ(x) ≤ α 2 ∩ x, ϕ(x) ≥ 0 we have, using now (3.4)–(3.5), 1 X (ψ) − (α 2 − ψ)+ (div X + c) ≥ ρ0 − 2 kX k L ∞ ( B(x ¯ 0 ,α)) α 2

1 − α 2 (div X + c)+ L ∞ ( B(x ¯ 0 ,α)) ≥ ρ0 /2 2 if α is chosen small enough with respect to ρ0 and kX k L ∞ (V0 ) . On the other hand, the term Z 2 χ u 0 (α 2 − ψ)+ (div X + c)− dm makes sense and is nonnegative. This yields Z 0 ≥ χu 0 (α 2 − ψ)+ dm, and since the integrand is nonnegative, we get χ u 0 (α 2 − ψ)+ = 0. Since on a neighborhood of x0 we have χ = 1 and α 2 − ψ > 0, we indeed obtain that u 0 vanishes near x0 . The proof of Lemma 3.1 is complete. Remark. Lemma 3.1 provides a local uniqueness result for noncharacteristic bounded vector fields with bounded positive part of the divergence; the assumptions are invariant∗ by C 2 -change of coordinates. The local problem makes sense since the speed of propagation is finite. p

p

∞ 2 assumptions X ∈ L ∞ loc , div X ∈ L loc , c ∈ L loc , (div X +c)+ ∈ L loc are indeed invariant by C -change of coordinates and local in the sense of the footnote on page 364. ∗ The


367

The proof of the following lemma is identical to the proof of Lemma 3.1, except for replacing integrals by brackets of duality. Note also that (3.2) makes sense for u continuous, a j , div X ∈ D 0(0) (). LEMMA 3.2 Let be an open set in Rn , and let X be a bounded real-valued vector field with divergence in D 0(0) (). Let S be a C 1 -oriented hypersurface, noncharacteristic for X as in (2.3). Let u be a continuous function such that, for some measure c ∈ D 0(0) (),

supp u ⊂ S+ ,

X u ≤ cu,

and

u ≥ 0.

Then if (c + div X )+ ∈ L ∞ loc , the function u vanishes in a neighborhood of S. The proof of this lemma is obtained by copying the proof of Lemma 3.1 and properly replacing integrals by brackets of duality; inequality (3.6) should be replaced by Z 2 1

0 ≥ χu 0 X (ψ)(α 2 − ψ)+ dm − div X + c (α 2 − ψ)+ χ u 0 D 0(0) (),C 0 () c 2 Z Z 2 1 ≥ χ u 0 X (ψ)(α 2 − ψ)+ dm − (div X + c)+ (α 2 − ψ)+ χu 0 dm. {z } 2 | ∈L ∞

The end of the proof is identical. 3.2. Global results for transport equations with integrable coefficients Let us consider a vector field X X = ∂t + a j (t, x)∂ j

(3.7)

1≤ j≤d

defined on (0, T0 ) × Rd where T0 > 0. We assume that the coefficients a j are real valued and such that (3.8) a j ∈ L 1 (0, T0 ); L 1loc (Rd ) . P We assume moreover that the divergence div X = 1≤ j≤d ∂ j (a j ) is such that div X ∈ L 1 (0, T0 ); L 1loc (Rd ) .

(3.9)

Let us consider a real-valued function c such that c ∈ L 1 (0, T0 ); L 1loc (Rd ) .

(3.10)

368


We examine the weak solutions of ( P ∂t u + 1≤ j≤d a j ∂ j u = cu + f

in (0, T0 ) × Rd ,

(3.11)

in Rd .

u |t=0 = u 0

This means that for all ϕ ∈ Cc∞ ([0, T0 ) × Rd ), using the notation ϕ0 (x) = ϕ(0, x), T0

Z

hZ Rd

0

i

u −X (ϕ) − ϕ div X − cϕ d x dt =

T0

Z

hZ

0

i

Rd

f ϕ d x dt +

Z Rd

u 0 ϕ0 d x.

(3.12) In particular, (3.11) makes sense when the a j , c, and div X satisfy (3.8) – (3.10) and u d belongs to L ∞ ((0, T0 ), L ∞ loc (R )). We assume for j = 1, . . . , d the global condition (2.11). This condition is satisfied in particular whenever the functions a j (t, x)/(1 + |x|) belong to L 1 ((0, T0 ) × Rd ). 3.3 Let X be a transport equation as in (3.7). We assume that the coefficients a j and c satisfy (3.8)– (3.10). Moreover, we assume (2.11) and (c + div X )+ ∈ L 1 (0, T0 ); L ∞ (Rd ) . (3.13) LEMMA

Let u be a nonnegative L ∞ ((0, T0 ) × Rd )-function such that ( P ∂t u + 1≤ j≤d a j ∂ j u ≤ cu in (0, T0 ) × Rd , in Rd .

u |t=0 = 0

(3.14)

Then u vanishes in (0, T0 ) × Rd . Proof Relationship (3.14) implies that, for any nonnegative ϕ ∈ Cc1 ([0, T0 ) × Rd ), T0

Z

Z Rd

0

u∂t ϕ d x dt +

T0

Z

XZ

0

j

Rd

ua j ∂ j ϕ d x dt T0

Z

Z

+ 0

Rd

uϕ (div X + c) d x dt ≥ 0. (3.15)

Since u and ϕ are nonnegative, using (3.13) we get, with 0 ≤ ν0 ∈ L 1 (0, T0 ), T0

Z 0

Z Rd

u∂t ϕ d x dt +

Z 0

T0

XZ j

Rd

ua j ∂ j ϕ d x dt +

T0

Z 0

Z Rd

uϕν0 (t) d x dt ≥ 0. (3.16)


369

We can choose ϕ(t, x) = τ (t)σ (|x|2 λ−2 ), where 0 ≤ τ ∈ Cc1 ([0, T0 )), λ > 0, and ( 1 on [0,1], 1 C R+ ; [0, 1] 3 σ = with −1/2 ≤ σ 0 ≤ 0. 0 on [4, ∞), Plugging this into (3.16), we obtain ZZ u(t, x)σ |x|2 λ−2 − τ˙ (t) − ν0 (t)τ (t) dt d x (0,T0 )×Rd X ZZ ≤ kuk L ∞ τ (t) a j (t, x)x j λ−2 dt d x. (3.17) (0,T0 )×{λ≤|x|≤2λ}

j

We want now to choose the function τ by solving the ODE τ˙ (t) + ν0 (t)τ (t) = −(T1 − t)+ , We have τ (t) =

T1

Z

where T1 is given in (0, T0 ).

Z t exp − ν0 (r ) dr (T1 − s)+ ds

t

(3.18)

(3.19)

s

and in particular τ (t) = 0 for t ≥ T1 . The function (3.19) is nonnegative and continuous with a derivative in L 1 , so that, approximating τ by a smooth function, we obtain∗ that (3.17) is satisfied for τ given in (3.19). This gives ZZ u(t, x)σ |x|2 λ−2 (T1 − t)+ dt d x ≤ kuk L ∞ (0,T1 )×Rd

·

X ZZ j

(0,T1 )×{λ≤|x|≤2λ}

1 (T1 − t)2 a j (t, x)x j λ−2 dt d x exp 2

T1

Z

ν0 (r ) dr . (3.20)

0

Using the Beppo Levi monotone convergence theorem (note that σ 0 takes nonpositive values), the left-hand side of (3.20) tends, for λ → +∞, to ZZ u(t, x)(T1 − t)+ dt d x, (3.21) (0,T1 )×Rd

whereas the lim sup of the right-hand side is bounded above by ZZ Z T1 X 1 2 kuk L ∞ T1 lim sup a j (t, x) dt d x exp ν0 (r ) dr , λ→+∞ λ (0,T1 )×{λ≤|x|≤2λ} 0 j

which is zero by hypothesis (2.11). As a consequence, since u is nonnegative, we get that u vanishes on (0, T1 ) × Rd for all T1 < T0 . The proof of Lemma 3.3 is complete. fact, if τ is a standard regularization of τ , we keep τ ≥ 0 and τ (t) = 0 for t ≥ T1 + in such a way that τ˙ − τ˙ and (τ − τ )ν0 both converge to 0 in L 1 (0, T0 ).

∗ In

370


The following lemma is similar to Lemma 3.3. 3.4 Let X be a transport equation as in (3.7). We assume that a j , divX, c belong to L 1 ((0, T0 ); M ). Moreover, we assume (3.13) and Z 1 T0 lim sup N{λ≤|x|≤2λ} a j (t, ·) dt = 0 (3.22) λ→+∞ λ 0 LEMMA

(see (2.4) for the definition of N ). Let u be a nonnegative L ∞ ((0, T0 ); Cb0 (Rd ))function such that (3.14) is satisfied. Then u vanishes in (0, T0 ) × Rd . Proof R It is easy to check that (3.15) is satisfied, with the integral Rd ua j ∂ j ϕ d x dt replaced by the bracket of duality ha j (t, ·), u(t, ·)∂ j ϕ(t, ·)iM ,Cc0 and the integral R Rd uϕ (div X + c) replaced by h(div X + c)(t, ·), u(t, ·)ϕ(t, ·)iM ,Cc0 . Assumption (3.13) implies (3.16) as well. The sequel of the proof is identical, with the right-hand side of (3.17) replaced by kuk L ∞ λ−1

T0

XZ j

τ (t)N{λ≤|x|≤2λ} a j (t, ·) dt.

0

Remark. It seems possible to relax assumption (2.11) and, in particular, to assume only that the lim sup is finite (see also the footnote on page 363 and the appendix for a different kind of weakening of (2.11)). However, it is not clear that this generalization is worthwhile, since the proof gets much more complicated. On the other hand, the fact that the vector field here is a priori unbounded forces somehow a global assumption, which should replace a standard convexification procedure workable in the bounded case. On the other hand, in Lemmas 3.3 and 3.4, we could have required global integrability conditions for the coefficients of the vector field; this would have led to slightly different results.

4. Commutation lemmas 4.1. The DiPerna-Lions commutation argument Since we want to use this now classical argument in a slightly different context involving BV functions, we start with a quick review of some identities attached to commutation of vector fields with a standard mollifier. Let us consider a vector field


371

X defined in Rn , X

X=

1≤ j≤n

a j (x)

∂ , ∂x j

(4.1) 1, p

so that, for some p ∈ [1, +∞], the coefficients a j belong to Wloc (Rn ). For u ∈ p0

L loc (Rn ), we define Xu =

X

∂ j (a j u) − u div X.

(4.2)

j

R Let ρ be a nonnegative function supported in the unit ball of Rn such that ρ = 1. We set, for > 0, ρ (·) = −n ρ(·/). We consider the linear operator defined on p0 L loc (Rn ) by R u = X (u ∗ ρ ) − (X u) ∗ ρ . (4.3) Cc∞

We define also the translation operator (τz u)(x) = u(x − z).

(4.4)

The following identity holds: Z n o 1 X (R u)(x) = (τz div X )(x)ρ(z) + (Id − τz )a j (x)∂ j ρ(z) j · (τz − Id)u (x) dz. (4.5) From this identity,∗ we get immediately, for K 0 compact in Rn , K 1 = K 0 + unit ball, 0 < ≤ 1, 1/q + 1/q 0 = 1, Z

kR uk L 1 (K 0 ) ≤ kdiv X k L q (K 1 ) (τz − Id)u L q 0 (K ) ρ(z) dz 0 Z X

∇a j q

(τz − Id)u q 0 ∂ ρ(z)kz dz. (4.6) + L (K ) L (K ) j 1

0

j ∗ The

sum in (4.5) can be written as X ZZ 1 (∂k a j )(x − θ z)z k (∂ j ρ)(z) u(x − z) − u(x) dz dθ, 0

1≤ j,k≤n

so that if ρ(z) = ρ0

(|z|2 ),

this term is in fact X ZZ 1 (∂k a j )(x − θ z)2z k z j ρ00 (|z|2 ) u(x − z) − u(x) dz dθ.

1≤ j,k≤n

0

As pointed out in [De3], we thus have only to deal with X X (∂k a j )(x − θ z)2z k z j = z k z j (∂k a j )(x − θ z) + (∂ j ak )(x − θ z) . 1≤ j,k≤n

1≤ j,k≤n

This means that choosing a radial mollifier ρ allows us to control only the symmetric part of the gradient, ∂ j ak + ∂k a j .

372


LEMMA 4.1 1, p Let p be in [1, ∞], and let X be a vector field in Wloc (Rn ). Then with R defined in p0

(4.3), we have, for u ∈ L loc (1/ p + 1/ p 0 = 1), lim R u = 0

→0

in L 1loc .

Proof If p 0 < ∞, estimate (4.6) with q = p suffices to get the result. Let us assume now ∞ u ∈ L∞ loc , and let Y be a C c vector field. Denoting R in (4.3) by R,X , we get from 0 (4.6), for R,Y with any q < ∞,

lim R,Y u L 1 (K ) = 0. →0

0

This yields from (4.6), for X − Y with q = 1, with C0 depending only on ρ,

lim sup R,X u L 1 (K ) ≤ lim sup R,X −Y u L 1 (K ) 0

→0

→0

0

≤ C0 kX − Y kW 1,1 (K 1 ) kuk L ∞ (K 1 ) , and the result by density of Cc∞ in W 1,1 . 4.2. Commutation for a BV vector field As was noticed in a paper by Bouchut (see [Bo, Remark 3.2]), the convergence to zero in L 1 of R u does not follow from the previous commutation lemma. We prove below a weak convergence result that is enough for our uniqueness theorem. LEMMA 4.2 Let X be a vector field in BVloc (Rn ), that is, with coefficients in L 1loc with first derivatives in D 0(0) . Then with R defined in (4.3), we have, for u ∈ C 0 (Rn ),

lim R u = 0 weakly in D 0(0) ,

→0

that is, ∀ψ ∈ Cc0 (Rn ),

lim hR u, ψiD 0(0) ,Cc0 = 0.

→0


373

Proof Going back to (4.5) and using notation (4.4), we get∗ Z h

R u, ψ = τz div X, ψ(τz − Id)u ρ(z) +

i 1 X

(Id − τz )a j , ψ(τz − Id)u ∂ j ρ(z) dz j

Z =

τz div X, ψ(τz u − u) ρ(z) dz X Z Z 1

+ τθ z ∂k a j , ψ(τz u − u) z k ∂ j ρ(z) dz dθ 1≤ j,k≤n

Z =

0

τz div X, ψ(τz u − u) ρ(z) dz X Z Z 1

+ ∂k a j , τ−θ z ψ (τz−θ z − τ−θ z )u z k ∂ j ρ(z) dz dθ. 1≤ j,k≤n

0

This implies, with K 0 = supp ψ, K 1 = K 0 + unit ball, 0 < ≤ 1, χ1 ∈ Cc0 (Rn ) identically 1 near K 1 , Z hR u, ψi ≤ kτ−z u − uk ∞ (4.7) L (K 1 ) ρ(z) dz kχ1 div X kMb kψk L ∞ X ZZ 1 kτz−θ z u − τ−θ z uk L ∞ (K 1 ) z k ∂ j ρ(z) dz dθ + 1≤ j,k≤n

0

× χ1 ∂k a j

Mb

kψk L ∞ ,

which gives the result of the lemma since u is continuous and the integration in the variables z, θ takes place on compact sets. Remark. In fact, if div X ∈ L 1loc , for χ ∈ Cc0 (Rn ), there exists χ˜ ∈ Cc0 (Rn ) such that, for all u ∈ C 0 (Rn ), kχ R uk L 1 ≤ kχ˜ ∇akMb kχuk ˜ L∞ . This implies that the sequence χ R u is bounded in L 1 and thus that one can extract 0 ) a convergent subsequence for the weak∗ topology on Mb , that is, the σ (Mb , C(0)

topology. On the other hand, the convergence of R u to zero in D 0 (1) (Rn ) is obvious. ∗ The

computation is more transparent using integrals instead of brackets of duality. Also, operator (4.4) acts on distributions, and for w ∈ D 0 (Rn ), z ∈ Rn , X Z 1 τz w = w − zj τθ z (∂ j w) dθ. j

0

374


5. Uniqueness results 5.1. Local results Proof of Theorem 2.1 Let be an open set of Rn , and let X=

X 1≤ j≤n

a j (x)

∂ ∂x j

be a vector field whose coefficients a j are locally bounded with derivatives in D 0(0) . P As a consequence, the divergence j ∂ j (a j ) belongs to D 0(0) . Let c be in D 0(0) . Let S be a C 1 -oriented hypersurface, noncharacteristic for X as in (2.3). We assume also that (2c + div X )+ ∈ L ∞ (5.1) loc . Let u be a continuous function such that supp u ⊂ S+ ,

X u = cu.

(5.2)

We want to prove that the function u actually vanishes in a neighborhood of S. We prove that X (u 2 ) = 2cu 2 and apply Lemma 3.2 to get the answer. Let us calculate for ϕ ∈ Cc1 () the bracket Z

X (u 2 ), ϕ D 0(1) (),C 1 () = − u 2 X (ϕ) dm − hdiv X, u 2 ϕiD 0(0) (),Cc0 () c Z = − χ 2 u 2 X (ϕ) dm − hϕ div X, χ 2 u 2 iD 0(0) ,Cc0 , where χ ∈ Cc1 () is identically 1 on the support of ϕ. We obtain, using the notation of Section 4,

X (u 2 ), ϕ

D 0(1) ,Cc1

= − lim

hZ

→0+

i

χ u(χu ∗ ρ )X (ϕ) dm + ϕ div X, χu(χu ∗ ρ ) D 0(0) ,C 0 . c

Since the function χu ∗ ρ is C 1 , we get

X (u 2 ), ϕ D 0(1) ,C 1 = lim χ u X (χ u ∗ ρ ) + (χ u ∗ ρ )X (χu), ϕ D 0(1) ,C 1 . c

→0+

c

We have, since χ ∈ Cc1 (), X (χ u) = χ X (u) + u X (χ) = χcu + u X (χ),

(5.3)


375

and since ϕ∇χ = 0 and χϕ = ϕ, we obtain

X (u 2 ), ϕ D 0(1) ,C 1 = lim χu X (χu ∗ ρ ) + (χ u ∗ ρ )cu, ϕ D 0(1) ,C 1 . c

→0+

c

(5.4)

Now we write, using the notation of Section 4 and (5.3), X (χ u ∗ ρ ) = R (χ u) + X (χu) ∗ ρ = R (χ u) + (χ cu) ∗ ρ + u X (χ ) ∗ ρ , yielding

X (u 2 ), ϕ D 0(1) ,C 1 = lim χ u R (χu) + χ u(χcu ∗ ρ ) + (χu ∗ ρ )cu, ϕ D 0(1) ,C 1 c c →0+

2 = lim χ u R (χu), ϕ D 0(1) ,C 1 + 2hc, u ϕiD 0(0) ,Cc0 c →0+

= lim R (χ u), χuϕ D 0(0) ,C 0 + 2hc, u 2 ϕiD 0(0) ,Cc0 . →0+

c

From Lemma 4.2, we get, since χuϕ is continuous,

lim R (χu), χuϕ D 0(0) ,C 0 = 0, →0+

c

which gives X (u 2 ) = 2cu 2 . Since u 2 is nonnegative and supported in S+ , we can apply Lemma 3.2, provided that (2c + div X )+ ∈ L ∞ loc . The proof of Theorem 2.1 is complete. Remark. If X is a W 1,1 ∩ L ∞ vector field, c ∈ L 1 , the L ∞ -solutions of X u = cu are such that X u 2 = 2cu 2 (a consequence of Lemma 4.1). On the other hand, for 0 1 < p ≤ ∞, if X is a W 1, p ∩ L ∞ vector field, c ∈ L p , the L p -solutions of X u = cu are such that X (α(u)) = cα 0 (u)u with α(u) = u 2 /(1 + u 2 ) (a consequence of Lemma 4.1). Lemma 3.1 yields uniqueness of these solutions, provided that (div X )+ and c+ are bounded.

5.2. Global results for transport equations Proof of Theorem 2.2 We consider now a transport equation X = ∂t +

X 1≤ j≤d

a j (t, x)∂ j

376


defined on (0, T0 ) × Rd where T0 > 0. We assume that the coefficients a j (t, x) are real valued and such that a j (t, x) ∈ L 1 (0, T0 ); BVloc (Rd ) . (5.5) P We assume also that the divergence div X = j ∂ j (a j ) is such that div X ∈ L 1 (0, T0 ); L 1loc (Rd ) . (5.6) Let us consider a real-valued function c such that c ∈ L 1 (0, T0 ); L 1loc (Rd ) .

(5.7)

We assume also that global condition (2.11) is satisfied and that d (2c + div X )+ ∈ L 1 (0, T0 ); L ∞ loc (R ) . Let us check a function u(t, x) ∈ L ∞ (0, T0 ); Cb0 (Rd )

(5.8)

such that X u = cu and u |t=0 = 0. This means in fact that ∂t v + Y (t)v = dv

on (−T0 , T0 ) × Rd ,

(5.9)

where v is the extension of the function u by zero on {t < 0}, Y (t) the extension of P the vector field j a j (t, x)∂ j by zero on {t < 0}, and d the extension of the function c by zero on {t < 0}. We do not assume here that u is nonnegative, and we want to prove that u ≡ 0. It is enough to prove that (∂t +Y (t))(v 2 ) = 2dv 2 on (−T0 , T0 )×Rd , since Lemma 3.3 could then be applied to the nonnegative bounded function v 2 . We write, for 8 ∈ Cc1 ((−T0 , T0 ) × Rd ),

E(v, 8) = ∂t (v 2 ) + Y (t)(v 2 ), 8 D 0(1) (R1+d ),C 1 (R1+d ) c ZZ ZZ ˙ d x dt + =− v28 v 2 Y ∗ (8) d x dt. Denoting by v (t)(x) = [v(t, ·)∗ρ ](x) the x-regularization of the function v, we get, using v ∈ L ∞ (Cb0 (Rd )), that v is bounded uniformly in and converges for almost all t to v(t), ZZ ZZ ˙ d x dt + lim v2 (t)Y ∗ (t) 8(t) d x dt, E(v, 8) = − v28 →0

and thus E(v, 8) = −

ZZ

˙ d x dt + lim v28

→0

ZZ

2Y (t) v (t) v (t)8(t) dt.

(5.10)


377

From the equality ∈L 1loc (Rd )

∈L 1loc (Rd )

z }| { X z }| { Y (t)v(t) ∗ ρ = [a j (t)v(t)] ∗∂ j ρ − [v(t) div Y (t)] ∗ρ

(5.11)

j

and assumptions (5.5)–(5.8), we get [Y (t)v(t)] ∗ ρ ∈ L 1loc (R1+d ) and thus, using equation (5.9), v(t) ˙ ∗ ρ = −Y (t)v(t) + d(t)v(t) ∗ ρ ∈ L 1loc (R1+d ). (5.12) On the other hand, we have

Y (t) v (t) v (t)8(t) D 0(0) (Rd ),C 0 (Rd ) = R,Y (t) v(t) v (t)8(t) D 0(0) ,C 0 c c

+ Y (t)v(t) ∗ ρ v (t)8(t) D 0(0) ,C 0 . c

(5.13) From Lemma 4.2 and estimate (4.7), we get (since v (t)8(t) → v(t)8(t) in Cc0 (Rd )) Z T0

R,Y (t) v(t) v (t)8(t) D 0(0) ,C 0 dt → 0 with . (5.14) c

0

We are left with Z T0

Y (t)v(t) ∗ ρ v (t)8(t) D 0(0) ,C 0 dt c

0

T0

Z = 0

−v(t) ˙ + d(t)v(t) ∗ ρ v (t)8(t) D 0(0) ,C 0 dt. (5.15) c

We get then from (5.10) and (5.13)–(5.15), Z ZZ ZZ ˙ d x dt − lim E(v, 8) = int2dv 2 8 d x dt − v28 2v˙ v 8 d x dt. (5.16) →0

1+d ) and ∂ v ∈ L 1 (R1+d ), we have ∂ (v 2 ) = On the other hand, since v ∈ L ∞ t t loc (R loc n ), ∂ w ∈ L 1 (Rn ), one checks∗ with a 2∂t (v )v ; as a matter of fact, if w ∈ L ∞ (R x 1 loc loc standard mollifier that ∂x1 (w2 ) = 2w∂x1 (w). Applying this to our situation, we get ZZ 2v˙ v 8 d x dt = h2v ∂t v , 8iD 0(1) (R1+d ),Cc1 (R1+d ) ZZ

= ∂t (v2 ), 8 D 0(1) ,C 1 = − v2 ∂t 8 d x dt. (5.17) c

0(0) 0 D , Cc brackets of duality, one has Z

∂x1 (w2 ), ϕ = − w 2 ∂x1 ϕ d x = lim ∂x1 (ww ), ϕ = lim hw∂x1 w , ϕi + hw ∂x1 w, ϕi . R R R The last term tends to ϕw∂x1 w, whereas the other is equal to (∂x1 w) wϕ and goes to wϕ∂x1 w, so that eventually ∂x1 (w2 ) = 2w∂x1 w. ∗ With

378


Eventually, taking the limit when goes to zero, we get ZZ E(v, 8) = 2dv 2 d x dt,

(5.18)

which is the desired result. Assuming (5.5) and (5.8) and replacing (5.6) – (5.7) by c, div X ∈ L 1 (0, T0 ); M , (5.19) let us check that the previous proof gives the result for Theorem 2.2, using Lemma 3.4 instead of Lemma 3.3. In fact, equality (5.10) is unchanged and (5.11) gives Y (t)v(t) ∗ ρ ∈ L 1loc (Rt , M ), so that equality (5.12) gives v˙ (t) ∈ L 1loc (Rt , M ). Equality (5.13) is unchanged, and (5.14) – (5.15) still hold, whereas (5.16) should be replaced by ZZ Z

˙ d x dt v28 (5.20) E(v, 8) = 2 d(t), v 2 (t)8(t) M ,C 0 dt − c Z

− lim 2 v˙ (t), v (t)8(t) M ,C 0 dt. →0

c

0 d To get the result, we need to prove that if w ∈ L ∞ loc (Rt , C (Rx )) and ∂t w ∈ then

L 1loc (Rt , M ),

∂t (w2 ) = 2w∂t w, that is, ZZ −

w ∂t 8 d x dt = 2 2

Z

w(t), ˙ w(t)8(t) M ,C 0 dt. c

(5.21)

In fact, we have

∂t (w2 ), ψ D 0(1) (R1+d ),C 1 (R1+d ) = − c

ZZ

w2 ∂t (ψ) d x dt ZZ = − lim w(w ∗ ρθ )∂t ψ d x dt, θ →0

(5.22)

since kwwθ ∂t ψ − w2 ∂t ψk L 1 (R1+d ) ≤ kw∂t ψk L ∞ kw − wθ k L 1 (supp ψ) → 0

with θ . (5.23)

Consequently, we have

∂t (w2 ), ψ D 0(1) (R1+d ),C 1 (R1+d ) c ZZ wψ∂t wθ d x dt . (5.24) = lim h∂t w, wθ ψiD 0(1) (R1+d ),Cc1 (R1+d ) + θ →0


379

But ∂t wθ → ∂t w in L 1loc (R, M ), so the second term in (5.24) converges to Z

∂t w(t), w(t)ψ(t) M ,C 0 dt. c

On the other hand, we have h∂t w, wθ ψiD 0(1) (R1+d ),Cc1 (R1+d ) =

ZZ

w(ψ∂t w)θ d x dt Z

→ ψ(t)∂t w(t), w(t) M ,C 0 dt, c

(5.25)

yielding (5.21). Consequently, from (5.20), Z ZZ ZZ

˙ d x dt + lim ˙ d x dt, E(v, 8) = 2 v 2 (t)d(t), 8(t) M ,C 0 dt − v28 v2 8 →0

c

implying the result (∂t + Y (t))v 2 = 2dv 2 . The proof of Theorem 2.2 is complete.

Appendix A.1. On the divergence of a vector field Let (M, ω) be a smooth oriented manifold, and let X be a smooth vector field on M. Using the flow of the vector field, it is possible to define the Lie derivative L X (ω), and the fundamental formula of the calculus of variations gives, since ω is a nondegenerate n-form, L X (ω) = d(ωcX ) + dωcX = d(ωcX ) = ω div X, (A.1) where c stands for the interior product. The last equality makes sense without any regularity assumption on X , for example, for a smooth ω and a vector field with distribution coefficients. Note that Green’s formula can be written as Z Z Z ω div X = d(ωcX ) = ωcX. M

∂M

M

In particular, working with coordinates in Rn and with a nonvanishing ν, ω = ν(x) d x1 ∧ · · · ∧ d xn ,

X=

X

a j (x)

1≤ j≤n

∂ , ∂x j

we obtain X div X = X ln |ν| + ∂ j (a j ). 1≤ j≤n

(A.2)

380


As a matter of fact,∗

L X (ω), Y1 ∧ · · · ∧ Yn = X hω, Y1 ∧ · · · ∧ Yn i X

ω, Y1 ∧ · · · ∧ [X, Y j ] ∧ · · · ∧ Yn . − 1≤ j≤n

If we choose Y j = e j the constant jth canonical vector, we get X ν div X = X (ν) + ν ∂ j (a j ), j

which is (A.2). A similar point of view is related to taking adjoints. We have X ∗ = −X − div X.

(A.3)

In fact, with ϕ, ψ ∈ Cc∞ (M), Z Z Z X (ϕ)ψω = L X (ϕψω) − ϕ L X (ψω) M M Z Z Z = d(ϕψωcX ) − ϕ X (ψ)ω − ϕψω div X, M M M | {z } =0

yielding (A.3). A.2. A Log-Lipschitz function is not in W 1,1 The so-called Log-Lipschitz functions satisfying (1.3) for ω1 (r ) = r ln(1/r ) fail to be W 1,1 , as shown by the following one-dimensional example. The function defined on R by ( x for x > 0, f (x) = x+ e−1/x sin(e1/x ), x+ = (A.4) 0 for x ≤ 0, 1,1 does not belong to Wloc but satisfies with some constant C (and for |x1 |, |x2 | smaller than 1) f (x1 ) − f (x2 ) ≤ C|x1 − x2 | ln |x1 − x2 | . (A.5) ∗ The

Lie derivative of a tensor K is defined as

d (8t )∗ (K )|t=0 . dt X It is a derivation for the tensor product, it commutes with the contraction, and for a function f and a vector field Y we have L X ( f ) = X f, L X (Y ) = [X, Y ]. Note that if ω is a one-form, X, Y vector fields, we have

hdω, X ∧ Y i = hdωcX, Y i = L X (ω)cY − d(ωcX ), Y = X hω, Y i − ω, [X, Y ] − Y hω, X i .

L X (K ) =


381

In fact, for x > 0, f 0 (x) = e−1/x sin(e1/x ) + x −1 e−1/x sin(e1/x ) − x −1 cos(e1/x ) = −x −1 cos(e1/x ) + L ∞ , and more precisely, since for x > 0, (x −1 + 1)e−1/x ≤ 1, | f 0 (x)| ≤ |x −1 cos(e1/x )| + 1. It is elementary to check that f 0 ∈ / L 1loc since Z +∞ Z 1 | cos u| +∞ = du = x −1 cos(e1/x ) d x. u ln u e 0 We consider now 0 ≤ x0 < x1 such that x1 − x0 < 1/e, and we define for θ ∈ (0, 1), xθ = x0 + θ(x1 − x0 ). We need to check that cos(e1/xθ ) | f (x1 ) − f (x0 )| +1 ≤ Q(x1 , x0 ) = |x1 − x0 | |ln(x1 − x0 )| xθ |ln(x1 − x0 )| for some θ ∈ (0, 1). So we get that Q is bounded if |xθ ln(x1 − x0 )| ≥ 1. However, denoting x1 − x0 = k −1 , x0 = x, we have k ≥ e and xθ ln(x1 − x0 ) = (x + θk −1 ) ln k ≥ x ln k. If x ln k ≥ 1, we obtain Q(x1 , x0 ) ≤ 2. If x < 1/ ln k, we get, using the fact that the function x 7 → xe−1/x is nondecreasing on R+ , | f (x + k −1 ) − f (x)| ln k 2k −1 (x + k −1 )e−1/(x+k ) ln k 2k −1 −1 (ln k)−1 + k −1 e−1/[(ln k) +k ] ln k 2k 1 2 + 2 exp − −1 ln k k + (ln k)−1 ln k 2 2 ln2 k + 2 exp →0 if k → +∞. ln k k + ln k ln k

Q=k ≤ ≤ ≤ =

Remarks. The function f defined in (A.4) also provides an example of a function that has “exactly” the Log-Lipschitz regularity; that is, it is such that | f (x1 ) − f (x2 )| . |x1 −x2 |→0 ω1 (|x 1 − x 2 |)

0 < lim inf

(A.6)

|x1 |,|x2 |≤1

We say that a function u is Log{k} -Lipschitz if, for each compact set K , there exists a positive constant C such that for x1 , x2 ∈ K , |x1 − x2 | ≤ 1/C, we have 1 u(x1 ) − u(x2 ) ≤ C|x1 − x2 | (ln ◦ · · · ◦ ln) . (A.7) | {z } |x1 − x2 | k terms

382


One can prove that the set of Log{k} -Lipschitz functions, although decreasing when k 1,1 1,1 increases, is not included in Wloc . (On the other hand, it is trivial to check that Wloc 6⊂ Log-Lipschitz.) The proof is similar to the previous discussion on the function f given by (A.4). Let us simply provide a formula; for an integer k ≥ 1, the following 1,1 function f k is Log{k} -Lipschitz but is not in Wloc (nor in Log{k+1} -Lipschitz). We define f k (x) = 0 for x ≤ 0, and for x positive we set x sin[exp{k} (1/x)] f k (x) = Q { j} 1≤ j≤k exp (1/x)

with exp{k} = exp ◦ · · · ◦ exp . | {z } k terms

A.3. A weakened version of condition (2.11) Let β be a positive L 1loc -function defined on [1, +∞) such that Z

+∞

β(r ) dr = +∞.

(A.8)

1

Condition (2.11) in Theorem 2.2 can be replaced by the more general integral condition Z Z T0

|a j (t, x)|β(|x| + 1) d x dt < ∞.

(A.9)

0

Note that (A.8) and (A.9) are satisfied, for instance, when a j (t, x) ∈ L 1 (0, T0 ) × Rd . (1 + |x|) ln(1 + |x|) Let us check the proof of Lemma 3.3, with assumptions (A.8) and (A.9) replacing (2.11). After (3.16), we choose ϕ(t, x) = τ (t)σ (λ−1 B(|x| + 1)), where the function B is defined for r ≥ 1 by Z r

B(r ) = exp

β(ρ) dρ.

1

In the right-hand side of (3.17), |a j (t, x)x j |λ−2 should be replaced by 1 a j (t, x) λ−1 β |x| + 1 B |x| + 1 2 and the domain of integration by (0, T0 ) × {λ ≤ B(1 + |x|) ≤ 4λ}. We are then left with Z T0 Z 2 a j (t, x) β |x| + 1 d x dt, 0

{λ≤B(1+|x|)≤4λ}

which goes to zero with λ−1 from (A.9), and the fact that the inverse function B −1 goes to infinity with λ, thanks to (A.8).


383

References [BC]

[Bo] [BD]

[BJ] [ChL]

[CoL] [De1]

[De2]

[De3] [DL] [Fe] [Fl]

[H1]

[H2] [Li] [PP]

[PR]

´ H. BAHOURI and J.-Y.CHEMIN, Equations de transport relatives a` des champs de vecteurs non-lipschitziens et mécanique des fluides, Arch. Rational Mech. Anal. 127 (1994), 159–181. MR 95g:35164 359, 360 F. BOUCHUT, On transport equations and the chain rule, preprint, 1999. 372 F. BOUCHUT and L. DESVILLETTES, On two-dimensional Hamiltonian transport equations with continuous coefficients, Differential Integral Equations 14 (2001), 1015–1024. MR 1 827 101 357 F. BOUCHUT and F. JAMES, One-dimensional transport equations with discontinuous coefficients, Nonlinear Anal. 32 (1998), 891–933. MR 2000a:35243 357 J.-Y. CHEMIN and N. LERNER, Flot de champ de vecteurs non lipschitziens et e´ quations de Navier-Stokes, J. Differential Equations 121 (1995), 314–328. MR 96h:35153 359 F. COLOMBINI and N. LERNER, Hyperbolic equations with non-Lipschitz coefficients, Duke Math. J. 77 (1995), 657–698. MR 95j:35127 360 B. DESJARDINS, Global existence results for the incompressible density-dependent Navier-Stokes equations in the whole space, Differential Integral Equations 10 (1995), 587–598. MR 2001c:76027 357 , Linear transport equations with initial values in Sobolev spaces and application to the Navier-Stokes equations, Differential Integral Equations 10 (1995), 577–586. MR 2000k:35234 357 , A few remarks on ordinary differential equations, Comm. Partial Differential Equations 21 (1996), 1667–1703. MR 97j:34005 357, 371 R. J. DIPERNA and P. L. LIONS, Ordinary differential equations, transport theory and Sobolev spaces, Invent. Math. 98 (1989), 511–547. MR 90j:34004 357 H. FEDERER, Geometric Measure Theory, Grundlehren Math. Wiss. 153, Springer, New York, 1969. MR 41:1976 362 T. M. FLETT, Differential Analysis: Differentiation, Differential Equations, and Differential Inequalities, Cambridge Univ. Press, Cambridge, 1980. MR 82e:26021 359 ¨ L. HORMANDER , The Analysis of Linear Partial Differential Operators, I, II; III, IV, Grundlehren Math. Wiss. 256, 257; 274, 275, Springer, Berlin, 1983; 1985. MR 85g:35002a, MR 85g:35002b; MR 87d:35002a, MR 87d:35002b 357 , Lectures on Nonlinear Hyperbolic Differential Equations, Math. Appl. 26, Springer, Berlin, 1997. MR 98e:35103 P. L. LIONS, Sur les e´ quations différentielles ordinaires et les e´ quations de transport, C. R. Acad. Sci. Paris Sér. I Math. 326 (1998), 833–838. MR 99i:34004 357 G. PETROVA and B. POPOV, Linear transport equations with discontinuous coefficients, Comm. Partial Differential Equations 24 (1999), 1849–1873. MR 2000e:35228 357 F. POUPAUD and M. RASCLE, Measure solutions to the linear multi-dimensional transport equation with non-smooth coefficients, Comm. Partial Differential Equations 22 (1997), 337–358. MR 98e:35111 357

384


[T]

` F. TREVES , Topological Vector Spaces, Distributions and Kernels, Pure Appl. Math.

[V]

A. I. VOL’PERT, Spaces BV and quasilinear equations (in Russian), Mat. Sb (N.S.) 73

[Z]

(115) (1967), 255–302; English translation in Math. USSR-Sb. 2 (1967), 225–267. MR 35:7172 W. P. ZIEMER, Weakly Differentiable Functions: Sobolev Spaces and Functions of Bounded Variation, Grad. Texts in Math. 120, Springer, New York, 1989. MR 91e:46046 362

25, Academic Press, New York, 1967. MR 37:726

Colombini Dipartimento di Matematica, Università di Pisa, Via F. Buonarroti 2, 56127 Pisa, Italy; [email protected] Lerner Université de Rennes 1, Institut de Recherche Mathématique de Rennes, Campus de Beaulieu, 35042 Rennes CEDEX, France; [email protected]


MORPHISMS OF F-ISOCRYSTALS AND THE FINITE MONODROMY THEOREM FOR UNIT-ROOT F-ISOCRYSTALS NOBUO TSUZUKI

Abstract We discuss Tate-type problems for F-isocrystals, that is, the full faithfulness of the natural restriction functors between categories of overconvergent F-isocrystals on schemes of positive characteristic. We prove it in the cases of unit-root F-isocrystals. Using this result, we prove that an overconvergent unit-root F-isocrystal has a finite monodromy. 1. Introduction In this paper we give several results on the Tate-type problem for F-isocrystals on a scheme of positive characteristic. The Tate-type problem asks the analyticity of formal horizontal sections of overconvergent F-isocrystals. Such a problem appears neither in the classical cases nor in the l-adic cases. We prove that the Tate-type problem is valid for unit-root F-isocrystals. We apply this result to the finite monodromy theorem for unit-root F-isocrystals. We explain our results briefly. DUKE MATHEMATICAL JOURNAL c 2002 Vol. 111, No. 3, Received 7 December 1998. Revision received 7 March 2001. 2000 Mathematics Subject Classification. Primary 14F30; Secondary 14F10. Author’s work partially supported by Centre Internationale des Etudiants et Stagiaires.

385

386

NOBUO TSUZUKI

1.1 We fix notation as follows: p : a rational prime; k : a perfect field of characteristic p; V : a complete discrete valuation ring of mixed characteristics with residue field k; K : the field of fractions of V ; π : a uniformizer of K ; σ : the Frobenius map on k. We also denote by σ a lift of the Frobenius map on V and K if it exists. In the case where we mention F-isocrystals or Frobenius structures, we assume that the Frobenius map lifts on K and we fix it. Let j : X → Y be an open immersion of separated schemes of finite type over Spec k. Suppose that X is smooth over Spec k and that X is dense in Y . Put Z = Y −X . Denote by Isoc(X/K ) resp., Isoc† ((X, Y )/K ) the category of convergent isocrystals on X/K (resp., the category of overconvergent isocrystals on X/K along Z ). Let a be a positive integer. We denote by F a -Isoc(X/K ) resp., F a -Isoc† ((X, Y )/K ), F a -Isoc(X/K )0 , F a -Isoc† ((X, Y )/K )0 the category of convergent F a -isocrystals resp., the category of overconvergent F a -isocrystals on X/K along Z , the category of convergent unit-root F a -isocrystals on X/K , the category of overconvergent unit root F a -isocrystals on X/K along Z with respect to the ath-power Frobenius σ a . Here unit-root means that, for each geometric point in X , all slopes of the Frobenius structure are zeros. 1.2 The Tate-type problems for isocrystals (resp., F a -isocrystals) are the following ones (see [3, Rem. 2.3.9] and [5, Sec. 1.13]). 1.2.1 The natural forgetful functors CONJECTURE

(TC) (TF) (TU) are fully faithful.

j ∗ : Isoc† (X, Y )/K → Isoc(X/K ), j ∗ : F a -Isoc† (X, Y )/K → F a -Isoc(X/K ), 0 j ∗ : F a -Isoc† (X, Y )/K → F a Isoc(X/K )0

FINITE MONODROMY THEOREM FOR UNIT-ROOT F-ISOCRYSTALS

387

The faithfulness is a natural property of the functors j ∗ (see [3, Cor. 2.2.9]), and we have obvious implications (TC) ⇒ (TF) ⇒ (TU). The proof of the left implication above is the same as the proof of Corollary 4.1.2, and the right one is trivial. J. Tate proved that the natural functor from the category of p-divisible groups over the spectrum of a local normal domain of mixed characteristics (0, p) to that of p-divisible groups over the spectrum of the field of fractions is fully faithful (see [16]). After his result the similar problem in the case of characteristic p naturally arose. This problem was translated into the problem of Dieudonné modules by Dieudonné theory (see [4], [1]), and the problem on homomorphisms of p-divisible groups was finally solved by A. de Jong [9]. The Dieudonné module is one of p-adic local systems over a scheme of characteristic p. P. Berthelot [3] constructed the theory of p-adic local systems, which one calls F-isocrystals, over a scheme of characteristic p. Especially, an overconvergent F-isocrystal is important since the p-adic local system that has a geometric origin is thought to be an overconvergent F-isocrystal. Hence, the Tatetype problems as above are natural. In the case of curves, R. Crew proved Conjecture 1.2.1 for objects of rank 1 in [5, Th. 4.10], and the author proved it for unit-root F a -isocrystals of arbitrary rank in [17, Th. 5.1.1]. In Section 6 we prove the following theorem. 1.2.2 With the notation as above, let U be an open dense subscheme of X over Spec k. If Y is smooth over Spec k, then the natural functor 0 0 j ∗ : F a -Isoc† (U, Y )/K → F a -Isoc† (U, X )/K THEOREM

is fully faithful. In the case where U = X , we have the following corollary. COROLLARY 1.2.3 Conjecture (TU) is true if Y is smooth over Spec k.

1.3 Keep the notation as in Section 1.1. Let M be an overconvergent isocrystal on X/K along Z . We consider the problem of whether M can extend on Y after taking a generically finite e´ tale covering. Our result for unit-root F a -isocrystals is the following theorem, and we prove it in Section 7.

388

NOBUO TSUZUKI

THEOREM 1.3.1 Let M be an object in F a -Isoc† ((X, Y )/K )0 . There exists a smooth scheme Y 0 of finite type over Spec k and a proper surjective morphism w : Y 0 → Y over Spec k such that w is generically e´ tale and, if we put X 0 = w−1 (X ), Z 0 = Y 0 − X 0 , and the open immersion j 0 : X 0 → Y 0 , there exists a unique object N in F a -Isoc(Y 0 /K )0 with w∗ M ∼ = ( j 0 )† N as overconvergent F a -isocrystals on X 0 /K along Z 0 .

Since X 0 is not finite e´ tale but generically finite e´ tale over X in our result, we should call Theorem 1.3.1 a “generically” finite monodromy theorem. But if we allow a resolution of singularities for varieties of characteristic p, then we can find a proper surjective morphism w : Y 0 → Y such that X 0 is finite e´ tale over X in Theorem 1.3.1 (see Rem. 7.3.1). In the case where X is a curve, Theorem 1.3.1 was proved for rank M = 1 in [5, Th. 4.12] and for a unit-root F-isocrystal of arbitrary rank in [18, Th. 7.2.3]. A general isocrystal or an F a -isocrystal M might not extend on Y 0 . But we expect that we can find a proper surjective generically e´ tale morphism w : Y 0 → Y such that Z 0 = Y 0 − X 0 is a strict normal crossing divisor in Y 0 and, locally on Y 0 , the connection of M extends to a connection with logarithmic poles on a rigid analytic lifting of Y 0 . In the case of F-isocrystals on a curve, this problem is equivalent to the quasi-unipotent conjecture of Crew in [7, Sec. 10.1]. It is unsolved even in the case of curves in general. This problem is one of the important steps in showing the finiteness of the rigid cohomology with coefficients of overconvergent F-isocrystals (see [2]).


389

2. Local studies 2.1 We fix notation as follows: | | : an absolute value of K ; S = V [[x]] : the ring of formal power series with coefficients in V ; h1i SQ = S ; p X ∞ n an ∈ K , there exists 0 < η < 1 such that an x ; R= |an |ξ n → 0 (n → ±∞) for any η < ξ < 1 n=−∞ A = R ∩ K [[x]]; X ∞ an x n E =

an ∈ K , sup |an | < ∞, ; |an | → 0 (n → −∞) n=−∞ X ∞ |an |ηn → 0 (n → −∞) † n E = an x ∈ E . for some 0 < η < 1 n=−∞ For any ring R as above, put R = R d x and let d : R → R be the derivation P P P d( an x n ) = ((d/d x) an x n ) d x = ( nan x n−1 ) d x. Then ker d = K . R includes S, SQ , and E † , but R and E have no inclusion relation with each P other. Define | | on E by | an x n | = supn |an |. E (resp., E † ) is a complete discrete valuation field (resp., a Henselian discrete valuation field) under the absolute value | | with residue field k((x)). The valuation of E is an extension of that on K . We denote by OE (resp., OE† ) the integer ring of E (resp., E † ). Let R be one of K -algebras SQ , E † , and E , and denote by O R the integer rings S, OE† , and OE , respectively. We recall the definition of several categories in [19, Sec. 3]. Define a category M∇R whose objects are free R-modules of finite rank with a K connection ∇ : M → M ⊗ R R and whose morphisms are R-linear homomorphisms that commute with connections. The category M∇R is a K -linear abelian category with tensor products and duals, and its unit object is R = (R, d/d x). Define a K -vector space H 0 R[∇], M = HomM∇ (R, M) R

M∇R .

for an object M in Let σ be a Frobenius on R, that is, a lifting of the Frobenius on O R /π O R and a a σ on K . Let a be a positive integer. Let K σ (resp., R σ ) be the fixed subfield of K a a a (resp., R) by σ a . Then R σ = K σ and [K σ : Q p ] < ∞.

390

NOBUO TSUZUKI

We say that a free R-module of finite rank with a σ a -linear homomorphism ϕ : M → M is a ϕ-module with respect to σ a if and only if the induced R-linear homomorphism (σ a )∗ ϕ : (σ a )∗ M → M is an isomorphism. A morphism of ϕ-modules is an R-linear homomorphism that is compatible with Frobenius structures ϕ. We say that an object M is e´ tale if and only if there exists a ϕ-stable O R -lattice L such that ϕ(L) generates L over O R . In other words, all slopes of the Frobenius structure ϕ of M are zeros. We denote by MF R,σ a the category of ϕ-module over R with respect t ´ tale objects. to σ a and by MFe´R,σ a the full subcategory of MF R,σ a which consists of e a ´t e σ The category MF R,σ a is a K -linear abelian category with tensor products and duals, a and its unit object is R = (R, σ a ). Define a K σ -vector space by H 0 R[σ ], M = HomMF a (R, M) R,σ for any object M in MF R,σ a . We say that a free R-module of finite rank with a K -connection ∇ : M → M ⊗ R R has a compatible Frobenius structure ϕ : M → M with respect to σ a t if and only if the pair (M, ϕ) belongs to MFe´R,σ a and ∇ and ϕ satisfy the relation a ∇ ◦ ϕ = (ϕ ⊗ σ ) ◦ ∇. A morphism of free R-modules of finite rank with a K connection and a compatible Frobenius structure is an R-linear homomorphism that commutes with connections ∇ and Frobenius structures ϕ. We denote by MF∇R,σ a the category of free R-modules of finite rank with a K -connection and with a compatible Frobenius structure. We say that an object M in MF∇R,σ a is e´ tale if and only if the et Frobenius structure ϕ of M is e´ tale, and we denote by MF∇,´ R,σ a the full subcategory of a MF∇R,σ a which consists of e´ tale objects. The category MF∇R,σ a is a K σ -linear abelian category with tensor products and duals, and its unit object is R = (R, d/d x, σ a ). et Moreover, the subcategory MF∇,´ R,σ a is closed under subquotients, tensor products, and a a duals, and it is also a K σ -linear abelian category. Define a K σ -vector space by H 0 R[∇, ϕ], M = HomMF∇ a (R, M) R,σ

for any object M in MF∇R,σ a . Since SQ and E are p-adically complete, we have the following lemma (see [11, A, Sec. 2.2.4]). 2.1.1 The forgetful functor LEMMA

et e´ t MF∇,´ R,σ a → MF R,σ a

is an equivalence of categories if R = SQ or E . The following lemma is obvious.


391

LEMMA 2.1.2 Let M1 and M2 be objects in M∇R . Then there is a natural K -isomorphism H 0 R[∇], M1∨ ⊗ M2 ∼ = HomM∇ (M1 , M2 ). R

The same holds for the categories MF R,σ a and MF∇R,σ a . Remark 2.1.3 Suppose that k includes F pa . Let E be a normal domain that includes k, and let A be a p-adically complete domain that is flat over O K with A/π A = E. Assume that there exists a lift σ of Frobenius. Then there is a natural equivalence of categories O K σ a -modules of finite type with continuous π1 (Spec E)-action   projective A-modules L of finite type →  with σ a -linear homomorphism ϕ : L → L  such that ϕ(L) generates L over A t by [13, Sec. 4.1]. We define the category MFeÁ[1/ p],σ a as the category of the righthand side tensored with Q (see [11, A, Sec. 1.1]). In the case where A = S (resp., A = OE ), this definition coincides with our definition of MFeS´ tQ ,σ a (resp., MFeE´ t ,σ a ) above.

2.2 Let M be an object in M∇SQ (resp., M∇ E ). If we fix a basis of M, then a norm k k on M is determined as a module over the normed ring SQ (resp., E ). Fix a norm k k on M. We say that the connection ∇ satisfies condition (C) if and only if the condition

1 d n

(e) ηn → 0 (n → ∞)

∇ n! dx holds for any element e ∈ M and any η < 1. Condition (C) is independent of the choice of a basis of M. If a connection on a free SQ -module (resp., E -module) of finite rank admits a Frobenius structure with ∇ ◦ ϕ = (σ ⊗ ϕ) ◦ ∇, then the connection satisfies condition (C). The subcategory of M∇SQ (resp., M∇ E ) which consists of objects whose connection satisfies condition (C) is closed under subquotients, tensor products, and duals and contains the unit object. PROPOSITION 2.2.1 Let M be an object in M∇SQ . Assume that the connection ∇ of M satisfies condition (C). Then the natural map H 0 SQ [∇], M → H 0 E † [∇], M ⊗ SQ E †

392

NOBUO TSUZUKI

is bijective. Proof The injectivity is trivial. Let e1 , . . . , er be a system of basis of M, and let M ∨ (resp., e1∨ , . . . , er∨ ) be the dual of M (resp., the dual basis). Then the connection of M ∨ also satisfies condition (C). Put ∇(d/d x)n (t (e1 , . . . , er )) = G n t (e1 , . . . , er ) resp., ∨ t ∨ ∇ ∨ (d/d x)n (t (e1∨ , . . . , er∨ )) = G ∨ n (e1 , . . . , er ) . Then the matrix ∞ X 1 ∨ Y = 1r + G (0)x n n! n n=1

in Mr (K [[x]]) is a solution of the system of linear differential equations d Y + YG = 0 dx

(*)

in Mr (K [[x]]). By condition (C), Y is contained in Mr (A ). By the existence of the solution for the dual module M ∨ , Y is contained in GLr (A ). If f : E † → M ⊗ SQ E † is an E † -linear homomorphism that commutes with connections, then f determines a solution of (∗) in (E † )r . Since E † is included in R as a K -algebra and Y is invertible, f is a linear sum of Y t (e1 , . . . , er ) over K . The image f (1) of 1 through f is contained both in M ⊗ SQ E † and in M ⊗ SQ A . Hence, f is included in H 0 (SQ [d/d x], M). THEOREM 2.2.2 ([17, Prop. 4.1.1, Rem. 4.1.2]) Let σ be a Frobenius on E † . If M is an object in MF∇ such that all slopes of the E † ,σ a Frobenius structure of M are less than or equal to zero, then the natural map H 0 E † [ϕ], M → H 0 E [ϕ], M ⊗E † E

is bijective. In particular, the functor et et MF∇,´ → MF∇,´ E ,σ a , E † ,σ a

M 7 → M ⊗E † E

is fully faithful. In [9] de Jong proved the integral version of the following theorem in the case where K is absolutely unramified and a = 1. THEOREM 2.2.3 Let σ be a Frobenius on SQ . Let M be an object in MF∇SQ ,σ a . Then the natural functor

MF∇SQ ,σ a → MF∇ E ,σ a ,

M 7 → M ⊗ SQ E


393

is fully faithful. In other words, the natural map H 0 SQ [∇, ϕ], M → H 0 E [∇, ϕ], M ⊗ SQ E is bijective for any object M in MF∇SQ ,σ a . Proof Let us consider a functor defined by (M, ∇, ϕ) 7→ where

MF∇R,σ a → MF∇R,σ La−1 i ∗ La−1 i ∗ i=0 (σ ) M, i=0 (σ ) ∇, ψ for R = SQ and E ,

ψ(m 0 ⊗ 1, m 1 ⊗ 1, . . . , m a−1 ⊗ 1) = (σ a )∗ ϕ(m a−1 ⊗ 1), m 0 ⊗ 1, m 1 ⊗ 1, . . . , m a−2 ⊗ 1 . Then one can easily reduce the full faithfulness of the functor MF∇SQ ,σ a → MF∇ E ,σ a to the case where a = 1. Let us denote by S0,Q (resp., E0 ) the subring of SQ (resp., E ) whose field of scalars is the maximally absolutely unramified subfield of K . Since the category MF∇R,σ does not depend on the choice of Frobenius σ (see [19, Th. 3.4.10]), we may assume that S0,Q is stable under the Frobenius σ . Now consider the natural functor MF∇R,σ → MF∇R0 ,σ induced by the restriction R0 → R for (R0 , R) = (S0,Q , SQ ) and (E0 , E ). Then one can easily see that de Jong’s theorem (see [9, Th. 9.1]) implies the full faithfulness for a = 1. 2.3 We state several conjectures as follows.

CONJECTURE

2.3.1

be an E † -module of finite rank with a connection. We say that the connection of

Let M P n n M satisfies condition (OC) if and only if the infinite sum ∞ n=0 (w /n!)∇(d/d x) (e) † converges in M for any element e ∈ M and w ∈ E with |w| < 1. Denote by ∇ ∇ (resp., M∇,C M∇,OC E ) the full subcategory of ME † (resp., ME ) which consists of obE† jects whose connection satisfies condition (OC) (resp., (C)). (The subcategory M∇,OC E† is closed under subquotients, tensor products, and duals and contains the unit object (E † , d/d x). Hence, it is K -linear abelian.) Then the functor M∇,OC → M∇,C E , E†

M 7→ M ⊗E † E

394

NOBUO TSUZUKI

is fully faithful. In other words, the natural map H 0 E † [∇], M → H 0 E [∇], M ⊗E † E is bijective for any object M in MF∇,OC . E† CONJECTURE 2.3.2 Let σ be a Frobenius on E † . Then the functor

→ MF∇ MF∇ E ,σ a , E † ,σ a

M 7 → M ⊗E † E

is fully faithful. In other words, the natural map H 0 E † [∇, ϕ], M → H 0 E [∇, ϕ], M ⊗E † E is bijective for any object M in MF∇ . E † ,σ a Remark 2.3.3 (1) We also expect the bijectivity of Conjecture 2.3.1 without any assumptions on the connections. In the case where the rank of M over E † is 1, the bijectivity of Conjecture 2.3.1 is true without the assumptions on connections (see [5, Sec. 4.1]). (2) If a connection admits a compatible Frobenius structure, then the connection satisfies condition (OC) (see [19, Prop. 3.4.4]) and Conjecture 2.3.1 implies Conjecture 2.3.2. Hence, Conjecture 2.3.2 is true for objects of rank 1. (3) If the connection over E † (resp., SQ ) comes from an overconvergent isocrystal, then the connection satisfies condition (OC) (resp., (C)) (see [3, Prop. 2.2.13] (resp., [3, Cor. 2.2.14])).

3. Some properties of isocrystals 3.1 Let j : X → Y be an open immersion of separated schemes of finite type over Spec k. Suppose that X is smooth over Spec k and that X is dense in Y . For an object M in Isoc† ((X, Y )/K ) resp., F a -Isoc† ((X, Y )/K ) for a positive integer a , we define H 0 (X, Y )/K , M = HomIsoc† ((X,Y )/K ) ( j † O]Y [ , M ) resp., H F0 a (X, Y )/K , M = Hom F a -Isoc† ((X,Y )/K ) ( j † O]Y [ , M ) . In other words, H 0 ((X, Y )/K , M ) resp., H F0 a ((X, Y )/K , M ) is the K -space of a horizontal sections of M (resp., the K σ -space of horizontal sections of M which is


395

fixed by the Frobenius of M ). Here K σ is the fixed subfield of K by σ a . If X = Y , we simply denote H 0 (X/K , M ) resp., H F0 a (X/K , M ) for H 0 ((X, X )/K , M ) resp., H F0 a ((X, X )/K , M ) . Since the category Isoc† ((X, Y )/K ) resp., F a -Isoc† ((X, Y )/K ) is abelian with tensor products, duals, and the unit object j † O]Y [ (see [3, Cor. 2.2.10]), we have the following proposition. a

PROPOSITION 3.1.1 For any objects M1 and M2 in Isoc† ((X, Y )/K ), we have a natural bijection H 0 (X, Y )/K , M1∨ ⊗ M2 ∼ = HomIsoc† ((X,Y )/K ) (M1 , M2 ).

Here M1∨ is the dual of M1 . The same holds for F-isocrystals. For any finite extension k 0 over k, denote by X 0 , Y 0 , U 0 (resp., K 0 ) the extension of the scalar of X , Y , U (resp., K ⊗W (k) W (k 0 ), where W (k) is the Witt ring) by k → k 0 . Since K 0 is unramified over K , the Frobenius σ on K extends uniquely on K 0 . 3.1.2 For any object M in Isoc† ((X, Y )/K ), if we denote the scalar extension M 0 = M ⊗ j † O]Y [ j † O]Y 0 [ , we have a natural bijection H 0 (X 0 , Y 0 )/K 0 , M 0 ∼ = H 0 (X, Y )/K , M ⊗ K K 0 .

PROPOSITION

(1)

(2)

For any object M in F a -Isoc† ((X, Y )/K ), if we denote the scalar extension M 0 = M ⊗ j † O]Y [ j † O]Y 0 [ , we have a natural bijection a H F0 a (X, Y )/K , M ⊗ K σ a (K 0 )σ a ∼ = H F0 a (X 0 , Y 0 )/K 0 , M 0 ∩ H F0 a (X/K , j ∗ M ) ⊗ K σ a (K 0 )σ .

Proof The proof is the same as in [2, Prop. 1.8]. 3.2 We recall the gluing lemma of isocrystals (see [3, Prop. 1.2.5, Prop. 2.1.12, Déf. 2.3.2(iii)]). Let j : X → Y be an open immersion of separated k-schemes of finite type such b be a formal scheme of finite type over Spf V that X is smooth over Spec k, and let P b b is smooth over Spf V around X . with a closed immersion Y → P such that P 3.2.1 Let {X i } be a finite open covering of X , and put ji : X i → Y . PROPOSITION

396

NOBUO TSUZUKI

Suppose that, for all i, Mi is an overconvergent isocrystal on X i /K along † Y − X i and that, for any i and i 0 , there is an isomorphism ji†0 Mi ∼ = ji Mi 0 which satisfies the usual cocycle condition. Then there exists an overconvergent isocrystal M on X/K along Y − X with ji† M ∼ = Mi for all i which † induces ji†0 Mi ∼ = ji Mi 0 . (2) Suppose that M and N are overconvergent isocrystals on X/K along Y − X and that, for all i, there is a morphism αi : ji† M → ji† N as overconvergent isocrystals with ji†0 αi = ji† αi 0 for any i and i 0 . Then there exists a unique morphism α : M → N with ji† α = αi as overconvergent isocrystals. The same holds for F-isocrystals. (1)

3.2.2 With the same notation as above, let {Yi } be an open covering of Y , and put X i = bi of P b with Yi = X ∩ Yi and ji : X i → Yi . Choose an open formal subscheme P b Y ×P P . c i (1) Suppose that, for all i, Mi is an overconvergent isocrystal on X i /K along ∼ Yi − X i and that, for any i and i 0 , there is an isomorphism Mi |]Yi ∩Yi 0 [P c ∩P c0 = i i Mi 0 |]Yi ∩Yi 0 [P c ∩P c 0 which satisfies the usual cocycle condition. Then there exi i ∼ ists an overconvergent isocrystal M on X/K along Y − X with M |]Yi [P c = i ∼ Mi for all i which induces Mi |]Yi ∩Yi 0 [ = Mi 0 |]Yi ∩Yi 0 [ . (2) If there is a morphism αi : M |]Yi [ → N |]Yi [ in Isoc† ((X i , Yi )/K ) which sat0 0 0 c c for i, i , then there isfies the relation αi |]Yi ∩Yi 0 [P c ∩P c 0 = αi ]Yi ∩ Yi [P i ∩P 0 PROPOSITION

i

i

i

† exists a unique morphism α : M → N on ]Y [P c in Isoc ((X, Y )/K ) with α|]Yi [P c = αi . i The same holds for F-isocrystals.

The next corollary follows from the faithfulness of the functor jU∗ : Isoc(X/K ) → Isoc(U/K ) (see [3, Cor. 2.1.11]) and Propositions 3.2.1 and 3.2.2. COROLLARY 3.2.3 With the same notation as above, let U be an open dense subscheme of X with jU : U → Y . Let {Yi } be a open covering of Y , and let {X im } be a finite open covering {X im } of X i with jim : X im → Yi for each i. Suppose that M and N are objects in Isoc† ((X, Y )/K ) and that β : jU∗ M → jU∗ N is a morphism in Isoc(U/K ). † † If αim : jim (M |]Yi [ ) → jim (N |]Yi [ ) is a morphism in Isoc† ((X im , Yi )/K ) and if αim |]U ∩X im [ = β]U ∩X im [ for each i and m, then there exists a unique morphism † α : M → N in Isoc† ((X, Y )/K ) with jim α|]Yi [P c = αim and α|]U [ = β. The i a same holds for F -isocrystals.


397

4. The full faithfulness of the functor j † 4.1 Let j : X → Y be an open immersion of separated k-schemes of finite type such that X is smooth over Spec k and dense in Y . Put Z = Y − X . Let U be an open dense subscheme of X over Spec k, and put D = X − U and jU : U → Y . THEOREM 4.1.1 With the notation as above, if Y is smooth over Spec k, then the natural functor jU† : Isoc† (X, Y )/K → Isoc† (U, Y )/K

is fully faithful. The faithfulness of the functor jU† follows from the faithfulness of the functor j ∗ : Isoc† ((X, Y )/K ) → Isoc(X/K ) (see [3, Cor. 2.2.9]) and that of jU∗ (see [3, Cor. 2.1.11]). We prove Theorem 4.1.1 in this section. First we give corollaries. 4.1.2 Let a be a positive integer. Under the same assumption as in Theorem 4.1.1, the natural functor jU† : F a -Isoc† (X, Y )/K → F a -Isoc† (U, Y )/K COROLLARY

is fully faithful. Proof Let α : jU† M1 → jU† M2 be a morphism in F a -Isoc† ((U, Y )/K ). Then there is a morphism β : M1 → M2 in Isoc† ((X, Y )/K ) such that jU† β = α by Theorem 4.1.1. Denote by 8i : σ a ∗ Mi → Mi the Frobenius structure of Mi for i = 1, 2, where σ a is the ath power of the absolute Frobenius. By definition, 8i is a horizontal section as a morphism of overconvergent isocrystals. Since jU† is faithful, we have 82 ◦ β = β ◦ 81 . By [3, Th. 2.3.5], we have the following corollary. 4.1.3 With the notation as above, assume furthermore that there is an open immersion j1 : X → Y1 of smooth k-schemes of finite type and a proper morphism w : Y1 → Y with w ◦ j1 = j. Then the natural functor jU† : Isoc† (X, Y )/K → Isoc† (U, Y )/K COROLLARY

398

NOBUO TSUZUKI

is fully faithful. The same holds for overconvergent F-isocrystals. 4.2 Let Y be an affine integral scheme of finite type over Spec k, and let Y = Spec R be an affine integral scheme of finite type over Spec V with Y ⊗V k = Y . Suppose that j : X → Y is an open immersion of affine schemes such that the complement Z = Y − X is defined by the element y¯ , which is a product of different prime elements y¯1 , . . . , y¯e of R/π R. Choose a lift yi of y¯i in R for each i, and put y = y1 · · · ye . Assume that X = Spec R[1/y] is smooth over Spec V and that there exists a system of coordinates x, x2 , . . . , xn of X over Spec V such that the section x¯ of the image of x in R/π R defines a nonempty smooth irreducible divisor in X . Assume furthermore that x is contained in R and that x¯ is not divisible by y¯i in R/π R for all i. Let D be a smooth divisor in X defined by the section x, and put U = X − D and U = U ⊗V k with jU : U → Y . Then U = Spec R[1/(x y)]. Denote by Yb (resp., Ub) the p-adic formal completion of Y (resp., U ). Denote by W X n (resp., WU n ) the affinoid open neighborhood of ]X [Yb (resp., n n ]U [Yb) in Yban which is defined by |y| = π −1/ p (resp., |x y| = π −1/ p ) for any positive integer n. Then {W X n } (resp., {WU n }) is a system of fundamental strict neighborhoods of ]X [Yb (resp., ]U [Yb) in Yban (see [3, Prop. 1.2.2]). Put pn d R X n = R[t]/(y t − π ),

d (x y) pn s − π . RU n = R[s]/ Here b means the p-adic completion. Then 0(W X n , O]Y [Yc) = R X n ⊗ Q resp., 0(WU n , O]Y [Yc) = RU n ⊗ Q . The inclusion W X n+1 → W X n is defined by t¯ 7→ n+1 n y p − p t¯, where t¯ is the image of the parameter t on W X n (resp., W X n+1 ). The same n holds for WU n , which is the open affinoid in W X n by t¯ 7→ x p s¯ . Then we have 0 ]Y [Yb, j † O]Y [Yc = lim R X n ⊗ Q, n→∞ † 0 ]Y [Yb, jU O]Y [Yc = lim RU n ⊗ Q, n→∞

and h 1 i \ 0 ]Y [Yb, j † O]Y [Yc ⊂ 0 ]Y [Yb, jU† O]Y [Yc ⊂ 0 ]U [Yb, O]U [Yc = R ⊗ Q. xy p −n

Choose a system of {x j

p −n

} of pth-power roots of x j ( j = 2). Add {x j

; n =

p −∞ 1, j = 2} to R[1/y], localize R[1/y][x j ; j = 2] at the prime ideal (x, π), and p −∞ p −∞ then take the completion (R[1/y][x j ; j = 2](x,π) )b of R[1/y][x j ; j = 2](x,π) p −∞ at the maximal ideal (x, π). Then (R[1/y][x j ; j = 2](x,π) )b is the same as the


399

ring S in Section 2.1; that is, it is isomorphic to V 0 [[x]] for some complete discrete valuation ring V 0 with a perfect residue field. Denote it by S, and define V [1/ p]algebras SQ , E , and E † as in Section 2.1. By our construction, we have a natural commutative diagram of homomorphisms of K -algebra: 0 ]Y [Yb, j † O]Y [Yc → 0 ]Y [Yb, jU† O]Y [Yc → 0 ]U [Yb, O]Y [Yc ↓ ↓ ↓ SQ → E† → E The vertical arrows are injective. Under the inclusion 0(]U [Yb, O]Y [Yc) → E , the derivation ∂/∂ x is compatible with the derivation d/d x. If we normalize a Banach norm | |]U [ on 0(]U [Yb, O]Y [Yc) by | p|]U [ = p −1 , then |a|]U [ = |a| for any a ∈ 0(]U [Yb, O]Y [Yc), where | | is the p-adic norm on E since π is a prime divisor on the p-adic formal affine integral scheme Ub. LEMMA 4.2.1 With the notation as above, we have

0 ]Y [Yb, j † O]Y [Yc = SQ ∩ 0 ]Y [Yb, jU† O]Y [Yc in E . Proof P∞ m Note that, for a ∈ RU n (resp., a ∈ R X n ), we can write a = m=0 am s¯ (resp., P∞ m a = m=0 am t¯ ) for some am ∈ R such that |am |]U [ → 0 (n → ∞). Of course, this expression is not unique. P Suppose that a is contained in S ∩ RU n and has an expression a = am s¯ m as P above. We show that a has an expression of the sum a = bm t¯m such that bm ∈ R and |bm |]U [ 5 |am |]U [ . If a has such an expression, then a is always convergent in E and is contained in R X n . Assume that a has an expression a=

mX 1 −1

bm t¯m +

∞ X

0 m am s¯

m=m 1

m=0

0 ,b 0 for some am m ∈ R such that |bm |]U [ 5 |am |]U [ (m < m 1 ) and |am |]U [ 5 m 0 m 0 1 |am |]U [ (m = m 1 ). If |π am 1 |]U [ 5 supm>m 1 |π am |]U [ , then we can rewrite a = Pm 1 P∞ 00 m 00 0 ¯m m=m 1 +1 am s¯ such that bm 1 = 0 and |am |]U [ 5 |am |]U [ (m > m 1 ) m=0 bm t + since π is a prime element in R. Indeed, we have 0 am s¯ m 1 = 1

(x y) p

n (m

2 −m 1 )

π (m 2 −m 1 )

0 am 1

s¯ m 2

and

(x y) pn (m 2 −m 1 ) a 0 m1 0 | 5 |am 2 ]U [ ]U [ π (m 2 −m 1 )

400

NOBUO TSUZUKI

for a suitable m 2 > m 1 , and we put 00 0 + am = am 2 2

(x y) p

n (m

2 −m 1 )

0 am 1

π (m 2 −m 1 ) 00 = a 0 (m 6 = m ). Assume that |π m 1 a 0 | m 0 0 and am 2 m m 1 ]U [ > supm>m 1 |π am |]U [ . Put am 1 = lm 1 0 0 ∈ (R − π R). Since R is a Noetherian domain, such an l π a˜ m 1 with a˜ m m 1 always P∞ 1 0 s¯ m ∈ π lm 1 +m 1 +1 O , x − p n m 1 a 0 is contained in S modulo π exists. As m=m 1 +1 am ˜ E m1 by our assumption on a and bm . Since x¯ is a prime element in R[1/y]/π R[1/y], h1i h1i h1i h 1 i n n x¯ p m 1 (S/π S) ∩ R /π R = x¯ p m 1 R /π R . y y y y Hence, we have 0 a˜ m = xp 1

nm

1

0 b˜m 1

yr

(mod π )

0 ∈ R and r = 0. Since x¯ is prime to the prime divisor y¯ for all i, b˜ 0 is for some b˜m i m1 1 divisible by y r modulo π. Therefore, we have

x−p

nm

1

0 a˜ m = b˜m 1 + π x − p 1

nm

1

00 a˜ m 1

00 ∈ R. Continuing these arguments, we obtain the formula for some b˜m 1 , a˜ m 1 0 00 m 1 am s¯ m 1 = bm 1 t¯m 1 + am s¯ 1 1 00 ∈ R such that |b | 0 m 1 00 for some bm 1 , am m 1 ]U [ = |am 1 |]U [ 5 |am 1 |]U [ and |π am 1 |]U [ 5 1 P P m ∞ 1 00 m 0 | ¯m supm>m 1 |π m am ]U [ . Then we can rewrite a = m=m 1 +1 am s¯ such m=0 bm t + 00 0 that |am |]U [ 5 |am |]U [ (m = m 1 + 1) since π is a prime element in R. This completes the proof.

4.3 Proof of Theorem 4.1.1 We have only to prove the fullness. Hence, it is sufficient to prove that the natural injection H 0 (X, Y )/K , M → H 0 (U, Y )/K , M is bijective for any object M in Isoc† ((X, Y )/K ) by Proposition 3.1.1 (see the notation H 0 in Section 3.1). By Proposition 3.1.2, we can replace k by a finite extension k 0 , and we may assume that Y is geometrically irreducible. We may assume the situation of Section 4.2 by Corollary 3.2.3. Now we keep the notation as in Section 4.2 and assume furthermore that R is smooth over Spec V . Define an SQ -module M = 0(]Y [Yb, M ) ⊗0(]Y [ c, j † O]Y [ ) SQ and a connection by Y

c Y

d ∂ da ∇ (m ⊗ a) = a∇ (m) + m ⊗ dx ∂x dx


401

for m ∈ 0(]Y [Yb, M ) and a ∈ SQ . Since M is an overconvergent isocrystal on (X, Y )/K , M is locally free (see [3, Prop. 2.2.3]) and M is free of finite rank over SQ . So M is an object in M∇SQ and the connection ∇ of M satisfies condition (C) in Section 2.2 by [3, Cor. 2.2.14]. Consider the following commutative diagram: 0

→

0

→

0(]Y [Y c, M ) ↓ M

→ →

† 0(]Y [Y c, jU M ) ↓ M ⊗ SQ E †

→ →

† 0(]Y [Y c, jU M )/ 0(]Y [Y c, M ) ↓ M ⊗ SQ E † /SQ

→

0

→

0

Since ]Y [Yb is affinoid and W X n is an affinoid strict neighborhood of ]X [Yb for all n, 0(]Y [Yb, M ) is a direct limit of an inductive system {L n }n such that L n is a flat R X n -module. Hence, 0(]Y [Yb, M ) is flat over 0(]Y [Yb, j † O]Y [Yc) and we have ∼ 0 ]Y [ b, M ⊗ 0 ]Y [Yb, jU† M = † 0(]Y [Y Y c, j O]Y [

c)

Y

0 ]Y [Yb, jU† O]Y [Yc .

So both horizontal rows in the diagram above are exact, and the right vertical arrow is injective by Lemma 4.2.1. Let f : jU† O]Y [ → jU† M be a horizontal section in Isoc† ((U, Y )/K ). The image of f (1) through the middle vertical arrow is contained in M by Proposition 2.2.1. Hence, f (1) is contained in 0(]Y [Yb, j † M ). Therefore, it is horizontal in Isoc† ((X, Y )/K ) by [3, Prop. 2.2.7]. Now we prove the general cases by induction on the dimension of D. Let r be the codimension D in X . By Proposition 3.1.2, we can replace k by a finite extension k 0 , and therefore, we may assume that D is reduced and there is an open subscheme U1 of X such that U1 ⊃ U , U1 ∩ D is smooth over k, and the dimension of D1 = X − U1 in X is less than the dimension of D. By the induction hypothesis, we may assume that D = D1 . By Corollary 3.2.3, we may assume that both X and Y are affine, and D is defined by a part x1 , . . . , xr of coordinates xi of X . Moreover, we may assume that Z = Y − X is defined by a product of different prime divisors in 0(Y, OY ) since 0(Y, OY ) is a locally factorial domain (see [14, Th. 48]). By [10, Th. 6], there is an open immersion X → Y of affine smooth schemes of finite type over Spec V with X = X ⊗V k, Y = Y ⊗V k. Denote by D2 the smooth divisor defined by x1 = 0 in X , and put U2 = X − D2 and jU2 : U2 → Y . By Corollary 3.2.3, we may assume that D2 is connected. Then we have a natural sequence of injections: H 0 (X, Y )/K , M → H 0 (U, Y )/K , jU† M → H 0 (U2 , Y )/K , jU† 2 M . We have already proved the bijectivity of the composition; hence, the left arrow is bijective. This completes the proof.

402

NOBUO TSUZUKI

5. Trace map 5.1 Consider the following commutative diagram of integral separated k-schemes of finite type: j1

X 1 → Y1 v↓ ↓w X 2 → Y2 j2

such that X 1 , X 2 , Y2 are smooth over Spec k, Y1 is normal, v : X 1 → X 2 is finite e´ tale, and w : Y1 → Y2 is proper. In this case we define a direct image functor w∗ : Isoc† (X 1 , Y1 )/K → Isoc† (X 2 , Y2 )/K as follows. In the case where X 1 = Y1 or X 1 is a curve, the direct image functor w∗ was constructed by Crew [6, Sec. 1.7]. By the gluing lemma (see Props. 3.2.1 and 3.2.2), the construction is local both on X 2 and on Y2 . We may assume that X 1 and X 2 are affine. Since the category of Isoc† ((X 1 , Y1 )/K ) is independent of the choice of Y1 up to canonical equivalence (see [3, Th. 2.3.5]), we may assume that Y1 and Y2 are affine, Y1 is finite over Y2 , and X 1 ∼ = X 2 ×Y2 Y1 . Choose a lift P2 smooth of finite type over Spec V of Y2 . By construction, we can find a normal integral flat V -scheme P1 which is finite over b1 is smooth over Spf V around X 1 since X 1 / X 2 P2 with Y1 = Y2 ×P2 P1 . Then P 1 2 is e´ tale. Denote by qi , qi : ]Yi [P c2 →]Yi [P ci two natural projections of the tubes for i i = 1, 2. Let M be an object in Isoc† ((X 1 , Y1 )/K ). Since e´ taleness is an open condib1 /P b2 is finite, we can choose a strict neighborhood U2 of ]X 2 [ such that tion and P an −1 U1 = (w ) (U2 ) is a strict open neighborhood of ]X 1 [ (see [3, Prop. 1.2.7]) and M is defined on U1 . Denote jUi : ]X i [→ Ui for i = 1, 2. Choose a coherent OU1 -module N with an overconvergent connection such that M = jU† 1 N . Define w∗ M = jU† 2 (wan |U1 )∗ N . Since wan |U1 is finite, w∗ M is coherent over j2† O]Y2 [ . We construct an overconvergent connection on w∗ M . Since the connection of N is overconvergent and wan × wan : ((q11 )−1 (U1 ) ∩ (q12 )−1 (U1 )) → ((q21 )−1 (U2 ) ∩ (q22 )−1 (U2 )) is finite e´ tale, there is a strict neighborhood U20 of ]X 2 [P c2 such that 2

(i) (ii) (iii)

U10 = (wan × wan )−1 (U20 ) is included in (q11 )−1 (U1 ) ∩ (q12 )−1 (U1 ), there exists an isomorphism : (q11 )∗ N ∼ = (q12 )∗ N on U10 which satisfies the usual cocycle condition, and induces the overconvergent connection of M by [3, Prop. 2.2.6].

LEMMA

5.1.1


403

With the notation as above, the commutative diagram q1l

U10 (wan )2 ↓ U20

U1 ↓ wan U2

→ → q2l

is Cartesian for l = 1, 2. Proof The proof is as in [6, Sec. 1.7]. Consider the commutative diagram ]X 1 [P 2

U10

→

1

k ]X 1 [P 2 1

→

↓ ((w ) , q ) ×q l U1 → an 2

→ (wan 2 ,q1l )

U20

ban )2 (P 1

l 1

2

↓ (w , id) ban ×P 1 an

ban P 2

Here U20 ×q l U1 means the fiber product for the map q2l : U20 → U2 . Since wan |U1 is 2

finite e´ tale, ((wan )2 , id) induces an isomorphism between U100 and U200 ×q l U1 by [3, 2 Th. 1.3.5]. Since N is coherent, wan |U1 is finite, and q2l |U 0 (l = 1, 2) is flat, induces the 2 isomorphism 2 2 (q21 |U 0 )∗ (wan |U1 )∗ N ∼ = (wan |U10 )∗ (q11 |U10 )∗ N ∼ = (wan |U10 )∗ (q12 |U10 )∗ N 2

∼ = (q22 |U20 )∗ (wan |U1 )∗ N by Lemma 5.1.1. One can check that this isomorphism satisfies the cocycle condition by the same method. Hence, this isomorphism induces an overconvergent connection on w∗ M . We have finished constructing the direct image functor w∗ . We also recall the inverse image functor w∗ : Isoc† (X 2 , Y2 )/K → Isoc† (X 1 , Y1 )/K in [3, Déf. 2.3.2(iv)]. Let M be an object in Isoc† ((X 2 , Y2 )/K ). Since e´ taleness is an b1 /P b2 is finite, we can find a strict neighborhood U2 of ]X 2 [ open condition and P such that M2 is defined over U2 . Put U1 = (wan )−1 (U2 ) and jUi : ]X [→ Ui for i = 1, 2. Denote by N a coherent OU2 -module with an overconvergent connection such that jU† 2 N ∼ = M . Then we define w∗ M = jU† 1 (wan |U1 )∗ N . One can easily see that the connection of M induces an overconvergent connection † on w∗ M . Of course, w∗ j2† O]Y2 [ ∼ = j1 O]Y1 [ .

404

NOBUO TSUZUKI

b2 . We define In the case of F a -isocrystals, we fix a lift of the Frobenius σ on P the Frobenius structure as follows. Put λ < 1. Let U2,λ (resp., U2,λ pa ) be a strict a neighborhood of ]X 2 [ of radius λ (resp., λ p ), and let U1,λ (resp., U1,λ pa ) be the b2 induces pullback of U2,λ (resp., U2,λ pa ) by wan . Note that the Frobenius σ on P a the Frobenius σ : U2,λ → U2,λ pa (see [3, 2.4.1.3]). Since ]X 1 [/]X 2 [ is e´ tale, U1,λ (resp., U1,λ pa ) is e´ tale over U2,λ (resp., U2,λ pa ) and the Frobenius extends to the map σ a : U1,λ → U1,λ p if one chooses λ sufficiently close to 1. In other words, we have a commutative diagram U1,λ ↓ U2,λ

σa

→ U1,λ pa ↓ →a U2,λ pa σ

if λ is sufficiently close to 1. Therefore, we can define the direct image functor w∗ : F a -Isoc† (X 1 , Y1 )/K → F a -Isoc† (X 2 , Y2 )/K and the inverse image functor w∗ : F a -Isoc† (X 2 , Y2 )/K → F a -Isoc† (X 1 , Y1 )/K as before. By construction, the functor w∗ (resp., w∗ ) commutes with functors j † and j ∗ . We define an adjoint map ad : M → w∗ w∗ M by m 7 → 1 ⊗ m for m ∈ M . Then one can easily check that the adjoint map ad is a morphism in Isoc† ((X 2 , Y2 )/K ) resp., F a -Isoc† ((X 2 , Y2 )/K ) . The functors w∗ and w∗ are adjoint to each other by the adjoint map ad. PROPOSITION 5.1.2 With the notation as above, let M be an object in Isoc† ((X 1 , Y1 )/K ). Then we have a natural isomorphism H 0 (X 1 , Y1 )/K , M ∼ = H 0 (X 2 , Y2 )/K , w∗ M

by the adjoint map ad. The same holds for F-isocrystals.

5.2 Keep the notation as in Section 5.1. For an object M in Isoc† ((X 2 , Y2 )/K ), we define a trace map tr : w∗ w∗ M → M


405

which is a morphism in Isoc† ((X 2 , Y2 )/K ) as follows. In the case of convergent b1 → P b2 of forisocrystals (i.e., X i = Yi ), we can find a finite e´ tale morphism P mally smooth schemes of finite type over Spec V which is a lifting of X 1 → X 2 . Then we can define the trace map to be the usual one. In general, the construction of the trace map is a local problem by Corollary 3.2.3. Hence, we may assume the local situation is as in Section 5.1. Since U1 is finite e´ tale over U2 , we can define a trace map tr : w∗an OU1 → OU2 and a trace map tr : w∗ w∗ M → M for M by tr ⊗ id wan |U1 ∗ wan |U1 ∗ N ∼ = wan |U1 ∗ (OU1 ⊗OU2 N ) ∼ = (wan |U1 )∗ OU1 ⊗OU2 N → N . One can easily check that the trace map tr commutes with connections. If we denote by d the degree of X 1 over X 2 , then the composition ad

tr

M → w∗ w ∗ M → M

of the adjoint map and the trace map is d × idM for any object M in Isoc† ((X 2 , Y2 )/K ), where idM is the identity map on M . One can easily see that the trace map tr commutes with Frobenius structures for F a -isocrystals. PROPOSITION

5.2.1

Let

j1

X 1 → Y1 v↓ ↓w X 2 → Y2 j2

be a commutative diagram of separated k-schemes of finite type with X 1 , X 2 , Y2 smooth, Y1 normal, v : X 1 → X 2 finite e´ tale of degree d, and w : Y1 → Y2 proper. Let M be an object in Isoc† ((X 2 , Y2 )/K ). Then the diagram H 0 (X 2 , Y2 )/K , M ↓ H 0 (X 2 /K , j2∗ M )

ad

→ → ad

H 0 (X 2 , Y2 )/K , w∗ w ∗ M ↓ H 0 (X 2 /K , w∗ w ∗ M )

tr

→ → tr

H 0 (X 2 , Y2 )/K , M ↓ H 0 (X 2 /K , j2∗ M )

is commutative, and both composite maps of the top and seconds rows are d × id. The same holds for F-isocrystals.

406

NOBUO TSUZUKI

6. Proof of Theorem 1.2.2 6.1 We fix notation as follows. Let Y = Spec R be a smooth integral affine scheme of finite type over Spec V with a system x1 , x2 , . . . , xn of coordinates such that Y = Pd Zi be a divisor in Y for d 5 n Y ×Spec V Spec k is irreducible, and let Z = i=1 such that each component Zi = Zi ⊗V k is smooth irreducible and is defined by sections xi . Put X = Y − Z , X = X ⊗V k, and j : X → Y . Denote by Yb the p-adic formal completion of Y . p −n Fix an integer i with 1 5 i 5 d. Choose a system of pth-power roots {x j } of p −n

x j ( j 6= i). Add {x j

p −∞

; n = 1, j 6 = i} to R, localize R[x j

ideal (xi , π ), and then take the completion of p −∞ ideal (xi , π ). Then (R[x j ; j 6 = that is, it is isomorphic to Vi0 [[xi ]]

p −∞ R[x j ;

; j 6= i] at the prime

j 6= i](xi ,π) at the maximal

)b is isomorphic to the ring

i](xi ,π ) S in Section 2.1; for some complete discrete valuation ring Vi0 with a perfect residue field. Denote it by Si , and define Vi0 [1/ p]-algebras SQi , Ei , and Ei† as in Section 2.2 for each 1 5 i 5 d. Then we have a commutative diagram of ring homomorphisms 0 ]Y [Yb, O]Y [Yc → 0 ]Y [Yb, j † O]Y [Yc → 0 ]X [Yb, O]Y [Yc ↓ ↓ ↓ † SQi → Ei → Ei By construction, the vertical arrows are injective. Under the inclusion 0(]X [Yb, O]Y [Yc) → Ei , the derivation ∂/∂ xi is compatible with the derivation d/d xi on Ei and the Banach norm | |]X [ on 0(]X [Yb, O]Y [Yc) is compatible with the norm | | on Ei of the usual p-adic norm. LEMMA 6.1.1 With the notation as above, we have M 0 ]Y [Yb, O]Y [Yc = SQi ∩ 0 ]Y [Yb, j † O]Y [Yc , i

0 ]Y [Yb, j † O]Y [Yc =

M

Ei† ∩ 0 ]X [Yb, O]Y [Yc .

i

Proof b ⊗ Q, 0(]Y [ b, j † O]Y [ c) = Put x = x1 · · · xd ; then we have 0(]Y [Yb, O]Y [Yc) = R Y Y n p \ ⊗ Q. Assume that d lim ( R[t]/(x t − π)) ⊗ Q, and 0(]X [Yb, O]Y [Yc) = R[1/x] n→∞ L \ If a 6 ≡ 0 (mod π), then x m a is contained in R modulo π for a ∈ ( i Si ) ∩ R[1/x].


407

some nonnegative integer m. Since xi is a prime divisor in R/π and x1 , . . . , xd are Q L coordinates, we have ( xim i )R/π = R/π ∩ ( i xim i Si /π). Hence, there exists an L \ Continuing this construction, element a0 ∈ R such that a−a0 ∈ π(( i Si )∩ R[1/x]). P n b Since R b is p-adically complete, we have we can write a = π an for an ∈ R. M 0 ]Y [Yb, O]Y [Yc = SQi ∩ 0 ]X [Yb, O]Y [Yc i

=

M

SQi ∩ 0 ]Y [Yb, j † O]Y [Yc .

i

L \ where O † is the valuation ring of E † , and Assume that a ∈ ( i OE † ) ∩ R[1/x], Ei i P i assume that there exists c > 0 and η < 1 such that, if we develop a = n∈Z αin xin in OE † , then |αin | < cη−n (n < 0). If a 6 ≡ 0 (mod π), then there is a nonnegative i integer m 0 such that x m 0 a is included in R modulo π and not included in x R modulo L \ π. Then we can choose a0 ∈ R such that x m 0 a − a0 ∈ π(( O † ) ∩ R[1/x]), i

Ei† .

Ei

in each If we see a − 0| < 0 as an element in OE † , then the i coefficients of its Laurant series satisfy the same condition as those of a. Continuing P n −m n a in R[1/x] \ such that this construction, we have a development a = ∞ n n=0 π x n −m n {m n } is a sequence of nonnegative integers, an ∈ R, and |π x an | < cηm n . Hence, † we know that a is a section of j O]Y [ c on the strict neighborhood of ]X [ b in Yban |x −m 0 a

cηm 0

which is defined for |x| = η.

x −m 0 a

Y

Y

Keep the situation as above. Let M be an object in Isoc† ((X, Y )/K ) (resp., Isoc(Y/K )). Define an Ei† -module (resp., an SQi -module) by Mi = 0 ]Y [Yb, M ⊗0(]Y [ c, j † O]Y [ ) Ei† Y c Y resp., Mi = 0 ]Y [Yb, M ⊗0(]Y [ c, j † O]Y [ ) SQi Y

c Y

and a connection on Mi by ∂ d da (m ⊗ a) = a∇ (m) + m ⊗ ∇ d xi ∂ xi d xi for m ∈ Mi and a ∈ Ei† (resp., a ∈ SQi ). Then Mi is an object in M∇ † (resp., M∇SQ i ) Ei

and the connection of M satisfies condition (OC) (resp., (C)) by [3, Prop. 2.2.13] b (resp., [3, Cor. 2.2.14]). In the case of F a -isocrystals, we assume furthermore that R p allows a lift of Frobenius map σ with σ (xi ) = xi for all i which is compatible with the Frobenius on K . Such a Frobenius σ always exists under our assumption, and σ can be extended on SQi . Hence, the Frobenius structure on M induces a Frobenius structure ϕ on Mi by the scalar extension, and Mi is an object in MF∇ † (resp., Ei ,σ

M∇SQi ,σ ) for each i. If M is unit-root, then Mi is e´ tale (see the definition in Sec. 2.1).

408

NOBUO TSUZUKI

PROPOSITION 6.1.2 With the notation as above, suppose that the natural map H 0 Ei† [∇], Mi → H 0 Ei [∇], Mi ⊗E † Ei i 0 0 resp., H SQi [∇], Mi → H Ei [∇], Mi ⊗ SQi Ei

is bijective for all i. Then the natural map H 0 (X, Y )/K , M → H 0 (X/K , j ∗ M ) resp., H 0 (Y/K , M ) → H 0 (X/K , j ∗ M ) is bijective. The same holds for F-isocrystals. Proof Consider the following commutative diagram: 0

→

0

→

0 ]Y [Y c, M ↓ L i Mi

→ →

∗ 0 ]X [Y c, j M ↓ L i Mi ⊗E † Ei i

→ →

∗ 0 ]X [Y c, j M / 0 ]Y [Y c, M ↓ L † i M ⊗E † Ei /Ei

→

0

→

0

i

0(]Y [Yb, M ) is flat over 0(]Y [Yb, j † O]Y [Y ), and we have 0 ]X [Yb, j ∗ M ∼ = 0 ]Y [Yb, M ⊗0(]Y [ c, j † O]Y [ ) 0 ]X [Yb, O]Y [Yc Y

c Y

by the same reason as in Section 4.3. So both horizontal rows in the diagram above are exact, and the right vertical arrow is injective by Lemma 6.1.1. Let f : O]X [Yc → j ∗ M be a horizontal section in Isoc(X/K ). The imL age of f (1) through the middle vertical arrow is contained in i Mi by our hypothesis, and f (1) is contained in 0(]Y [Yb, M ) by the diagram chase. Hence, f comes from a j † O]Y [Yc-homomorphism f 0 : j † O]Y [Yc → M , and f 0 commutes with connections and is horizontal by [3, Prop. 2.2.7]. Therefore, the natural map H 0 ((X, Y )/K , M ) → H 0 (X/K , j ∗ M ) is bijective. The rest is the same. COROLLARY 6.1.3 Suppose that Conjecture 2.3.1 is valid. Then the natural map H 0 (X, Y )/K , M → H 0 (X/K , j ∗ M )

is bijective for any object M in Isoc† ((X, Y )/K ). The same holds for F-isocrystals. 6.2 Let j : X → Y be an open immersion of smooth separated schemes of finite type over Spec k. Assume that Z = Y − X is a strict normal crossing divisor in Y ; that is, assume


409

S that Z is a reduced scheme and, if we denote Z = i∈I Z i the sum of irreducible T components, the closed subscheme Z J = j∈J Z i is regular of codimension ]J for any subset J ⊂ I . PROPOSITION 6.2.1 Assume the situation is as above. If Conjecture 2.3.1 (resp., Conjecture 2.3.2 for a positive integer a) is true, then the functor j ∗ : Isoc† (X, Y )/K → Isoc(X/K ) resp., j ∗ : F a -Isoc† (X, Y )/K → F a -Isoc(X/K )

is fully faithful. Proof By Corollary 3.2.3, the problem is local on both X and Y . We may assume that Y is affine irreducible and that there is a system of coordinates y1 , . . . , yn of Y (n is the dimension of Y over Spec k) such that each component Z i of Z is defined by yi for 1 5 i 5 d. Then there is an affine smooth scheme Y of finite type over Spec V with Y = Y ×Spec V Spec k by [10, Th. 6]. Choose a lift xi of yi in 0(Y , OY ), and define f : Y → AnV by the system of sections {xi }. If one takes a covering of the e´ tale locus of f which consists of open affine subschemes of Y , then it covers Y since f is e´ tale on Y . Therefore, we may assume that the situation is as in Proposition 6.1.2 by Corollary 3.2.3. This completes the proof. Theorem 2.2.2 implies the following proposition. 6.2.2 With the notation as above, the functor PROPOSITION

j ∗ : F a -Isoc (X, Y )/K

0

→ F a -Isoc(X/K )0

is fully faithful. 6.3 Let j : X → Y be an open immersion of smooth separated k-schemes of finite type such that X is dense in Y . Put Z = Y − X . Let a be a positive integer. THEOREM 6.3.1 With the notation as above, the functor

j ∗ : F a -Isoc(Y/K ) → F a -Isoc(X/K ) is fully faithful.

410

NOBUO TSUZUKI

Proof The faithfulness of j ∗ follows from the fact that a convergent isocrystal is locally free. We prove the fullness of j ∗ by induction on the dimension of Z . By Proposition 3.1.2, we can replace k by a finite extension k 0 ; therefore, we may assume that Z is reduced and that there is an open subscheme U of Y such that U ⊃ X , U ∩ Z is smooth over k and the dimension of Z 1 = Y − U in Z is less than the dimension of Z . By the induction hypothesis, we may assume that Z = Z 1 . By Corollary 3.2.3, we may assume that both X and Y are affine and that Z is defined by a part x1 , . . . , xr of coordinates xi of X . By [10, Th. 6], there is an open immersion X → Y of affine smooth schemes of finite type over Spec V with X = X ⊗V k, Y = Y ⊗V k. Denote by Z 2 the smooth divisor defined by x1 = 0 in Y , and put U2 = Y − Z 2 . By Corollary 3.2.3, we may assume that Z 2 is connected. Then we have a natural sequence F a -Isoc(Y/K ) → F a -Isoc(X/K ) → F a -Isoc(U2 /K ). The composite map is bijective by Theorem 2.2.3 and Corollary 6.1.2. Hence, j ∗ is full. 6.4 Consider the following commutative diagram of separated k-schemes of finite type: U 1 → X 1 → Y1 u↓ v↓ ↓w U 2 → X 2 → Y2 where we assume that X 1 and X 2 are smooth over Spec k, Y1 and Y2 are normal, all horizontal arrows are open immersions, u is finite e´ tale, and w is proper. Denote by j X 1 (resp., j X 2 , jU1 , jU2 ) the open immersion X 1 → Y1 (resp., X 2 → Y2 , U1 → Y1 , U2 → Y2 ). By [3, Déf. 2.3.2(iv)], there is a natural commutative diagram Isoc† Isoc†

(X 2 , Y2 )/K w∗ ↓

jU†

→

2

Isoc† (U2 , Y2 )/K ↓ w∗

(X 1 , Y1 )/K

jU†

1

Isoc† (U1 , Y1 )/K

→

of an inverse image of categories of overconvergent isocrystals. PROPOSITION 6.4.1 Assume that Y1 is smooth over Spec k and that U2 is dense in Y2 . If M is an overconvergent F a -isocrystal on X 2 /K along Y2 − X 2 , then the overconvergent isocrystal † w∗ M on X 1 /K along Y1 − X 1 is an F a -isocrystal such that jU† 1 w∗ M ∼ = w∗ jU2 M


411

as overconvergent F a -isocrystals U1 /K along Y1 − U1 . Moreover, if M is unit-root, then w∗ M is also. Proof Denote by σ the absolute Frobenius on all schemes of characteristic p. Since σ a ◦w = w ◦ σ a , the horizontal isomorphism of Frobenius structure 8 : jU† 1 (σ a )∗ w∗ M → jU† 1 w∗ M in Isoc† ((U1 , Y1 )/K ) extends uniquely to a horizontal isomorphism 8 : (σ a )∗ w∗ M → w∗ M by Theorem 4.1.1. 6.4.2 With the notation as above, assume furthermore that Y1 and Y2 are smooth over Spec k and that U2 is dense in Y2 . If Conjecture (TC) is true for the pair (X 1 , Y1 ), it is also true for the pair (X 2 , Y2 ). The same holds for Conjectures (TF) and (TU). PROPOSITION

Proof By the commutativity of w∗ (resp., w∗ ) and j X† 1 (resp., jU† 1 , j X∗ 1 ), we have a natural diagram H 0 (U2 , Y2 )/K , jU† 2 M ad ↓

∼ = H 0 (X 2 , Y2 )/K , M

→ H 0 (X 2 /K , j X∗ 2 M ) → H 0 (U2 /K , jU∗ 2 M )

∼ = H 0 (X 1 /K , j X∗ 1 w∗ M ) → H 0 (U1 /K , jU∗ 1 w ∗ M )

↓

H 0 (U1 , Y1 )/K , jU† 1 w ∗ M

∼ = H 0 (X 1 , Y1 )/K , w∗ M

↓

tr ↓ H 0 (U2 , Y2 )/K , jU† 2 M

↓ ad

↓ tr

→

H 0 (U2 /K , jU∗ 2 M )

such that all squares are commutative by Proposition 5.1.2. The first horizontal arrows both in the top line and in the middle line are bijective by Theorem 4.1.1. Since both composite maps of the left and right vertical arrows are deg(U1 /U2 ) × id by Proposition 5.2.1, the bijection H 0 ((X 1 , Y1 )/K , w∗ M ) ∼ = H 0 (X 1 /K , j X∗ 1 w∗ M ) implies the bijectivity of the map H 0 (X 2 , Y2 )/K , M → H 0 (X 2 /K , j X∗ 2 M ). The rest is the same. 6.5 Let j : X → Y be an open immersion of separated k-schemes of finite type such that X is smooth over Spec k and X is dense in Y . PROPOSITION 6.5.1 Assume that Conjecture 2.3.1 (resp., Conjecture 2.3.2) is true. If Y is smooth over k, then Conjecture (TC) (resp., (TF)) is true for the pair (X, Y ).

412

NOBUO TSUZUKI

Proof There exist a smooth scheme Y 0 over Spec k, a proper surjective morphism w : Y 0 → Y , and an open subscheme U of X over k with X 0 ⊂ w−1 (X ) and U 0 = w−1 (U ) such that Y 0 − X 0 is a strict normal crossing divisor in Y 0 and U 0 is finite e´ tale over U by the alteration theorem (see [8, Th. 4.1]). The assertion follows from Propositions 6.2.1 and 6.4.2. COROLLARY 6.5.2 With the notation as above, assume furthermore that there are an open immersion j1 : X → Y1 of smooth k-schemes of finite type and a proper morphism w : Y1 → Y with w ◦ j1 = j. If Conjecture 2.3.1 (resp., Conjecture 2.3.2) is true, then conjecture (TC) (resp., (TF)) is true for the pair (X, Y ).

Proof By [3, Th. 2.3.5], it is sufficient to see the assertion for the pair (X, Y1 ). The assertion follows from Proposition 6.5.1. 6.6 Proof of Theorem 1.2.2 In the case where X = U , one can prove the assertion by an argument similar to that of Proposition 6.5.1 using Proposition 6.2.2. In general, consider the natural commutative diagram 0 0 j∗ F a -Isoc† (U, Y )/K → F a -Isoc† (U, X )/K & . F a -Isoc(U/K )0 of categories. Since both slanting arrows are fully faithful by the previous step, the functor j ∗ is also. This completes the proof. 6.6.1 Let j : X → Y be an open immersion of separated k-schemes of finite type such that X is smooth over Spec k, and let U be an open subscheme of X which is dense in Y . Assume that there are an open immersion j1 : X → Y1 of smooth k-schemes of finite type and a proper morphism w : Y1 → Y with w ◦ j1 = j. Then the functor 0 0 j ∗ : F a -Isoc† (U, Y )/K → F a -Isoc† (U, X )/K COROLLARY

is fully faithful.


413

7. Finite monodromy theorem of unit-root F-isocrystals 7.1 a a Let K σ be the fixed subfield of K by σ a ; then [K σ : Q p ] < ∞. Let T be a scheme, and let t¯ be a geometric point of T . Denote by Rep K σ a (π1 (T, t¯)) the category of a K σ -vector spaces of finite dimension with a continuous π1 (T, t¯)-action. The category Rep K σ a (π1 (T, t¯)) is independent of the choice of geometric point t¯ up to the canonical equivalence if T is connected. First we recall the theorem of Crew [5, Th. 2.1]. THEOREM 7.1.1 Assume that k includes F pa . Let X be a smooth connected scheme of finite type over Spec k, and let x¯ be a geometric point of X . Then there is a natural equivalence of categories G X,x¯ : Rep K σ a π1 (X, x) ¯ → F a -Isoc(X/K )0 .

The functor G X,x¯ is independent of the choice of geometric point x¯ of X up to the canonical equivalence. The natural equivalence means that G X,x¯ commutes with tensor products, duals, and the functor of inverse image for any morphism of smooth k-schemes with a base point. 7.2 In this section we assume that k includes F pa . Denote by O K σ a the ring of integers a in K σ . Let Y be a smooth connected separated scheme of finite type over Spec k, let j : X → Y be a dense open subscheme of Y over Spec k, denote by η the generic point of Y , and fix a geometrically generic point η¯ of Y . Let M be an object in F a -Isoc† ((X, Y )/K )0 , and let V be the object in Rep K σ a (π1 (X, η)) ¯ which ∗ corresponds to j M under the functor G X,η¯ in Theorem 7.1.1. PROPOSITION 7.2.1 With the notation as above, assume furthermore that there exists an O K σ a -lattice 0 in V which is stable under the action of π1 (X, η) ¯ such that π1 (X, η) ¯ acts trivially on a 0 0/2 p0. Then there exists a unique object N in F -Isoc(Y/K ) such that

M ∼ = j †N .

Proof The uniqueness follows from Theorem 4.1.1. Suppose that the representation π1 (X, η) ¯ → GL(V ) factors through the surjection π1 (X, η) ¯ → π1 (Y, η). ¯ If we denote by N the object in F a -Isoc(Y/K )0 corresponding to the induced representation

414

NOBUO TSUZUKI

V of π1 (Y, η) ¯ under the functor G Y,η¯ , then we have an isomorphism j ∗ M ∼ = j ∗ N in a 0 F -Isoc(X/K ) by Theorem 7.1.1. Corollary 1.2.3 implies the desired isomorphism M ∼ = j †N . Therefore, it remains to prove that the representation π1 (X, η) ¯ → GL(V ) factors through the surjection π1 (X, η) ¯ → π1 (Y, η). ¯ By the theorem of purity (see [12, X, Cor. 3.3]), we deduce the factorization from the following lemma. LEMMA 7.2.2 Under the assumption in Proposition 7.2.1, let t be a point of height 1 in Y . Then the representation π1 (Spec OY,η , η) ¯ → π1 (X, η) ¯ → GL(V ) factors through the surjection π1 (Spec OY,η , η) ¯ → π1 (Spec OY,t , η). ¯

Proof In the case where dim X = 0, there is nothing to prove, so we assume that dim X = 1. e the field of fractions of the completion of OY,t at the valuation Denote by F (resp., F) defined by the point t of height 1 (resp., an extension of F as a complete discrete e is an valuation field with the same uniformizer of F such that the residue field of F e e e algebraic closure of that of F). Let k (resp., K ) be the residue field of F (resp., a field esep of F e which includes η, W (e k) ⊗W (k) K ). Fix a separable closure F ¯ and let F sep be esep . Let Fn (resp., F en ) be the fixed field in F sep (resp., the separable closure of F in F sep e F ) by the kernel of the homomorphism Gal(F sep /F) → GL(0/2 pπ n 0) (resp., esep / F) e → GL(0/2 pπ n 0)) for any positive integer n (π is a uniformizer of the Gal( F e, K , and K σ ). To prove the assertion, we have only to show that the restricted triple K representation Gal(F sep /F) → π1 (Spec OY,η , η) ¯ → GL(V ) is unramified. This is equivalent to the assertion that the extension Fn over F is unramified for all n. First esep / F) e acts trivially on V , in other words, that F en = F e for all n. we show that Gal( F Choose a smooth affine scheme Spec R of finite type over Spec V such that Spec R ⊗V k is an open integral subscheme of Y which contains the point t of height 1. If we choose a sufficiently small affine scheme Spec R, we may assume that there is a system x1 , . . . , xn of coordinates of Spec R over Spec V such that the closure of the point t is defined by the image of x1 in Spec R ⊗V k. Then we can define a lift of b with σ (xi ) = x p for all i. Frobenius σ on the p-adic formal scheme Spf R i Let Rtˆ be the x1 -adic completion of R, and let Rd X,tˆ be the p-adic completion of d Rtˆ[1/x1 ]. Then Rd X,tˆ is a regular local domain of the maximal ideal (π ), and R X,tˆ ⊗V b d k = F. The Frobenius σ on R extends uniquely to R X,tˆ. We define two rings as


415

follows: E Fe =

∞ n X

o e, sup |an | < ∞, |an | → 0 (n → −∞) an x1n an ∈ K

n=−∞

resp., E Fe† =

∞ n X

o an x1n ∈ E Fe |an |ξ n → 0 (n → −∞) for some ξ < 1 .

n=−∞

E Fe (resp., E Fe† ) is a complete (resp., Henselian) discrete valuation field with residue e We define a Frobenius on E e (resp., E † ) by x1 7 → x p and as usual on K e. field F. F

1

e F

Then one can easily see that there are continuous injective ring homomorphisms h1i \ → Rd R X,tˆ → OE Fe x1 which induce the inclusions in the special fiber and commute with the Frobenius map. We also denote the Frobenius on E Fe (resp., E Fe† ) by σ . One can easily see that the above inclusions commute with the derivation ∂/∂ x1 . Moreover, the inclusions above induce the commutative diagram 0 ]Y [Spf Rb, j † O]Y [Spf Rb → E Fe† ↓ ↓ 0 ]X [Spf Rb, O]X [Spf Rb → Rd [1/ p] → E e X,tˆ F All maps preserve norms and commute with Frobenius σ and the derivation ∂/∂ x1 . Hence, we have a commutative diagram 0 et → MF∇,´ F a -Isoc† (X, Y )/K † a j∗ ↓ F a -Isoc(X/K )0

E Fe,σ

↓ →

t MFe´Rd a X,tˆ [1/ p],σ

→

(*)

MFeE´ te,σ a F

of functors by the extension of the scalar (see the definition of connection on E Fe† in t Sec. 6.1 and the definition of the category MFe´Rd in Rem. 2.1.3). On E Fe we [1/ p],σ a X,tˆ

can forget the structure of connections by Lemma 2.1.1. By the construction of the functor from p-adic representations to Frobenius modules (see [13, Sec. 4.1], [5, Sec. 2], and see also Rem. 1.1.3), we have a commutative diagram esep / F) e Rep K σ a π1 (X, η) ¯ → Rep K σ a Gal(F sep /F) → Rep K σ a Gal( F G X,η¯ ↓ ↓ ↓ Dσ ´t e a 0 F -Isoc(X/K ) → MF Rd [1/ p],σ a → MFeE´ te,σ a X,tˆ

F

(**)

416

NOBUO TSUZUKI

Here both functors in the top horizontal sequence are defined by the restriction. It is known that all vertical functors are equivalences of categories by Theorem 7.1.1 and [11, A, Prop. 1.2.6]. To prove the assertion, we need two lemmas. 7.2.3 ([18, Lem. 5.2.4]) † et Let N be an object in MF∇,´ † . If there exists a basis of N over E F e whose Frobenius LEMMA

E Fe,σ

matrix with respect to the basis satisfies the condition |A − 1| < | p|1/( p−1) , then N is trivial; that is, N is a finite sum of the trivial object E Fe† . Note that |2 p| < | p|1/( p−1) . Since Dσ can be constructed using integral structures and since E Fe is the p-adic completion of E Fe† , Lemma 7.2.3 implies the following lemma. LEMMA 7.2.4 esep / F)) e which satisfies the following conditions. Let W be an object of Rep K σ a (Gal( F esep / F)-stable e esep / F) e acts triv(1) W has a Gal( F O K σ a -lattice 1 such that Gal( F ially on 1/2 p1. et ∼ (2) W is overconvergent; that is, there is an object N in MF∇,´ † a with Dσ (W ) =

ν∗ N . esep / F) e acts trivially on W . Then Gal( F

Et ,σ

We continue the proof of Lemma 7.2.2. By the assumption, the corresponding local object of our overconvergent unit-root F-isocrystal M satisfies the assumption of Lemma 7.2.4 by the commutativity of diagrams (∗) and (∗∗). Hence, Lemma 7.2.4 esep / F) e acts trivially on V . implies that Gal( F To complete the proof, we have only to check that the residue extension of Fn b for over F is separable since the valuation group of F coincides with that of F b Fn ⊂ Fn . By the construction of the quasi-inverse of Dσ (see [13, Sec. 4.1], [11, A, Prop. 1.2.6]), we know that the extension Fn+1 over Fn is obtained by several Artin-Schreier extensions that are determined by the Frobenius structure for any n. Hence, the residue extension of Fn+1 over Fn is separable for any n. This completes the proof of Lemma 7.2.2. This also completes the proof of Proposition 7.2.1.


417

7.3 Proof of Theorem 1.3.1 Since it is allowable to change a base Spec k by a finite extension of k, we may assume that k includes F pa . We may also assume that Y is normal and integral. (Hence, X is connected.) Let M be an object in F a -Isoc† ((X, Y )/K )0 , and denote by V the corresponding representation of π1 (X, η) ¯ to j † M under the equivalence of categories in Theorem 7.1.1, where η¯ is a geometrically generic point of X . Choose an O K σ a submodule 0 in V which is stable under the action of π1 (X, η). ¯ Take a finite e´ tale covering X 1 of X which corresponds to the kernel of π1 (X, η) ¯ → GL(0/2 p0), and let Y1 be the normalization of Y in X . By the alteration theorem (see [8, Th. 4.1]), we can find a proper surjective morphism w : Y 0 → Y over Spec k such that Y 0 is smooth of finite type over Spec k and such that X 0 = w−1 (X ) is a dense open subscheme of Y 0 . If Y 0 is not connected, one considers the problem of extension of w∗ M as a convergent F-isocrystal on each connected component of Y 0 . Hence, we may assume that Y 0 is connected. By the commutativity of the diagram Rep K σ a π1 (X, η) ¯ v∗ ↓

Rep K σ a π1 (X 0 , η) ¯

G X,η¯

→

G X 0 ,η¯

→

F a -Isoc(X/K )0 ↓ v∗ F a -Isoc(X 0 /K )0

(see Th. 7.1.1), where v = w| X 0 , the inverse image w∗ M corresponds to the restricted representation π1 (X 0 , η) ¯ → π1 (X, η) ¯ → GL(V ). 0 is a π1 (X 0 , η)-stable ¯ 0 O K σ a -lattice, and π1 (X , η) ¯ acts trivially on 0/2 p0 since it acts through π1 (X 1 , η). ¯ Therefore, the assertion follows from Proposition 7.2.1. Remark 7.3.1 If one allows the resolution of singularity of Y1 in the proof of Theorem 1.3.1, then one can find a proper surjective generically e´ tale morphism w : Y 0 → Y such that X 0 = X 1 is finite e´ tale over X . Acknowledgments. The author would like to thank F. Baldassarri, P. Berthelot, B. Chiarellotto, A. J. de Jong, S. Matsuda, and T. Saito for useful conversations and advice. This paper was written during his stay at l’Institut Henri Poincaré in the semester of “Cohomologies p-adiques et applications arithmetiques” in 1997. He would like also to thank members of the institute. References [1]

P. BERTHELOT, Théorie de Dieudonné sur un anneau de valuation parfait, Ann. Sci.

418

[2]

[3]

[4]

[5]

[6] [7]

[8] [9]

[10] [11]

[12]

[13]

[14] [15] [16]

[17]

NOBUO TSUZUKI

´ Ecole Norm. Sup. (4) 13 (1980), 225–268. MR 82b:14026 387 , Finitude et pureté cohomologique en cohomologie rigide, avec un appendice en Anglais par Aise Johan de Jong, Invent. Math. 128 (1997), 329–377. MR 98j:14023 388, 395 , Cohomologie rigide et cohomologie rigide a` support propre: Première partie, preprint, Institut de Recherche Mathématique de Rennes, no. 96-03, 1996, http://maths.univ-rennes1.fr/˜berthelo/ 386, 387, 394, 395, 396, 397, 398, 401, 402, 403, 404, 407, 408, 410, 412 P. BERTHELOT and W. MESSING, “Théorie de Dieudonné cristalline, I” in Journées de Géométrie Algébrique de Rennes (Rennes, France, 1978), Vol. I, Astérisque 63, Soc. Math. France, Montrouge, 1979, 17–37. MR 82f:14041 387 R. CREW, “F-isocrystals and p-adic representations” in Algebraic Geometry (Brunswick, Maine, 1985), Proc. Sympos. Pure Math. 46, Part 2, Amer. Math. Soc., Providence, 1987, 111–138. MR 89c:14024 386, 387, 388, 394, 413, 415 ´ , F-isocrystals and their monodromy groups, Ann. Sci. Ecole Norm. Sup. (4) 25 (1992), 429–464. MR 94a:14021 402, 403 , Finiteness theorems for the cohomology of an overconvergent isocrystal on a ´ curve, Ann. Sci. Ecole Norm. Sup. (4) 31 (1998), 717–763. MR 2000a:14023 388 ´ A. J. DE JONG, Smoothness, semi-stability and alterations, Inst. Hautes Etudes Sci. Publ. Math. 83 (1996), 51–93. MR 98e:14011 412, 417 , Homomorphisms of Barsotti-Tate groups and crystals in positive characteristic, Invent. Math. 134 (1998), 301–303, MR 2000f:14070a; Erratum, Invent. Math. 138 (1999), 225. MR 2000f:14070b 387, 392, 393 R. ELKIK, Solutions d’équations a` coefficients dans un anneau hensélian, Ann. Sci. ´ Ecole Norm. Sup. (4) 6 (1973), 553–603. MR 49:10692 401, 409, 410 J.-M. FONTAINE, “Représentations p-adiques des corps locaux, I” in The Grothendieck Festschrift, Vol. II, Progr. Math. 87, Birkhäuser, Boston, 1990, 249–309. MR 92i:11125 390, 391, 416 A. GROTHENDIECK and M. RAYNAUD, Revêtements e´ tales et groupe fondamental, Séminaire de Géométrie Algébrique du Bois Marie 1960–1961 (SGA 1), Lecture Notes in Math. 224, Springer, Berlin, 1971. MR 50:7129 414 N. M. KATZ, “ p-adic properties of modular schemes and modular forms” in Modular Functions of One Variable, III (Antwerp, 1972), Lecture Notes in Math. 350, Springer, Berlin, 1973, 69–190. MR 56:5434 391, 415, 416 H. MATSUMURA, Commutative Algebra, Benjamin, New York, 1970. MR 42:1813 401 A. OGUS, F-isocrystals and de Rham cohomology, II: Convergent isocrystals, Duke Math. J. 51 (1984), 765–850. MR 86j:14012 J. T. TATE, “ p-divisible groups” in Proceedings of a Conference on Local Fields (Driebergen, Netherlands, 1966), ed. T. A. Springer, Springer, Berlin, 1967, 158–183. MR 38:155 387 N. TSUZUKI [T. NOBUO], The overconvergence of morphisms of e´ tale ϕ-∇-spaces on a local field, Compositio Math. 103 (1996), 227–239. MR 97m:14024 387, 392


[18] [19]

419

, Finite local monodromy of overconvergent unit-root F-isocrystals on a curve, Amer. J. Math. 120 (1998), 1165–1190. MR 99k:14038 388, 416 , Slope filtration of quasi-unipotent overconvergent F-isocrystals, Ann. Inst. Fourier (Grenoble) 48 (1998), 379–412. MR 99e:14023 389, 393, 394

Department of Mathematics, Faculty of Science, Hiroshima University, Higashi-Hiroshima 739-8526, Japan; [email protected]


TORIC HILBERT SCHEMES IRENA PEEVA and MIKE STILLMAN

Abstract We introduce and study the toric Hilbert scheme that parametrizes all ideals with the same multigraded Hilbert function as a given toric ideal. 1. Introduction The classical Hilbert scheme, introduced by A. Grothendieck [Gr], parametrizes subschemes of Pr with fixed Hilbert polynomial. We introduce and study the toric Hilbert scheme that parametrizes all ideals with the same multigraded Hilbert function as a given toric ideal. Note that the Hilbert polynomial does not exist in the toric (multigraded) case. A toric variety is a variety parametrized by finitely many monomials. Let n and d be positive integers with n > d, and let A = {a1 , . . . , an } be a subset of Nd \ {0} with n different vectors. Suppose that the matrix with columns ai has rank d. Denote by NA the subsemigroup of Nd spanned by A . Consider the polynomial ring S = k[x1 , . . . , xn ] over a field k generated by variables x1 , . . . , xn in Nd degrees a1 , . . . , an , respectively. The toric ideal IA is the kernel of the homomora phism k[x1 , . . . , xn ] → k[t1 , . . . , td ] mapping xi to tai = t1 i1 · · · tdaid for 1 ≤ i ≤ n. d This is a prime N -graded ideal. A homogeneous ideal M is called A -graded if, for all b ∈ Nd , ( 1 if b ∈ NA , dimk (S/M)b = 0 otherwise. This means that S/M has the same multigraded Hilbert function as the toric ring S/IA . The paradigms of A -graded ideals are the toric ideal and its initial ideals. The study of A -graded ideals was initiated by V. Arnold [Ar]; he discovered that in the case d = 1, n = 3 the structure of such ideals is encoded into continued fractions; further work in this case was completed by E. Korkina, G. Post, and M. Roelofs [Ko], [KPR]. General A -graded algebras (for arbitrary d and n) are studied in [St2]. DUKE MATHEMATICAL JOURNAL c 2002 Vol. 111, No. 3, Received 10 December 1999. Revision received 31 January 2001. 2000 Mathematics Subject Classification. Primary 13A02; Secondary 14J10. Peeva’s work supported by a Sloan Research Fellowship. Both authors’ work partially supported by the National Science Foundation. 419

420

PEEVA and STILLMAN

In Section 4 we construct the toric Hilbert scheme HA that parametrizes all ideals with the same multigraded Hilbert function as IA . By Corollary 4.6, HA has S a finite open affine cover i Ui such that each Ui is defined by binomial equations. We prove the following. 1.1 satisfies the universality property in Theorem 4.4.

THEOREM

HA

A significant virtue of universality is that it makes it possible to describe easily the tangent space to HA in Theorem 5.1. For a relation to moduli spaces of abelian varieties, see [Al]. At first glance, toric Hilbert schemes might seem very similar to classical Hilbert schemes. However, many ideas and techniques used by R. Hartshorne, A. Reeves, M. Stillman, K. Pardue, and others (cf. [Ha], [Re], [RS], [Pa]) to study classical Hilbert schemes are not applicable to toric Hilbert schemes. For example, • changing the variables preserves the Hilbert polynomial, so it can be used to study a classical Hilbert scheme; however, changing the variables usually does not preserve the multigraded Hilbert function, so it cannot be used to study a toric Hilbert scheme; • on a classical Hilbert scheme there exists one special point—the lexicographic ideal—which is often very useful; usually there is no lexicographic ideal on a toric Hilbert scheme. In particular, Hartshorne’s proof that the classical Hilbert scheme is connected cannot be applied and the following question is open: Is the toric Hilbert scheme connected? Most of the tools and ideas used in our paper are specific for the toric case. The lexicographic ideal (the special point on a classical Hilbert scheme) is a smooth point by [RS]. Although there is no lexicographic ideal on a toric Hilbert scheme, we have another special point on it: the toric ideal. We obtain the following theorem. 1.2 There exists exactly one component containing the point [IA ]. If char (k) = 0, then this component is reduced and so the point [IA ] on HA is smooth. THEOREM

In Sections 6 and 7 we consider the case when codim (S/IA ) = 2. The main result in [GP] generalizes a result of Arnold, Korkina, Post, and Roelofs and can be reformulated as follows. THEOREM

1.3

TORIC HILBERT SCHEMES

421

Suppose that codim (S/IA ) = 2. The toric Hilbert scheme has one component. It is the closure of the orbit of the toric ideal under the torus action. Each point on such a toric Hilbert scheme is an initial (not necessarily monomial) ideal of an ideal obtained from IA by scaling the variables. We obtain a thorough description of the structure of the initial ideals of IA in Theorem 6.2. Using this description, local equations from [PSti], and results from [GP] and [PStu2], we prove the following results. THEOREM 1.4 Suppose that codim (S/IA ) = 2. The toric Hilbert scheme is 2-dimensional and smooth.

1.5 If codim (S/IA ) = 2, then HA is the toric variety of the Gröbner fan of IA . COROLLARY

In particular, we show that the toric Hilbert scheme is reduced if codim(S/IA ) = 2. Note that there are no assumptions on the characteristic of k in Theorem 1.4. In contrast to Theorem 1.4, the structure of the classical Hilbert scheme of a codimension 2 toric variety is usually very complicated. The best known result describes the scheme for the twisted cubic curve. The toric Hilbert scheme and the classical Hilbert scheme of the twisted cubic curve compare as follows: R. Piene and M. Schlessinger [PiSc] proved that the classical Hilbert scheme of the twisted cubic curve has two components of dimensions 12 and 15; each component is smooth, but the scheme is not; the two components intersect transversally, and their intersection is smooth of dimension 11. By Theorems 1.3 and 1.4, the toric Hilbert scheme of the twisted cubic curve has one component and is 2-dimensional and smooth. B. Sturmfels has introduced a quotient of a polynomial ring parametrizing all ideals with the same Hilbert function as IA in the unpublished paper [St1]; he uses quadratic equations. We are introducing and using different defining equations; our equations involve fewer variables, are determinantal, make it possible to prove the universality property, and make it possible to obtain local equations for HA . Using the local equations obtained in [PSti], D. MacLagan is studying whether the two schemes (Sturmfels’s and ours) are isomorphic. 2. Generators of weakly A -graded or A -graded ideals In this section we study the properties of minimal generators of weakly A -graded or A -graded ideals.

422

PEEVA and STILLMAN

A homogeneous ideal E is weakly A -graded if, for all b ∈ Nd , ( 1 if b ∈ NA , dimk (S/E)b ≤ 0 otherwise. Note that a weakly A -graded ideal is generated by binomials (that is, polynomials with at most two terms). Proposition 2.1 shows that a weakly A -graded ideal is generated by special binomials. A binomial u − v in the toric ideal IA is called primitive if there exist no proper monomial factors u 0 of u and v 0 of v such that u 0 − v 0 ∈ IA . The set of all primitive binomials is finite and is called the Graver basis (cf. [St2]). The structure of the weakly A -graded ideals is described by the following result. PROPOSITION 2.1 Let E be an ideal in S. The following are equivalent: (1) the ideal E is weakly A -graded; (2) if u − v is a primitive binomial, then there exists a (α : β) ∈ P1k such that αu − βv ∈ E.

The Graver basis in the case d = 1, n = 3 considered by [Ar], [Ko], and [KPR] is the star (see [Ko, Def. 2.9]); in this case, Proposition 2.1 corresponds to [Ko, Prop. 2.10]. Proof First, we show that (1) implies (2). Let u − v be a primitive binomial. Both monomials u and v have the same degree b ∈ Nd . As E is weakly A -graded, we have dim((S/E)b ) ≤ 1; hence, the images of u and v in S/E are k-linearly dependent. Suppose that (2) holds. Let u and v be two monomials of the same degree. We have that u −v ∈ IA . Suppose that u −v is not primitive. Then there exists a primitive binomial u 0 −v 0 ∈ I such that u = u 0 u 00 and v = v 0 v 00 . The Nd -degree of u 00 is smaller than the Nd -degree of u; hence, by induction we can assume that at least one of the elements u 00 , v 00 , u 00 −β 00 v 00 (for some nonzero constant β 00 ) is in E. Also, at least one of the elements u 0 , v 0 , u 0 − β 0 v 0 (for some nonzero constant β 0 ) is in E. Therefore, at least one of the elements u, v, u − β 0 β 00 v = u 00 (u 0 − β 0 v 0 ) + β 0 v 0 (u 00 − β 00 v 00 ) is in E. Hence, the images of u and v are k-linearly dependent in E. It follows that E is weakly A -graded. Let M be a monomial A -graded ideal. For b ∈ NA , we denote by sb the unique M-standard monomial in degree b, that is, the monomial of degree b that is not in M. If m is a monomial, we denote by sm the M-standard monomial in the degree of m.


423

If A is a commutative Noetherian ring, then we say that the quotient A[x1 , . . . , xn ]/J is A -homogeneous (or J is A -homogeneous) if the ideal J is homogeneous with respect to the grading deg(xi ) = ai . The following lemma is useful in the next section. 2.2 Let A be a commutative Noetherian ring, and let J be an A -homogeneous ideal in A[x1 , . . . , xn ]. Let M be an A -graded monomial ideal in k[x1 , . . . , xn ]. We denote by u a monomial in k[x1 , . . . , xn ]. (1) Suppose that for each primitive binomial of the form u − su there exists an αu ∈ A such that u − αu su ∈ J . Then for all b ∈ NA the A-module A[x1 , . . . , xn ]/J b is generated by the image of sb . (2) Suppose that for all b ∈ NA the A-module A[x1 , . . . , xn ]/J b is free of rank one. Furthermore, suppose that A[x1 , . . . , xn ]/J b is generated by sb whenever there is a primitive binomial of the form u − sb . Then the A-module A[x1 , . . . , xn ]/J b is generated by sb for all b ∈ NA , and there exist unique αu ∈ A such that J = u − αu su¯ | u − su¯ is primitive . LEMMA

(3)

Suppose that M is an initial ideal of IA . Suppose that for each primitive binomial of the form u − su , where u is a minimal monomial generator of M, there exists an αu ∈ A such that u − αu su ∈ J . Then for all b ∈ NA the A-module A[x1 , . . . , xn ]/J b is generated by the image of sb .

Proof First, we prove (1). Let m be a monomial. Set s = sm . We show that there exists an α ∈ A such that m − αs ∈ J . If m − s is primitive, then m − αm s ∈ J . Suppose that m − s is not primitive. By the definition of primitive elements, it follows that m = m 0 m 00 and s = s 0 s 00 , so that m 0 − s 0 is primitive. Since s is M-standard, it follows that both s 0 and s 00 are M-standard. By induction on the degree, we have that there exist α 0 , α 00 ∈ A such that m 0 − α 0 s 0 ∈ J and m 00 − α 00 s 00 ∈ J . Then m − α 0 α 00 s = m 00 (m 0 − α 0 s 0 ) + α 0 s 0 (m 00 − α 00 s 00 ) ∈ J. Now we prove (2). Let u − su be a primitive binomial. Since the free A-module A[x1 , . . . , xn ]/J deg(u) is generated by su and has rank one, it follows that there exists a unique αu ∈ A such that u − αu su ∈ J . Let J 0 be the ideal generated by the elements {u − αu su | u − su is primitive}. By (1) it follows that A[x1 , . . . , xn ]/J 0 b is generated by sb for each b ∈ NA . Since J 0 ⊆ J and A[x1 , . . . , xn ]/J b is a free module of rank one for each b ∈ NA , we conclude that J 0 = J .

424

PEEVA and STILLMAN

Finally, we prove (3). Let m be a monomial. Set s = sm . We show that there exists an α ∈ A such that m − αs ∈ J . By induction we assume that for each monomial m 0 ≺ m of the same degree as m (here ≺ is the monomial order with respect to which M is an initial ideal) there exists an α 0 ∈ A such that m 0 − α 0 s ∈ J . Since C = u − su | u − su is primitive and u is a minimal generator of M is a Gröbner basis of IA , there exist a binomial u − su ∈ C and a monomial h such that m = hu. Set m 0 = hsu . Then m 0 ≺ m. By induction we have that there exists an α 0 ∈ A such that m 0 − α 0 s ∈ J . Hence, m − α 0 αu s = h(u − αu su ) + αu (m 0 − α 0 s) ∈ J. COROLLARY 2.3 If E is an A -graded ideal in S, then there exist unique constants αuv ∈ k such that E is generated by {u − αuv v | u − v is primitive and v ∈ / E}.

In the terminology of [PSti], the corollary states that if E is A -graded, then there exists a minimal system of generators of E consisting of E-distinguished binomials. Proof Let M be a monomial initial ideal of E. Hence, M is A -graded. Applying Lemma 2.2(2), we get E = ( u − αu su | u − su is primitive ) for some αu ∈ A. Clearly, su ∈ / E since su ∈ / M. 3. Families of A -graded ideals Fix an A ⊂ Nd \ 0. In this section we introduce some constructions needed for the definition of the toric Hilbert scheme and the proof of universality. We use the following notation. Let b ∈ NA . The set of all monomials in S of degree b is called the fiber of b; we denote by |b| the number of monomials in the fiber and by m b1 , . . . , m b|b| the monomials. Denote by P the set of all b such that there exists a primitive binomial of degree b; we call P the set of primitive degrees. Definition 3.1 Suppose that S is a scheme (over k), and X is a subscheme of S × An . Let π : X −→ S be the projection map. We say that π : X −→ S is A -homogeneous if there exists an affine open cover {Ui } of S such that if Ui = Spec Ai , and π −1 (Ui ) = Spec Bi with Bi = Ai [x1 , . . . , xn ]/Ji , then Bi is A -homogeneous for all i (in particular, π is of finite type).


425

If π : X −→ S is A -homogeneous, then M X = Spec Lb , b∈NA

where each L b is a coherent OS -module and L 0 = OS . The coherent OS -module L b is globally generated by the sections that are the images of all the monomials in S of degree b. If E is a coherent OS -module, define the ith Fitting ideal of E, Fitti (E) ⊂ OS , to be the ideal sheaf that is locally the ith Fitting ideal (see, e.g., [Ei, Def. 20.4] for the definition of Fitting ideal). We extend the definitions of A -gradedness and weak A -gradedness for ideals to the corresponding notions for families as follows. Definition 3.2 Let π : X −→ S be A -homogeneous. (1) Call π weakly A -graded if Fitt1 (L b ) = OS , for all b ∈ NA . (2) Call π A -graded if L b is locally free of rank one, for all b ∈ NA . The map π is flat if and only if L b is locally free for all b. In this case, if S is connected, all fibers of π have the same multigraded Hilbert function: b 7→ rankL b . A basic property of Fitting ideals is that L b is locally free of rank one if and only if Fitt0 (L b ) = 0 and Fitt1 (L b ) = OS . Thus, if π is A -graded, it is both weakly A -graded and flat. Remark 3.3 The map π : X → S is A -graded if and only if it is A -homogeneous and flat, and all the fibers X P ⊂ Ank(P) are A -graded (here P ∈ S ). We use the following determinants in the definition of the toric Hilbert scheme. Definition 3.4 Let π : X −→ S be A -homogeneous. Define the ideal of O S , X dets(π) = Fitt0 (L b ). b∈NA

Definition 3.5 Let M ⊂ S = k[x1 , . . . , xn ] be a monomial A -graded ideal. For each b ∈ NA , denote by sb ∈ S the M-standard monomial. Let π : X −→ S be A -graded, and

426

let

PEEVA and STILLMAN

M X = Spec Lb . b∈NA

Call π A -graded in M if for each b ∈ NA we have that L b is a free O S -module of rank one generated by the image of sb . The following proposition is a restatement of Lemma 2.2(2). 3.6 Let π : X −→ S be A -graded, with S = Spec A affine and X = Spec B, where M B = A[x1 , . . . , xn ]/J = Lb. PROPOSITION

b∈NA

Suppose that M ⊂ S is an A -graded monomial ideal and that, for all b ∈ P , the A-module L b is generated by the image of sb . Then (1) π is A -graded in M; (2) there exist αu ∈ A such that J = u − αu su | u has primitive degree, su is M-standard . PROPOSITION 3.7 If π : X −→ S is A -graded, then there exist an affine open cover {Si } of S and A -graded monomial ideals Mi ⊂ S, such that, for each i, the induced family

π −1 (Si ) −→ Si is A -graded in Mi . We need the next lemma for the proof of the above proposition. LEMMA 3.8 Let A be a commutative Noetherian ring, and let J be an A -homogeneous ideal in A[x1 , . . . , xn ]. Let P ∈ Spec(A), and suppose that J P is a weakly A -graded ideal in k(P)[x1 , . . . , xn ]. Let M be an initial monomial ideal of J P . For each b ∈ NA , denote by sb the M-standard monomial if it exists, or, otherwise, set sb = 0. Then, for each b ∈ NA , there exists an f b ∈ / P such that (A fb [x1 , . . . , xn ]/J fb )b is generated by sb .

Proof Fix a b ∈ NA . By assumption, there exist αi0 ∈ k(P) such that J P k(P)[x1 , . . . , xn ] ⊇ m bi − αi0 sb | 1 ≤ i ≤ | b | .


427

Therefore, there exist h i ∈ / P, αi00 ∈ A, and βi j ∈ P, such that X h i m bi − αi00 sb + βi, j m bj ∈ J. j

So in A[x1 , . . . , xn ]/J we have the equality     00   α1 sb h1 m b1       .  . ..  + (βi, j )  ..  =  ..  .  . 00 s α|b| h |b| m b|b| b Let f b be the determinant of the left matrix. Since h i ∈ / P and βi, j ∈ P, it follows that f b ∈ / P. After we localize at f b , the left matrix is invertible, and therefore, J A fb [x1 , . . . , xn ] ⊇ m bi − αbi sb | 1 ≤ i ≤ | b | , for some αbi ∈ A fb . Proof of Proposition 3.7 By choosing an appropriate open affine cover, we may reduce to the case when L b = OS , for all b ∈ P . For an A -graded monomial ideal M, define VM = P ∈ S | L b is generated at P by the image of sb , for b ∈ P . Note that VM is an open affine subset of S . We prove the following. CLAIM

{VM } is an open cover of S , where M runs over all monomial A -graded ideals. Let P ∈ S be a point, and let Spec(A) be an affine neighborhood of P. Consider the fiber X P = Spec(k(P)[x1 , . . . , xn ]/J P ). Choose a monomial initial ideal M of J P . Hence, M is an A -graded monomial ideal. For each b ∈ P , take the element Q f b constructed in Lemma 3.8. Set f = b∈P f b . By Lemma 3.8 it follows that (A f [x1 , . . . , xn ]/J f )b is generated by sb for each b ∈ P . Thus, P ∈ U ( f ) = { f 6= 0}, and for each b ∈ P , the module L b is free on U ( f ) and is generated by the image of sb . So P ∈ U ( f ) ⊂ VM , proving the claim. Now apply Proposition 3.6 to the maps π M : π −1 (VM ) −→ VM to obtain that each π M is A -graded in M. We prove one more lemma that we use in the next section. 3.9 Let A be a commutative Noetherian ring, and let J be an A -homogeneous ideal in LEMMA

428

PEEVA and STILLMAN

A[x1 , . . . , xn ]. Let P ∈ Spec(A), and suppose that J P is a weakly A -graded ideal in k(P)[x1 , . . . , xn ]. Let M be an initial monomial ideal of J P . For each b ∈ NA , denote by sb the M-standard monomial if it exists, or, otherwise, set sb = 0. There exists an f ∈ / P such that for each b ∈ NA we have that (A f [x1 , . . . , xn ]/J f )b is generated by sb . In particular, A f [x1 , . . . , xn ]/J f is weakly A -graded. Proof Denote by M the set of degrees in which M has a minimal monomial generator. For Q each b ∈ M , take the element f b constructed in Lemma 3.8. Set f = b∈M f b . Now fix a b ∈ NA . Let m be a monomial of degree b. We prove that there exists an αm ∈ A f such that m − αm sb ∈ J f . By Lemma 3.8 it follows that there exist αci ∈ A f such that J f ⊃ m ci − αci sc | c ∈ M , 1 ≤ i ≤ |c| . Denote by the monomial order with respect to which M is the initial ideal of J P . Denote by α¯ ci the image of αci in k(P). Note that {m ci − α¯ ci sc | c ∈ M , 1 ≤ i ≤ |c| } is a Gröbner basis of J P with respect to . Consider the same monomial order in A f [x1 , . . . , xn ]. Choose a reduction of m by {m ci − αci sc | c ∈ M , 1 ≤ i ≤ |c| } to αm sb , where αm ∈ A f . Then m − αm sb ∈ J f , as desired. 4. The toric Hilbert scheme and its universality In this section we define the toric Hilbert scheme and prove the universality property. Definition 4.1 Consider P=

Y

P|b|−1 .

b∈P

We denote by z b1 , . . . , z b|b| the coordinates in P|b|−1 . Let Y ⊂ P × An be the subscheme defined by the ideal I (Y ) = z bi m bj − z bj m bi | b ∈ P , 1 ≤ i < j ≤ |b| . Denote by φ the projection map φ : Y −→ P, and note that it is A -homogeneous. Define the toric Hilbert scheme to be HA = V dets(φ) ⊂ P (recall Definition 3.4 of dets(φ)). Let WA = Y ×P HA


429

be the pullback, and let ψ : WA −→ HA be the projection map. PROPOSITION 4.2 The map Y −→ P is weakly A -graded.

Proof L Write Y = Spec( b∈NA Nb ). Fix a b ∈ NA and a P ∈ P. We show that there is an open set U of P containing P such that Fitt1 (Nb ) restricted to U is OU . First, restrict to an open affine set Spec(B) containing P, and let I be such that φ −1 (Spec(B)) = Spec(B[x1 , . . . , xn ]/I ). By construction, the fiber Spec(k(P)[x1 , . . . , xn ]/I P ) is weakly A -graded. Apply Lemma 3.9, and set U = { f 6= 0}. COROLLARY 4.3 The map ψ : WA −→ HA is A -graded.

Proof L Let i : HA −→ P be the inclusion map. Write Y = Spec( b∈NA Nb ), and write L WA = Spec( b∈NA Mb ). By base change, Fitt1 Mb ) = Fitt1 (i ∗ (Nb ) = i ∗ OP = OHA . Also, Fitt0 (Mb ) = i ∗ Fitt0 (Nb ) = 0 by the definition of HA . Therefore, Mb is locally free of rank one, for each b. THEOREM 4.4 (Universality of the toric Hilbert scheme) If π : X −→ S is A -graded, then there exists a unique morphism g : S −→ HA such that X = WA ×HA S .

Note that the condition that π is A -graded implies that it is flat and that we do not need S to be reduced. We consider the following diagram: X  π y S

W A  ψ y

g −→ HA

Define a contravariant functor h A : (schemes over k) −→ (sets)

430

PEEVA and STILLMAN

that associates to any S the set of subschemes X ⊂ S × An | π : X → S is A -homogeneous and flat, and all the fibers X P ⊂ Ank(P) are A -graded . A restatement of Theorem 4.4 says that HA represents the functor h A . Proof of Theorem 4.4 Let WA = Spec

M

Mb

b∈NA

and

M X = Spec Lb . b∈NA

By flatness, both L b and Mb are locally free rank one sheaves on their respective bases. For each b, we have that L b and Mb are generated by the global sections m b1 , . . . , m b|b| . For any b, let gb : S −→ P|b|−1 and h b : HA −→ P|b|−1 be the maps corresponding to these sections. Notice that, for b ∈ P , the map h b is the projection map onto the b-factor of P. Also, note that L b = gb∗ (O (1)) and Mb = h ∗b (O (1)). Given a morphism g : S −→ HA , we have M M W A ×H S = g ∗ Spec Mb = Spec g ∗ (Mb ) . b

b

Thus, we wish to show that there exists a unique morphism g : S −→ HA such that, for all b ∈ NA , we have g ∗ (Mb ) = L b . Uniqueness. If g, g 0 : S −→ HA are two morphisms that satisfy g ∗ (Mb ) = 0 0 b ) = L b , let gb = h b g, gb = h b g . These two maps have gb∗ O (1) = g ∗ h ∗b O (1) = g ∗ Mb = L b ,

g 0∗ (M

and, similarly, g 0 ∗b (O (1)) = L b . Both of these maps use the images of the same Q sections, and therefore gb = gb0 , for all b ∈ P . But g = b∈P gb , so g = g 0 . Existence. The above argument also shows that if such a map g exists, then it must be Y gb : S −→ P. g= b∈P

Q

So set g = L b , for all b.

b∈P

gb . We must show that g factors through HA , and that g ∗ (Mb ) =


431

By the uniqueness of the desired map g, it suffices to prove the theorem for each πi : π −1 (Si ) −→ Si , where {Si } is some affine open cover of S . By Proposition 3.7, it suffices to consider the case when π is A -graded in M, for some A -graded monomial ideal M of S. Thus, consider the local situation: π : X = Spec A[x1 , . . . , xn ]/J −→ S = Spec(A), where A is a commutative Noetherian ring. By Proposition 3.6 we have J = m bi − αbi sb | b ∈ P , 1 ≤ i ≤ |b|, m bi 6= sb , for some αbi ∈ A, and the A-module L b is L b = A[x1 , . . . , xn ]/J b . Let U M ⊂ HA be the open affine subscheme U M = HA ∩ z bi 6 = 0 | m bi = sb . Set Z = {z bi | m bi 6 = sb }. Then ψ : ψ −1 (U M ) = Spec k[x1 , . . . , xn , Z ]/(I + F) −→ Spec k[Z ]/F , where by construction, I = m bi − z bi sb | b ∈ P , 1 ≤ i ≤ |b|, m bi 6= sb , X F= Fitt0 k[Z ][x1 , . . . , xn ]/I b . b∈NA

Then the k[Z ]/F-module Mb is Mb = k[Z ][x1 , . . . , xn ]/(I + F) b . The map g is given locally by the ring homomorphism g ∗ : k[Z ] → A that sends z bi to αbi . We have g ∗ (F) = 0 because for each b the module L b is free of rank one and generated by sb . Therefore, we have a well-defined homomorphism g ∗ : k[Z ]/F → A. Thus, g factors through HA . Furthermore, g ∗ (I ) = J , so for each b ∈ NA we have a well-defined map µ : Mb = k[Z ][x1 , . . . , xn ]/(I + F) b −→ L b = A[x1 , . . . , xn ]/J b that maps the k[Z ]/F-module Mb to the A-module L b . Note that both Mb and L b are free modules of rank one generated by sb and that µ maps the generator of Mb to the generator of L b . Since g ∗ (Mb ) is the A-module generated by µ(Mb ), we have the desired equality g ∗ (Mb ) = L b , for all b ∈ NA .

432

PEEVA and STILLMAN

We use the notation from the proof of Theorem 4.4. The following is a corollary of the proof of Theorem 4.4. COROLLARY 4.5 Let M be a monomial A -graded ideal. Let U M ⊂ HA be the open affine subscheme U M = HA ∩ z bi 6 = 0 | m bi = sb .

Set Z = {z bi | m bi 6= sb }. The coordinate ring of U M is k[Z ]/F, where I = m bi − z bi sb | b ∈ P , 1 ≤ i ≤ |b|, m bi 6 = sb , X F= Fitt0 k[Z ][x1 , . . . , xn ]/I b . b∈NA

The ideal F is generated by the maximal minors of matrices of the form   1 rb1  1 rb2     ..  ..  . .      1 rb(|b|−1)      rb|b|    ..    .  rbt (here we assume that sb = m b, | b | corresponds to the last column in the matrix). Therefore, we have X Ib : sb ; F= b∈NA

in particular, F is a binomial ideal. Proof By Lemma 2.2, it follows that F is generated by the maximal minors of matrices of the desired form. COROLLARY 4.6 The open sets U M (where M runs through all monomial A -graded ideals) form a finite open affine cover of HA such that the defining ideal of each U M is defined by binomial equations.

5. Tangent spaces and reducibility The universality proved in Theorem 4.4 makes it possible to obtain the tangent space to HA .


433

THEOREM 5.1 Let F be an A -graded ideal. The Zariski tangent space to the toric Hilbert scheme at F is T[F] HA = Hom(F/F 2 , S/F)0 . (Note that 0 ∈ Nd ; i.e., we consider the zeroth component of Hom in the multigrading.)

Let M be a monomial A -graded ideal minimally generated by monomials m 1 , . . . , m r . Denote by s1 , . . . , sr the standard monomials of the corresponding degrees. The monomial m i is called a flip of M if the ideal (m 1 , . . . , m i−1 , m i − si , m i+1 , . . . , m r ) is A -graded. 5.2 If M is a monomial A -graded ideal, then dim T[M] HA equals the number of flips of M. COROLLARY

Proof Consider Hom S (M, S/M)0 . If ν ∈ Hom S (M, S/M)0 , then for each i we have ν(m i ) = ci si , for some ci ∈ k. On the other hand, given (c1 , . . . , cr ) ∈ k r , we have that ν mapping m i to ci si is in Hom S (M, S/M)0 if and only if, for every syzygy f m i − gm j = 0 with f, g ∈ S, we have f ν(m i ) − gν(m j ) = ci f si − c j gs j ∈ M (here we can assume that f, g are monomials). CLAIM

If ci 6 = 0, then m i is a flip. Proof We show that m 1 , . . . , m i−1 , m i − si , m i+1 , . . . , m r form a Gröbner basis. We take an s-pair f (m i − si ) − gm j , where f and g are relatively prime monomials. First, we show that f si ∈ M. Since we have the syzygy f m i − gm j = 0, we get f ν(m i ) − gν(m j ) = ci f si − c j gs j ∈ M. Then either f si ∈ M or ci f si = c j gs j . We show that the latter case cannot occur. The equality f si = gs j implies f (m i −si ) = g(m j −s j ); hence, m i − si = m j − s j , which is a contradiction. Thus, f si ∈ M; that is, some minimal monomial generator m l divides f si . Suppose that l = i. Since m i and si are relatively prime (by Lemma 2.2), it follows that m i divides f . Furthermore, since f and g are relatively prime, the equality f m i − gm j = 0 implies that m i divides m j , which is a contradiction. Hence, l 6 = i, and we are done. CLAIM

The monomial m i is a flip if and only if νi defined by ci = 1 and c j = 0 for j 6= i is a homomorphism in Hom S (M, S/M)0 .

434

PEEVA and STILLMAN

Proof The monomial m i is a flip if and only if the ideal (m 1 , . . . , m i−1 , m i − si , m i+1 , . . . , m r ) is A -graded; if and only if m 1 , . . . , m i−1 , m i − si , m i+1 , . . . , m r form a Gröbner basis; if and only if, for every s-pair f (m i − si ) − gm j , we have f si ∈ M; if and only if, for every syzygy f m i − gm j = 0, we have f νi (m i ) − gνi (m j ) = f si ∈ M. The claim is proved. Now consider the homomorphism ν. The first claim above implies that X X ν= ci νi = ci νi . 1≤i≤r ci 6=0

1≤i≤r m i is a flip

Furthermore, the second claim gives that the νi in the sum are homomorphisms. Hence, dim(Hom S (M, S/M)0 ) is the number of flips, and Corollary 5.2 is proved. Applying Corollary 4.6 and results by D. Eisenbud and Sturmfels [ES], we prove Theorem 1.2. First, we recall the statement of the theorem. THEOREM 5.3 There exists exactly one component containing the point [IA ]. If char (k) = 0, then this component is reduced and so the point [IA ] on HA is smooth.

Proof We can assume that k is algebraically closed. Let M be a monomial initial ideal of IA . Note that UM ⊂ A N . Then [IA ] corresponds to the point (1, . . . , 1), and [M] corresponds to the point (0, . . . , 0). Since J is binomial, it follows from [ES] that the ideal J : (y1 · · · yq )∞ is a lattice ideal. By [ES, Cor. 2.2], we get that the point (1, . . . , 1) lies on a single component (corresponding to the partial character that maps each lattice element to 1). Furthermore, if char (k) = 0, then [ES, Cor. 2.2] shows that this component is a toric variety. The scheme H A has a canonical component that is the closure of the orbit of the toric ideal under the torus action. This component contains all initial ideals of IA . Since the state polytope has dimension n − d, it follows that this component has dimension at least n − d = codim (S/IA ). Example 5.4 Consider the generic monomial curve defined by A = {20, 24, 25, 31} from [PStu1, Exam. 4.5]. The toric Hilbert scheme HA is not reduced. In this case, HA has one


435

4-dimensional component and thirty-one 3-dimensional components. The Graver basis contains 75 elements, and there are 48 noncoherent monomial A -graded ideals. There is one nonreduced component; we describe this component below. An expository paper explaining how to compute examples like this one is [SST]. Consider the noncoherent monomial A -graded ideal M = x1 x2 x4 , x14 , x1 x22 x3 , x13 x32 , x12 x23 , x25 , x13 x42 , x1 x45 , x24 x44 , x12 x3 x4 , x2 x34 , x12 x35 , x23 x48 , x22 x412 , x1 x320 , x2 x421 , x331 . Three irreducible components of HA go through M, and all of them are 3dimensional. Two of the components are isomorphic to P1 × P1 × P1 . The third component is nonreduced and isomorphic to a double P1 × P1 × P1 . Two noncoherent and six coherent monomial A -graded ideals lie on this component. Next we use local equations from [PSti]. Let T = b2 , ab, a 2 , da − eb and E = k[a, b, c, d, e]/T. The ideal Q = x1 x2 x4 , x14 , x1 x22 x3 , x13 x32 , x12 x23 , x25 , x13 x42 , x1 x45 , x24 x44 , − x24 c + x12 x3 x4 , −x44 da + x2 x34 , −x33 x43 cda + x12 x35 , −x1 x312 d 2 a + x23 x48 , − x1 x316 d + x22 x412 , x1 x320 − x2 x416 a, −x327 b + x2 x421 , x331 − x425 e in E[x1 , . . . , x4 ] gives a family of A -graded ideals over Spec(E). Here T is the defining ideal of the nonreduced component through M. 6. Initial ideals of a codimension 2 toric ideal In this section we suppose that codim (S/IA ) = 2. We obtain a description of the structure of the initial monomial ideals of IA . This description is of interest on its own, and it is also used in Section 7 in the proof of Theorem 1.4. Fix an A = {a1 , . . . an } ⊆ Nd \ 0 such that codim (S/IA ) = n − d = 2. Set I = IA , and let A be the matrix with columns a1 , . . . , an . Let B = (bi j ) be an integer (n × 2)-matrix such that the following sequence is exact: B

A

0 → Z2 −→ Zn −→ Z2 . A vector u ∈ Zn can be written uniquely as u = u + − u − , where u + and u − have nonnegative coordinates and supp(u + ) ∩ supp(u − ) = ∅ (here supp(u) = {i | the ith coordinate of u is not 0}). Given a vector v = (v1 , . . . , vn ) ∈ Nn , we denote xv = x1v1 · · · xnvn . Each vector α in Z2 corresponds to a binomial x(Bα)+ − x(Bα)− in I , and every binomial in I without monomial factors can be represented uniquely

436

PEEVA and STILLMAN

in this way. By [PStu2, Rem. 3.2 and Th. 3.7], we can choose the matrix B so that the binomials corresponding to (1, 0) and (0, 1) are minimal generators of I . We recall the following result from [PStu2, Th. 6.1]. THEOREM 6.1 ([PStu2]) The ideal I is a complete intersection if and only if it is minimally generated by two elements. The ideal I is Cohen-Macaulay, but not a complete intersection, if and only if it is minimally generated by three elements (which can be chosen to correspond to either (1, 0), (0, 1), (1, 1) or (1, 0), (0, 1), (−1, 1) ). The ideal I is not CohenMacaulay if and only if it is minimally generated by more than three elements if and only if there exists a syzygy quadrangle for I . If I is not Cohen-Macaulay, then it has a unique (up to multiplying by a constant) minimal system of binomial generators.

We call α generating if one of the following two conditions is satisfied: • I is not a complete intersection, and the binomial corresponding to α is in the unique minimal system of generators of I (cf. [PStu2, Th. 3.7]); • I is a complete intersection, and α is either (1, 0) or (0, 1). We call α primitive if its binomial is primitive. Now fix a monomial initial ideal M of I . We say that α is a head-vector if x(Bα)+ ∈ M; we say that α is a tail-vector if x(Bα)− ∈ M; we say that α is standard if one of the monomials x(Bα)+ and x(Bα)− is M-standard. By [PStu2, Th. 3.4], x(Bα)+ and x(Bα)− form a fiber if α is generating, so a generating vector is standard. However, a primitive vector can be nonstandard. If α ∈ Z2 is standard, then set ( α if α is a head-vector, αE = −α if α is a tail-vector. After renumbering the quadrants if necessary, we can assume that (1, 0) and (0, 1) are head-vectors. We also use terminology from [GP] and [PStu2] about the syzygies of I : the syzygies are represented by vectors, triangles, and quadrangles in Z2 with integer vertices and one vertex fixed at the origin (0, 0). For a sequence of syzygy quadrangles T = P1 , . . . , Pr in the first or second quadrant, denote by αi , βi the edges of Pi and by γi the longer diagonal of Pi , for 1 ≤ i ≤ r . The sequence T is a chain if, for each 1 ≤ i ≤ r − 1, the edges of Pi+1 are either αi , γi or βi , γi . We say that T starts with P1 . When we say that the vectors α, β are edges of a quadrangle, we always mean “oriented edges,” so that α + β is the longer diagonal of the quadrangle. For this reason we say that the vector (1, 1) is the longer diagonal of the unit square with edges (1, 0), (0, 1) and that the vector (−1, 1) is the longer diagonal of the unit square with edges (−1, 0), (0, 1). The syzygy quadrangles for I form the syzygy tree described in [PStu2, Const. 4.4 and Th. 4.5].


437

By [Ei, Prop. 15.16], there exists a weight vector w ∈ Nn such that M is the initial ideal of I with respect to the weight order ≺w with weight vector w. Following the construction before [Ei, Prop. 15.17], we construct a flat family S[t]/It whose fiber over zero is S/M and whose fiber over 1 is S/I . The ideal M is minimally generated by the monomials (B αE ) + | α is generating for I . x t As explained in [PStu2, Alg. 8.2], It is the toric ideal corresponding to the matrix e B obtained from B by adding the row w B. Therefore, It has codimension 2, and its Gale diagram is obtained from the Gale diagram of I by adding one new vector w B. By [PStu2] we see that the syzygy tree of I is a subtree of the syzygy tree of It . The next theorem describes the structure of M. 6.2 Let M be a monomial initial ideal of I . (1) Suppose that It is not Cohen-Macaulay. There exists a chain M in the second quadrant consisting of syzygy quadrangles for It which are not syzygy quadrangles for I , such that the syzygy tree of It consists of M and the syzygy tree of I . In particular, the ideal M is minimally generated by the elements (B αE ) + | α ∈ M or α is generating . x THEOREM

(2)

If It is Cohen-Macaulay, then the ideal M is minimally generated by the elements (B αE ) + | α ∈ M or α is generating , x where M = {(−1, 1)} if I is a complete intersection and (−1, 1) is generating for It , or M = ∅ otherwise.

We call the chain M from Theorem 6.2(1) the M-chain. The rest of this section is devoted to the proof of Theorem 6.2. First, we need to introduce some more notation and background. The Lawrence lifting I L of I is a codimension 2 toric ideal, and the primitive vectors for I are exactly the generating vectors for I L (see [GP, Sec. 2]). If G is the Gale diagram of I , then G ∪ (−G) is the Gale diagram of I L . LEMMA 6.3 Suppose that I is not Cohen-Macaulay. The syzygy tree of I is a subtree of the syzygy tree of It , which is a subtree of the syzygy tree of I L .

438

PEEVA and STILLMAN

Proof We have the following inclusions of sets of vectors: {generating vectors of I } ⊆ {generating vectors of It } ⊆ {generating vectors of I L }. By [PStu2, Cor. 4.3] we obtain the following inclusions: {syzygy quadrangles for I } ⊆ {syzygy quadrangles for It } ⊆ {syzygy quadrangles for I L }. We describe which of the syzygy quadrangles for I L are syzygy quadrangles for It . We say that two vectors ill match if exactly one of them is a head-vector and exactly one of them is a tail-vector (note that both vectors are standard in this case). We say that two vectors α and β well match if either they are both head-vectors or they are both tail-vectors. Note that the properties “ill-matching” and “well-matching” are with respect to the fixed initial ideal M. LEMMA 6.4 Let P be a syzygy quadrangle for I L in the first or second quadrant such that P is not a syzygy quadrangle for I . If the edges of P well match, then P is not a syzygy quadrangle for It .

Proof Denote by α and β the edges of P, and denote by γ = α + β the longer diagonal of P. Suppose that α and β are head-vectors. By [GP, Lem. 3.10], it follows that the initial term of x(Bγ )+ − x(Bγ )− is x(Bγ )+ , and x(Bγ )+ ∈ x(Bα)+ , x(Bβ)+ . Hence, γ is not a generating vector of It . If α and β are tail-vectors, then we apply the above argument to −α, −β. We use the following result. LEMMA 6.5 ([GP, Lems. 3.2(c) and 3.12(a)]) Suppose that α and β are the edges of a syzygy quadrangle for I L . If α and β well match, then the longer diagonal α + β of the quadrangle well matches α and β.

Proof of Theorem 6.2(1) We call a chain ill-matching if the two edges in each quadrangle in the chain ill match. We show that there exists a unique maximal ill-matching chain T of syzygy quadrangles for I L starting with the unit quadrangle with edges (−1, 0), (0, 1). Let r be the


439

number of quadrangles in the longest ill-matching chain. We show by induction on i that there exists a unique ill-matching chain of i syzygy quadrangles for I L which starts with the unit quadrangle with edges (−1, 0), (0, 1). Clearly, P1 is determined. If r = 1, we are done; suppose that r ≥ 2. Let i < r . By the induction hypothesis, let P1 , . . . , Pi be the unique ill-matching chain of i syzygy quadrangles for I L in the second quadrant which starts with the unit quadrangle with edges (−1, 0), (0, 1). Denote by α the tail-vector edge of Pi , by β the head-vector edge of Pi , and by γ the longer diagonal of Pi . By Lemma 6.5 it follows that • if γ is a standard head-vector, then the edges of Pi+1 are α, γ ; • if γ is a standard tail-vector, then the edges of Pi+1 are β, γ ; • if γ is not standard, then Pi+1 cannot have ill-matching edges, which is a contradiction. Thus, there exists a single choice for the quadrangle Pi+1 . Next, we show that if P is a syzygy quadrangle for I L in the second quadrant with ill-matching edges, then P is contained in the ill-matching chain T. There exists a chain P1 , . . . , Ps of syzygy quadrangles for I L in the second quadrant which starts with the unit quadrangle with edges (−1, 0), (0, 1) and such that P = Ps . By [GP, Lem. 3.13], the chain P1 , . . . , Ps is ill-matching. Hence, P1 , . . . , Ps is in T. Denote by M the set of syzygy quadrangles for It which are not syzygy quadrangles for I . By Lemma 6.4 it follows that each quadrangle in M has ill-matching edges. By Lemma 6.5 it follows that each syzygy quadrangle for I L in the first quadrant has well-matching edges. Hence, M is in the second quadrant. Therefore, M is a subchain of T. This finishes the proof of Theorem 6.2(1). Proof of Theorem 6.2(2) Use Theorem 6.1. If the generating vectors of I and It coincide, then we are done. It remains to consider the case when I is a complete intersection and It is not. By Theorem 6.1, the generating vectors of It are (1, 0), (0, 1), (−1, 1) or (1, 0), (0, 1), (1, 1). In the former case we are done. We show that the latter case never occurs; that is, (1, 1) cannot be generating for It . As in [PStu2, Const. 5.2], we write the binomials corresponding to (1, 0), (0, 1), (1, 1) as xu+ xt xp − xu− xs xr , xv+ xs xp − xv− xt xr , and (xu+ xp )(xv+ xp ) −(xu− xr )(xv− xr ). By [PStu2, Rem. 3.2], we have that either xs or xt is 1 since I is a complete intersection. It follows that the initial term of x(B(1,1))+ −x(B(1,1))− is x(B(1,1))+ and that x(B(1,1))+ ∈ (x(B(1,0))+ , x(B(0,1))+ ). Hence, (1, 1) is not a generating vector of It . 7. The toric Hilbert scheme of a codimension 2 toric variety In this section we prove Theorem 7.1. It immediately implies Theorem 1.4. We also prove Corollary 1.5.

440

PEEVA and STILLMAN

Recall that for a monomial A -graded ideal M we have the open affine subscheme U M = HA ∩ z bi 6 = 0 | where m bi is M-standard ⊂ HA . THEOREM 7.1 Suppose that codim (S/IA ) = 2. Denote by J the finite set of monomial initial ideals S of IA . Then M∈J U M is a finite affine open cover of the toric Hilbert scheme. The coordinate ring of each U M is k[u, v, y1 , . . . , yr ]/ y j − u p j v q j | 1 ≤ j ≤ r ∼ = k[u, v]

(here r, p j , q j depend on M). In particular, the theorem shows that the toric Hilbert scheme is reduced in this case. We use the following construction from [PSti], which provides local equations around a monomial initial ideal of IA . Construction 7.2 (Local equations) Let M be a monomial initial ideal of IA . As in Section 2, consider the set G M = { f i | 1 ≤ i ≤ p M }, where the monomials f i generate M minimally, and | p M | is the number of minimal generators of M. For each 1 ≤ i ≤ p M , denote by si the Mstandard monomial in the degree of f i . Consider the ring k[x1 , . . . , xn , y1 , . . . , y p M ] and the ideal G = f i − yi si | 1 ≤ i ≤ p M . Define J=

X

G b : sb

in k[y1 , . . . , y p M ],

b∈NA

H (M) = Spec k[y1 , . . . , y p M ]/J . S Theorem 1.3 implies that M∈J U M is a finite affine open cover of HA . Fix an initial monomial ideal M. By [PSti] we have U M ∼ = H (M). The rest of this section is devoted to obtaining the defining ideal of the scheme H (M). We use the first syzygies, Theorem 6.2, and Construction 7.2 to obtain the ideal. We use the notation from Section 6. 7.3 Let P be a syzygy quadrangle for It in the first or second quadrant. Denote by α, β the edges of P and by γ the longer diagonal of P. Let yα , yβ , yγ be the variables corresponding to α, β, γ in Construction 7.2. (1) If γE = αE + βE , then yγ − yα yβ is in the defining ideal of H (M). LEMMA


(2)

441

E then yα − yβ yγ is in the defining ideal of H (M). If αE = γE + β,

Example 7.4 Consider the twisted cubic curve. The rows of the matrix B can be chosen to be (1, 0), (−2, 1), (1, −2), (0, 1). The generating vectors are α = (1, 0), β = (0, 1), and γ = α + β = (1, 1); they correspond to the binomials ac − b2 , bd − c2 , ad − bc, respectively. Let M be the initial ideal (b2 , c2 , bc). In this case, αE = −α, βE = −β, γE = −γ . We illustrate how the equality γE = αE + βE yields the equation yγ = yα yβ . The equality γE = αE + βE corresponds to the first syzygy on the generators of I : c(b2 − ac) − b(bc − ad) + a(c2 − bd) = 0; therefore, c(b2 − yα ac) − b(bc − yγ ad) + yα a(c2 − yβ bd) = (yγ − yα yβ )abd ∈ G . Since the monomial adb is M-standard and (yγ − yα yβ )abd ∈ G, it follows by Construction 7.2 that we have the desired equation yγ = yα yβ . Proof of Lemma 7.3 The syzygy tree of It is contained in the syzygy tree of I L by Lemma 6.3. Therefore, α, β, γ are primitive vectors. Since they are generating for It , Proposition 2.3(1) E γE are well defined. implies that they are standard vectors. Thus, αE , β, First, we prove (1). Suppose that α and β are head-vectors. By Lemma 6.5 it follows that γ is a head-vector as well. As in [PStu2, Const. 5.2], we write the binomials corresponding to α, β, γ as u + x xt xp − xu− xs xr , xv+ xs xp − xv− xt xr , and (xu+ xp )(xv+ xp ) − (xu− xr )(xv− xr ). We consider a = xu+ xt xp − yα xu− xs xr , b = xv+ xs xp − yβ xv− xt xr , c = (xu+ xp )(xv+ xp ) − yγ (xu− xr )(xv− xr ). We have the equalities (corresponding to syzygies of S/IA ) xv+ xp a − xt c + yα xu− xr b = (yγ − yα yβ )xu− xv− xt x2r ∈ G , xu+ xp b − xs c + yβ xv− xr a = (yγ − yα yβ )xu− xv− xs x2r ∈ G. By Construction 7.2 it follows that it suffices to show that at least one of the monomials xu− xv− xt x2r and xu− xv− xs x2r is a standard monomial. If γ is not generating, then by [GP, Lem. 3.10] we have that either xs or xt is 1. In this case, (yγ − yα yβ )xu− xv− x2r ∈ G. Since the monomial xu− xv− x2r is M-standard, we conclude that yγ − yα yβ = 0.

442

PEEVA and STILLMAN

If γ is generating, then the three monomials xu− xv− xt x2r , xu+ xv+ xt x2p , and form a fiber (the fiber of a second syzygy of S/I ) by [PStu2, Const. 5.2]. Note that xu+ xt xp and xv+ xs xp are in M. Hence, the monomial xu− xv− xt x2r is M-standard. Therefore, (yγ − yα yβ )xu− xv− xt x2r ∈ G implies that yγ − yα yβ = 0. This finishes the proof of (1) in the case when α and β are headvectors. If α and β are tail-vectors, then −α, −β are head-vectors and we can apply the above argument. Now we prove (2). Suppose that α and γ are head-vectors, but β is a tail-vector. We consider xu− xv+ xp xr xs

a = xu+ xt xp − yα xu− xs xr , b0 = yβ xv+ xs xp − xv− xt xr , c = (xu+ xp )(xv+ xp ) − yγ (xu− xr )(xv− xr ). We have the equalities (corresponding to syzygies of S/IA ) xv+ xp a − xt c + yγ xu− xr b0 = −(yα − yβ yγ )xu− xv+ xs xr xp ∈ G, xv− xr a + xu+ xp b0 − yβ xs c = −(yα − yβ yγ )xu− xv− xs x2r ∈ G. By Construction 7.2 it follows that it suffices to show that at least one of the monomials xu− xv+ xs xr xp and xu− xv− xs x2r is a standard monomial. Suppose that γ is not generating. Then by [GP, Lem. 3.10] we have that either xs t or x is 1. However, xt = 1 is impossible because xv− xt xr ∈ M and (xu− xr )(xv− xr ) is M-standard. Therefore, xs = 1 in this case. Hence, (yα − yβ yγ )xu− xv− x2r ∈ G. Since the monomial xu− xv− x2r is M-standard, we conclude that yα − yβ yγ = 0. Suppose that γ is generating. We give an argument similar to the one in (1). Then the three monomials xu− xv− xt x2r , xu+ xv+ xt x2p , and xu− xv+ xp xr xs form a fiber (the fiber of a second syzygy) by [PStu2, Const. 5.2]. Note that xu+ xt xp and xv− xt xr are in M. Hence, xu− xv+ xp xr xs is M-standard. Therefore, (yα − yβ yγ )xu− xv+ xs xr xp ∈ G implies that yα − yβ yγ = 0. This finishes the proof of (2) in the case when α, γ are head-vectors and β is a tail-vector. Suppose that α and γ are tail-vectors, but β is a head-vector. Then −α, −γ are head-vectors and −β is a tail-vector, so we can apply the above argument. 7.5 Suppose that It is Cohen-Macaulay, but not a complete intersection. Set α = (1, 0) and β = (0, 1), and let γ be the third generating vector for It in the first or second quadrant. Then (1) and (2) in Lemma 7.3 hold. LEMMA


443

Proof Write the binomials corresponding to α, β, and γ as in the proof of Lemma 7.3. First, suppose that I is a complete intersection. In this case, γ is not generating. By [PStu2, Rem. 3.2] we have that either xs or xt is 1. This allows us to apply the same argument as in the proof of Lemma 7.3. Now suppose that I is not a complete intersection; it has to be Cohen-Macaulay. In this case, α, β, and γ are generating vectors. The same argument as in the proof of Lemma 7.3 can be applied. We apply Construction 7.2 in slightly different notation: if α ∈ Z2 is a standard vector in the first or second quadrant such that f i − si = x(B αEi )+ − x(B αEi )− , for some 1 ≤ i ≤ p M , then set yαE = yi . So for a vector in α | x(B αE )+ is a minimal generator of M , denote by yα the variable corresponding to x(B αE )+ − yα x(B αE )− in Construction 7.2. (Note that the set {α | x(B αE )+ is a minimal generator of M} is well defined.) We construct two variables u and v such that each variable yα in Construction 7.2 is a product of a power of u and a power of v. Suppose that It is not Cohen-Macaulay. Let P be the last quadrangle in the Mchain M. Denote by σ the longer diagonal of P and by τ the edge in this quadrangle which ill matches σ . Denote by u and v the variables corresponding to σ and τ in Construction 7.2. If It is a complete intersection, denote by u and v the variables corresponding to (1, 0) and (0, 1) in Construction 7.2. If It is Cohen-Macaulay, but not a complete intersection, then denote by u and v the variables corresponding to • (−1, 0) and (−1, 1) if (−1, 1) appears in M and is a head-vector, • (0, 1) and (−1, 1) if (−1, 1) appears in M and is a tail-vector, • (−1, 0) and (0, 1) if (−1, 1) does not appear in M. 7.6 For each variable yα in Construction 7.2, there exist p, q ∈ N such that yα − u p v q is in the defining ideal of H (M). LEMMA

Proof We say that a variable yα is a uv-power if there exist p, q ∈ N such that yα − u p v q is in the defining ideal of H (M). By Construction 7.2 and Theorem 6.2 it follows that the variables yα in Construction 7.2 are yα | α is generating or α ∈ M ,

444

PEEVA and STILLMAN

where M is the M-chain. The claim is obvious if It is a complete intersection. If It is Cohen-Macaulay, but not a complete intersection, then apply Lemma 7.5. Now suppose that It is not Cohen-Macaulay. Let T be the unique ill-matching chain of syzygy quadrangles for It . Denote by σi the longer diagonal of Pi in T and by τi the edge in this quadrangle which ill matches σi . Then the other edge of Pi is δi = σi − τi . Since δi and τi ill match, it follows that δEi = σEi + τEi for each i. We show by induction on s − i that yσi , yτi , and yδi are uv-powers. For i = s we have yσi = y and yτi = z. Since δEs = σEs + τEs , by Lemma 7.3(2) we get that yδi − uv is in the defining ideal of H (M). Suppose that the desired property holds for i + 1. Consider Pi . Since the edges of Pi+1 ill match, they have to be σi and τi , so the induction hypothesis applies to them. It remains to consider δi . As δEi = σEi + τEi , by Lemma 7.3(2) we get that yδi − yσi yτi is in the defining ideal of H (M). Combining this equation with the fact that yσi and yτi are uv-powers, we get that yδi is a uv-power, as desired. In particular, y(1,0) and y(0,1) are uv-powers. By [PStu2, Lem. 3.1(b)] and Lemma 7.3(1), it follows that the variable yα is a uv-power for every generating vector α in the first quadrant. Now let α be a generating vector in the second quadrant, and let α not appear as an edge or diagonal in T; hence, α does not appear in the maximal ill-matching chain. Therefore, α is the longer diagonal of a syzygy quadrangle whose edges β and γ well match. All the vectors α, β, γ are standard because they are generating. Hence, αE = βE + γE . By induction (the induction argument starts at the root of the master tree and moves outward), we can assume that yβ and yγ are uv-variables. It follows from Lemma 7.3(1) that yα is a uv-variable as well. Finally, we are ready to complete the proof of Theorem 7.1. Proof of Theorem 7.1 Theorem 1.3 shows that HA has one component. Note that HA is 2-dimensional since the state polytope of IA is 2-dimensional. S Theorem 1.3 implies that M∈J U M is a finite affine open cover of HA . Fix a monomial initial ideal M of IA . [PSti] provides that the coordinate ring of U M is isomorphic to the coordinate ring of H (M). Lemma 7.6 implies that the coordinate ring of H (M) is isomorphic to a quotient of k[u, v]. Since H (M) is 2-dimensional, we conclude that its coordinate ring is isomorphic to k[u, v]. It remains to prove Corollary 1.5. We start with a construction that works in any codimension.


445

Construction 7.7 Fix an A = {a1 , . . . , an } ⊆ Nd \ 0, and let A be the matrix with columns a1 , . . . , an . Let B = (bi j ) be an integer (n × (n − d))-matrix such that the following sequence is exact: B A 0 → Zn−d −→ Zn −→ Zd . Each vector α in Zn−d corresponds to a binomial x(Bα)+ − x(Bα)− in IA , and every binomial in IA without monomial factors can be represented uniquely in this way. Now fix a monomial initial ideal M of IA . By Proposition 2.2 there exist u1 , . . . , ur primitive vectors such that M is minimally generated by xu1+ , . . . , xur+ and the monomials xu1− , . . . , xur− are standard in S/M. Let α1 , . . . , αr ∈ Zn−d be such that ui = Bαi for all i. Denote by τ the cone in Rn−d generated by α1 , . . . , αr , and let X M be the toric variety of the dual of this cone. The cone σ in the Gröbner fan of IA that corresponds to M is σ = {w ∈ Rn | (ui · w) ≥ 0 for 1 ≤ i ≤ r }. Therefore, the dual cone σ ∨ is generated by the vectors u1 , . . . , ur . Hence, the toric variety of σ is exactly X M . We would like to compare X M to the open affine subscheme U M of the toric Hilbert scheme. Jointly with R. Thomas, we have obtained an example of a 3codimensional toric ideal and a monomial ideal M such that X M and U M are not isomorphic (see [SST]). Proof of Corollary 1.5 We use the notation in Construction 7.7. First, note that, since the Gröbner fan is smooth in this case, we have that X M is a 2-dimensional smooth toric variety. By the proof of Theorem 1.4, we have the isomorphism X M ∼ = U M . However, this does not suffice to prove the desired corollary because we have to make sure that the open sets S S in the two covers M X M and M U M are glued in the same way. The coordinate ring of X M is k u, v, yα | α ∈ {α1 , . . . , αr } and yα 6 = u, v /(E), where E is some ideal containing each equation yα − u p v q from Lemma 7.6. Since X M is a 2-dimensional smooth toric variety, we conclude that E is generated by these equations. The proof of Theorem 7.1 shows that the coordinate ring of H (M) is the same. Let L be the monomial initial ideal of IA corresponding to a cone µ adjacent to σ . Suppose that u is the variable corresponding to the common edge of σ and µ. Then gluing X M and X L amounts to inverting u in the coordinate ring of X M . By the construction of the toric Hilbert scheme, gluing U M and U L amounts to inverting u as well.

446

PEEVA and STILLMAN

8. Extremal Betti numbers Questions about extremal Betti numbers have been of continuous interest. Denote by Ph the set of all ideals with a fixed Hilbert function h. We introduce a partial order in Ph as follows: for I1 , I2 ∈ Ph , we have I1 ≥ I2 if we have inequalities for the Betti numbers bi (S/I1 ) ≥ bi (S/I2 ), for all i. Note that we are comparing total (not graded) Betti numbers. Also, note that two different ideals can be equal in the partial order. We say that I ∈ Ph is a top ideal if it is a biggest element in Ph ; we say that I ∈ Ph is a bottom ideal if it is a smallest element in Ph . The set Ph may not have a top or bottom ideal. Fix a Hilbert function with respect to the standard grading deg(xi ) = 1 (for all i). The lexicographic ideal is a top ideal in Ph . This result was proved by A. Bigatti and H. Hulett for char (k) = 0. Using the classical Hilbert scheme, Pardue [Pa] proved the result for any characteristic. In contrast, in the toric (multigraded) case there are examples when no top ideal exists. Example 8.1 Take 1 1 1 1 1 A= . 0 1 3 5 8 There exist two monomial ideals that are maximal ideals, and their Betti numbers are 1, 12, 28, 27, 12 and 1, 15, 35, 31, 11. Thus, none of the ideals is a top ideal. On the other hand, in the case deg(xi ) = 1 (for all i), G. Evans constructed an example of a Hilbert function h such that Ph has no bottom ideal. In contrast, it seems that often IA is a bottom ideal in the toric (multigraded) case, so we address the following problem. Question 8.2 Fix the multigraded Hilbert function h of a toric ideal. Under what conditions does there exist a bottom ideal in Ph ? We show that in the codimension 2 case the extremal Betti numbers are attained. 8.3 Set I = IA , and suppose that codim (S/I ) = 2. (1) If the binomials corresponding to the generating vectors of I form a Gröbner basis, then the Betti numbers of S/I and S/in I coincide. (2) Fix the multigraded Hilbert function h of I . Let L be a monomial initial ideal of I with the biggest possible number of minimal generators. The ideal L is a top ideal in Ph . The ideal I is a bottom ideal in Ph . PROPOSITION


(3) (4)

447

Let M be a monomial initial ideal of I . Either S/M is a complete intersection or it is Golod. Let L be a monomial initial ideal of I with the biggest possible number of minimal generators. Among all A -graded ideals, k has the greatest Betti numbers over the ring S/L and the smallest Betti numbers over the ring S/I .

Before giving the proof, we recall the definition of Golodness. Let T be an Nd -graded ideal in S. The ring S/T is called Golod if X i≥0 a∈Nd

S/T

dim Tor i,a (k, k)t i xa =

1 − t2

(1 + t x1 ) · · · (1 + t xn ) ; P S i a i≥0 dim Tor i,a (T, k)t x a∈Nd

the left-hand side in this formula is the generating function of the (infinite) minimal free resolution of k over S/T . Proof We use the notation in Section 6. (1) If the binomials corresponding to the generating vectors of I form a Gröbner basis, then the generating vectors of I and It coincide. By [PStu2, Th. 6.1], it follows that the Betti numbers of S/I and S/It coincide. On the other hand, the Betti numbers of S/It and S/in I coincide by [Ei, Th. 15.17]. (2) By [GP], any A -graded ideal is an initial ideal of an ideal obtained from I by scaling the variables. By [Ei, Th. 15.17] it follows that I is a bottom ideal in Ph . The Betti numbers of a monomial initial ideal M coincide with the Betti numbers of It . Therefore, we get the biggest Betti numbers if and only if the M-chain is longest if and only if M has the biggest possible number of minimal generators. (3) Suppose that S[t]/It is not a complete intersection. Then S[t]/It is Golod by [PStu2, Lem. 6.2]. Since S/M = S[t]/It ⊗ S[t]/t and since t is S[t]/It -regular by [Ei, Th. 15.17], it follows that S/M is Golod. (4) By [GP], any A -graded ideal is an initial ideal of an ideal obtained from I by scaling the variables. By [Ei, Th. 15.17], it follows that, among all A -graded ideals, k has the smallest Betti numbers over the ring S/I . Let M be a monomial initial ideal of I . By (3) we have that S/M is either a complete intersection or Golod. Therefore, k has the biggest Betti numbers over the ring S/M if and only if M is a top ideal in Ph . Apply (2). Acknowledgments. We thank D. Eisenbud and B. Sturmfels for helpful conversations.

448

PEEVA and STILLMAN

References [Al]

V. ALEXEEV, Complete moduli in the presence of semiabelian group action, to appear

[Ar]

V. ARNOLD, A-graded algebras and continued fractions, Comm. Pure Appl. Math. 42

[Ei]

D. EISENBUD, Commutative Algebra: With a View toward Algebraic Geometry, Grad.

[ES]

D. EISENBUD and B. STURMFELS, Binomial ideals, Duke Math. J. 84 (1996), 1–45.

[GP]

V. GASHAROV and I. PEEVA, Deformations of codimension 2 toric varieties,

in J. Algebraic Geom. 420 (1989), 993–1000. MR 90h:32025 419, 422 Texts in Math. 150, Springer, New York, 1995. MR 97a:13001 425, 437, 447 MR 97d:13031 434 Compositio Math. 123 (2000), 225–241. MR 1 794 859 420, 421, 436, 437, 438, 439, 441, 442, 447 [Gr] A. GROTHENDIECK, Techniques de construction et théorèmes d’existence en géométrie algébrique, IV: Les schémas de Hilbert, Séminaire Bourbaki 6, exp. no. 221, Soc. Math. France, Montrouge, 1995, 249–276. MR 1 611 822 419 ´ [Ha] R. HARTSHORNE, Connectedness of the Hilbert scheme, Inst. Hautes Etudes Sci. Publ. Math. 29 (1966), 5–48. MR 35:4232 420 [Ko] E. KORKINA, Classification of A-graded algebras with 3 generators, Indag. Math. (N.S.) 3 (1992), 27–40. MR 93c:13013 419, 422 [KPR] E. KORKINA, G. POST, and M. ROELOFS, Classification of generalized A-graded algebras with 3 generators, Bull. Sci. Math. 119 (1995), 267–287. MR 96d:13001 419, 422 [Pa] K. PARDUE, Deformation classes of graded modules and maximal Betti numbers, Illinois J. Math. 40 (1996), 564–585. MR 97g:13029 420, 446 [PSti] I. PEEVA and M. STILLMAN, Local equations for the toric Hilbert scheme, Adv. in Appl. Math. 25 (2000), 307–321. MR 2001m:13019 421, 424, 435, 440, 444 [PStu1] I. PEEVA and B. STURMFELS, Generic lattice ideals, J. Amer. Math. Soc. 11 (1998), 363–373. MR 98i:13022 434 [PStu2] , Syzygies of codimension 2 lattice ideals, Math. Z. 229 (1998), 163–194. MR 99g:13020 421, 436, 437, 438, 439, 441, 442, 443, 444, 447 [PiSc] R. PIENE and M. SCHLESSINGER, On the Hilbert scheme compactification of the space of twisted cubics, Amer. J. Math. 107 (1985), 761–774. MR 86m:14004 421 [Re] A. REEVES, The radius of the Hilbert scheme, J. Algebraic Geom. 4 (1995), 639–657. MR 97g:14003 420 [RS] A. REEVES and M. STILLMAN, Smoothness of the lexicographic point, J. Algebraic Geom. 6 (1997), 235–246. MR 98m:14003 420 [SST] M. STILLMAN, B. STURMFELS, and R. THOMAS, Equations of the toric Hilbert scheme, preprint, arXiv:math.AG/00010130, to appear in Computations in Algebraic Geometry Using Macaulay 2, ed. D. Eisenbud, D. Grayson, M. Stillman, and B. Sturmfels. 435, 445 [St1] B. STURMFELS, The geometry of A -graded algebras, preprint, arXiv:alg-geom/9410032 421 [St2] , Gröbner Bases and Convex Polytopes, Univ. Lecture Ser. 8, Amer. Math.


Soc., Providence, 1996. MR 97b:13034 419, 422

Peeva Department of Mathematics, Cornell University, Ithaca, New York 14853, USA; [email protected] Stillman Department of Mathematics, Cornell University, Ithaca, New York 14853, USA; [email protected]

449


LOCAL GEOMETRIZED RANKIN-SELBERG METHOD FOR GL(n) SERGEY LYSENKO

Abstract Following G. Laumon [12], to a nonramified `-adic local system E of rank n on a curve X one associates a complex of `-adic sheaves n K E on the moduli stack of rank n vector bundles on X with a section, which is cuspidal and satisfies the Hecke property for E. This is a geometric counterpart of the well-known construction due to J. Shalika [19] and I. Piatetski-Shapiro [18]. We express the cohomology of the tensor product n K E 1 ⊗n K E 2 in terms of cohomology of the symmetric powers of X . This may be considered as a geometric interpretation of the local part of the classical RankinSelberg method for GL(n) in the framework of the geometric Langlands program. 0. Introduction This is the first in a series of two papers, where we propose a geometric version of the classical Rankin-Selberg method for computation of the scalar product of two cuspidal automorphic forms on GL(n) over a function field. This geometrization fits in the framework of the geometric Langlands program initiated by V. Drinfeld, A. Beilinson, and Laumon. Let X be a smooth, projective, geometrically connected curve over Fq . Let ` be a prime invertible in Fq . According to the Langlands correspondence for GL(n) over ¯ `function fields (proved by L. Lafforgue), to any smooth geometrically irreducible Q sheaf E of rank n on X is associated a (unique up to a multiple) cuspidal automorphic ¯ ` , which is a Hecke eigenvector with respect to E. The form ϕ E : Bunn (Fq ) → Q function ϕ E is defined on the set Bunn (Fq ) of isomorphism classes of rank n vector bundles on X . The classical method of Rankin and Selberg for GL(n) may be divided into two parts: local and global. The global result calculates for any integer d the scalar product DUKE MATHEMATICAL JOURNAL c 2002 Vol. 111, No. 3, Received 9 June 2000. Revision received 21 February 2001. 2000 Mathematics Subject Classification. Primary 11R39; Secondary 11S37, 11F70, 14H60.

451

452

SERGEY LYSENKO

of two (appropriately normalized) automorphic forms X 1 ϕ E 1∗ (L)ϕ E 2 (L), # Aut L d

(1)

L∈Bunn (Fq )

where Bundn (Fq ) is the set of isomorphism classes of vector bundles L on X of rank n and degree d, and # Aut L stands for the number of elements in Aut L. More precisely, this scalar product vanishes if and only if E 1 and E 2 are nonisomorphic. In the case E 1 → f E2 → f E, the answer is expressed in terms of the action of the geometric Frobenius endomorphism on H1 (X ⊗ F¯ q , End E). The computation of (1) is based on the equality of formal series X X 1 ϕ E 1∗ (L)ϕ E 2 (L)t d = L(E 1∗ ⊗ E 2 , q −1 t). n−1 # Aut( ,→ L) n−1 d≥0 (

,→L)∈ n Md (Fq )

(2) Here n Md (Fq ) is the set of isomorphism classes of pairs ,→ L), where L is a vector bundle on X of rank n and degree d + n(n − 1)(g − 1), and is the canonical invertible sheaf on X (n−1 is embedded in L as a subsheaf; i.e., the quotient is allowed to have torsion). We have denoted by L(E 1∗ ⊗ E 2 , t) the L-function attached to the local system E 1∗ ⊗ E 2 on X . Recall that the existence of the automorphic form ϕ E is a descent problem (cf. [12]). Using an explicit construction due to Shalika [19] and Piatetski-Shapiro [18], ¯ ` -sheaf E of rank n on X a function ϕ˜ E : n Md (Fq ) → one associates to a smooth Q ¯ ` , which is cuspidal and satisfies the Hecke property with respect to E. The LangQ lands conjecture predicts that when E is geometrically irreducible, ϕ˜ E is constant d+n(n−1)(g−1) along the fibres of the projection n Md (Fq ) → Bunn (Fq ); that is, ϕ˜ E is the pullback of a function ϕ E on Bunn (Fq ). So, (2) is a statement independent of the Langlands conjecture. In fact, (2) is of local nature: it is true for any local systems E 1 and E 2 of rank n on X after replacing ϕ E by ϕ˜ E . The main result of this paper is a strengthened geometric version of the equality (n−1

X (n−1 ,→L)∈ n Md (Fq )

1 ϕ˜ E ∗ (L)ϕ˜ E 2 (L) # Aut(n−1 ,→ L) 1 X (d) = q −d tr Fr, (E 1∗ ⊗ E 2 ) D (3) D∈X (d) (Fq )

of coefficients in (2) for each d ≥ 0. Here X (d) is the dth symmetric power of X , ¯ ` -sheaf on X (d) (cf. Section 1), and Fr is the geo(E 1∗ ⊗ E 2 )(d) is a constructable Q metric Frobenius endomorphism. s Let n Md denote the moduli stack of pairs (n−1 ,→ L), where L is a vector bundle of rank n and degree d + n(n − 1)(g − 1) on X , and s is an inclusion of

LOCAL GEOMETRIZED RANKIN-SELBERG METHOD FOR GL(n)

453

O X -modules. Following Drinfeld [3] (n = 2) and P. Deligne (n = 1), Laumon [12] ¯ ` -sheaves n K d on n Md , which is a geometric counterhas defined a complex of Q E ∗ part of ϕ˜ E . The geometric Langlands conjecture predicts that when E is a smooth ¯ ` -sheaf of rank n on X , n K d descends with respect to the geometrically irreducible Q E projection n Md → Bunn , where Bunn is the moduli stack of rank n vector bundles on X . ¯ ` -sheaves E 1 , E 2 of rank n on X and any d ≥ 0 a We establish for any smooth Q canonical isomorphism

f R0 X (d) , (E 1∗ ⊗ E 2 )(d) (d)[2d], R0 c (n Md , n K Ed∗ ⊗ n K Ed2 ) → 1

which is a geometric version of (3). In fact, a more general statement is proved. 0.1. Conventions and notation 0.1.1. Fix an algebraically closed ground field k of characteristic p > 0, a prime ` 6 = p, and ¯ ` of Q` . All the schemes and stacks we use are defined over k. an algebraic closure Q Throughout the paper, X denotes a fixed smooth projective connected curve of genus g ≥ 1 (over k). ¯ `We work with algebraic stacks in smooth topology and with (perverse) Q sheaves on them. If X is an algebraic stack locally of finite type, then the notion ¯ ` -sheaf on X localizes in smooth topology and, hence, makes perof a (perverse) Q fect sense. However, the corresponding derived category is problematic. We adopt the point of view that an appropriate formalism exists. (It is partially established in [13].) Let f : X → Y be a morphism of algebraic stacks. The functors f ∗ , f ∗ , f ! are understood in the derived category sense. We say that f is a generalized affine fibration of rank m in the following cases: first, if locally in smooth topology on Y there exists a homomorphism L → L 0 of locally free coherent sheaves on Y and an L 0 -torsor Y 0 → Y such that f is identified with Y 0 /L → Y , the quotient being taken in stack sense, and rk L 0 − rk L = m; second, if the map f can be written as the composition of generalized affine fibrations P of first type of ranks m 1 , . . . , m k with m i = m. We essentially use the fact that for ¯`→ ¯ ` (−m)[−2m]. a generalized affine fibration f of rank m one has f ! Q fQ ∗ ¯ and denote by Lψ the ArtinWe fix a nontrivial additive character ψ : F p → Q ` Schreier sheaf on A1k associated to ψ (see [2, Section 1.7]). Fix also a square root of ¯ ` , and use it to define the sheaf Q ¯ ` (1/2) over Spec F p and, hence, over Spec k. p in Q normalize n K Ed as in Remark 1 (cf. Section 2.1). This also gives a normalization of ϕ˜ E as the function “trace of Frobenius” of n K Ed .

∗ We

454

SERGEY LYSENKO

0.1.2. When we say that a stack Y classifies something, it should always be clear what an S-family of something is for any k-scheme S; that is, what the groupoid Hom(S, Y ) and what the functors Hom(S2 , Y ) → Hom(S1 , Y ) are for each morphism S1 → S2 . For example, if Y is the stack that classifies pairs M1 ,→ M2 with M1 (resp., M2 ) being a coherent sheaf on X of generic rank i 1 and of degree d1 (resp., of generic rank i 2 and of degree d2 ), then Hom(S, Y ) is the groupoid whose objects are inclusions M1 ,→ M2 of coherent sheaves on S × X which are S-flat and such that the quotient M2 /M1 is also S-flat, and for any point s ∈ S the conditions on the generic rank and on the degree of Mi |s×X (i = 1, 2) hold. Morphisms from an object M1 ,→ M2 to an object M10 ,→ M20 are by definition the isomorphisms M1 → f M10 and M2 → f M20 , making the natural diagram commutative. We denote by Shi the moduli stack of coherent sheaves on X of generic rank i. This is an algebraic stack locally of finite type. Its connected components are numbered by d ∈ Z; the component Shid classifies coherent sheaves of rank i and of degree d on X . The stack Shd0 is, in fact, of finite type. By Pic X ⊂ Sh1 we denote the open substack classifying invertible O X -modules. This is the Picard stack of X . Its connected component Picd X classifies line bundles of degree d on X . Denote by ≤n Shd0 ⊂ Shd0 the open substack given by the following property: For a scheme S an object F of Hom(S, Shd0 ) lies in Hom(S, ≤n Shd0 ) if the geometric fibre of F at any point of X × S is of dimension at most n. We write X (d) for the dth symmetric power of X . The morphism norm is denoted by div : Shd0 → X (d) (cf. [10, Section 6]). It sends the O X -module O D1 +···+Ds ⊕ O D2 +···+Ds ⊕ · · · ⊕ O Ds to D1 + 2D2 + · · · + s Ds if D1 , . . . , Ds are effective divisors on X . 0.1.3. Fix the maximal torus of diagonal matrices in GL(n) and the Borel subgroup of uppertriangular matrices. Then the set of weights of GL(n) is identified with Zn . The fundamental weights are given by ωi = (1, . . . , 1, 0, . . . , 0) ∈ Zn , where 1 occurs i times (i = 1, . . . , n). p Define the following semigroups 3+ n ⊂ 3n ⊂ 3n , consisting of weights. Let p 3n = Zn+ and 3n = {λ ∈ Zn | λ1 + · · · + λi ≥ 0 for all i}. The superscript p should p designate that 3n contains the Z+ -span of positive roots. Set also 3+ n = {λ = (λ1 ≥ · · · ≥ λn ≥ 0) | λi ∈ Z}. Similarly, we let 3− = {λ = (0 ≤ λ ≤ · · · ≤ λn ) | λi ∈ 1 n Z}. For d ≥ 0 we also introduce 3n,d ⊂ 3n , 3+ ⊂ 3+ n , and so on, where the Pn,d subscript d means that we impose the condition λi = d. The half sum of positive roots is denoted by ρ.


455

λ , X λ , X λ , and X λ , which For a weight λ of GL(n) we introduce the schemes X + − p p + should be thought of as the moduli schemes of 3n (resp., of 3− , 3 , 3 )-valued n n n divisors on X of degree λ. The precise definition is as follows. Qn Set X λp = i=1 X (λ1 +···+λi ) . A point of X λp is a collection of (not necessarily effective) divisors (D1 , . . . , Dn ) on X with D1 + · · · + Di ∈ X (λ1 +···+λi ) . Let X λ ,→ λ (resp., X λ ) be the X λp be the closed subscheme given by Di ≥ 0 for all i. Let X + − closed subscheme of X λ given by D1 ≥ · · · ≥ Dn (resp, D1 ≤ · · · ≤ Dn ). P Given a closed point (Di ) of X iλ with Di = x di,x x, we associate to it a divisor p on X with values in 3n . The value of this divisor at x is the weight (d1,x , . . . , dn,x ). λ , X λ ) can be viewed as a 3 (resp., In the same way, a closed point of X λ (resp., X + n − + − 3n , 3n )-valued divisor on X .

0.1.4. λ ¯ For λ ∈ 3+ n,d , define the polynomial functor V of a Q` -vector space V as follows. Let λ = (λ1 , . . . , λn 0 , 0, . . . , 0) with λn 0 > 0. Denote by U λ the irreducible representation of Sd (over Q) associated to λ. So, for example, if λ = (d, 0, . . . , 0), then Uλ → f Q is trivial, and if λ = (1, . . . , 1), then U λ is the signature representation. Set V λ = (V ⊗d ⊗Q U λ ) Sd , where it is understood that Sd acts by permutations on V ⊗d and diagonally on the tensor product. If m = dim V < n 0 , then V λ = 0; otherwise, V λ is the irreducible representation of GL(V ) of the highest weight (λ1 , . . . , λn 0 , 0, . . . , 0) ∈ 3+ m,d . 1. Laumon’s perverse sheaf L Ed ¯ ` -sheaf on X . Recall the definition of Laumon’s perverse sheaf Let E be a smooth Q L Ed on Shd0 associated to E (see [12]). Denote by sym : X d → X (d) the natural ¯ ` -sheaf E d on X d . Notice that sym! (E d )[d] is a map, and consider the smooth Q perverse sheaf. Set S E (d) = sym! (E d ) d . Since E (d) is a direct summand of sym! (E d ), E (d) [d] is also a perverse sheaf. Denote by F l 1,...,1 (1 occurs d times) the stack of complete flags (F1 ⊂ · · · ⊂ Fd ), where Fi is a coherent torsion sheaf on X of length i. The morphism p : F l 1,...,1 → Shd0 that sends (F1 ⊂ · · · ⊂ Fd ) to Fd is representable and proper. The morphism q : F l 1,...,1 → Sh10 × · · · × Sh10 that sends (F1 ⊂ · · · ⊂ Fd ) to (F1 , F2 /F1 , . . . , Fd /Fd−1 ) is a generalized affine fibration. This, in particular, implies that F l 1,...,1 is smooth. Springer’s sheaf SprdE on Shd0 is defined as SprdE = p! q∗ (div×d )∗ (E d ).

456

SERGEY LYSENKO

Since p is small, SprdE is a perverse sheaf that coincides with the Goresky-MacPherson extension of its restriction to any nonempty open substack of Shd0 . It also carries a natural Sd -action (cf. [12, Theorem 3.3.1]). Set L Ed = Hom Sd (triv, SprdE ),

where triv denotes the trivial representation of the symmetric group Sd . Again, L Ed is a direct summand of SprdE , so L Ed is perverse and coincides with the GoreskyMacPherson extension of its restriction to any nonempty open substack of Shd0 . We have a smooth morphism X (d) → Shd0 that sends a divisor D to O D , and the pullback of L Ed under this map is identified with E (d) .

2. Main results 2.1 Fix n > 0, d ≥ 0. Let be the canonical invertible sheaf on X . Denote by n Qd the stack that classifies collections 0 = L 0 ⊂ L 1 ⊂ · · · ⊂ L n ⊂ L , (si ) , (4) where L n ⊂ L is a modification of rank n vector bundles on X with deg(L/L n ) = d, (L i ) is a complete flag of subbundles on L n , and si : n−i → f L i /L i−1 is an isomorphism (i = 1, . . . , n). We have a map µ : n Qd → A1k that at the level of k-points sends the above collection to the sum of n − 1 classes in k→ f Ext1 (n−i−1 , n−i ) → f Ext1 (L i+1 /L i , L i /L i−1 ) which correspond to the succesive extensions 0 → L i /L i−1 → L i+1 /L i−1 → L i+1 /L i → 0. Let β : n Qd → ≤n Shd0 be the map that sends (4) to L/L n . It is of finite type and Pn−1 2 smooth of relative dimension b = b(n, d) = nd + (1 − g) i=1 i . Therefore, n Qd is ¯ ` -sheaf on X , then on n Qd we have smooth and of finite type. So, if E is a smooth Q ¯ ` -sheaf a perverse Q b d ∗ d ∗ . n F E,ψ = β L E ⊗ µ Lψ [b] 2 Let π0 : n Qd → X (d) be the map that sends (4) to the divisor D ∈ X (d) for which the inclusion of invertible sheaves ∧n L n ,→ ∧n L induces an isomorphism ∧n L n (D) → f ∧n L. We also have a map X (d) → Picd X that sends a divisor D to O X (D). Let n Md be the stack classifying pairs (n−1 ,→ L), where L is an n-bundle on X with deg L − deg((n−1)+(n−2)+···+(n−n) ) = d.


457

The forgetful map ζ : n Qd → n Md is representable, and the following diagram commutes: π0 → X (d) n Qd ↓ζ ↓ n Md

θ

→ Picd X

where θ is the map that sends (n−1 ,→ L) to det L ⊗(1−n)+(2−n)+···+(n−n) . Denote by π : n Qd ×n Md n Qd → X (d) ×Picd X X (d) the morphism π0 × π0 . Since X (d) → Picd X is representable and separated, the diagonal map i : X (d) → X (d) ×Picd X X (d) is a closed immersion. Our main result is the next theorem. MAIN LOCAL THEOREM

¯ ` -sheaves E, E 0 on X of ranks m, m 0 , respectively, with min{m, m 0 } For any smooth Q ≤ n, there exists a canonical isomorphism d π! (n F E,ψ n F Ed 0 ,ψ −1 ) → f i ∗ (E ⊗ E 0 )(d) (d)[2d]

in the derived category on X (d) ×Picd X X (d) . Remark 1 (i) The stack n Qd ×n Md n Qd is of finite type, though n Md is not, so that π is of finite type but not representable. d ). The geometric (ii) Define the complex n K Ed on n Md as n K Ed = ζ! (n F E,ψ ¯ ` -sheaf of Langlands conjecture claims that if E is a smooth irreducible Q d rank n on X , then for each d ≥ 0 the complex n K E descends with respect to the projection n Md → Bunn . 2.2 ¯ ` -vector spaces E, E 0 Actually, we prove a more general statement. Recall that for Q 0 of dimensions m, m , respectively, we have M E λ ⊗ (E 0 )λ , Symd (E ⊗ E 0 ) = + λ∈3r,d

where r = min{m, m 0 }. To formulate the version of the Main Local Theorem that we actually prove, we globalize the above equality as follows. ¯ ¯ For λ ∈ 3+ n,d and a smooth Q` -sheaf E on X , we define a constructable Q` -sheaf λ λ E + on X + (cf. Section 3.1) which is a global analog of the corresponding polynomial

458

SERGEY LYSENKO

P λ at D = functor. The fibre of E + x λx x is the tensor product over closed points of X , O (E x )λx , x∈X

where E x denotes the fibre of E at x. For example, for λ = (d, 0, . . . , 0) we have λ = X (d) and E λ = E (d) . Another example is that for λ = ω we obtain X λ = X X+ i + + λ = ∧i E. and E + λ → X (d) the map that sends (D ≥ · · · ≥ D ≥ 0) ∈ X λ to Denote by π λ : X + n 1 + P Di . 1 ¯ ` -sheaves E, E 0 on X of ranks m, m 0 , respectively, there is a canonFor any smooth Q ical filtration 0 = ≤0 (E ⊗ E 0 )(d) ⊂ ≤1 (E ⊗ E 0 )(d) ⊂ · · · LEMMA

on (E ⊗ E 0 )(d) by constructable subsheaves with the following property. First, if min{m, m 0 } ≤ n, then ≤n (E ⊗ E 0 )(d) = (E ⊗ E 0 )(d) . Second, there is a canonical refinement of this filtration such that M λ gr ≤n (E ⊗ E 0 )(d) → f π∗λ (E + ⊗ E 0+λ ) λ∈3+ n,d

for each n. MAIN LOCAL THEOREM n

¯ ` -sheaves E, E 0 on X there exists a canonical isomorphism For any smooth Q d π! (n F E,ψ n F Ed 0 ,ψ −1 ) → f i ∗ ≤n (E ⊗ E 0 )(d) (d)[2d]

in the derived category on X (d) ×Picd X X (d) . 2.3 The proof consists of the following steps. Let us denote by n Xd the stack classifying collections (L , (ti )), where L is a vector bundle on X of rank n, ti : (n−1)+(n−2)+···+(n−i) ,→ ∧i L is an inclusion of O X -modules (i = 1, . . . , n), and deg L − deg((n−1)+(n−2)+···+(n−n) ) = d. Given an object of n Qd , we get the morphisms ti : (n−1)+···+(n−i) → f ∧i L i ,→ ∧i L .


459 ϕ

This defines a map ϕ : n Qd → n Xd . Notice that ζ : n Qd → n Md factors as n Qd → n Xd → n Md , where the second arrow is the forgetful map. Since n Xd → n Md is representable and separated, the natural map n Qd ×n Xd n Qd → n Qd ×n Md n Qd is a closed immersion. Let π 0 : n Qd ×n Xd n Qd → X (d) ×Picd X X (d) be the restriction of π to n Qd ×n Xd n Qd . The first step is to establish the following result. THEOREM A ¯ ` -sheaves E, E 0 on X , the natural map For any smooth Q d d π! (n F E,ψ n F Ed 0 ,ψ −1 ) → π!0 (n F E,ψ n F Ed 0 ,ψ −1 )

is an isomorphism. Our proof of Theorem A is based on Proposition 1, which is a corollary of the geometric Casselman-Shalika formula for GL(n) (cf. [16], [7], [17]). We present it in Section 3, written independently of the rest of the paper. The second step is as follows. Let φ : n Xd → X (d) be the map that sends (L , (ti )) to the divisor D ∈ X (d) such that tn induces an isomorphism (n−1)+···+(n−n) (D) → f ∧n L , so that φ ◦ ϕ = π0 . We write f : n Qd ×n Xd n Qd → X (d) for the composition φ

×n Xd n Qd → n Xd → X (d) , where the first map is the natural projection. The morphism f is of finite type but not representable. Since the diagram n Qd

n Qd

×n Xd n Qd ↓ f X (d)

,→ i

→

n Qd

×n Md n Qd ↓π

X (d) ×Picd X X (d)

commutes, the Main Local Theorem is just a combination of Theorem A with the following result. THEOREM B ¯ ` -sheaves E, E 0 on X , there is a canonical isomorphism For any smooth Q d f ! (n F E,ψ n F Ed 0 ,ψ −1 ) → f

≤n

(E ⊗ E 0 )(d) (d)[2d].

(5)

We present two different proofs of Theorem B. In the first proof, which occupies Sections 6.1–6.5, we derive Theorem B from the following result.

460

SERGEY LYSENKO

THEOREM C Let ≤n div : ≤n Shd0 → X (d) denote the restriction of div : Shd0 → X (d) . For any ¯ ` -sheaves E, E 0 on X , the complex (≤n div)! (L d ⊗ L d0 ) is placed in degrees smooth Q E E less than or equal to −2d, and for the highest cohomology sheaf of this complex we have canonically

f ≤n (E ⊗ E 0 )(d) . R−2d (≤n div)! (L Ed ⊗ L Ed0 )(−d) → In Section 6.6 we present an alternative proof of Theorem B. The idea of this proof was communicated to the author by D. Gaitsgory. This proof requires the additional assumption min{rk E, rk E 0 } ≤ n. The reader interested in the proof of the Main Local Theorem under this assumption may skip Sections 6.1–6.5.

3. Around the geometric Casselman-Shalika formula for GL(n) 3.1 The purpose of Section 3 is to present Proposition 1, which is a corollary of the geometric Casselman-Shalika formulae for GL(n) (cf. [16], [7], [17]). To formulate it we introduce some notation. λ Fix λ ∈ 3− n,d . Recall that X − is the scheme of collections (D1 , . . . , Dn ), where Di is an effective divisor on X of degree λi with D1 ≤ · · · ≤ Dn . Let λ iλ : X − → ≤n Shd0

be the map that sends (D1 , . . . , Dn ) to n−1 (D1 )/ n−1 ⊕ n−2 (D2 )/n−2 ⊕ · · · ⊕ O (Dn )/O . ¯ ` -sheaf on X , then the complex According to [12, Theorem 3.3.8], if E is a smooth Q d ∗ i λ L E is placed in degrees less than or equal to 2a(λ) with respect to the usual tstructure, where def

a(λ) = λ, (n − 1, n − 2, . . . , 0) . Moreover, if m ∈ N is such that λ = (0, . . . , 0, λn−m+1 , . . . , λn ) with λn−m+1 > 0, then the 2a(λ) th cohomology sheaf of i λ∗ L Ed vanishes if and only if rk E < m. For a weight λ = (λ1 , . . . , λn ), set λt = (λn , . . . , λ1 ). Also, denote by t : λ λt the isomorphism that sends (D , . . . , D ) to (D , . . . , D ). X− → f X+ n n 1 1 Definition 1 λ on X λ by ¯ ` -sheaf E on X , define the sheaf E − For any smooth Q − λ E− = H 2a(λ) (i λ∗ L Ed ) a(λ) .


461

λ on X λ by E λ = t E λ . Also, define the sheaf E + ∗ − + + t

t

t

λ be the vector bundle whose fibre over (D , . . . , D ) is the vector space Let S λ → X − n 1 of collections (σ1 , . . . , σn−1 ), where σi ∈ Hom n−i−1 , n−i (Di )/n−i .

By µ S : S λ → A1 we denote the map that at the level of k-points sends (σ1 , . . . , σn−1 ) to the sum of n −1 classes in k → f Ext1 (n−i−1 , n−i ) corresponding to the pullbacks of 0 → n−i → n−i (Di ) → n−i (Di )/n−i → 0 with respect to σi : n−i−1 → n−i (Di )/n−i . λ and a flag Let W λ be the following stack of collections: (D1 , . . . , Dn ) ∈ X − (F 1 ⊂ · · · ⊂ F n ) of coherent torsion sheaves on X with trivializations F i /F i−1 → f n−i (Di )/n−i λ is a generalized affine fibration of for i = 1, . . . , n. The projection τ : W λ → X − rank zero. λ defined as follows. Given an S-point Let κ : W λ → S λ be the morphism over X − λ of W , consider for i = 1, . . . , n − 1 the exact sequence

0 → n−i−1 → n−i−1 (Di+1 ) → n−i−1 (Di+1 )/n−i−1 → 0. (Here should be understood as the sheaf of relative differentials S×X/S .) It induces a map H om n−i−1 , n−i (Di )/n−i → E xt 1 n−i−1 (Di+1 )/n−i−1 , n−i (Di )/ n−i (6) which is an isomorphism of O S×X -modules, because Di+1 ≥ Di . The map κ sends this point of W λ to (σ1 , . . . , σn−1 ), where σi is the global section of H om(n−i−1 , n−i (Di )/n−i ), whose image under (6) corresponds to the extension 0 → F i /F i−1 → F i+1 /F i−1 → F i+1 /F i → 0. Denote also by βW : W λ → to F n .

≤n

Shd0 the morphism that sends (F 1 ⊂ · · · ⊂ F n )

PROPOSITION 1 ¯ ` -sheaf E on X , there is a canonical isomorphism For any smooth Q ∗ λ τ! (βW L Ed ⊗ κ ∗ µ∗S Lψ ) → f E− .

The proof is given in Section 3.3.

(7)

462

SERGEY LYSENKO

3.2. Local lemma For the convenience of the reader, we begin with a local counterpart of Proposition 1, working completely in a local setting. Let O be a complete local k-algebra with residue field k, which is regular of dimension one (that is, choosing a generator ω of the maximal ideal m ⊂ O , one identifies O with the ring k[[ω]] of formal power series of one variable). Denote by K the field of fractions of O . Let be the completed module of relative differentials of O over k (so, is a free O -module generated by dω). For i ≥ 0 we write i for the ith tensor power of (over O ). For an integer m, denote by i (m) ⊂ i ⊗O K the O -submodule generated by ω−m dω⊗i . λ Recall that we have fixed λ ∈ 3− n,d . Consider the stack Wloc classifying collections: a flag of torsion sheaves (F 1 ⊂ · · · ⊂ F n ) over Spf O with trivializations F i /F i−1 → f n−i (λi )/n−i for i = 1, . . . , n. (The subscript “loc” stands for local counterparts of certain stacks λ → Spec k is a generalized affine fibration of rank zero. or morphisms.) Clearly, Wloc λ whose set of k-points is the set of (σ , . . . , σ We also have the scheme Sloc 1 n−1 ) with σi ∈ Hom n−i−1 , n−i (λi )/n−i . λ → A1 that at the level of k-points sends Besides, we have a map (µ S )loc : Sloc P λ → S λ in the (σ1 , . . . , σn−1 ) to res σi . One also defines a morphism κloc : Wloc loc same way as κ. Let ≤n Shd0 (O ) be the stack classifying coherent torsion sheaves F on Spf O of length d for which dim(F ⊗O k) ≤ n. It is stratified by locally closed substacks ν Shν (O ) indexed by ν ∈ 3+ n,d . The stratum Sh (O ) classifies sheaves isomorphic to

O /mν1 ⊕ · · · ⊕ O /mνn .

Let Bν be the intersection cohomology sheaf associated to the constant sheaf on the stratum Shν (O ). Let λ βW ,loc : Wloc → ≤n Shd0 (O ) be the map that sends (F 1 ⊂ · · · ⊂ F n ) to F n . A local version of Proposition 1 can be stated as follows. LEMMA 2 For any ν ∈ 3+ n,d , we have canonically λ R0c Wloc ,

∗ βW ,loc Bν

∗ ⊗ κloc (µ S )∗loc Lψ

( 0 → f ¯ ` [−d](a(λ)) Q

if ν t 6= λ, if ν t = λ.


463

Proof λ the stack of collections: a flag of torsion sheaves ( Fˇ 1 ⊂ · · · ⊂ Fˇ n ) Denote by Wˇloc on Spf O with trivializations Fˇ i / Fˇ i−1 → f n−i /n−i (−λn−i+1 ) λ that sends (F 1 ⊂ · · · ⊂ F n ) λ → for i = 1, . . . , n. We have an isomorphism Wloc f Wˇloc 1 n to the flag ( Fˇ ⊂ · · · ⊂ Fˇ ) with

Fˇ i = E xt 1 (F n /F n−i, n−1 ) for i = 1, . . . , n. This duality allows us to switch between dominant and antidominant weights of GL(n). Put Lˇ i = n−1 ⊕ · · · ⊕ n−i for i = 1, . . . , n. Denote by G r d,+ ( Lˇ n ) the moduli scheme of O -sublattices Rˇ ⊂ Lˇ n such that dim( Lˇ n /Rˇ ) = d. Choosing a trivialization Lˇ n → f O n , one identifies this scheme with the connected d,+ d,+ component G r = G r (O n ) of the positive part of the affine grassmanian for GL(n). We have a locally closed subscheme Sˇ ,→ G r d,+ ( Lˇ n ) whose set of k-points consists of Rˇ with the following property. If Rˇi = Rˇ ∩ Lˇ i , then the image of the inclusion Rˇi /Rˇi−1 ,→ n−i is n−i (−λn−i+1 ) for i = 1, . . . , n. λ given by Fˇ i = L ˇ i /Rˇi for i = 1, . . . , n. We have a map ηˇ loc : Sˇ → Wˇloc t One checks that ηˇ loc is an affine fibration of rank a(λ ). We also have a smooth and surjective map G r d,+ ( Lˇ n ) → ≤n Shd0 (O ) that sends Rˇ ⊂ Lˇ n to Lˇ n /Rˇ , and we denote by G r ν ( Lˇ n ) the preimage of the stratum ¯ ` [2hν, ρi](hν, ρi) Shν (O ) under this map. The Goresky-MacPherson extension of Q ν ˇ from G r ( L n ) to its closure is a perverse sheaf denoted by Aν (cf. [6]). So, our assertion is nothing but the geometric Casselman-Shalika formulae (cf. [16], [7], [17]): ( 0 if ν t 6= λ, ∗ ∗ ∗ ˇ R0c S, Aν ⊗ ηˇ loc κloc (µ S )loc Lψ → f ¯ ` [−2hν, ρi](−hν, ρi) if ν t = λ. Q We need Lemma 2 in a slightly different form. Put L i = n−1 (λ1 ) ⊕ · · · ⊕ n−i (λi ) for i = 1, . . . , n. Let G r d,+ (L n ) be the moduli scheme of O -sublattices R ⊂ L n such that dim(L n /R ) = d. As in the proof of Lemma 2, on G r d,+ (L n ) we get a stratification by locally closed subschemes G r ν (L n ) indexed by ν ∈ 3+ n,d , and the perverse sheaves Aν .

464

SERGEY LYSENKO

By S ⊂ G r d,+ (L n ) we denote the locally closed subscheme whose set of kpoints consists of sublattices R with the following property. Let Ri = R ∩ L i . The condition is that the image of the inclusion Ri /Ri−1 ,→ L i /L i−1 → f n−i (λi )

is the sublattice n−i ⊂ n−i (λi ) for i = 1, . . . , n. We also have a map ηloc : S → λ given by F i = L /R for i = 1, . . . , n. This is an affine fibration of rank a(λ). Wloc i i Let i : Spec k → S be the distinguished point that corresponds to R = n−1 ⊕ · · · ⊕ O . Put µloc = (µ S )loc ◦ κloc ◦ ηloc . We use Lemma 2 under the following form. LEMMA 3 For any ν ∈ 3+ n,d we have canonically

( 0 R0c (S, Aν ⊗ µ∗loc Lψ ) → f ¯ ` [−2hλ, ρi](−hλ, ρi) Q

if ν t 6= λ, if ν t = λ.

Moreover, this isomorphism is obtained by applying the functor R0c to the composition of the canonical maps Aν ⊗ µ∗loc Lψ → i ∗ i ∗ (Aν ⊗ µ∗loc Lψ ) → τ≥ 2hλ,ρi i ∗ i ∗ (Aν ⊗ µ∗loc Lψ ) . Proof As is easy to see, if ν t 6= λ, then the fibre i ∗ (Aν ⊗ µ∗loc Lψ ) is placed in (usual) degrees strictly less then 2hλ, ρi. For ν t = λ this fibre equals ¯ ` −2hλ, ρi −hλ, ρi . Q Since the closure of G r λ (L n ) in G r d,+ (L n ) is the union of strata G r ν (L n ) with ν ≤ λt , our assertion follows from Lemma 2 combined with the geometric statement due to B. C. Ngo (cf. [17, Lemma 5.2]): (i) S ∩ G r ν (L n ) = ∅ for ν < λt , t

(ii)

i

S ∩ G r ν (L n ) is the point Spec k ,→ S for ν = λt .

Remark 2 The shift in degree and the twist are calculated using the following two formulae. For any weight ν of GL(n) we have a(ν)−a(ν t ) = 2hν, ρi and 2a(ν)−2hν, ρi = d(n−1), P where d = νi . 3.3. Proof of Proposition 1 We return to our notation in the global case (as in Section 3.1).


465

λ , a diagram Step 1. Denote by Weλ the scheme of collections: (D1 , . . . , Dn ) ∈ X −

L1 ∪ R1

Ln ∪ ⊂ · · · ⊂ Rn ⊂ ··· ⊂

(8)

where L i = n−1 (D1 ) ⊕ · · · ⊕ n−i (Di ) for i = 1, . . . , n, and (Ri ) is a complete flag of vector subbundles on an n-bundle Rn such that the natural map Ri /Ri−1 ,→ L i /L i−1 → f n−i (Di ) λ induces an isomorphism Ri /Ri−1 → f n−i . We have a map η : Weλ → W λ over X − given by F i = L i /Ri

for i = 1, . . . , n. This is an affine fibration of rank a(λ), so that ¯`→ ¯ ` − a(λ) − 2a(λ) . η! Q fQ Put τ˜ = τ ◦ η. We replace the functor τ! (·) by τ˜! η∗ (·) a(λ) 2a(λ) . The advantage is that τ˜ is representable whereas τ is not. λ admits a canonical section ξ : X λ → W eλ defined The morphism τ˜ : Weλ → X − − by Ri = n−1 ⊕ · · · ⊕ n−i for i = 1, . . . , n. Notice that ξ is a closed immersion. The following diagram commutes: η τ˜ λ Wλ ← Weλ → X − ↓ βW ↑ ξ % id ≤n

Shd0

iλ

←

λ X−

Besides, the composition µS κ λ ξ eλ η X− → W → W λ → S λ → A1

is the zero map. Now applying the functor τ˜! to the canonical morphism ∗ ∗ L Ed ⊗ κ ∗ µ∗S Lψ ), η∗ (βW L Ed ⊗ κ ∗ µ∗S Lψ ) → ξ∗ ξ ∗ η∗ (βW

we get a map ∗ L Ed ⊗ κ ∗ µ∗S Lψ → i λ∗ L Ed a(λ) 2a(λ) . τ! β W

(9)

466

SERGEY LYSENKO

Define (7) as the composition of (9) with the canonical map i λ∗ L Ed a(λ) 2a(λ) → H 2a(λ) (i λ∗ L Ed ) a(λ) . Now we check that (7) is an isomorphism fibre by fibre. P λ . Let D = Step 2. Fix a k-point (D1 , . . . , Dn ) of X − i x λi,x x. So, the corresponding 3− -valued divisor on X associates to x ∈ X the antidominant weight n λx = (λ1,x , . . . , λn,x ) ∈ 3− n,dx P with dx = i λi,x . For i = 1, . . . , n, put L i = n−1 (D1 ) ⊕ · · · ⊕ n−i (Di ). For every closed point x ∈ X , let (L 1 )x ⊂ · · · ⊂ (L n )x be the restriction of the flag (L 1 ⊂ · · · ⊂ L n ) to Spec Oˆ X,x . Let G r dx ,+ ((L n )x ) denote the moduli scheme of sublattices R ⊂ (L n )x such that dim (L n )x /R = dx . By Sx ⊂ G r dx ,+ ((L n )x ) we denote the locally closed subscheme whose set of kpoints consists of sublattices R ⊂ (L n )x with the following property. Let Ri = R ∩ (L i )x . The condition is that the image of the natural inclusion Ri /Ri−1 ,→ (L i )x /(L i−1 )x → f n−i (λi,x x) x is the sublattice n−i ⊂ (n−i (λi,x x))x for i = 1, . . . , n. x For every x ∈ X , one defines the stack Wxλ and the scheme Sxλ with morphisms ηx

κx

(µ S )x

Sx → Wxλ → Sxλ → A1 , which are local counterparts of the corresponding stacks and morphisms for the λ , the diagram weight λx . So, after the base change (D1 , . . . , Dn ) : Spec k → X − η κ Weλ → W λ → S λ

becomes Y x∈X

Sx →

Y x∈X

Wxλ →

Y

Sxλ ,

x∈X

the morphisms being the product of morphisms ηx and κx , respectively. The restricQ tion of µ S : S λ → A1 to x∈X Sxλ is the sum of morphisms (µ S )x . dx ,+ For ν ∈ 3+ ((L n )x ) is n,dx , the perverse sheaf Aν considered as a sheaf on G r denoted Aν,x (cf. Section 3.2).


467

From [6, Proposition 3.1 and Lemma 4.2] it follows that the restriction of Q ∗ L d to η∗ βW x∈X Sx is identified with x∈X Fx , where Fx is the restriction of E M ν

−dx (n − 1) ⊗ E xν Aν,x −dx (n − 1) 2

under the inclusion Sx ,→ G r dx ,+ ((L n )x ) (the sum being taken over ν ∈ 3+ n,dx ). Now combining Lemma 3 with [12, Theorem 3.3.8]), we get the desired assertion. This concludes the proof of Proposition 1. 2

4. Geometric Whittaker models for GL(n) 4.1. The stack n Yd Consider the stack n Xd defined in Section 2.3. We impose Plücker’s relations on a point (L , (ti )) of n Xd , which means that generically (ti ) come from a complete flag of vector subbundles of L. Our definition is justified by the following simple observation. Let V be a vector space of dimension n (over any field). For n ≥ k > i ≥ 1, let αk,i : ∧k V ⊗ ∧i V → ∧k+1 V ⊗ ∧i−1 V be the contraction map that sends u ⊗ (v1 ∧ v2 ∧ · · · ∧ vi ) to i X (−1) j (u ∧ v j ) ⊗ (v1 ∧ · · · ∧ vˆ j ∧ · · · ∧ vi ). j=1

LEMMA 4 Given nonzero elements ti ∈ ∧i V for 1 ≤ i ≤ n, the following are equivalent: (1) there exists a complete flag of vector subspaces 0 = V0 ⊂ V1 ⊂ · · · ⊂ Vn = V such that ti ∈ ∧i Vi ⊂ ∧i V ; (2) for n ≥ k > i ≥ 1, we have αk,i (tk ⊗ ti ) = 0.

Proof The statement is obvious in characteristic zero. Let us give an argument that holds in any characteristic. Write e1 ∧· · ·∧ei for the image of e1 ⊗· · ·⊗ei under V ⊗i → ∧i V . We construct by induction on k the elements e1 , . . . , ek ∈ V such that ti = e1 ∧ · · · ∧ ei for i = 1, . . . , k. Let e1 = t1 , and assume that e1 , . . . , ek−1 are already constructed. To construct ek , we show by induction on i that tk = e1 ∧ · · · ∧ ei ∧ ωk−i for some ωk−i ∈ ∧k−i V , and we define ek as ω1 .

468

SERGEY LYSENKO

First, since αk,1 (tk ⊗ t1 ) = −tk ∧ e1 = 0, we get tk = e1 ∧ ωk−1 for some ωk−1 ∈ ∧k−1 V . Now assume that tk = e1 ∧ · · · ∧ ei−1 ∧ ωk−i+1 for some ωk−i+1 ∈ ∧k−i+1 V with i < k. Then αk,i (tk ⊗ ti ) = αk,i tk ⊗ (e1 ∧ · · · ∧ ei ) = (−1)i (tk ∧ ei ) ⊗ (e1 ∧ · · · ∧ ei−1 ) = 0. It follows that tk ∧ ei = 0. So, there exists ωk−i ∈ ∧k−i V such that tk = e1 ∧ · · · ∧ ei ∧ ωk−i . We are done. Now we define the closed substack n Yd ,→ n Xd by the conditions αk,i (tk ⊗ ti ) = 0 for n ≥ k > i ≥ 1, where αk,i : ∧k L ⊗ ∧i L → ∧k+1 L ⊗ ∧i−1 L are the contraction maps defined as above. Then the map ϕ factors through n Qd → n Yd ,→ n Xd . p We stratify n Yd by locally closed substacks V pλ ⊂ n Yd numbered by λ ∈ 3n,d . λ The stratum V p is defined by the following condition: the degree of the divisor of zeros of ti : (n−1)+···+(n−i) ,→ ∧i L equals λ1 + · · · + λi for i = 1, . . . , n. Recall that a point of X λp is a collection of divisors (D1 , . . . , Dn ) on X with deg(Di ) = λi and D1 + · · · + Di ≥ 0 for all i. So, the stack V pλ classifies collections 0 = L 00 ⊂ L 01 ⊂ · · · ⊂ L 0n = L , (si ), (Di ) ∈ X λp ,

(10)

where (L i0 ) is a complete flag of subbundles on a rank n vector bundle L and si : (n−1)+···+(n−i) (D1 + · · · + Di ) → f ∧i L i0 is an isomorphism. Define the closed substack V λ ,→ V pλ as V pλ × X λp X λ . So if λ ∈ / 3n , then V λ is

empty. Notice that the projection V pλ ×n Yd n Qd → V pλ factors through V λ ,→ V pλ , and for λ ∈ 3n,d the corresponding morphism V pλ ×n Yd n Qd = V λ ×n Yd n Qd → V λ is an affine fibration of rank a(λ). d 4.2. The sheaves n P E,ψ

Definition 2 ¯ ` -sheaf E on X , put n P d = ϕ! (n F d ). For any smooth Q E,ψ E,ψ d Clearly, the restriction of n P E,ψ to a stratum V pλ of n Yd vanishes outside the closed d λ substack V λ of V pλ . For λ ∈ 3n,d , denote by n P E,ψ the restriction of n P E,ψ to V λ .


469

λ . Recall that the subscheme Define the closed substack V−λ ,→ V λ as V λ × X λ X − λ of X is given by the condition 0 ≤ D1 ≤ · · · ≤ Dn , where (Di ) ∈ X λ . Let µλ : V−λ → A1 be the map that at the level of k-points sends (10) to the sum of n − 1 classes in k→ f Ext1 n−i−1 (Di ), n−i (Di ) λ X−

corresponding to the pullbacks of the successive extensions 0→

0 0 /L 0 0 /L 0 L i0 /L i−1 → L i+1 L i+1 →0 i i−1 → ↓o ↓o n−i (Di ) n−i−1 (Di+1 )

with respect to the inclusion n−i−1 (Di ) ,→ n−i−1 (Di+1 ). PROPOSITION 2 ¯ ` -sheaf on X of rank m and λ ∈ 3n,d . Then Let E be a smooth Q λ (1) n P E,ψ vanishes unless

λ1 = · · · = λn−m = 0; (2)

(∗)

λ under condition (∗) the complex n P E,ψ is supported at V−λ ,→ V λ , and its restriction to V−λ is isomorphic to the tensor product of

µ∗λ Lψ

b − 2a(λ) b − 2a(λ) 2

λ under V λ → X λ . with the inverse image of E − − −

Remark 3 d (i) The sheaf n P E,ψ was also considered by E. Frenkel, Gaitsgory, and K. Vilonen [8, ¯ ` -sheaf E on X , n P d is a perverse Section 4.3]. They show that for any smooth Q E,ψ sheaf and the Goresky-MacPherson extension of its restriction to any nonempty open d substack of n Yd . Besides, the Verdier dual of n P E,ψ is canonically isomorphic to d P (see [8, Sections 4.6 and 4.7]). We notice that the stratification of n Yd used n E ∗ ,ψ −1 in [8, Section 4.10] is different from ours, so that our Proposition 2 is a strengthened version of [8, Proposition 4.12]. d According to [7], [8], in the case rk E = n the perverse sheaf n P E,ψ can be thought of as a geometric counterpart of the Whittaker function canonically attached to E. (ii) For m > 0, let m n Yd ⊂ n Yd denote the open substack given by the following conditions: the image of ti is a line subbundle in ∧i L for i = 1, . . . , n − m. In d particular, m n Yd = n Yd for m ≥ n. Then Proposition 2(1) claims that n P E,ψ is the extension by zero of its restriction to m n Yd ⊂ n Yd .

470

SERGEY LYSENKO

d (iii) The relation between the sheaves n P E,ψ for different n is as follows. Let m be the preimage of n Yd under ϕ : n Qd → n Yd . So, m n Qd is the open substack of n Qd parametrizing collections (4) such that L/L n−m is locally free. Denote by m n E xt the stack of collections (L 1 ⊂ · · · ⊂ L n−m+1 ⊂ L , (si )), where L/L n−m is a vector bundle on X of rank m, and si : n−i → f L i /L i−1 is an m E xt → A1 be the composition isomorphism (i = 1, . . . , n − m + 1). Let µm : n n µ m E xt → 1 , where the first arrow is the map that forgets L. Q → A n−m+1 0 n Let m M be the stack of pairs (m−1 ,→ L), where L is a vector bundle on X of rank m. Taking the quotient by L n−m , we get a map m n E xt → m M , which is a generalized affine fibration. For 1 ≤ m ≤ n there is a commutative diagram mQ n d

mQ n d

→ f

↓ mY n d

→ f

m Qd

×m M m n E xt ↓ ϕ×id m m Yd ×m M n E xt

d where the left vertical arrow is the restriction of ϕ. So, the restriction of n P E,ψ to m Y is isomorphic to n d d m P E,ψ

b(m, d) − b(n, d) ∗ (µm . n ) Lψ b(m, d) − b(n, d) 2

λ 4.3. The support of n P E,ψ In this section we prove the following lemma.

LEMMA 5 λ The complex n P E,ψ vanishes outside the closed substack V−λ ,→ V λ .

This may be derived from the geometric Casselman-Shalika formulae, but we give a direct proof. We start with the following sublemma. Given λ ∈ 3− n and ν ∈ 3n , λ , (D 0 , . . . , D 0 ) ∈ X ν and denote by Uλν the stack of collections: (D1 , . . . , Dn ) ∈ X − n 1 a diagram L 01 ⊂ · · · ⊂ L 0n (11) ∪ ∪ L1 ⊂ · · · ⊂ Ln where (L i ) (resp., (L i0 )) is a complete flag of vector subbundles on a rank n vector bundle L n (resp., L 0n ) on X with trivializations 0 → f n−i (Di + Di0 ) L i0 /L i−1


471

0 such that the image of the inclusion L i /L i−1 ,→ L i0 /L i−1 → f n−i (Di + Di0 ) equals n−i (Di ) for i = 1, . . . , n. Let λ ϕλν : Uλν → (X − × X ν ) × X λ+ν V λ+ν λ × X ν → X λ+ν denotes the summation be the map that forgets the flag (L i ). Here X − ν of divisors. The map ϕλ is an affine fibration of rank a(ν). λ × X ν be the closed subscheme defined by Let X λ,ν ,→ X − 0 Di ≥ Di−1 + Di−1 λ × X ν → X λ+ν factors through for i = 2, . . . , n. The composition X λ,ν ,→ X − λ+ν λ+ν X − ,→ X . We also have a map Uλν → V−λ that forgets (L i0 ) and (Di0 ). By abuse of notation, the composition of this map with µλ : V−λ → A1 is also denoted µλ .

SUBLEMMA 1 The complex (ϕλν )! µ∗λ Lψ is supported at the closed substack X λ,ν × X λ+ν V−λ+ν of

λ × Xν) × λ+ν and is isomorphic to the inverse image of (X − X λ+ν V µ∗λ+ν Lψ −2a(ν) −a(ν)

−

from V−λ+ν . Proof

ϕ˜λν

λ × X ν )× λ+ν Let us decompose ϕλν into two affine fibrations Uλν → U˜λν → (X − X λ+ν V defined as follows. Let U˜λν be the stack of collections: λ , (D 0 ) ∈ X ν ; • (Di ) ∈ X − i • a complete flag of vector bundles (L 01 ⊂ · · · ⊂ L 0n ) on X with trivializations 0 L i0 /L i−1 → f n−i (Di + Di0 )

•

(12)

for i = 1, . . . , n; for i = 1, . . . , n − 1 diagrams 0 L i0 /L i−1 → ∪ 0 → n−i (Di ) →

0→

0 /L 0 L i+1 i−1 ∪ Fi

0 /L 0 L i+1 →0 i ∪ → n−i−1 (Di+1 ) → 0

→

where each row is an exact sequence of O X -modules, and both left and right vertical arrows are compatible with (12).

472

SERGEY LYSENKO

Now define the morphism Uλν → U˜λν by Fi = L i+1 /L i−1 for i = 1, . . . , n − 1, where L i are from diagram (11). One checks that this is an affine fibration of rank (n − 2)ν1 + (n − 3)ν2 + · · · + νn−2 . Define also ϕ˜λν as the map that forgets all Fi . This is an affine fibration of rank ν1 + · · · + νn−1 . Clearly, µλ : Uλν → A1 is constant along the fibres of Uλν → U˜λν . So, it suffices to prove the sublemma in the case n = 2. In this case a fibre of ϕλν is the affine space of maps ξ : L 2 /L 1 → L 02 /L 1 such that the diagram commutes: 0 → L 01 /L 1 →

L 02 /L 1 ↑ξ L 2 /L 1

→ L 02 /L 01 → 0 %i

where i is the canonical inclusion compatible with trivializations. On this affine space we have a free and transitive action of Hom(L 2 /L 1 , L 01 /L 1 ). The restriction of µ∗λ Lψ to this affine space is a sheaf that changes under the action of Hom(L 2 /L 1 , L 01 /L 1 ) by a local system, say, µ˜ ∗ Lψ , where µ˜ : Hom(L 2 /L 1 , L 01 /L 1 ) → k is the following linear functional. It associates to s ∈ Hom(L 2 /L 1 , L 01 /L 1 ) the class of the pullback of 0 → L 1 → L 01 → L 01 /L 1 → 0 (13) s

under the composition O (D1 ) ,→ O (D2 ) → f L 2 /L 1 → L 01 /L 1 . The sequence (13) is just 0 → (D1 ) → (D1 + D10 ) → (D1 + D10 )/ (D1 ) → 0. So, µ˜ = 0 if and only if D2 ≥ D1 + D10 . Besides, under this condition the pullback of 0 → L 01 → L 02 → L 02 /L 01 → 0 under O (D1 + D10 ) ,→ O (D2 + D20 ) → f L 02 /L 01 is identified (after tensoring by O (−D10 )) with the pullback of 0 → L 1 → L 2 → L 2 /L 1 → 0 under O (D1 ) ,→ O (D2 ) → f L 2 /L 1 . Our assertion follows. For m ≥ 0 and ν ∈ 3m,d , denote by F l ν the stack of flags (F 1 ⊂ · · · ⊂ F m ), where F i is a coherent torsion sheaf on X with deg(F i /F i−1 ) = νi for i = 1, . . . , m. Let divν : F l ν → X ν denote the composition ν

ν

F l ν → Sh01 × · · · × Sh0m

div ×···×div

→

Xν.


473

Set also n Q ν = n Qd ×Shd F l ν , where F l ν → Shd0 sends (F 1 ⊂ · · · ⊂ F m ) to F m . 0

j

Denote by m n Jd the set of (n × m)-matrices e = (ei ) (1 ≤ i ≤ n, 1 ≤ j ≤ m) P j j with ei ∈ Z+ , i, j ei = d. We have a map h : m n Jd → 3n,d × 3m,d that sends e to j P j P j Q (ei ) m e (λ, ν), where λi = . So, j ei and ν j = i ei . For e ∈ n Jd , put Y = i, j X j

j

j

Y e classifies matrices of effective divisors (Di ) on X such that deg(Di ) = ei . For every λ ∈ 3n,d , the stack V λ ×n Yd n Q ν is stratified by locally closed substacks Q e ,→ V λ ×n Yd n Q ν indexed by e ∈ m n Jd such that h(e) = (λ, ν). The stratum Q e is the stack classifying the following collections: • a diagram Lm 1 ∪ L m−1 1 ∪ .. . ∪ L 01

⊂ ⊂

⊂

Lm 2 ∪ L m−1 2 ∪ .. .

⊂ ···

⊂

⊂ ···

⊂

∪ L 02

⊂ ···

⊂

Lm n ∪ L m−1 n ∪ .. .

(14)

∪ L 0n

j

• •

where L i is a vector bundle of rank i on X , and all the maps are inclusions of O X -modules; j a matrix (Di ) ∈ Y e ; m → isomorphisms L im /L i−1 f n−i (Di1 + · · · + Dim ) such that the image of the inclusion j

j

m L i /L i−1 ,→ L im /L i−1 → f n−i (Di1 + · · · + Dim ) j

equals n−i (Di1 + · · · + Di ) (i = 1, . . . , n; j = 0, . . . , m). We have a natural map ϕ e : Qe → Y e × X λ V λ that forgets all the rows in (14) except the top one. (Here Y e → X λ sends (Di ) to P j (Di ) with Di = j Di .) The morphism ϕ e is an affine fibration of rank a(λ). Denote by Y−e ,→ Y e the closed subscheme given by the following conditions: j (10 ) for i ≤ n − j, we have Di = 0; j 1 +···+ (20 ) for 1 ≤ j ≤ m − 1 and 2 ≤ i ≤ n , we have Di1 + · · · + Di ≥ Di−1 j

j+1

Di−1 .

µ

The composition Q e ,→ n Q ν → n Qd → A1 is denoted by µe .

474

SERGEY LYSENKO

SUBLEMMA 2 The complex (ϕ e )! µ∗e Lψ is supported at Y−e × X λ V−λ ,→ Y e × X λ V λ and is isomorphic − to the inverse image of µ∗λ Lψ −a(λ) −2a(λ)

from V−λ . Proof Apply Sublemma 1 m times forgetting successively the rows in diagram (14) starting from the lowest one and moving up. Proof of Lemma 5 Since L Ed is a direct summand of the Springer sheaf SprdE (cf. Section 1), it suffices to show that the restriction of ϕ! (β ∗ SprdE ⊗µ∗ Lψ ) to V λ vanishes outside V−λ . Put ν = (1, . . . , 1) ∈ 3d,d . The composition n Q ν → µ

→ A1 is also denoted by µ. By the projection formulae, we have to consider the direct image with respect to the projection

n Qd

V λ ×n Yd n Q ν → V λ

(15)

of pr∗2 µ∗ Lψ tensored by some local system that comes from X ν . The stack V λ ×n Yd ν e d n Q is stratified by locally closed substacks Q indexed by e ∈ n Jd such that h(e) = (λ, ν). The restriction of (15) to Q e can be decomposed as ϕe

Q e → Y e × X λ V λ → X ν × X (d) V λ → V λ .

So, our assertion follows from Sublemma 2 because the composition Y−e ,→ Y e → λ ,→ X λ . X λ factors through X − Remark 4 ¯ Using Sublemma 2, one may also check that for any λ ∈ 3− n,d and any smooth Q` λ λ sheaf E on X , E − [λn ] is a perverse sheaf on X − . 4.4. Proof of Proposition Ê2 Recall that V−λ ×n Yd n Qd is the stack classifying the following collections: (D1 , λ and a diagram . . . , Dn ) ∈ X − L 01 ∪ L1

⊂ ··· ⊂ ⊂ ··· ⊂

L 0n ∪ Ln

(16)


475

where (L i0 ) (resp., (L i )) is a complete flag of vector subbundles on a rank n vector bundle L 0n (resp., L n ) on X with trivializations 0 L i0 /L i−1 → f n−i (Di ) 0 such that the image of the natural inclusion L i /L i−1 ,→ L i0 /L i−1 → f n−i (Di ) equals n−i for i = 1, . . . , n. Denote by

η : V−λ ×n Yd n Qd → V−λ × X λ W λ −

V−λ

the morphism over , whose composition with the projection V−λ × X λ W λ → W λ − sends (16) to the flag (L 01 /L 1 ⊂ L 02 /L 2 ⊂ · · · ⊂ L 0n /L n ). One checks that η is a (representable) affine fibration of rank a(λ). Further, the composition η

id ×κ

V−λ ×n Yd n Qd → V−λ × X λ W λ → V−λ × X λ S λ −

µλ ×µ S

→

−

sum

A1 × A1 → A1

coincides with the restriction of µ : n Qd → A1 to the substack V−λ ×n Yd n Qd ,→ 2 n Qd . So, our assertion follows from Proposition 1. 5. Proof of Theorem A Recall the map φ : n Xd → X (d) (cf. Section 2.3). By abuse of notation, its restriction to n Yd ,→ n Xd is also denoted φ. We let π˜ : n Yd ×n Md n Yd → X (d) ×Picd X X (d) be the morphism φ × φ. By π˜ 0 we denote the restriction of π˜ to the diagonal n Yd ,→ n Yd ×n Md n Yd . Clearly, Theorem A is equivalent to the fact that the natural map d d π˜ ! (n P E,ψ n P Ed 0 ,ψ −1 ) → π˜ !0 (n P E,ψ ⊗ n P Ed 0 ,ψ −1 )

is an isomorphism. For λ, ν ∈ 3n,d , we denote by π˜ λ,ν the restriction of π˜ to the substack V λ ×n Md V ν ,→ n Yd ×n Md n Yd . In the case λ = ν we write (π˜ λ,λ )0 for the restriction of π˜ λ,λ to the diagonal V λ ,→ V λ ×n Md V λ . Using the stratification of n Yd ×n Md n Yd induced by both stratifications of the first and the second multiple (cf. Section 4.1), Theorem A is reduced to the following statement. 3 λ For any λ, ν ∈ 3n,d , the direct image (π˜ λ,ν )! (n P E,ψ n P Eν 0 ,ψ −1 ) vanishes unless λ = ν. Under the condition λ = ν the natural map PROPOSITION

λ λ (π˜ λ,λ )! (n P E,ψ n P Eλ 0 ,ψ −1 ) → (π˜ λ,λ )0! (n P E,ψ ⊗ n P Eλ 0 ,ψ −1 )

is an isomorphism.

476

SERGEY LYSENKO

Proof Put V1 = V−λ ×n Md V−ν . The restriction of π˜ λ,ν to V1 can be decomposed as 1π

λ ν V1 → X − ×Picd X X − → X (d) ×Picd X X (d) , λ and V ν → X ν . In the case where 1 π is the product of two projections V−λ → X − − − λ λ = ν we denote by diag : V− ,→ V1 the diagonal map. By Proposition 2, our assertion is reduced to the following lemma.

6 ∗ 1 ∗ For any λ, ν ∈ 3− n,d , the direct image π! (µλ Lψ µν Lψ −1 ) vanishes unless λ = ν. Under the condition λ = ν the natural map LEMMA

1

π! (µ∗λ Lψ µ∗λ Lψ −1 ) → 1 π! (diag)∗ (diag)∗ (µ∗λ Lψ µ∗λ Lψ −1 )

is an isomorphism. We need the next straightforward sublemma. Given a divisor D and a coherent sheaf s M on X with a section O (D) → M, denote by E xt M,D the stack classifying extensions of O X -modules 0 → (D) → ? → M → 0, and by µs : E xt M,D → A1 the map that sends this extension to the class of its pullback under s. 3 If s 6= 0, then R0 c (E xt M,D , µ∗s Lψ ) = 0. SUBLEMMA

Proof of Lemma 6 λ , (D 0 , . . . , D 0 ) The stack V1 classifies the following collections: (D1 , . . . , Dn ) ∈ X − n 1 ν 0 0 ∈ X − , two flags (L 1 ⊂ · · · ⊂ L n = L) and (L 1 ⊂ · · · ⊂ L n = L) of subbundles on a rank n vector bundle L on X with trivializations si : n−i (Di ) → f L i /L i−1

and

0 si0 : n−i (Di0 ) → f L i0 /L i−1

for i = 1, . . . , n such that (L 1 , s1 ) and (L 01 , s10 ) coincide (in particular, we have D1 = D10 ). Let Vi ,→ V1 be the closed substack defined by the condition that the flags L 1 ⊂ · · · ⊂ L i , (s j ) j=1,...,i and L 01 ⊂ · · · ⊂ L i0 , (s 0j ) j=1,...,i coincide. Let i Lψ be the restriction of µ∗λ Lψ µ∗ν Lψ −1 under Vi ,→ V1 . Also, let i

λ ν ×Picd X X − π : Vi → X −


477

be the restriction of 1 π to Vi . Arguing by induction, we show that for every i = 1, . . . , n − 1, the natural map (i π)! (i Lψ ) → (i+1 π)! (i+1 Lψ ) is an isomorphism. To do so, denote by Ni the stack of the following collections: (D1 , . . . , Dn ) ∈ 0 λ , (D 0 , . . . , D 0 ) ∈ X ν , two flags (M X− i+1 ⊂ · · · ⊂ Mn = M) and (Mi+1 ⊂ · · · ⊂ − n 1 Mn0 = M) of subbundles on a rank n − i vector bundle M on X with trivializations s j : n− j (D j ) → f M j /M j−1

and

s 0j : n− j (D 0j ) → f M 0j /M 0j−1

for j = i + 1, . . . , n. Let also 0 Ni ,→ Ni be the closed substack defined by the condition that 0 0 (Mi+1 , si+1 ) and (Mi+1 , si+1 ) coincide. Taking the quotient by L i = L i0 , we get a morphism γ : Vi → Ni , which is a generalized affine fibration. Further, we have a commutative diagram Vi+1 ↓ 0N i

,→ ,→

Vi ↓ γ & iπ ν λ × Ni → X − Picd X X −

0 : where the square is Cartesian. Applying Sublemma 3 for the section si+1 − si+1 n−i−1 i 0 (Di ) → M, one checks that the complex γ! ( Lψ ) is supported at Ni , and our assertion follows.

Proposition 3 also follows. This concludes the proof of Theorem A.

6. Proof of Theorems B and C 6.1. Plan of the proof Proposition 2 admits the following corollary. COROLLARY 1 ¯ ` -sheaves E, E 0 on X , the complex For any smooth Q def d n S E,E 0 =

d f ! (n F E,ψ n F Ed 0 ,ψ −1 )(−d)[−2d]

478

SERGEY LYSENKO

is a sheaf on X (d) placed in degree zero. It has a canonical filtration by constructable subsheaves such that M d λ gr n S E,E π∗λ (E + ⊗ E 0+λ ). 0 = λ∈3+ n,d d d For each r ≤ n there is a canonical inclusion r S E,E 0 ⊂ n S E,E 0 compatible with filtrations.

Proof d d ⊗ Pd By the projection formulae, n S E,E f φ! (n P E,ψ 0 → n E 0 ,ψ −1 )(−d)[−2d]. Calculate this direct image with respect to the stratification of n Yd by locally closed substacks p λ is a V pλ indexed by λ ∈ 3n,d (cf. Section 4.1). Since the natural map V−λ → X − generalized affine fibration of rank b − d − 2a(λ), our first assertion follows from Proposition 2. Recall the open substacks rn Yd ⊂ n Yd for r ≤ n introduced in Section 4.2, Remark 3(iii). Let ≤r φ be the restriction of φ to rn Yd . By the same remark, d d . Our second assertion follows. ≤r φ ( P d f r S E,E 0 ! n E,ψ ⊗ n P E 0 ,ψ −1 )(−d)[−2d] → This reduces our proof of Theorem B to the following steps. For ν ∈ 3m,d , ν 0 ∈ def

3m 0 ,d and c = (ν, ν 0 ), set V c = X ν × X (d) X ν . Recall our notation n Q ν = n Qd ×Shd 0 F l ν (cf. Section 4.3). Let 0

f c : n Q ν ×n Yd n Q ν → V c 0

divν × divν

0

denote the composition n Q ν ×n Yd n Q ν → F l ν × X (d) F l ν → V c . The morphism f c is of finite type. Let also n f c be the restriction of f c to the closed substack ν ν0 ν ν0 n Q ×n Qd n Q ⊂ n Q ×n Yd n Q . 0

0

The first step is as follows. 4 The morphism f c is of relative dimension less than or equal to b − d, and the natural map of the highest cohomology sheaves PROPOSITION

¯` R2(b−d) ( f c )! (µ∗ Lψ µ∗ Lψ −1 ) → R2(b−d) (n f c )! Q is an isomorphism. Further, set W c = F l ν ×Shd F l ν . Let divc : W c → V c be the map divν × divν . Set 0

≤n W c

W c.

= W c ×Shd 0

≤n Shd . 0

0

Let also ≤n divc denote the restriction of divc to

0

≤n W c

⊂


479 βc

≤n divc

The morphism n f c is decomposed as n Q ν ×n Qd n Q ν → ≤n W c → V c , where c β is the natural projection. The second step is the following lemma. 0

7 β c is smooth and surjective with connected fibres of dimension b; divc is of relative dimension less than or equal to −d, so that

LEMMA

(i) (ii)

¯ ` (b − d) → ¯ ` (−d). R2(b−d) (n f c )! Q f R−2d (≤n divc )! Q c Now assume c = (ν, ν) with ν = (1, . . . , 1) ∈ 3d,d . Let n S E,E 0 denote the direct c (d) image under V → X of the sheaf

R2(b−d) ( f c )! (µ∗ Lψ µ∗ Lψ −1 )(b − d) tensored by the local system (E d )(E 0 d ) on V c . The group Sd ×Sd acts naturally c d c on n S E,E 0 . By Corollary 1, n S E,E 0 is the sheaf of (Sd × Sd )-invariants of n S E,E 0 . Combining Lemma 7 and Proposition 4, we learn that the complex ≤n ( div)! (SprdE ⊗ SprdE 0 ) is placed in degrees less than or equal to −2d, and there is a canonical (Sd × Sd )-equivariant isomorphism c n S E,E 0

→ f R−2d (≤n div)! (SprdE ⊗ SprdE 0 )(−d).

Thus, Theorem B is reduced to Theorem C. 6.2. The stack i Z˜d For 0 ≤ i ≤ n, denote by i Q˜d the stack classifying collections 0 = L 0 ⊂ L 1 ⊂ · · · ⊂ L i ⊂ F, (s j ) ,

(17)

where F ∈ Shi , (L j ) is a complete flag of vector subbundles on a rank i vector bundle L i , deg(F/L i ) = d, and s j : i− j → f L j /L j−1 is an isomorphism ( j = 1, . . . , i). We have the open substack i Qd ⊂ i Q˜d given by the following condition: F is locally free. We also have a map i Q˜d → Shi that sends (17) to F. Define a substack i Z˜d

,→ i Q˜d ×Shi i Q˜d

as follows. If S is a scheme, then an object F, (L j , s j ), (L 0j , s 0j )

(18)

of Hom(S, i Q˜d ×Shi i Q˜d ) lies in Hom(S, i Z˜d ) if the collections (L j , s j ) and (L 0j , s 0j ) coincide outside a closed subscheme of S × X finite over S.

480

SERGEY LYSENKO

LEMMA 8 The map i Z˜d ,→ i Q˜d ×Shi i Q˜d is a closed immersion. In particular, the stack i Z˜d is algebraic.

Proof An object (18) of Hom(S, i Q˜d ×Shi i Q˜d ) gives rise to a pair of sections t j : (i−1)+···+(i− j) → f ∧j L j → ∧j F and

t 0j : (i−1)+···+(i− j) → f ∧ j L 0j → ∧ j F.

Clearly, (18) lies in Hom(S, i Z˜d ) if and only if the support of t j − t 0j is a closed subscheme of S × X finite over S (for all j = 1, . . . , i). Since F is S-flat, F (as well as its exterior powers) is locally free outside some closed subscheme of S × X finite over S. So, our assertion is a consequence of the following sublemma, communicated to the author by Drinfeld. 4 Let F be any coherent sheaf on S × X , which is locally free outside a closed subscheme of S × X finite over S. Let s be a global section of F. Consider the following subfunctor Z of S (on the category of S-schemes): a morphism S 0 → S belongs to Z (S 0 ) if the pullback of s to S 0 × X vanishes outside a closed subscheme of S 0 × X finite over S 0 . Then the subfunctor Z is closed. Suppose in addition that F is locally free. Then S 0 → S belongs to Z (S 0 ) if and only if the pullback of s to S 0 × X vanishes.

SUBLEMMA

(1)

(2)

Proof If r : S × X → S ×P1 is a finite morphism over S, then the functor Z does not change if we replace (X, F, s) by (P1 , r∗ F, r∗ s). After localizing with respect to S, we may assume that S is affine and that there is r as above with r∗ F locally free over S × A1 . (Recall that if S is Noetherian, then any finite morphism S × X → S × P1 over S is flat (cf. [9, expose IV, Section 5.9])). So, we are reduced to the case X = P1 with F locally free over S × A1 . If S = Spec R, then we have the projective R[t]-module M = H0 (S ×A1 , F) and its element s. Represent M as a direct summand of a free R[t]-module M 0 . Clearly, M 0 is also a free R-module. If si ∈ R are the coordinates of s ∈ M 0 (over R), then Z is the closed subscheme of S defined by the equations si = 0. We have an open substack i Zd ⊂ i Z˜d given by the condition that F is locally free. In particular, by Sublemma 4(2), n Zd = n Qd ×n Yd n Qd . Besides, if (18) is a point


481

of n Z˜d , then the sections (n−1)+···+(n−n) → f det L n ,→ det F and

(n−1)+···+(n−n) → f det L 0n ,→ det F

coincide, where det : Shn → Pic X denotes the determinant map (cf. [11]). This yields a map f˜ : n Z˜d → X (d) whose restriction to n Zd coincides with f . We prove in Section 6.3 that f˜ is of relative dimension less than or equal to b − d but not of finite type. (The stack n Z˜d even has infinitely many irreducible components.) For k = 0, . . . , i we have a closed substack ik Z˜d ,→ i Z˜d given by the following condition: for a point (18) of i Z˜d the flags 0 = L 0 ⊂ · · · ⊂ L k , (s j ) j=1,...,k and 0 = L 00 ⊂ · · · ⊂ L 0k , (s 0j ) j=1,...,k

coincide. Notice that taking the quotient by L k = L 0k , one gets a map ik Z˜d → i−k Z˜d , which is a generalized affine fibration. This observation is a key point in the proof of Proposition 4. 6.3. Dimensions counting Proof of Lemma 7 (i) The map β c is obtained by base change from the map β : n Qd → ≤n Shd0 , which is surjective and extends to a generalized affine fibration n Q˜d → Shd that sends (17) 0

to F/L n . 0 (ii) We stratify W c by locally closed substacks U e ,→ W c indexed by e ∈ m m Jd with h(e) = (ν, ν 0 ). A point 0 F 1 ⊂ · · · ⊂ F m = F, (F 1 )0 ⊂ · · · ⊂ (F m )0 = F (19) of W c lies in U e if j

deg(Fi ) =

X

ekl

for 1 ≤ i ≤ m, 1 ≤ j ≤ m 0 ,

k≤i, l≤ j j

where Fi = F i ∩ (F j )0 . If (19) is a point of U e , then for 1 ≤ i ≤ m, 1 ≤ j ≤ m 0 , j define F˜i ∈ Sh0 from the co-Cartesian square j

Fi−1 ↑ j−1 Fi−1

→ →

j F˜i ↑ j−1 Fi

482

SERGEY LYSENKO

j j j and put G i = Fi / F˜i . Set also

We=

Y

e

j

Sh0i .

i, j j

The map U e → W e that sends (19) to the collection (G i ) is a generalized affine j fibration of rank zero. We have a map W e → Y e that sends (G i ) to the collection j (div G i ), and we define dive : U e → Y e as the composition U e → W e → Y e . Since for any i ≥ 0 the morphism div : Shi0 → X (i) is of relative dimension less than or equal to −i, our assertion follows. Define the stack i Z˜ c by the Cartesian square i Z˜

c

↓ 0 F l ν × X (d) F l ν

→

i Z˜d

↓ → Shd0 × X (d) Shd0

where the right vertical arrow sends (18) to (F/L i , F/L i0 ). Let i Z c ⊂ i Z˜ c denote the preimage of i Zd under i Z˜ c → i Z˜d . In particular, we have n Z c = n Q ν ×n Yd ν0 n Q . Let f˜c : n Z˜ c → V c ν

ν0

0 div × div denote the composition n Z˜ c → F l ν × X (d) F l ν → V c . The restriction of f˜c c c c ˜ to n Z coincides with f . Notice that f is locally of finite type but not of finite type in general.

LEMMA 9 The map f˜c is of relative dimension less than or equal to b − d.

Proof Step 1. The stack n Q ν ×n Yd n Q ν is stratified by locally closed substacks Q e ×n Yd Q e 0 m0 indexed by pairs e ∈ m n Jd , e ∈ n Jd such that there exists λ ∈ 3n,d with h(e) = (λ, ν), h(e0 ) = (λ, ν 0 ). (cf. Section 4.3). 0 The restriction of f c : n Z c → V c to a stratum Q e ×n Yd Q e is written as the composition 0

Q e ×n Yd Q e

0

ϕe × ϕe

→

0

0

(Y e × X λ Y e ) × X λ V λ → Y e × X λ Y e → V c . 0

0

Since ϕ e : Q e → Y e × X λ V λ is an affine fibration of rank a(λ), and V λ → X λ is a generalized affine fibration of rank b − d − 2a(λ), it follows that f c is of relative dimension less than or equal to b − d.


483

Step 2. Stratify Shn by fixing the degree of the maximal torsion subsheaf of F ∈ Shn . Consider the induced stratification of n Z˜ c . A stratum n Z˜kc ⊂ n Z˜ c classifies the following data: a point (18) of n Z˜d , two flags of subsheaves (L n ⊂ L 1n ⊂ · · · ⊂ L m n = F)

(20)

0 0 L 0n ⊂ (L 1n )0 ⊂ · · · ⊂ (L m n ) = F ,

(21)

and and an exact sequence 0 → F0 → F → M → 0 of O X -modules, where F0 ∈ Shk0 and M is a vector bundle on X of rank i. The preimages of flags (20) and (21) in F0 give rise to a point of F l λ ×Shk F l λ for 0 some λ ∈ 3m,k , λ0 ∈ 3m 0 ,k . This yields a stratification of n Z˜kc by locally closed substacks n Z˜ c 0 ,→ n Z˜ c indexed by pairs λ ∈ 3m,k , λ0 ∈ 3m 0 ,k . 0

λ,λ

k

c the vector bundle M together with the images of the corFor an object of n Z˜λ,λ 0

responding flags on F defines a point of n Q ν−λ ×n Yd−k n Q ν −λ . The natural forgetful map 0 0 c λ λ0 ν−λ ×n Yd−k n Q ν −λ ) n Z˜λ,λ0 → (F l ×Shk F l ) × (n Q 0

0

0

is a generalized affine fibration of rank nk. Recall that b = b(n, d) depends on n and d (cf. Section 2.2). By Step 1, nQ

ν−λ

×n Yd−k n Q ν −λ → X ν−λ × X (d−k) X ν −λ 0

0

0

0

is of relative dimension less than or equal to b(n, d−k)−(d−k) = b(n, d)−d−nk+k. By Lemma 7(ii), 0 0 F l λ ×Shk F l λ → X λ × X (k) X λ 0

is of relative dimension less than or equal to −k. Our assertion follows. 6.4. Proof of Proposition 4 Let ik Zd be the preimage of i Zd under ik Z˜d ,→ i Z˜d . For k = 0, . . . , i, define the stacks ik Z c ⊂ ik Z˜ c by the Cartesian squares k c iZ

,→

k ˜c iZ

,→

k ˜ i Zd

↓ k i Zd

,→

i Z˜

,→

i Z˜d

↓

c

↓

Denote by i Lψ the restriction of µ∗ Lψ µ∗ Lψ −1 under the composition i c nZ

,→ n Z c → n Zd → f n Qd ×n Yd n Qd .

484

SERGEY LYSENKO

Let also i f c be the restriction of f c to inZ c ,→ n Z c . Arguing by induction, we show that the natural map R2(b−d) (i f c )! (i Lψ ) → R2(b−d) (i+1 f c )! (i+1 Lψ ) is an isomorphism for i = 1, . . . , n − 1. The map µ : n Qd → A1 extends naturally to a morphism n Q˜d → A1 defined in the same way; it is also denoted by µ. This allows us to extend i Lψ to a local system i L˜ on i Z˜ c , where i L˜ is defined as the restriction of µ∗ L µ∗ L ψ ψ ψ ψ −1 under the n composition i ˜c ˜c ˜ ˜ ˜ n Z ,→ n Z → n Zd ,→ n Qd ×Shi n Qd . We have the diagram i+1 Z c n

↓ 1 Z˜ c n−i

,→ &α ,→

δ

,→ .γ

iZc n ↓β n−i Z˜

i Z˜ c n

c

in which the square is Cartesian, and γ is a generalized affine fibration of rank b(n, d) − b(n − i, d). Since δ is an open immersion, Rtop β! (i Lψ ) → Rtop γ! (i L˜ψ ) is an isomorphism over the image of (the smooth map) β. Applying Sublemma 3 for 0 M = F/L i , D = 0, and the section si+1 − si+1 : n−i−1 → M, we learn that Rtop γ! (i L˜ψ ) is supported at 1n−i Z˜ c . Therefore, Rtop β! (i Lψ ) → Rtop α! (i+1 Lψ ) β is an isomorphism. So, i f c is decomposed as inZ c → n−i Z˜ c → V c , where, by Lemma 9, the second map is of relative dimension less than or equal to b(n −i, d)−d. Though n−i Z˜ c → V c is not of finite type in general, its restriction to the image of β is a morphism of finite type. This concludes the proof of Proposition 4.

6.5. Proof of Theorem C 0 Recall that we have the map h : m m Jd → 3m,d × 3m 0 ,d , and for e ∈ j Q Y e = i, j X (ei ) (cf. Section 4.3). Let G norm : Ye → Vc

m0 J m d

we write

e∈h −1 (c) j

be the map that sends a matrix (Di ) ∈ Y e to the collection ((Di ), (D 0j )), where P P j j Di = j Di and D 0j = i Di .


485

10 The scheme V c is of pure dimension d, and its irreducible components are numbered by the set h −1 (c). Namely, to e ∈ h −1 (c) there corresponds the component norm(Y e ). The map norm is the normalization of V c . (More precisely, it is a finite morphism, an isomorphism over an open dense subscheme of V c , and the scheme F e e∈h −1 (c) Y is smooth.) So,

LEMMA

(i)

(ii)

¯ ` [d] → norm∗ Q f IC, where IC is the intersection cohomology sheaf on V c . Proof Stratify V c by locally closed subschemes e V c ⊂ V c indexed by e ∈ h −1 (c). First, define e V c as the open subscheme of Y e given by the following condition: j

if i > k, l > j, then Di ∩ Dkl = ∅. norm

Then the composition e V c ,→ Y e → V c is a locally closed immersion. As a subscheme of V c , e V c is given by the condition that X X X = ekl . for all i, j, we have deg Dk ∩ Dl0 k≤i

l≤ j

k≤i, l≤ j

For any e ∈ h −1 (c), the scheme e V c is smooth, nonempty, and irreducible of dimension d. This concludes the proof. LEMMA 11 ¯ ` (−d) → ¯ `. There is a canonical isomorphism R−2d (divc )! Q f norm∗ Q

We need the following straightforward sublemma. 5 Let r : Y → Y 0 be a separated morphism of schemes of finite type. Assume that the fibres of r are of dimension less than or equal to d. Let F be a smooth ¯ ` -sheaf on Y , let U ⊂ Y be an open subscheme, and let rU be the restriction Q of r to U . Then the natural map R2d (rU )! F → R2d r! F is injective. Let (U j ) j∈J be a stratification of Y by locally closed subschemes which comes from a filtration of Y by closed subschemes. Let r j be the restriction of r to U j . Then R2d r! F admits a filtration by subsheaves with successive quotients j being R2d r! F ( j ∈ J ).

SUBLEMMA

(i)

(ii)

486

SERGEY LYSENKO

Proof of Lemma 11 Recall that W c is stratified by locally closed substacks U e ,→ W c indexed by e ∈ h −1 (c) and that we have the maps dive : U e → Y e (cf. the proof of Lemma 7). The diagram commutes: U e ,→ W c ↓ dive ↓ divc norm Ye → Vc ¯ ` (−d) → ¯ ` canonically. Indeed, by Künneth formulae, this We have R−2d (dive )! Q fQ is reduced to the fact that for any i ≥ 0 the fibres of div : Shi0 → X (i) are connected of dimension −i. ¯ ` (−d) there is a filtration parametrized by By Sublemma 5(ii), on R−2d (divc )! Q −1 ¯ ` . We claim that any filtrathe set h (c) with successive quotients being (norme )∗ Q tion with these successive quotients degenerates canonically into a direct sum. Indeed, (i) the different successive quotients are supported on different irreducible components of V c , so our filtration degenerates into a direct sum over some open dense subscheme of V c ; ¯ ` [d] is perverse and the Goresky-MacPherson extension (ii) the sheaf (norme )∗ Q of its restriction to any open dense subscheme of V c ; (iii) the property “perverse and the Goresky-MacPherson extension of its restriction to a given open subscheme of V c ” is preserved for extensions.

Finally, assume c = (ν, ν) with ν = (1, . . . , 1) ∈ 3d,d . Then the set h −1 (c) is in natural bijection with Sd , and the map norm becomes G X σν → V c , σ ∈Sd

where X σν = X ν , and the map norm sends a point (x1 , . . . , xd ) ∈ X σν to ((x1 , . . . , xd ), (xσ 1 , . . . , xσ d )). The action of Sd × Sd on V c lifts naturally to an action on ¯ `, (E d E 0 d ) ⊗ norm∗ Q ¯ ` ) Sd ×Sd → and it is easy to see that pr! ((E d E 0 d ) ⊗ norm∗ Q f (E ⊗ E 0 )(d) canonic (d) cally, where pr : V → X denotes the projection. On the other hand, by Lemma 11, ¯` . R−2d div! (SprdE ⊗ SprdE 0 )(−d) → f pr! (E d E 0 d ) ⊗ norm∗ Q One checks that this isomorphism is (Sd × Sd )-equivariant. Taking the invariants, one gets R−2d div! (L Ed ⊗ L Ed0 )(−d) → f (E ⊗ E 0 )(d) .


487

¯ ` → R−2d (divc )! Q ¯ ` is an By Sublemma 5(i), the natural map R−2d (≤n divc )! Q inclusion. It follows that R−2d (≤n div)! (SprdE ⊗ SprdE 0 )(−d) → R−2d div! (SprdE ⊗ SprdE 0 )(−d)

(22)

d is an inclusion. Taking the Sd × Sd -invariants in (22), one gets an inclusion n S E,E 0 ⊂ 0 (d) ≤n 0 (d) (E ⊗ E ) , whose image is denoted (E ⊗ E ) . Since n and d were arbitrary, Lemma 1 follows now from Corollary 1, and the proof of Theorem C is complete. So, Theorem B and the Main Local Theorem are also proved.

6.6. Second proof of Theorem B In this section we present an alternative proof of Theorem B under the additional assumption min{rk E, rk E 0 } ≤ n. The idea of this proof was suggested to the author by Gaitsgory. Let n Modd denote the stack classifying modifications (L ⊂ L 0 ) of rank n vector bundles on X with deg(L 0 /L) = d. Let q : n Modd → Shd0 be the map that sends (L ⊂ L 0 ) to L 0 /L, and let supp : n Modd → X (d) denote div ◦ q. For d 0 ≥ 0, let pY : n Yd ×Bunn n Modd 0 → n Yd+d 0 be the map that sends ((ti ), L ⊂ L 0 ) to ((ti0 ), L 0 ), where ti0 is the composition ti

(n−1)+···+(n−i) ,→ ∧i L ,→ ∧i L 0 . The map pY is representable and proper. Let qY : n Yd ×Bunn n Modd 0 → n Yd denote the projection. The map qY is smooth of relative dimension nd 0 . The key ingredient is the Hecke property of Whittaker sheaves (see [8, Proposition 7.5]). It admits the following immediate corollary. (The argument given in [8, Proposition 7.5] for rk E = n holds, in fact, for rk E ≤ n.) 5 ¯ ` -sheaf E on X and any d ≥ 0, there is a natural map For any smooth Q PROPOSITION

d+1 d (qY × supp)! p∗Y (n P E,ψ ) → n P E,ψ E

2 − n [2 − n], 2

which is an isomorphism if rk E ≤ n. g d be the stack of flags (L 0 ⊂ · · · ⊂ L d ), where (L i ⊂ L i+1 ) ∈ n Mod1 Let n Mod g d → X d be the map that sends (L 0 ⊂ · · · ⊂ L d ) to for all i. Let supp g : n Mod g d 0 → n Yd ×Bunn n Modd 0 (div(L 1 /L 0 ), . . . , div(L d /L d−1 )). Let p : n Yd ×Bunn n Mod g be the projection, and let p˜ Y : n Yd ×Bunn n Modd 0 → n Yd+d 0 be the composition pY ◦ p.

488

SERGEY LYSENKO

COROLLARY 2 ¯ ` -sheaf E on X and any d, d 0 ≥ 0, there is a natural map For any smooth Q 0

d+d d (qY × supp) g ! p˜ ∗Y (n P E,ψ ) → n P E,ψ E d

0

2d 0 − nd 0 [2d 0 − nd 0 ], 2

(23)

which is an isomorphism if rk E ≤ n. Proof The map (23) is defined as follows. Let pQ : n Qd ×Bunn n Modd 0 → n Qd+d 0 be the map that sends (L 1 ⊂ · · · ⊂ L n ⊂ L ⊂ L 0 ) to (L 1 ⊂ · · · ⊂ L n ⊂ L 0 ). Let g d 0 → n Qd+d 0 denote the composition p˜ Q : n Qd ×Bunn n Mod n Qd

Q g d 0 → n Qd ×Bunn n Modd 0 p→ ×Bunn n Mod n Qd+d 0 ,

where the first arrow is the projection. Consider the commutative diagram n Qd n Yd

g d0 ×Bunn n Mod ↓ ϕ×id g d0 ×Bunn n Mod

p˜ Q

→

n Qd+d 0

↓ϕ p˜ Y

→

n Yd+d 0

0

d+d Since n F E,ψ is a direct summand of 0

d (p˜ Q )! (n F E,ψ supp g ∗ E d )[nd 0 ]

nd 0 , 2

0

d+d it follows that n P E,ψ is a direct summand of 0

d (p˜ Y )! (n P E,ψ supp g ∗ E d )[nd 0 ](nd 0 /2).

This yields a morphism 0

0

d+d d p˜ ∗Y (n P E,ψ ) → n P E,ψ supp g ∗ E d [nd 0 ]

nd 0 2

.

Since qY × supp g is smooth of relative dimension d 0 (n−1), the desired map is obtained from the last one by adjointness. To show that (23) is an isomorphism under the condition rk E ≤ n, apply d 0 times Proposition 5. 0

0

Denote by rss X d ⊂ X d the open subscheme that parametrizes pairwise different 0 g points (x1 , . . . , xd 0 ) ∈ X d (here “rss” stands for “regular semisimple”). Let rss n Modd 0 0 rss d rss g d 0 , and be the preimage of X under supp. g The symmetric group Sd 0 acts on n Mod


489

g the restriction of p˜ Y to n Yd ×Bunn rss n Modd 0 is Sd 0 -invariant. So, the action of Sd 0 on rss g n Yd ×Bunn n Modd 0 lifts to an action on 0

d+d p˜ ∗Y (n P E,ψ ).

Since the restriction complex

rss Mod g d0 n

→

rss X d 0

of supp g is Sd 0 -equivariant, Sd 0 acts on the 0

d+d (qY × supp) g ! p˜ ∗Y (n P E,ψ ) 0

0

restricted to n Yd × rss X d . On the other hand, Sd 0 acts on E d and, hence, on the right-hand side of (23). Using the explicit description of map (23), one easily proves the following lemma. LEMMA 12 ¯ ` -sheaf E on X , the map (23) restricted to n Yd × rss X d 0 is Sd 0 For any smooth Q equivariant.

¯ ` -sheaf E on X , the Verdier dual of n P d is canonically Recall that for any smooth Q E,ψ isomorphic to n P Ed ∗ ,ψ −1 (see [8, Lemma 4.7]). So, to prove Theorem B we must establish a canonical isomorphism d φ∗ RH om(n P E,ψ , n P Ed 0 ,ψ ) → f H om(E, E 0 )(d) ,

where φ : n Yd → X (d) is the map defined in Section 2.3. The statement of Theorem B being symmetric with respect to interchanging E and E 0 , we assume rk E ≤ n. ¯ ` -sheaf E on X , set n P˜ d = ϕ! (β ∗ Sprd ⊗µ∗ Lψ )[b](b/2). For any smooth Q E,ψ

E

d In other words, n P˜ E,ψ is a complex on n Yd obtained by replacing in the definition of d d d n P E,ψ Laumon’s sheaf L E by Springer’s sheaf Spr E . Theorem B follows now from the next statement.

PROPOSITION 6 ¯ ` -sheaves on X with rk E ≤ n. Then there exists a canonical Let E, E 0 be smooth Q Sd -equivariant isomorphism d φ∗ RH om(n P E,ψ , n P˜ Ed 0 ,ψ ) → f sym∗ H om(E, E 0 ) d .

Proof The idea is that Proposition 6 is a tautological consequence of Corollary 2 obtained by applying the formalism of six functors. The equivariance property follows from Lemma 12. The precise argument is as follows.

490

SERGEY LYSENKO

Consider the commutative diagram n Y0 n Y0

. qY ←

×Bunn n Modd ↓ qY ×supp Y × X (d) n 0

pY

→

n Yd

→

↓φ X (d)

0 Set 9 0 = n P E,ψ for brevity. (It does not depend on E, though it does depend on ψ.) By definition, nd ∗ 0 ∗ d ˜d0 → f p (q 9 ⊗ q Spr )[nd] . 0 nP Y ∗ Y E ,ψ E 2

LEMMA 13 We have q∗Y 9 0 ⊗ q ∗ SprdE 0 → f RH om(q ∗ SprdE 0 ∗ , q∗Y 9 0 ) canonically and Sd equivariantly.

Proof Using the fact that both qY and qY ◦ p are smooth of relative dimension nd, we get q∗Y 9 0 ⊗ q ∗ SprdE 0 → f p! (supp g ∗ E0

d

⊗ p ∗ q∗Y 9 0 ) ∗

→ f p∗ RH om supp g ∗ (E 0 ) d , p ∗ q∗Y 9 0

∗ → f p∗ RH om supp g ∗ (E 0 ) d , p ! q!Y 9 0 [−2nd](−nd) ∗ → f RH om p! supp g ∗ (E 0 ) d , q∗Y 9 0 . Using Lemma 13, we get d RH om(n P E,ψ , n P˜ Ed 0 ,ψ )

nd d → f pY ∗ RH om p∗Y (n P E,ψ ), q∗Y 9 0 ⊗ q ∗ SprdE 0 [nd] 2 −nd ∗ d ∗ d ! 0 . → f pY ∗ RH om pY (n P E,ψ ) ⊗ q Spr E 0 ∗ , qY 9 [−nd] 2 Let j : n Q0 ,→ n Y0 denote the natural open immersion. Since n Q0 → Spec k ¯ `) → ¯ ` . So, our assertion is is a generalized affine fibration, we have R0(n Q0 , Q fQ reduced to the following lemma. 14 There is a canonical Sd -equivariant isomorphism over n Y0 × X (d) : LEMMA

d (qY × supp)∗ RH om p∗Y (n P E,ψ ) ⊗ q ∗ SprdE 0 ∗ , q!Y 9 0

¯ ` sym∗ H om(E, E 0 ) d [nd] nd . → f ( j × id)∗ Q 2


491

Proof Let prY : n Y0 × X d → n Y0 denote the projection. Using the commutative diagram n Y0

p

×Bunn n Modd ↓ qY ×supp

←

× X (d)

id × sym

n Y0

←

n Y0

gd ×Bunn n Mod ↓ qY ×supp g n Y0

× Xd

we obtain d (qY × supp)∗ RH om p∗Y (n P E,ψ ) ⊗ q ∗ SprdE 0 ∗ , q!Y 9 0

∗ d → f (qY × supp)∗ RH om p! p˜ ∗Y (n P E,ψ ) ⊗ supp g ∗ (E 0 ) d , q!Y 9 0 ∗ d ) ⊗ supp g ∗ (E 0 ) d , p ! q!Y 9 0 → f (qY × supp)∗ p∗ RH om p˜ ∗Y (n P E,ψ ∗ d ) ⊗ supp g ∗ (E 0 ) d , → f (id × sym)∗ (qY × supp) g ∗ RH om p˜ ∗Y (n P E,ψ

(qY × supp) g ! pr!Y 9 0 ∗ d → f (id × sym)∗ RH om (qY × supp) g ! p˜ ∗Y (n P E,ψ ) ⊗ supp g ∗ (E 0 ) d , pr!Y 9 0 nd − 2d ∗ , → f (id × sym)∗ RH om 9 0 (E ⊗ E 0 ) d , pr!Y 9 0 [nd − 2d] 2 where the last isomorphism comes from Corollary 2. Since prY is smooth of relative dimension d, our assertion follows. This concludes the proof of Proposition 6. Acknowledgments. I am deeply grateful to V. Drinfeld, who has posed the problem we consider and explained to me its place in the geometric Langlands program. In this paper we attempt to realize certain of his ideas. I am grateful to D. Gaitsgory, who suggested to the author an idea of the second proof of Theorem B (cf. Section 6.6). I also wish to thank G. Laumon for constant support. References [1]

A. BRAVERMAN AND D. GAITSGORY, Geometric Eisenstein series, preprint,

[2]

P. DELIGNE, “Applications de la formule des traces aux sommes trigonométriques” in

arXiv:math.AG/9912097

[3]

Cohomologie étale, Séminaire de Géométrie Algébrique du Bois-Marie (SGA 4 1/2), Lecture Notes in Math. 569, Springer, Berlin, 1977, 168–232. MR 57:3132 453 V. DRINFELD, Two-dimensional `-adic representations of the fundamental group of a curve over a finite field and automorphic forms on GL(2), Amer. J. Math. 105 (1983), 85–114. MR 84i:12011 453

492

[4]

[5] [6]

[7]

[8] [9]

[10]

[11]

[12]

[13] [14] [15]

[16]

[17]

[18]

[19]

SERGEY LYSENKO B. FEIGIN, M. FINKELBERG, A. KUZNETSOV, and I. MIRKOVIC´ , Semiinfinite flags, II:

Local and global intersection cohomology of quasimaps’ spaces, preprint, arXiv:alg-geom/9711009 ´ , Semiinfinite flags, I: Case of global curve P 1 , M. FINKELBERG and I. MIRKOVIC preprint, arXiv:alg-geom/9707010 E. FRENKEL, D. GAITSGORY, D. KAZHDAN, and K. VILONEN, Geometric realization of Whittaker functions and the Langlands conjecture, J. Amer. Math. Soc. 11 (1998), 451–484. MR 99f:11148 463, 467 E. FRENKEL, D. GAITSGORY, and K. VILONEN, Whittaker patterns in the geometry of moduli spaces of bundles on curves, Ann. of Math. (2) 153 (2001), 699–748. MR CMP 1 836 286 459, 460, 463, 469 , On the geometric Langlands conjecture, preprint, arXiv:math.AG/0012255 469, 487, 489 A. GROTHENDIECK, Revêtements étales et groupe fondamental, Séminaire de Géométrie Algébrique du Bois-Marie 1960/61 (SGA 1), Lecture Notes in Math. 224, Springer, Berlin, 1971. MR 50:7129 480 , Techniques de construction et théorèmes d’existence en géométrie algébrique, IV: Les schémas de Hilbert, Sem. Bourbaki 221 (1960/61), Soc. Math. France, Paris, 1995, 249–276. MR CMP 1 611 822 454 F. KNUDSEN and D. MUMFORD, The projectivity of the moduli space of stable curves, I: Preliminaries on “det” and “Div”, Math. Scand. 39 (1976), 19–55. MR 55:10465 481 G. LAUMON, Correspondance de Langlands géométrique pour les corps de fonctions, Duke Math. J. 54 (1987), 309–359. MR 88g:11086 451, 452, 453, 455, 456, 460, 467 G. LAUMON and L. MORET-BAILLY, Champs algébriques, Ergeb. Math. Grenzgeb. (3) 39, Springer, Berlin, 2000. MR 2001f:14006 453 S. LYSENKO, Local geometrized Rankin-Selberg method for GL(n) and its application, C. R. Acad. Sci. Paris Sér. I Math. 329 (1999), 1065–1070. MR 2001h:11081 , Orthogonality relations between the automorphic sheaves attached to 2-dimensional irreducible local systems on a curve, Ph.D. thesis, Université Paris-Sud, 1999, http://math.u-psud.fr/˜biblio/the/1999/2111/the_1999_2111.html B. C. NGÔ, Preuve d’une conjecture de Frenkel-Gaitsgory-Kazhdan-Vilonen pour les groupes linéaires généraux, Israel J. Math. 120 (2000), 259–270. MR CMP 1 815 378 459, 460, 463 B. C. NGÔ and P. POLO, Résolutions de Demazure affines et formule de Casselman-Shalika géométrique, J. Algebraic Geom. 10 (2001), 515–547. MR CMP 1 832 331 459, 460, 463, 464 I. I. PIATETSKI-SHAPIRO, “Euler subgroups” in Lie Groups and Their Representations (Budapest, 1971), ed. I. M. Gelfand, Halstead, New York, 1975, 597–620. MR 53:10720 451, 452 J. A. SHALIKA, The multiplicity one theorem for GL(n), Ann. of Math. (2) 100 (1974), 171–193. MR 50:545


493

451, 452

Institut de Mathématiques de Jussieu, case 247, 4 place Jussieu, 75252 Paris CEDEX, France; [email protected]


SOME REMARKS ON LANDAU-SIEGEL ZEROS P. SARNAK and A. ZAHARESCU

Abstract In this paper we show that, under the assumption that all the zeros of the L-functions under consideration are either real or lie on the critical line, one may considerably improve on the known results on Landau-Siegel zeros. 1. Introduction Let D be a fundamental discriminant, and let χ D (n) = (D/n) be the corresponding Kronecker symbol. In particular, χ D is a real primitive character on N with conductor |D|. A well-known problem whose solution would have far-reaching implications is to give lower bounds for L(1, χ D ) =

∞ X

χ D (n)n −1 .

(1.1)

n=1

L. Dirichlet [5], who first faced this problem in his work on primes in arithmetic progressions, proved that L(1, χ D ) >

1 |D|1/2

if D < 0

(1.2)

and

log(D − 2) if D > 1. (1.3) |D|1/2 This problem is well known to be intimately connected to possible real zeros β of L(s, χ D ) near s = 1. Such zeros are generally referred to as Landau-Siegel zeros. Indeed, the best-known lower bound for L(1, χ D ), or for the distance of β to 1, is due to C. Siegel [21], who, after E. Landau [14], proved the following. Given 0 < < 1/2, there is c() > 0 which is ineffective (i.e., the proof, even in principle, does not allow for a determination of c()) such that L(1, χ D ) >

L(1, χ D ) ≥

c() . |D|

DUKE MATHEMATICAL JOURNAL c 2002 Vol. 111, No. 3, Received 14 June 2000. Revision received 18 December 2000. 2000 Mathematics Subject Classification. Primary 11M20. 495

(1.4)

496

SARNAK and ZAHARESCU

Effective lower bounds, which are crucial in many applications (see, e.g., [7], [6]), are much harder to come by. In fact, the only improvement over Dirichlet’s bounds above is for D < 0, where, combining the results of D. Goldfeld [7] and B. Gross and D. Zagier [8], one can show the following (see J. Oesterlé [18] and also the elegant treatment in H. Iwaniec [12]). For D < 0, √ Y [2 p] π log |D| 1− . (1.5) L(1, χ D ) > p+1 55|D|1/2 p|D

In fact, Goldfeld shows that if there is an elliptic curve E over Q whose L-function L(s, E) has a zero of order g at s = 1/2 (this in our normalization being the central point of this L-function), then there is an effective constant c(E) such that if µ = 1 or 2 when χ D (−1) = (−1)g−µ , then n o p g−µ−1 c(E) L(1, χ D ) > log |D| exp − 21 g log log |D| . (1.6) |D|1/2 Such E’s with g = 3 are known (see [8]), and candidates for such E’s with g as large as 24 are also known (see [16]). It seems likely that there are E’s with g arbitrarily large. The optimal lower bound for L(1, χ D ) was determined by J. Littlewood [15] assuming the generalized Riemann hypothesis (GRH) for L(s, χ D ): L(1, χ D ) ≥

c log log |D|

(1.7)

for an effective universal c. In this note we assume Hypothesis H below. It allows for the existence of such Landau-Siegel zeros for L(s, χ D ) but shows that they alone cannot do too much harm. HYPOTHESIS H All the zeros of any of the L-functions in question (namely, for L(s, χ D ) and L(s, E ⊗ χ D )) are either on the line Re(s) = 1/2 or are real.

Comments (C1) Hypothesis H is of course a weak form of GRH. However, in connection with lower bounds for L(1, χ D ) or, equivalently, with possible zeros β D ∈ (1/2, 1) (near s = 1) of L(s, χ D ), it appears, at least on the face of it, that Hypothesis H is not of much use. (C2) There are reasons to accept Hypothesis H as a natural hypothesis. For example, in the theory of the Selberg zeta function (see [20]) for a lattice 0 in SL(2, R), and also for the analogue in the p-adic setting (see Ihara [11]),

SOME REMARKS ON LAUDAU-SIEGEL ZEROS

(C3) (C4)

(C5)

497

the analogue of the Riemann hypothesis is true, except for possible real zeros (i.e., in (1/2, 1)). That is, Hypothesis H is true for these zeta functions, and in general the analogue of the Riemann hypothesis is not true. Also, certain investigations into the zeros of ζ D (s) = ζ (s)L(s, χ D ) by E. Bombieri [1] and H. Yoshida [23] divide naturally into cases with real zeros and complex zeros (off the line Re(s) = 1/2) being treated differently. Some people have expressed the opinion that GRH may be true, except perhaps for Landau-Siegel zeros. The Deuring-Heilbronn phenomenon (see [4], [10]) asserts that if L(1, χ D ) is abnormally small, then ζ D (s) satisfies Hypothesis H at least for its zeros up to a certain height. In recent work (see [13]), Hypothesis H arose naturally, and it together with another conjecture about GL(1) L-functions was shown to imply a quasiRiemann hypothesis for GL(2) L-functions (i.e., a zero-free strip 10/11 < Re(s) < 1).

Given these comments, it seems of some interest to determine what, if any, are the consequences of Hypothesis H (or of its local version Hypothesis H∗ ).∗ We show that under Hypothesis H much better lower bounds for L(1, χ D ) can be established. Basically, we can “exponentiate” the known results. The reason for this is that Hypothesis H allows us to work effectively with the logarithmic derivatives of the L-functions in question, in conjunction with some standard methods. THEOREM 1 Assume Hypothesis H (for Dirichlet L-functions only). Then for any > 0 there exists c() > 0 (ineffective) such that

L(1, χ D ) ≥

c() . log |D|

THEOREM 2 Assume Hypothesis H. Then for any η > 2/5 there is an effective c(η) > 0 such that, for any D with χ D (37) = −1, one has

L(1, χ D ) ≥ THEOREM ∗ This

c(η) . |D|η

3

assumption has no doubt occurred to others, but we are unaware of any work in this direction other than that H. Stark has also recently established Theorem 1 below. He suggests the name “modified GRH” for Hypothesis H.

498


Assume Hypothesis H. Given an elliptic curve E over Q whose L-function has a zero of order m at s = 1/2, for any > 0 there is an effective constant c(E, ) > 0 such that for any D one has c(E, ) L(1, χ D ) ≥ . |D|(2+)/(m+1) Remarks (R1) Theorem 1 shows that statement (C3) is impossible, at least if one takes the meaning of Landau-Siegel zeros to be as in, say, [9]; that is, there is a sequence of D j ’s, |D j | → ∞, and corresponding zeros β j of L(s, χ D j ) with (1 − β j ) log |D j | ≤ 0 , for some fixed positive 0 . (R2) In Theorem 2 we can deal with other progressions in D (besides χ D (37) = −1), but we do not see how to remove this secondary condition completely. The point is that this condition gives an extra zero for a certain L-function in the proof of Theorem 3, thus increasing m by 1 in that theorem. This allows us to effectively use an elliptic curve E with m = 3 which is known to exist. Put another way, Theorem 2 shows that, in a world where GRH is true except possibly for real zeros, one would still have very strong effective lower bounds. (R3) Since we expect that there are elliptic curves of arbitrary high rank over Q, we see that this together with Theorem 3 and Hypothesis H and the Birch and Swinnerton-Dyer conjectures would yield an effective Siegel-type bound (1.4). We turn to a local version of Hypothesis H. (In what follows, we choose local to mean |γ | ≤ 2; there is nothing special about this choice, and another choice would lead to different constants.) H∗ All nonreal zeros ρ = σ + iγ with |γ | ≤ 2 of all the L-functions under consideration are on the line Re(s) = 1/2. HYPOTHESIS

Note that the Deuring-Heilbronn phenomenon asserts that if L(1, χ D ) is abnormally small, then Hypothesis H∗ is valid for ζ D (s). 4 Assume Hypothesis H∗ . There is an effective constant c0 > 0 (e.g., with our choice in Hypothesis H∗ with the condition |γ | ≤ 2, one can take c0 = 100) such that, given an elliptic curve E over Q with L(s, E) having a zero of order m at s = 1/2, there is THEOREM


499

an effective c(E) > 0 such that for any D one has L(1, χ D ) ≥

c(E) . |D|c0 /m

2. Proofs Proof of Theorem 3 First note that, to even make sense of the statement of this theorem in the generality stated, we need to know that L(s, E) is defined at s = 1/2. That this is so is a consequence of the recent celebrated work of A. Wiles and R. Taylor and in particular the complete solution of the modularity problem (see [2]). According to this result, L(s, E) = L(s, f ) for a holomorphic modular form of weight 2 on the upper halfplane. So we proceed in this context, that is, assume that we are given a weight 2 newform f for 00 (N ) whose L-function L(s, f ) has a zero of order m at s = 1/2. In what follows, all implied constants depend effectively on f . We may write L(s, f ) =

∞ X

λ f (n)n −s =

Y

1 − α f ( p) p −s

−1

1 − β f ( p) p −s

−1

(2.1)

p

n=1

for ( p, N ) = 1, α f ( p)β f ( p) = 1, and, in fact, |α f ( p)| = |β f ( p)| = 1 (this being the well-known Ramanujan conjecture for f ). The corresponding completed L-function 3(s, f ) is defined by √ N s 1 3(s, f ) = 0 s+ L(s, f ). (2.2) 2π 2 It is entire and satisfies the functional equation 3(s, f ) = w( f )3(1 − s, f )

(2.3)

with w( f ) = ±1. We can twist f by our character χ D as follows: L(s, f ⊗ χ D ) =

∞ X

λ f (n)χ D (n)n −s

n=1

=

Y

1 − α f ( p)χ D ( p) p −s

−1

1 − β f ( p)χ D ( p) p −s

−1

.

(2.4)

p

L(s, f ⊗ χ D ) is also entire and satisfies √ M s 1 3(s, f ⊗ χ D ) := 0 s+ L(s, f ⊗ χ D ) 2π 2 = w( f ⊗ χ D )3(1 − s, f ⊗ χ D ),

(2.5)

500


where M is the corresponding conductor (M is a divisor of N D 2 ) and w( f ⊗ χ D ) = ±1. Consider now the L-function F(s) given by the product 2 F(s) = ζ (s)L(s, χ D ) L(s, f )L(s, f ⊗ χ D ) (2.6) and the corresponding completed function 3(s, F). F(s) has a double pole at s = 1 and is otherwise analytic. A simple calculation of the logarithmic derivative of F(s) yields −

∞ F 0 (s) X X log p k = 1 + χD ( p))(2 + α kf ( p) + β kf ( p) ks F(s) p p k=1 X := 3(n)b(n)n −s .

(2.7)

n

Clearly, this Dirichlet series has nonnegative coefficients, as does the series defining F(s) itself. We apply the explicit formula to F(s) (see [19] or [13]), obtaining the following relation. Let φ be an even function on R with Z ∞ ˆ φ(y) = φ(x)e−2πi x y d x −∞

and for which φˆ is continuous and of compact support. Let ρ = β + iγ run over the zeros of F(s) (with multiplicities). Then X

φ(γ ) =

ρ

i i 1 X 3(n) log n A + 2φ + 2φ − − , √ b(n)φˆ 2π 2 2 π n 2π n

(2.8)

where ˆ log Q + A = 2φ(0)

8 Z X

∞

j=1 −∞

α j = 0 or 1, and

1 00 α j + + i x φ(x) d x, 0 4

√

M N |D| . 4π 4 Let B be a parameter to be chosen later (it is of order log |D|), and choose φ in (2.8) to be B φ(x) = φ0 x , 2π where sin 2π x 2 φ0 (x) = . (2.9) 2π x Q=


Then

( φˆ 0 (y) =

1 2

1−

0

501

|y| 2

for |y| ≤ 2, otherwise

(2.10)

and

2π 2π φˆ 0 y . B B Our choice has been such as to arrange that ˆ φ(y) =

(2.11)

φ(0) = 1,

(2.12)

φ(γ ) ≥ 0

for γ ∈ R or iR,

(2.13)

ˆ φ(y) ≥0

for y ∈ R.

(2.14)

and

With this, (2.8) reads X

φ0

Bγ

ρ

2π

=

1 iB iB log Q + Of + 2φ0 + 2φ0 − B B 4π 4π log n 2 X 3(n) − . √ b(n)φˆ 0 B n B n

(2.15)

Using b(n) ≥ 0, we drop all terms in the n-sum except n = p 2 , which yields X ρ

φ0

Bγ 2π

≤

1 iB iB 2 log |D| + Of + 2φ0 + 2φ0 − B B 4π 4π −

2 log p 2 X log p 2 1 + χD ( p) 2 + α 2f ( p) + β 2f ( p) φˆ 0 . B p p B

Similarly, using Hypothesis H, we drop all zeros γ of 3(s, F) except γ = 0 and the potential Landau-Siegel zero of L(s, χ D ) (which occurs with multiplicity two in F(s)), and the inequality continues to hold. Inserting the formula for φ0 (x) yields e(β−(1/2))B − e−(β−(1/2))B 2 m+2 (2β − 1)B 1 e B/2 − e−(B/2) 2 2 log |D| + Of +2 ≤ B B B 2 log p 2 X log p 2 − 1 + χ D ( p) 2 + α 2f ( p) + β 2f ( p) φˆ 0 . B p p B

(2.16)

502


Now 2 log p 1 X log p 2 1 + χD ( p) 2 + α 2f ( p) + β 2f ( p) φˆ 0 B p p B log log |D| 2 X log p 2 log p = 2 + α 2f ( p) + β 2f ( p) φˆ 0 + Of . (2.17) B p p B B The sum in (2.17) no longer involves D and can be evaluated using the RankinSelberg L-function L(s, f ⊗ f ) (see [19]). One has X log2 p log2 x 2 + α 2f ( p) + β 2f ( p) = + O f (log x). p 2 p≤x From this one easily derives X log p p 2 + α 2f ( p) + β 2f ( p) = log x + O f log x log log x . p p≤x Hence 2 log p 2 X log p 2 + α 2f ( p) + β 2f ( p) φˆ 0 B p p B =

1 B

X log p 2 + α 2f ( p) + β 2f ( p) p

log p≤B

1 X log2 p 2 + α 2f ( p) + β 2f ( p) 2 p B log p≤B log B 1 = +O . 2 B 1/2 Returning to (2.16), we now have e(β−(1/2))B − e−(β−(1/2))B 2 2 log |D| e B/2 − e−(B/2) 2 m+1+2 ≤ +2 (2β − 1)B B B log log |D| log B + Of + 1/2 . (2.18) B B Choosing B = (1 + )((2 log |D|)/(m + 1)) with > 0, (2.18) yields −

1 − β ≥ c(, f )|D|−((2+)/(m+1))

(2.19)

for a suitable effective positive constant c(, f ). The passage from inequality (2.19) concerning the zero β D of L(s, χ D ) nearest to 1 to the lower bound claimed in Theorem 3 is known. For example, one can deduce this from Tatuzawa’s paper (see [22, Lem. 8]). This completes the proof of Theorem 3.


503

Proof of Theorem 2 To apply Theorem 3 we use the elliptic curve (see [8]) E : −139Y 2 = X 3 + 10X 2 − 20X + 8. It has conductor N = 37 · 1392 , and according to [8], L(s, E) has a zero of order 3 at s = 1/2. Moreover, under our assumption that χ D (37) = −1, the sign ( f ⊗ χ D ) of the functional equation of L(s, f ⊗ χ D ) = L(s, E ⊗ χ D ) is −1. Hence the function F(s) in (2.6) has a zero of order at least 4 at s = 1/2. Thus we may deduce the lower bound in Theorem 3 with m taken to be equal to 4; this yields Theorem 2. Proof of Theorem 1 The proof of Theorem 1 goes on the same lines as that of Theorem 3 and is technically simpler. Fix a small 1 > 0, and assume first that we have a fundamental discriminant D1 such that L(s, χ D1 ) has a real zero β1 ≥ 1 − 1 . Then take a large D for which L(s, χ D ) has a (Landau-Siegel) zero β, and consider the product used by Siegel: F(s) = ζ (s)L(s, χ D1 )L(s, χ D )L(s, χ D1 χ D ). Its coefficients are again nonnegative. The completed L-function has the form 3(s) =

M|D ||D| s/2 s s + a s + a s + a 1 2 3 1 0 0 0 F(s), 0 2 2 2 2 π4

where a1 , a2 , a3 ∈ {0, 1} according to the parity of the characters χ D1 , χ D and χ D1 χ D , and M is a divisor of D1 D. The function 3(s) is holomorphic everywhere, except for simple poles at s = 0, 1, and satisfies the functional equation 3(s) = 3(1 − s). Again we apply the explicit formula X 3(ρ)=0

φ0

γ B 2π

=

iB −i B A + 2φ0 + 2φ0 B 4π 4π −2

∞ XX p k=1

k log p log p k k 1 + χD ( p) 1 + χ D ( p) φˆ 0 1 B p k/2 B

with the same choice for the test function φ0 . As before, A = log Q + O(1), where this time

√ Q=

M|D1 ||D| . π2

Therefore A ≤ log |D| + log |D1 | + O(1).

504


Again we ignore the positive sum on the right-hand side of the explicit formula while from the left-hand side we pick up the contribution of the Landau-Siegel zeros of L(s, χ D1 ) and L(s, χ D ). One obtains e(β1 −(1/2))B − e−(β1 −(1/2))B 2 e(β−(1/2))B − e−(β−(1/2))B 2 + (2β1 − 1)B (2β − 1)B e B/2 − e−(B/2) 2 log |D| O D1 (1) ≤ + + . B B B Given 1 > 0, we choose B = (1 + 31 ) log log |D|, and from the above inequality we derive −41 1 − β ≥ c(1 , D1 ) log |D| for a positive (effective) constant depending on 1 and D1 . If there is no D1 with a zero for L(s, χ D1 ) in (1 − 1 , 1), then certainly Theorem 1 is true. So either way Theorem 1 is true, alas, with an ineffective constant, as stated.

Proof of Theorem 4 Let E and D be as in the statement of the theorem, and let β be a Landau-Siegel zero of L(s, χ D ). As in the proof of Theorem 3, we let f be the modular form corresponding to E and set 2 F(s) = ζ (s)L(s, χ D ) L(s, f )L(s, f ⊗ χ D ). Since we are assuming only the local hypothesis, Hypothesis H∗ , we proceed at this point in a somewhat different way. For Re(s) > 1, one has X 3(n) 2 + λ f (n))(1 + χ D (n) F0 . − (s) = F ns n≥1

On the other hand, we have the partial fraction expansion X 1 F0 2 1 − (s) = − + + B E,D . F s−1 s−ρ ρ F(ρ)=0

Here B E,D is a constant depending on E and D. For k ≥ 1, we differentiate the above identity (2k − 1) times and get X 2 1 − 2k (s − 1) (s − ρ)2k F(ρ)=0

X 3(n)(log n)2k−1 1 + χ D (n) 2 + λ f (n) 1 = . (2k − 1)! ns n≥1


505

We choose s = 2. Since the right-hand side is positive, by taking the real part on both sides, one has X 2 1 2− ≥ Re . 2k (2 − β) (2 − ρ)2k F(ρ)=0

If we remove from the right-hand side the real zeros of F(s) except ρ1 = 1/2, the inequality is still valid. We want to show that there exists k relatively small for which the right-hand side is large. This prevents β from being very close to 1. At this point we write the numbers 1/(2 − ρ)2 as a sequence of complex numbers z 1 , z 2 , . . . , with multiplicities m 1 , m 2 , . . . , arranged such that |z 1 | ≥ |z 2 | ≥ · · · . Then the right-hand P side of the above inequality equals Re j≥1 m j z kj . We need the following lemma of Turan type. 5 P Set H = ( j≥1 m j |z j |)/(m 1 |z 1 |). Then there exists 1 ≤ k ≤ 24H such that LEMMA

Re

X

m j z kj ≥

j≥1

1 m 1 |z 1 |k . 8

Proof For the proof, see H. Montgomery [17, Chap. 9, Lem. 2]. Note that Hypothesis H∗ implies that z 1 corresponds to ρ1 = 1/2 and equals 1/(2 − (1/2))2 = 4/9 and that m 1 ≥ m. By standard zero counting arguments (see [3, §16]), one has 1 X m j |z j | ≤ c1 log |D|N |z 1 | j≥1

for some effective absolute constant c1 . Hence H ≤ (c1 log(|D|N ))/m, and from Lemma 5 it follows that there exists k ≤ (24c1 log(|D|N ))/m such that Re

X

m j z kj ≥

j≥1

1 4 k . 8 9

For such a k, one has 2−

2 1 4 k ≥ , 8 9 (2 − β)2k

from which one derives the result stated.

506


References [1]

[2]

[3]

[4] [5]

[6] [7]

[8] [9] [10] [11]

[12] [13] [14] [15] [16]

[17]

[18]

E. BOMBIERI, Remarks on Weil’s quadratic functional in the theory of prime numbers,

I, Atti Accad. Naz. Lincei Cl. Sci. Fis. Mat. Natur. Rend. Lincei (9) Mat. Appl. 11 (2000), 183–233. MR CMP 1 841 692 497 C. BREUIL, B. CONRAD, F. DIAMOND, and R. TAYLOR, On the modularity of elliptic curves over Q: Wild 3-adic exercises, J. Amer. Math. Soc. 14 (2001), 843–939. MR 1 839 918 499 H. DAVENPORT, Multiplicative Number Theory, 2d ed., revised by Hugh L. Montgomery, Grad. Texts in Math. 74, Springer, New York, 1980. MR 82m:10001 505 M. DEURING, Imaginäre quadratische Zahlkörper mit der Klassenzahl Eins, Invent. Math. 5 (1968), 169–179. MR 37:4044 497 L. DIRICHLET, Recherches sur diverse applications de l’analyse infinitésimale a` la théorie des nombres, I, J. Reine Angew. Math. 19 (1839), 324–369; II, 21 (1840), 1–12. 495 W. DUKE, Some old problems and new results about quadratic forms, Notices Amer. Math. Soc. 44 (1997), 190–196. MR 97i:11035 496 D. M. GOLDFELD, The class number of quadratic fields and the conjectures of Birch and Swinnerton-Dyer, Ann. Scuola Norm. Sup. Pisa (4) 3 (1976), 624–663. MR 56:8529 496 B. GROSS and D. ZAGIER, Points de Heegner et dérivées de fonctions L, C. R. Acad. Sci. Paris Sér. I Math. 297 (1983), 85–87. MR 85d:11062 496, 503 D. R. HEATH-BROWN, Prime twins and Siegel zeros, Proc. London Math. Soc. (3) 47 (1983), 193–224. MR 84m:10029 498 H. HEILBRONN, On the class-number in imaginary quadratic fields, Quart. J. Math. Oxf. Ser. (2) 5 (1934), 150–160. 497 Y. IHARA, “Discrete subgroups of PL(2, k℘ )” in Algebraic Groups and Discontinuous Subgroups (Boulder, 1965), Proc. Sympos. Pure. Math. 9, Amer. Math. Soc., Providence, 1966, 272–278. MR 34:5777 496 H. IWANIEC, course lecture notes, Rutgers, 1999. 496 H. IWANIEC, W. LUO, and P. SARNAK, Low lying zeros of families of L-functions, Inst. ´ Hautes Etudes Sci. Publ. Math. 91 (2000), 55–131. MR 1 828 743 497, 500 E. LANDAU, Bemerkungen zum Heilbronnschen Satz, Acta Arith. 1 (1935), 1–18. 495 √ J. E. LITTLEWOOD, On the class-number of the corpus P( −k), Proc. London Math. Soc. (2) 27 (1928), 358–372. 496 R. MARTIN and W. MCMILLEN, An elliptic curve over Q with rank at least 24, preprint, 2000, http: //listserv.nodak.edu/scripts/wa.exe?A2=ind0005&L=nmbrthry&F=&S=&P=521 496 H. L. MONTGOMERY, Ten Lectures on the Interface between Analytic Number Theory and Harmonic Analysis, CBMS Reg. Conf. Ser. Math. 84, Amer. Math. Soc., Providence, 1994. MR 96i:11002 505 J. OESTERLE´ , Nombres de classes des corps quadratiques imaginaires, Astérisque


[19] [20]

[21] [22] [23]

507

121–122 (1985), 309–323, Séminaire Bourbaki, 1983/84, exp. no. 631. MR 86k:11064 496 Z. RUDNICK and P. SARNAK, Zeros of principal L-functions and random matrix theory, Duke Math. J. 81 (1996), 269–322. MR 97f:11074 500, 502 A. SELBERG, Harmonic analysis and discontinuous groups in weakly symmetric Riemannian spaces with applications to Dirichlet series, J. Indian Math. Soc. (N.S.) 20 (1956), 47–87. MR 19:531g 496 ¨ C. L. SIEGEL, Uber die Classenzahl quadratischer Zahlkörper, Acta Arith. 1 (1935), 83–86. 495 T. TATUZAWA, On a theorem of Siegel, Jap. J. Math. 21 (1951), 163–178. MR 14:452c 502 H. YOSHIDA, “On hermitian forms attached to zeta functions” in Zeta Functions in Geometry (Tokyo, 1990), Adv. Stud. Pure Math. 21, Kinokuniya, Tokyo, 1992, 281–325. MR 94f:11120 497

Sarnak Department of Mathematics, Princeton University, Fine Hall, Washington Road, Princeton, New Jersey 08544, USA; [email protected] Zaharescu Institute of Mathematics of the Romanian Academy, Post Office Box 1-764, Bucharest 70700, Romania, and Department of Mathematics, University of Illinois at Urbana-Champaign, Altgeld Hall, 1409 West Green Street, Urbana, Illinois 61801, USA; [email protected]


STANDARD MODULES OF QUANTUM AFFINE ALGEBRAS M. VARAGNOLO and E. VASSEROT

Abstract We give a proof of the cyclicity conjecture of Akasaka and Kashiwara for simply laced types, via quiver varieties. We also get an algebraic characterization of the standard modules. 1. Introduction Let g be a simple, simply laced, complex Lie algebra. If g is of type A, a geometric realization of the quantized enveloping algebra U of g[t, t −1 ] and of its simple modules was given a few years ago in [8] and [18]. This construction involved perverse sheaves and the convolution algebra in equivariant K -theory of partial flag varieties of type A. It was then observed in [14] and [15] that these varieties should be viewed as a particular case of the quiver varieties associated to any symmetric Kac-Moody Lie algebra. This leads to a geometric realization of U via a convolution algebra in equivariant K -theory of the quiver varieties for g of (affine) type A(1) in [17] and for a general symmetric Kac-Moody algebra g in [16]. For any symmetric Lie algebra g, one gets a formula for the dimension of the finite-dimensional simple modules of U in terms of intersection cohomology (see [16]). The standard modules are a basic tool in this geometric approach. They are the geometric counterparts of the Weyl modules of U (see Remark 7.19). In this paper we give an algebraic construction of the standard modules. It answers a question in [16, Corollary 7.16]. An immediate corollary is a proof of the cyclicity conjecture in [1, Corollary 7.17] for simply laced Lie algebras. The plan of the paper is the following. Sections 1 to 6 contain a review of quantum affine algebras and quiver varieties. The main results are given in Section 7. The proof of Theorem 7.4 uses Lemma 8.1. DUKE MATHEMATICAL JOURNAL c 2002 Vol. 111, No. 3, Received 29 June 2000. Revision received 16 March 2001. 2000 Mathematics Subject Classification. Primary 17B37; Secondary 14D21, 14L30, 16G20, 33D80. Both authors’ work partially supported by European Economic Community (EEC) grant number ERB FMRXCT97-0100.

509

510

VARAGNOLO and VASSEROT

2. The algebra U Let g be a simple, simply laced, complex Lie algebra. The quantum loop algebra ± ±1 ± ± (i ∈ I, r ∈ associated to g is the C(q)-algebra U0 generated by xir , kis , ki = ki0 × Z, s ∈ ±N ) modulo the following defining relations: ki ki−1 = 1 = ki−1 ki ,

± [ki,±r , kεj,εs ] = 0,

ki x±jr ki−1 = q ±ai j x±jr , (w − q ±a ji z)kεj (w)xi± (z) = (q ±a ji w − z)xi± (z)kεj (w), (z − q ±ai j w)xi± (z)x±j (w) = (q ±ai j z − w)x±j (w)xi± (z), + − ki,r +s − ki,r +s

, q − q −1 m XX p m ± ± ± (−1) xir x± · · · xir x± x± · · · xir = 0, w(1) irw(2) w( p) js irw( p+1) w(m) p w

+ − [xir , x js ] = δi j

p=0

where i 6= j, m = 1 − ai j , r1 , . . . , rm ∈ Z, and w ∈ Sm . We set [n] = q 1−n + q 3−n + · · · + q n−1 if n ≥ 0, [n]! = [n][n − 1] · · · [2], and [m]! m = . p [ p]![m − p]! We also set ε = + or −, ki± (z) =

X

± ki,±r z ∓r ,

xi± (z) =

r ≥0

X

± ∓r xir z .

r ∈Z

Put A = C[q, q −1 ]. Consider also the A-subalgebra U ⊂ U0 generated by the quan± (n) ±n tum divided powers xir = xir /[n]!, the Cartan elements ki±1 , and the coefficients P of the formal series exp(− s≥1 (hi,±s /[s])z ∓s ) such that X ki± (z) = ki±1 exp ±(q − q −1 ) hi,±s z ∓s . s≥1

Let 1◦ be the coproduct defined in terms of the Kac-Moody generators ei , fi , ki±1 , i ∈ I ∪ {0}, of U0 as follows: 1◦ (ei ) = ei 1 + ki ei ,

1◦ (fi ) = fi ki−1 + 1 fi ,

1◦ (ki ) = ki ki ,

where is the tensor product over the field C, or the ring A. Let τ be the antiautomorphism of U such that τ (ei ) = fi , τ (fi ) = ei , τ (ki ) = ki−1 , and τ (q) = q −1 . It is − + ± ∓ ) = xik and τ (ki,±r ) = ki,∓r . Let 1• be the coproduct known (see [2]) that τ (xi,−k ◦ ◦ • opposite to 1 . We have (τ τ )1 τ = 1 . Hereafter, ζ is an element of C× which is not a root of unity.

STANDARD MODULES OF QUANTUM AFFINE ALGEBRAS

511

3. The braid group Let W, P, Q be, respectively, the Weyl group, the weight lattice, and the root lattice e = W n P is generated by the simple reof g. The extended affine Weyl group W e by the flexions si and the fundamental weights ωi , i ∈ I . Let 0 be the quotient of W normal Coxeter subgroup generated by the simple affine reflexions. The group 0 is a group of diagram automorphisms of the extended Dynkin diagram of g. In partice . For any w ∈ W e , let l(w) denote its length. The braid ular, 0 acts on U and on W e is the group on generators Tw , w ∈ W e , with the relation group BW e associated to W Tw Tw0 = Tww0 whenever l(ww0 ) = l(w) + l(w0 ). The group BW acts on U by algebra e − + r r (ei ), automorphisms (see [12], [2]). Recall that xir = ν(i) Tωi (fi ), xir = ν(i)r Tω−r i for a fixed function ν : I → {±1} such that ν(i) + ν( j) = 0 if ai j < 0 (see [2, ± ± ± ± Definition 4.6]). We have Tω j (kir ) = kir for any i, j, and Tω j (xir ) = xir for any i 6= j. For any i ∈ I ∪ {0}, put X cl Tsi (fi )(l) Tsi (ei )(l) , Ri = l≥0

where cl = (−1)l q −l(l−1)/2 (q − q −1 )l [l]!. The element Ri belongs to a completed tensor product (U U)ˆ(see [12, Section 4.1.1], for instance). It is known that X Ri−1 = c¯l Tsi (fi )(l) Tsi (ei )(l) , l≥0

e where c¯l = q l(l−1)/2 (q − q −1 )l [l]!. If τ si1 · · · sir is a reduced expression of w ∈ W with τ ∈ 0, set (Ri2 )Ri1 , (3.1) Rw = τ Ti[2] · · · Ti[2] (Rir ) · · · Ti[2] 1 1 r −1 e are such that l(ww0 ) = where a [2] = a a for any a. In particular, if w, w0 ∈ W [2] 0 e , set 1◦w = Tw[2] 1◦ Tw−1 . l(w) + l(w ), then Rww0 = Tw (Rw0 )Rw . For any w ∈ W −1 = 1◦ x for all x ∈ U (see [12, Proposition 37.3.2] and [2, Then Rw · 1◦ x · Rw w Section 5]).

4. The quiver varieties 4.1 Let I (resp., E) be the set of vertices (resp., edges) of a finite graph (I, E) with no edge loops. For i, j ∈ I , let n i j be the number of edges joining i and j. Put ai j = 2δi j −n i j . The map (I, E) 7 → A = (ai j )i, j∈I is a bijection from the set of finite graphs with no edge loops onto the set of symmetric generalized Cartan matrices. Let αi , i ∈ I , be the simple roots of the symmetric Kac-Moody algebra g corresponding

512


to A. Hereafter, we assume that g is finite-dimensional, that is, that the matrix A is positive definite. Let H be the set of edges of (I, E) together with an orientation. For h ∈ H , let h 0 and h 00 be the incoming and the outcoming vertex of h, respectively. If h ∈ H , we denote by h ∈ H the same edge with opposite orientation. Given two L L I -graded finite-dimensional complex vector spaces V = i∈I Vi , W = i∈I Wi , set M M E(V, W ) = Hom(Vh 00 , Wh 0 ), L(V, W ) = Hom(Vi , Wi ). h∈H

i∈I

L Let P + , Q + be the semigroups Q + = i∈I Nαi and P + = i∈I Nωi . Let us fix once for all the following convention: the dimension of the graded vector space V is P identified with the element α = i∈I vi αi ∈ Q + (where vi is the dimension of Vi ), P while the dimension of W is identified with the weight λ = i wi ωi ∈ P + (where P P wi is the dimension of Wi ). We also put |λ| = i wi and |α| = i vi . We write α ≥ α 0 if and only if α − α 0 ∈ Q + . Set L

Mαλ = E(V, V ) ⊕ L(W, V ) ⊕ L(V, W ). For any (B, p, q) ∈ Mαλ , let Bh be the component of B in Hom(Vh 00 , Vh 0 ) and set X m αλ (B, p, q) = ε(h)Bh Bh + pq ∈ L(V, V ), h

where ε is a function ε : H → C× such that ε(h) + ε(h) = 0. A triple (B, p, q) ∈ ◦ m −1 subspace of Ker q. Let m −1 αλ (0) is stable if there is no nontrivial B-invariant αλ (0) Q be the set of stable triples. The group G α = i GL(Vi ) acts on Mαλ by g · (B, p, q) = (g Bg −1 , gp, qg −1 ). ◦ The action of G α on m −1 αλ (0) is free. Put ◦ Q αλ = m −1 αλ (0) /G α

and

Nαλ = m −1 αλ (0)//G α ,

where // denotes the categorical quotient. The variety Q αλ is smooth and quasiprojective. 4.2 Let π : Q αλ → Nαλ be the affinization map. Let L αλ = π −1 (0) ⊂ Q αλ be the zero fiber. It is known that dim Q αλ = 2 dim L αλ . If α, α 0 ∈ Q + are such that α ≥ α 0 , then the extension by zero of representations of the quiver gives an injection Nα 0 λ ,→ Nαλ (see [16, Lemma 2.5.3]). For any α, α 0 , we consider the fiber product Z αα 0 λ = Q αλ ×π Q α 0 λ .


513

It is known that dim Z αα 0 λ = (dim Q αλ + dim Q α 0 λ )/2. If α 0 = α + αi and V ⊂ V 0 have dimensions α and α 0 , respectively, let Cαi+0 λ ⊂ Z αα 0 λ be the set of pairs of triples 0 = B, p 0 = p, q 0 = q. If α 0 = α − α , (B, p, q), (B 0 , p 0 , q 0 ), such that B|V i |V i+ ⊂ Z αα 0 λ , where φ flips the components. The variety Cαi±0 λ is an put Cαi−0 λ = φ Cαλ irreducible component of Z αα 0 λ . Consider the following varieties: [ G G Nλ = Nαλ , Qλ = Q αλ , Zλ = Z αα 0 λ , α

α

Cλi± =

G α

i± Cαλ ,

α,α 0

Lλ =

G

L αλ ,

α

where α, α 0 take all the possible values in Q + . Observe that for a fixed λ the set Q αλ is empty except for a finite number of α’s. 4.3 eλ = G λ × C× . The group G eλ acts on Mαλ by Put G (g, z) · (B, p, q) = (z B, zpg −1 , zgq). eλ with t This action descends to Q αλ and Nαλ . For any element s = (t, ζ ) ∈ G Z e semisimple, let hsi ⊂ G λ be the Zariski closure of s . For any group homomorphism ρ ∈ Hom(hsi, G α ), let Q(ρ) ⊂ Q αλ be the subset of the classes of the triples (B, p, q), such that s · (B, p, q) = ρ(s) · (B, p, q). The fixed-point set Q sαλ is the disjoint union of the subvarieties Q(ρ). It is proved in [16, Theorem 5.5.6] that Q(ρ) is either empty or a connected component of Q sαλ . 4.4 Fix λ1 , λ2 ∈ P + , such that λ = λ1 + λ2 . The direct sum Mλ1 × Mλ2 → Mλ gives a closed embedding κ : Q λ1 × Q λ2 ,→ Q λ . eλ , Fix a semisimple element t = t 1 ⊕ t 2 ∈ G λ1 × G λ2 . Set s = (t, ζ ) ∈ G 1 2 Z 1 2 eλ1 , s = (t, ζ ) ∈ G eλ2 . If (ζ spec t ) ∩ spec t = ∅, then s = (t, ζ ) ∈ G

LEMMA

(i) (ii)

1

2

Q sλ = κ(Q sλ1 × Q sλ2 ).

Proof Fix I -graded vector spaces W, W 1 , W 2 , V, V 1 , V 2 , such that W = W 1 ⊕ W 2 , V = V 1 ⊕ V 2 , dim W 1 = λ1 , dim W 2 = λ2 , dim V 1 = α 1 , and dim V 2 = α 2 . Fix triples (0)◦ , x 2 , x 0 2 ∈ m −1 (0)◦ . The triple x = x 1 ⊕ x 2 is stable since if x 1 , x 0 1 ∈ m −1 α 1 λ1 α 2 λ2 V 0 ⊆ Ker q is a B-stable subspace, then V 1 ∩ V 0 = {0} by the stability of x 1 , and then V 0 embeds in V /V 1 . Thus it is zero by the stability of x 2 . Assume that g ∈ G α 1 +α 2 maps x to x 0 1 ⊕ x 0 2 . Then q g −1 (V 2 )∩V 1 ⊆ W 1 ∩W 2 = {0} and B g −1 (V 2 )∩V 1 ⊆ g −1 (V 2 )∩V 1 .

514


Thus the stability of x gives g −1 (V 2 ) ∩ V 1 = {0}. In the same way, we get g −1 (V 1 ) ∩ V 2 = {0}. Thus g ∈ G α 1 × G α 2 . Claim (i) is proved. Fix ρ such that Q(ρ) is nonempty. For any z ∈ C× , put V (z) = Ker ρ(s) − z −1 idV , W (z) = Ker(t − z idW ). If x ∈ Q(ρ) and (B, p, q) is a representative of x, then B V (z) ⊂ V (z/ζ ), q V (z) ⊂ W (z/ζ ), Moreover, the stability of (B, p, q) implies that V = follows.

p W (z) ⊂ V (z/ζ ). L

z∈ζ Z spec t

V (z). Claim (ii)

4.5 ◦ Let V = m −1 αλ (0) ×G α V and W be, respectively, the tautological bundle and the trivial W -bundle on Q αλ . The ith component of V , W is denoted by Vi , Wi . The eλ -equivariant. Let q be the trivial line bundle on Q αλ with the bundles V , W are G degree-one action of C× . We consider the classes X i = q −1 Wi − (1 + q −2 )Vi + q −1 Vh 00 , Fαλ h 0 =i

Tαλ = q E(V , V ) + q L(q W − V , V ) + L(V , q W − V ) 2

in KG λ (Q αλ ), and the classes e

0 2 0 0 2 Tαi+ 0 λ = q E(V , V ) + q L(q W − V , V ) + L(V , q W − V ) − q , 0 2 0 0 2 Tαi− 0 λ = q E(V , V ) + q L(q W − V , V ) + L(V , q W − V ) − q

in KG λ (Q αλ × Q α 0 λ ) (where α 0 = α ± αi ). It is known that Tαλ is the class of the i± tangent sheaf to Q αλ and that Tαi± 0 λ |C i± is the class of the normal sheaf of C α 0 λ in e

Q λ × Q λ (see [15]).

α0 λ

5. The convolution product 5.1 For any complex algebraic linear group G and any quasi-projective G-variety X , let KG (X ), KG (X ) be the complexified Grothendieck groups of G-equivariant coherent sheaves and of locally free sheaves, respectively, on X . We put R(G) = KG ( point). For simplicity, let f ∗ , f ∗ , ⊗ denote the derived functors R f ∗ , L f ∗ , ⊗ L (where ⊗ is the tensor product of sheaves of O X -modules). We use the same notation for a sheaf


515

and its class in the Grothendieck group. Hereafter, elements of KG (X ) may be identified with their image in KG (X ). The class of the structural sheaf is simply denoted by 1. Given smooth quasi-projective G-varieties X 1 , X 2 , X 3 , consider the projection pab : X 1 × X 2 × X 3 → X a × X b for all 1 ≤ a < b ≤ 3. Consider closed subvarieties −1 −1 Z ab ⊂ X a × X b such that the restriction of p13 to p12 Z 12 ∩ p23 Z 23 is proper and maps to Z 13 . The convolution product is the map ∗ ∗ ? : KG (Z 12 ) KG (Z 23 ) → KG (Z 13 ), E F 7 → p13 ∗ ( p12 E ) ⊗ ( p23 F) (see [4] for more details). The flip φ : X a × X b → X b × X a gives a map φ∗ : KG (Z ab ) → KG (Z ba ). This map anticommutes with ?; that is, φ∗ (x12 ? x23 ) = φ∗ (x23 ) ? φ∗ (x12 ),

∀x12 , x23 .

5.2 Let D X be the Serre-Grothendieck duality operator on KG (X ) (see [13, Section 6.10], for instance). Assume that X is the disjoint union of smooth connected subvarieties X (i) . Assume also that we have fixed a particular invertible element q ∈ KG (X ). Put P d X (i) = dim X (i) , D X (i) = q d X (i) D X (i) , D X = i D X (i) . Let X be the determinant of the cotangent bundle to X . Then D X (E ) = (−1)dim X E ∗ ⊗ X for any G-equivariant (i j)

(i)

( j)

locally free sheaf E . Put Z ab = Z ab ∩ (X a × X b ). Assume that X (i) = q a for all a, i. The operators X (i j) D Z ab = q dab D Z i j , i, j

−d

(i) Xa

ab

(i j) where dab = d X (i) + d X ( j) /2 are compatible with the convolution product ?; that a b is, D Z 12 (x12 ) ? D Z 23 (x23 ) = D Z 13 (x12 ? x23 ), ∀x12 , x23 (see [13, Lemma 9.5] for more details). 5.3 If E is a G-bundle on X , we have the element V

z (E ) =

rk E X V ( i E ) · z i ∈ KG (X )[z], i=0

Vi

V V V E is the ith wedge product. Clearly, z (E + F ) = z (E ) ⊗ z (F ) for where any E , F , and V V rk E Det(E ) ⊗ −1 (E ∗ ). (5.4) −1 (E ) = (−1)

516


V ¯ Observe also that z (E ) admits an inverse in KG (X )[[z]]. Let R(G) be the fraction G ¯ ¯ ¯ ¯ field of R(G), and set KG (X ) = KG (X )R(G) R(G), K (X ) = KG (X )R(G) R(G). Assume that G is a diagonalizable group and that the set of G-fixed points of the V ¯ G (X ) by the localization restriction E | X G is X G . Then −1 (E ) is invertible in K V theorem and [4, Proposition 5.10.3]. In particular, the element −1 (F − E ) = V V −1 is well defined in K ¯ G (X ) for any F . If G is a product of −1 (F ) ⊗ −1 (E ) V ¯ G (X ) if the set of GLn ’s and H ⊂ G is a torus, then −1 (E ) is still invertible in K H H -fixed points of E | X H is X (use [4, Theorem 6.1.22], which holds, although the group G is not simply connected). In the sequel we may identify a G-bundle and its class in the Grothendieck group. 5.5 Assume that the group G is Abelian. For any R(G)-module M and any s ∈ G, let Ms be the specialization of M at the maximal ideal in R(G) associated to s. The localization theorem gives isomorphisms of modules ι∗ : KG (X s )s → KG (X )s ,

¯ G (X s ) → K ¯ G (X ), ι∗ : K

where ι : X s → X is the closed embedding.

6. Nakajima’s theorem 6.1 We fix a subset H + ⊂ H such that H + ∩ H¯ + = ∅ and H + ∪ H¯ + = H . For any i, j ∈ I , let n i+j be the number of arrows in H + from i to j. Put n i−j = n i j − n i+j . Observe that n i+j = n −ji . Put i− Fαλ = −Vi + q −1

X

n i−j V j ,

i+ Fαλ = q −1 Wi − q −2 Vi + q −1

j

X

n i+j V j .

j

Let ( | ) : Q × P → Z be the pairing such that (αi |ω j ) = δi j for all i, j ∈ I . The i is (α |λ − α). Put F i± = L F i± and F i = L F i . Let f i , f i± rank of Fαλ i α αλ α αλ λ λ λ λ e i = rk F i and be the diagonal operators acting on KG λ (Q αλ ) by the scalars f αλ αλ i± i± f αλ = rk Fαλ . Let p, p 0 : (Q λ )2 → Q λ be the first and the second projection. We e denote by V , V 0 ∈ KG λ (Q λ )2 the pullback of the tautological sheaf (i.e., V = p ∗ V and V 0 = p 0 ∗ V ). Set L = q −1 (V 0 − V ). For any r ∈ Z, set xir± =

X α0

⊗r + f αi± 0λ

(±L )

|Cαi±0 λ

? δ∗ xαi±0 λ ,

i

ki± (z) = δ∗ q fλ

V

−1/z

(q −1 − q)Fλi

±

, (6.2)


517

i±

i± i±∗ where xαλ = (−1) fαλ Det(Fαλ ), the map δ is the diagonal embedding Q λ ,→ 2 (Q λ ) , and ± is the expansion at z = ∞ or zero. Hereafter, we may omit δ, hoping e eλ )that it causes no confusion. Let Uλ be the quotient of KG λ (Z λ ) by its torsion R(G submodule. The space Uλ is an associative algebra for the convolution product ? (with Z 12 = Z 23 = Z λ ). It is proved in [16, Theorems 9.4.1 and 12.2.1] that the map ± ± ± xir 7 → xir± , kir 7→ kir extends uniquely to an algebra homomorphism 8λ : U → Uλ .

Remark 6.3 The morphism 8λ is not the one used by H. Nakajima, although the operators h ir in (6.2) and in [16, Section 9.2] are the same. The proof of Nakajima still works in our case: the relations [16, (1.2.8) and (1.2.10)] are checked in the appendix, the relations involving only one vertex of the graph are proved as in [16, Section 11], and the Serre relations are proved as in [16, Section 10.4]. 6.4 Recall that dim Q αλ = (α|2λ − α). From the formula for Tαλ in Section 4.5, we get Q αλ = q −d Q αλ . Thus the hypotheses in Section 5.2 are satisfied. Consider the antiautomorphism γU = φ∗ D Z λ of Uλ . LEMMA 6.5 We have 8λ τ = γU 8λ .

Proof For any α 0 ∈ Q + , the Hecke correspondence Cαi±0 λ is smooth and i

i∗ C i± = q ∓ fαλ p ∗ Q αλ ⊗ p ∗ Det(Fαλ ) ⊗ (±L )

i f αλ

|Cαi±0 λ

α0 λ

,

where α = α 0 ∓ αi (see the proof of 7.4). Using the identities ∗

∗ i±∗ ∓1 p 0 Fαi±∗ L 0 λ − p Fαλ = −q

and

i dim Z αα 0 λ = ∓ f αλ + d Q α0 λ + 1

and the commutation of the Serre-Grothendieck duality with closed embeddings, we get γU (xir± ) =

X α0

= =

⊗ f αi∓ 0 λ −r

q −1 xαi∓0 λ ? (∓L )

|Cαi∓0 λ

X ⊗ f i∓ −r i∓ (∓L ) i∓αλ ? xαλ α ∓ xi,−r .

|Cαλ

518


6.6 e e eλ ). The Rλ -modules W0 , Put Wλ = KG λ (L λ ), W0λ = KG λ (Q λ ), and Rλ = R(G λ 0 Wλ are free. Thus Wλ (resp., Wλ ) may be viewed as a U-module via the algebra homomorphism U → Uλ → End Wλ (resp., U → End W0λ ), which is composed of 8λ and of the convolution product ? : Uλ Wλ → Wλ (resp., ? : Uλ W0λ → W0λ ) for Z 12 = Z λ and Z 23 = Q λ (resp., Z 23 = L λ ). The varieties L 0λ and Q 0λ are reduced to a point. Let [0] be their fundamental class in K-theory. By [16, Propositions 12.3.2 and 13.3.1], the U-module Wλ is cyclic and generated by [0], and we have ± V xi+ (z) ? [0] = 0, ki± (z) ? [0] = q (λ | αi ) −1/z (q −2 − 1)Wi ⊗ [0]. 6.7 eλ . Let hsi ⊂ G eλ be the Zariski closed Fix a semisimple element s = (t, ζ ) in G subgroup generated by s. Put Ws = Khsi (L λ )s ,

W0s = Khsi (Q λ )s ,

Us = Khsi (Z λ )s .

Let 8s : U → Us be the composition of 8λ and the specialization at s. Consider the C-algebra U|q=ζ = U ⊗A A/(q − ζ ) . The spaces Ws , W0s are U|q=ζ -modules. The U|q=ζ -module Ws is called a standard module. It is cyclic and generated by [0].

7. The coproduct 7.1 Fix λ1 , λ2 ∈ P + , such that λ = λ1 + λ2 . Put G λ1 λ2 = G λ1 × G λ2 × C× , Rλ1 λ2 = ¯ λ1 λ2 be the fraction field of Rλ1 λ2 . Set R(G λ1 λ2 ), and let R ¯ λ1 λ2 , ¯ 0λ = W0λ Rλ R W0λ1 λ2 = KG λ1 λ2 (Q λ1 × Q λ2 ), W ¯ 0 1 2 = W0 1 2 R 1 2 R ¯ λ1 λ2 . W λ λ λ λ λ λ ¯ λ , Wλ1 λ2 , W ¯ λ1 λ2 (using L λ instead of Q λ ) and U ¯ λ, U ¯ λ1 λ2 (using We define similarly W Z λ instead of Q λ ). By [16, Section 7] and [4, Section 5.6], we have two Kunneth isomorphisms ∼

W0λ1 W0λ2 → W0λ1 λ2 ,

∼

Wλ1 Wλ2 → Wλ1 λ2 .

¯ λ1 λ2 . We do not know if Let us denote them by θ. Set e Uλ1 λ2 = (Uλ1 Uλ2 ) Rλ1 λ2 R there is a Kunneth isomorphism KG λ1 (Z λ1 ) KG λ2 (Z λ2 ) ' KG λ1 λ2 (Z λ1 × Z λ2 ). e

e


519

¯ λ1 λ2 induced by the external However, it is easy to see that the map θ : e Uλ1 λ2 → U × tensor product is invertible: if H = Hλ1 × Hλ2 × C ⊆ G λ1 λ2 is a maximal torus, then ¯ ) ¯ R(H R

λ1 λ2

¯ Hλ2 ×C× (Z λ2 ) ¯ Hλ1 ×C× (Z λ1 ) K e Uλ1 λ2 ' K ×

×

¯ Hλ1 ×C (Q λ1 × Q λ1 ) K ¯ Hλ2 ×C (Q λ2 × Q λ2 ) 'K ¯ H (Q 2 1 × Q 2 2 ) 'K λ λ ¯ H (Z λ1 × Z λ2 ) 'K ¯ ) ¯ ' R(H R

λ1 λ2

¯ λ1 λ2 . U

Here we have used the localization theorem, the identity ×

(Z λb ) Hλb ×C = (Q λb × Q λb ) Hλb ×C

×

for b = 1, 2,

the Kunneth formula for Q λ1 × Q λ2 , and [4, Theorem 6.1.22] (which is valid, although G λ1 λ2 is not simply connected). Taking the invariants under the Weyl group, we get the required invertibility. This invertibility is not needed in the sequel. Set also T+ = q E + (V 1 , V 2 ) + L(V 1 , q W 2 − V 2 ) + q −1 E + (V 2 , V 1 )

+ q −2 L(V 2 , q W 1 − V 1 ), T+0 = q E(V 1 , V 2 ) + L(V 1 , q W 2 − V 2 ) + q 2 L(q W 1 − V 1 , V 2 )

in W0λ1 λ2 , where E ± (V 1 , V 2 ) =

L

i, j

∗

n i±j Vi 1 ⊗ V j2 .

LEMMA 7.2 The class of the normal bundle of Q λ1 × Q λ2 in Q λ is T+0 + q 2 T+0 ∗ . In particular, V ¯ 0 1 2. the class 0 = −1 (T+0 ∗ + q −2 T+0 ) is well defined and is invertible in W λ λ

Proof The proof follows from Lemma 4.4 and Sections 4.5 and 5.3. ¯0 → W ¯ 0 1 2 be the map induced by the pullback κ ∗ : W0 → W0 1 2 Let 10W 0 : W λ λ λ λ λ λ (which is well defined since Q λ1 × Q λ2 and Q λ are smooth). Since Z λ is a closed subvariety of (Q λ )2 , the restriction with support with respect to the embedding (Q λ1 )2 × (Q λ2 )2 ,→ (Q λ )2 gives a map ×

×

(κ × κ)∗ : KG λ ×C (Z λ ) → KG λ1 ×G λ2 ×C (Z λ1 × Z λ2 ). 0 = (1 0 −1 ) ? (κ × κ)∗ : U ¯λ → U ¯ λ1 λ2 , where 1 0 −1 is the pullPut 1U −1 back of 0 by the second projection Z λ1 × Z λ2 → Q λ1 × Q λ2 . There is a unique

520


i+ linear map Q → P, α 7→ α + , such that f αλ = (αi |λ − α + ). By (5.4) and V Lemma 7.2, the class −1 (T+ | Q α1 λ1 ×Q α2 λ2 ) is well defined and is invertible in the ¯ G λ1 ×G λ2 ×C× (Q α 1 λ1 × Q α 2 λ2 ). Set ring K

=

X

q (α

1 |α 2+ −λ2 )

α 1 ,α 2

−1 (T+ | Q α 1 λ1 ×Q α 2 λ2 )

V

¯ 0 1 2. ∈W λ λ

The class is invertible. We put 1W 0 = −1 ⊗ 10W 0 . (Here ⊗ is the tensor product ¯ λ1 λ2 . We put 1 = δ∗ −1 ? on Q λ1 × Q λ2 .) Let δ∗ ±1 be the image of ±1 in U U 0 1U ? δ∗ . Hereafter, δ∗ may be omitted. 7.3 e∈ e e We put Let R Uλ1 λ2 be the element defined in Lemma 8.1(iv), and set R¯ = θ ( R). −1 −1 ¯ ¯ ¯ 1W 0 = R ? 1W 0 and 1U = R ? 1U ? R. Recall that we have anti-involutions γU ¯ λ and U ¯ λ1 λ2 (see Section 6.4). We set 1γ = γU 1U γU . of U U

THEOREM 7.4 ¯λ → U ¯ λ1 λ2 satisfies The map 1U : U

1U 8λ = θ(8λ1 8λ2 )1◦

and

γ

1U 8λ = θ(8λ1 8λ2 )1• .

LEMMA 7.5 Assume that M, M 0 are smooth quasi-projective G-varieties. Let p : M × M 0 → M be the projection. Fix a semisimple element s ∈ G and a smooth closed G-subvariety X ⊂ M × M 0 . Put N = T X − ( p ∗ T M)| X and N s = T X s − ( p ∗ T M s )| X s . V (i) The element −1 (−N ∗ | X s + N s∗ ) ∈ Khsi (X s ) is well defined. Its image in V K(X s ) under the evaluation map is still denoted by −1 (−N ∗ | X s + N s∗ ). (ii) For any G-bundle E on X , the bivariant localization morphism r : Khsi (X )s → K(X s ) defined in [4, Section 5.11] maps E to E | X s ⊗ V ∗ s∗ s −1 (−N | X s + N ) (here ⊗ is the tensor product on X ).

Remark 7.6 eλ , s 1 = (t 1 , ζ ) ∈ G eλ1 , s 2 = (t 2 , ζ ) ∈ G eλ2 , such that t = t 1 ⊕ t 2 Fix s = (t, ζ ) ∈ G hsi and t is semisimple. Put Us 1 s 2 = K (Z λ1 × Z λ2 )s , and let rs : Us → K(Z λs ),

1

2

rs 1 s 2 : Us 1 s 2 → K(Z λs 1 × Z λs 2 )

be the bivariant localization maps. These maps are invertible and commute to the 1 2 convolution product ?. If (ζ Z spec t 1 ) ∩ spec t 2 = ∅, then Q sλ ' Q sλ1 × Q sλ2 , Z λs ' 1

2

0 at s is well defined and coincides with the Z λs 1 × Z λs 2 , and the specialization of 1U map r−1 · rs . s1s2


521

Proof of Lemma 7.5 Claim (i) is well known (see [4, Proposition 5.10.3], for instance). Claim (ii) is immediate from the Koszul resolution of O X by sheaves of locally free O M×M 0 -modules in a neighborhood, in M × M 0 , of each point of X s . Proof of Theorem 7.4 1 2 0 specializes to the map Assume that s is generic. Then κ(Q sλ1 × Q sλ2 ) = Q sλ and 1U −1 rs 1 s 2 · rs by Remark 7.6. Assume that α 0 = α ± αi ∈ Q + . Observe that T Cαi+0 λ − T Q αλ 1 = 1 Tα 0 λ − Tαi+ 0λ

= E(L , V 0 ) + L(L , W ) − (q + q −1 )L(L , V 0 ) + q 2 ∗

= q 2 + q L ∗ ⊗ p 0 Fαi 0 λ , ∗

T Cαi−0 λ − T Q αλ 1 = q 2 + q L ⊗ p 0 Fαi∗0 λ .

We have G

Cαi±0 λ ∩ (Z λ1 × Z λ2 ) =

C i±1 0

α=α 1 +α 2

α λ1

× δ Q α 2 λ2 t δ Q α 1 λ1 × C i±2 0

α λ2

,

0

where α b = α b ± αi . Fix α 1 , α 2 ∈ Q + such that α = α 1 + α 2 . Take M = Q αλ , M 0 = Q α 0 λ , and X = Cαi±0 λ in Lemma 7.5. Let θ : Us 1 Us 2 → Us 1 s 2 be the obvious map. The element r−1 · rs (xir+ ) ∈ Us1 s2 is the image by θ of s1s2 V f i+ (xir+ 1) ⊗ (−1) λ2 Det L δ∗ Fλi+∗ ⊗ −1 −q −1 L δ∗ Fλi∗2 2 V f i+ + −1 −q −1 δ∗ Fλi∗1 L ⊗ (−1) λ1 Det δ∗ Fλi+∗ L ⊗ (1 xir+ ). 1 In this formula, the element L δ∗ Fλi+∗ is identified with its restriction to C i+1 0 2

α λ1

)2

×

δ Q α 2 λ2 ⊆ (Q λ1 × δ(Q λ2 ). Thus Det is the maximal exterior power of a virtual V V bundle on C i+1 0 1 × δ Q α 2 λ2 . Similarly, −1 −q −1 L δ∗ Fλi∗2 is the −1 of a virα λ

tual bundle on C i+1 0

α λ1

× δ Q α 2 λ2 . It is well defined by [4, Proposition 5.10.3] since

s is generic. Here, ⊗ is the tensor product of sheaves on C i+1 0 smooth). In the same way,

r−1 s1s2

· rs (xir− )

α λ1

× δ Q α 2 λ2 (which is

is the image by θ of

V f i− ⊗ −1 −q −1 L ∗ δ∗ Fλi 2 (xir− 1) ⊗ (−1) λ2 Det −L δ∗ Fλi−∗ 2 V f i− + −1 −q −1 δ∗ Fλi 1 L ∗ ⊗ (−1) λ1 Det −δ∗ Fλi−∗ L ⊗ (1 xir− ). 1 ¯ 0 1 2 specializes to a class in Khsi (Q λ1 × Q λ2 )s . Since s is generic, the class ∈ W λ λ

522


Let s 1 s 2 be this class. A direct computation gives the following identities in Us 1 s 2 : V ∓ f i+ ∗ λ2 −1 q L ⊗ (xir± 1),

± s−1 1 s 2 ⊗ (x ir 1) ⊗ s 1 s 2 = q

−1 Fλi+ L Fλi− 2 +q 2

V ∓ f i− −1 i+ λ1 −1 q Fλ1 ⊗ (1 xir± ).

± s−1 1 s 2 ⊗ (1 x ir ) ⊗ s 1 s 2 = q

∗

∗

L ∗ + q Fλi− L 1

Using (5.4), we get fi V 1U (xir+ ) = θ xir+ 1 + q λ1 −1 (q −1 − q)Fλi 1 L ∗ ⊗ (1 xir+ ) , fi V (xir− ) = θ (xir− 1) ⊗ q λ2 −1 (q −1 − q)(−L )∗ Fλi 2 + 1 xir− . (7.7) 1U Using (7.7), it is proved in Lemma 8.1(iii) that ◦ ¯ R¯ s−1 1 s 2 ? (1U 8s ) ? Rs 1 s 2 = θ(8s 1 8s 2 )1 .

We are done. The second identity follows from the first one and Lemma 6.5 since 1◦,τ = 1• . 7.8 Let H ⊂ G λ1 × G λ2 × C× be the maximal torus of diagonal matrices. Assume that the elements s, s 1 , s 2 in Section 7.3 belong to H. Put W0λ,H = K H (Q λ ),

W0λ1 λ2 ,H = K H (Q λ1 × Q λ2 ),

¯ )-vector spaces are and do the same for Wλ,H , Wλ1 λ2 ,H . The corresponding R(H 0 H ¯ ¯ overlined (i.e., we set Wλ,H = K (Q λ ), etc.). Let θ denote the Kunneth isomorphisms ∼

Wλ1 ,H R(H ) Wλ2 ,H → Wλ1 λ2 ,H ,

∼

W0λ1 ,H R(H ) W0λ2 ,H → W0λ1 λ2 ,H .

Consider the bilinear pairing ( | ) : W0λ,H R(H ) Wλ,H → R(H ), E F 7 → q∗ (E ⊗ F ), where q is the projection to a point and ⊗ is the tensor product of sheaves on Q sλ . The following lemma is proved as in [16, Proposition 12.3.2 and Theorem 7.3.5]. 7.9 The U-module Wλ,H is generated by R(H ) ⊗ [0].

LEMMA

(i)


(ii)

523

The R(H )-modules W0λ,H , Wλ,H are free, and the pairing ( | ) is perfect.

¯ ), and the Let ( | ) denote also the pairing with the scalars extended to the field R(H 0 ¯ ¯ ¯ λ,H → pairing between Wλ1 λ2 ,H and Wλ1 λ2 ,H . The automorphism D L λ idR(H ¯ ) :W ¯ λ,H is still denoted by D L λ . By Section 5.2 we have W D Z λ (u) ? D L λ (m) = D L λ (u ? m),

¯ λ,H . ∀u ∈ Uλ , ∀m ∈ W

Given a Uλ -module M, let M [ be its contragredient module; that is, M [ = M ∗ as a vector space and (u f )(m) = f (φ∗ (u)m) for all f ∈ M [ , m ∈ M, u ∈ Uλ . The symbols ◦ and • denote the tensor product of U-modules relative to the coproduct 1◦ and 1• , respectively. To simplify, let 1W 0 denote also the map 1W 0 idR(H ¯ ) : 0 0 ¯ ¯ Wλ,H → Wλ1 λ2 ,H . 7.10 ¯0 ¯0 The map 1W 0 : W λ,H → Wλ1 λ2 ,H is invertible. The map 1U is an algebra homomorphism, and

PROPOSITION

(i) (ii)

1U (u) ? 1W 0 (m 0 ) = 1W 0 (u ? m 0 ),

∀u, m 0 . [

(iii) (iv)

The pairing identifies W0λ,H with the contragredient module Wλ,H . Set 1W = D L λ (t 1−1 W 0 )D L λ , where the transpose is relative to the pairing ( | ). We have γ 1U (u) ? 1W (m) = 1W (u ? m), ∀u, m.

(v)

The map 1W is an embedding of U-modules Wλ,H ,→ Wλ1 λ2 ,H .

Proof Claim (i) follows from the localization theorem since the fixed-point sets Q λH and Q λH1 × Q λH2 are equal. It suffices to check claim (ii) on a dense subset of spec Rλ1 λ2 . 0 = r−1 r (see Remark 7.6). Thus 1 is an algebra homoIf s is generic, then 1U U s1s2 s morphism since the map rs commutes to the convolution product. The case of 1W 0 is similar. Claim (iii) means that for any m ∈ Wλ,H , m 0 ∈ W0λ,H , u ∈ Uλ , we have (m 0 |u ? m) = (φ∗ (u) ? m 0 |m). It suffices to check the two identities below: (m 0 |u ? m) = (m 0 ? u|m)

and

m 0 ? u = φ∗ (u) ? m 0 .

These identities are standard. The first one is the associativity of the convolution product; the second one is essentially the fact that φ∗ is an antihomomorphism. Claim (iv)

524


is a direct computation: for any u, m, m 0 as above, we have 1W 0 (m 0 )|D L λ 1W (u ? m) = m 0 |D L λ (u ? m) = m 0 |D Z λ (u) ? D L λ (m) = γU (u) ? m 0 |D L λ (m) = 1W 0 (γU (u) ? m 0 )|D L λ 1W (m) = 1U γU (u) ? 1W 0 (m 0 )|D L λ 1W (m)

γ = 1W 0 (m 0 )|D Z λ 1U (u) ? D L λ 1W (m) γ = 1W 0 (m 0 )|D L λ (1U (u) ? 1W (m)) .

Since Wλ,H is a free R(H )-module, the restriction of 1W to Wλ,H is injective. The U-module Wλ,H is generated by R(H ) ⊗ [0] and 1W ([0]) = [0] [0]. Thus 1W (Wλ,H ) ⊆ Wλ1 λ2 ,H . 7.11 We can now state the main result of this paper. Using Theorem 7.4, Proposition 7.10, and the Kunneth isomorphisms ∼

θ −1 : Wλ1 λ2 ,H → Wλ1 ,H R(H ) Wλ2 ,H , ∼

θ −1 : W0λ1 λ2 ,H → W0λ1 ,H R(H ) W0λ2 ,H , we get morphisms of U|q=ζ -modules θ −1 1W : Ws → Ws 1 • Ws 2

and

θ −1 1W 0 : W0s → W0s 1 ◦ W0s 2 .

THEOREM 7.12 Assume that (ζ −1−N spec t 1 ) ∩ spec t 2 = ∅. Then the maps

θ −1 1W : Ws → Ws 1 • Ws 2

and

θ −1 1W 0 : W0s → W0s 1 ◦ W0s 2

are isomorphisms of U|q=ζ -modules. Remark 7.13 Let us first recall the following standard facts (see [16, Section 7.1], for instance). Let X be a smooth quasi-projective G-variety (G a linear group) with a finite partition into G-stable locally closed subsets X i , i ∈ I . Fix an order on I such that the subset S i 0 ≤i X i 0 ⊂ X is closed for all i. Let κ : Y ,→ X be the embedding of a smooth closed G-stable subvariety such that the intersections Yi = X i ∩ Y are the connected components of Y (in particular, Yi is smooth). Assume that there is a G-invariant G vector bundle map πi : X i → Yi for each i. Let Kr,top be the complexified equivariant


525

G (Y ) = {0}, topological K -group of degree r (r = 0, 1). Assume also that K1,top G G G K0,top (Y ) = K (Y ), and K (Y ) is a free R(G)-module. Let us consider the vector bundle Ni = (T X |Yi )/(T X i |Yi ) on Yi . For each i, fix a basis (Ei j , j ∈ Ji ) of the space KG (Yi ). Fix also an element E¯i j ∈ KG ( X¯ i ) whose restriction to X i is πi∗ Ei j . G (X ) = {0}, KG (X ) = KG (X ), and the E¯ ’s form a basis of KG (X ). Then K1,top ij V 0,top S Moreover, κ ∗ (E¯i j ) = −1 (Ni ∗ ) ⊗ Ei j modulo KG ( i 0 0, let Lsnb , b = 1, 2, be the image in Us b of the restriction of L n to Z λ . We want to prove that limn→∞ Lsn1 1

2

s s 1 2 Ls−n 2 = 0. Fix ρ , ρ such that the subset Q ρ 1 ρ 2 ⊂ Q λ1 × Q λ2 is nonempty. The set of the eigenvalues of s b acting on the bundle V | Q(ρ b ) is the spectrum of the semisimple element ρ b (s b )−1 . For any α ∈ spec ρ b (s b )−1 , let Vραb ⊆ V | Q(ρ b ) be the corre-

sponding eigensubbundle. The image of Lsn1 Ls−n 2 by rs 1 rs 2 is the restriction to 1

2

Z λs 1 × Z λs 2 of the product of XX α ⊗n β∗ ⊗n β∗ (α/β)n Vρα1 − V 0 ρ 1 Vρ 2 − V 0 ρ 2 ρ 1 ,ρ 2 α,β

by a constant that does not depend on n. Let us recall that if E is a line bundle on a smooth variety X , then the element E − 1 is a nilpotent element in the ring K(X ) (see [4, Proposition 5.9.4], for instance). Since (s 1 , s 2 ) ∈ S, we are done. Claim (ii) follows from claim (i), from the formula [2] [2] Rnρ = T(n−1)ρ (Rρ )T(n−2)ρ (Rρ ) · · · Rρ ,

and from [5, Theorem 4.4(2)], applied to the partial R-matrix Rρ . + We prove claim (iii) as in [10, Appendix B]. Fix r ∈ Z. Since R2nρ · 1◦ (xir )· −1 + + ◦ ◦ R2nρ = 12nρ (xir ) (see Section 3), the limit ìm = limn→∞ (8s 1 8s 2 )12nρ (xir ) is P + + well defined by claim (ii). Let us prove that the series xir+ 1 + s≥0 kis xi,r −s converges and coincides with this limit. We assume that r ≥ 0, the case r < 0 being + + + very similar. The element 1◦ (xir ) − xir 1 − ki xir is a linear combination of monomials of the form Q0 Qa Q b − + + E E k x x u=1 ju su u=1 h u qu u=a i u ru , Qa where E u=1 is the ordered product (the term corresponding to u = 1 is on the left), P P P P P 1 ≤ su ≤ r ≥ ru ≥ 0, u αiu − u α ju = αi , and u su + u ru + u qu = r + (see [5, Theorem 4.4(3)] and its proof in [6, Section 3.5]). Since 1◦2nρ (xir ) = [2] ◦ + + + + T2nρ 1 (xi,r +2n ), the element 1◦2nρ (xir ) − xir 1 − ki xir is a linear combination of monomials of the form Qa Q Q0 b − + + E E x k x (8.2) u=1 ju ,su +2n u=1 h u qu u=a i u ,ru −2n ,

P P P P where 0 ≤ su ≤ r + 2n ≥ ru ≥ 0, u αiu − u α ju = αi , and u su + u ru + P u qu = r + 2n. By claim (i) the image of the monomials (8.2) by 8s 1 8s 2 cannot

530


+ + − contribute to ìm unless a = 0 (use the relation kir = [xi0 , xir ] and the inequalities P P P (s + 2n) + q ≥ 2na, (r − 2n) ≤ r − 2na). Then i 0 = i. Let us consider u u u u u u the monomials Q + + [(h u ), (qu ), r0 ] = u kh u qu xir0 , P + + + + where r0 , qu ≥ 0 and r0 + u qu = r. We have 1◦ (xir ) = A(xir 1−ki0 xir )A−1 , [2] where A = T−rρ (Rrρ ). By (3.1), the partial R-matrix Rrρ is a sum of monomials in − + in the elements x−js x+j,−s with s ≥ 1. Moreover, the coefficients of xi,−r xir 0 0 − + −1 A, A are, respectively, c1 , c¯1 (because the coefficients of xi,r −r0 xi,r0 −r in Rrρ , −1 are, respectively, c , c¯ ; see Section 3). Thus the coefficients of [(h ), (q ), r ] Rrρ u u 1 1 0 + + + + in 1◦ (xir ) − xir 1 − ki0 xir and in − + − + + − + + + (c1 xi,−r x+ + c¯1 xir xi,−r0 ) xir = (q − q −1 )[xir , xi,−r0 ] xir = ki,r −r0 xir0 0 ir 0 0

coincide. We are done. A similar argument gives the equality (and the convergence of both sides) X − − − lim (8s 1 8s 2 )1◦2nρ (xir ) = 1 xir− + xi,r +s ki,−s n→∞

s≥0

for all r ∈ Z. By (6.2) the r th Fourier coefficient in + fi V xi+ (z) 1 + q λ1 −1 (q −1 − q) z −1 Fλi 1 xi+ (z) P P + + − − − is xir+ 1 + s≥0 kis xi,r s≥0 x i,r +s ki,−s is the r th −s . Similarly, 1 x ir + Fourier coefficient in − fi V xi− (z) 1 ⊗ q λ2 −1 (q −1 − q)z −1 Fλi 2 + 1 xi− (z). V Since (s 1 , s 2 ) ∈ S, the class −1 (q −1 − q)Fλi 1 L ∗ is well defined in Us 1 Us 2 . Recall that, by (7.7), the element 1U (xir± ) is the image by θ of the r th Fourier coefficient in fi V xi+ (z) 1 + q λ1 −1 (q −1 − q)Fλi 1 L ∗ ⊗ 1 xi+ (z) , fi V xi− (z) 1 ⊗ q λ2 −1 (q −1 − q)(−L )∗ Fλi 2 + 1 xi− (z). Since L xi± (z) = ±zxi± (z) by (6.2), we get ± lim θ(8s 1 8s 2 )1◦2nρ (xir ) = 1U (xir± ).

n→∞

(8.3)

± We are done because the algebra U is generated by the elements xir . 0 0 0 ¯ →W ¯ 1 2 is the bivariant localization Let us prove claim (iv). The map 1W 0 : W λ λ λ map (associated to the embedding (Q λ1 × Q λ2 ) × pt ,→ Q λ × pt). In particular, it


531

¯ 0 1 2 can be viewed as a Uis invertible. The map 1W 0 is invertible also. Thus W λ λ module in two different ways: via 1W 0 or via the coproduct 1◦ and the Kunneth ¯ 01 2 ' W ¯ 0 1 ¯ ¯ 0 2 . These representations of U are denoted isomorphism W W λ λ

λ

λ

Rλ1 λ2

¯ 0 1 2 and ◦ W ¯ 0 1 2 . Both U-modules are finite-dimensional R ¯ λ1 λ2 -vector spaces. by W λ λ λ λ They are simple by [16, Theorem 14.1.2] since the tensor product of two generic simple finite-dimensional U-modules is still simple. They also have the same Drinfeld polynomials (see Section 6.6). Thus they are isomorphic. Obviously, HomR¯

λ1 λ2

¯ 0 1 2 , W ¯ 0 1 2 ) = End ¯ (◦ W R λ λ λ λ

λ1 λ2

¯ 0 1 2 ). (W λ λ

Let Hλb ⊂ G λb be a maximal torus, b = 1, 2. Put H = Hλ1 × Hλ2 × C× . Then ×

×

(Z λb ) Hλb ×C = (Q λb × Q λb ) Hλb ×C . The localization theorem and the Kunneth formula give an isomorphism ¯ Hλb ×C× (Z λb ) ' End ¯ ¯ Hλb ×C× (Q λb ) . K R(H b ×C× ) K λ

Moreover, [4, Theorem 6.1.22] gives ¯ ) ¯ R(H R

λ1 λ2

¯ ) ¯ R(H R

λ1 λ2

¯ 01 2 ' K ¯ Hλ1 ×C× (Q λ1 ) K ¯ Hλ2 ×C× (Q λ2 ), W λ λ ¯ Hλ1 ×C× (Z λ1 ) K ¯ Hλ2 ×C× (Z λ2 ). e Uλ1 λ2 ' K

Thus ¯ ) ¯ R(H R

λ1 λ2

HomR¯

λ1 λ2

¯ 0 1 2 , W ¯ 0 1 2 ) ' R(H ¯ ) ¯ (◦ W R λ λ λ λ

λ1 λ2

e Uλ1 λ2 .

Taking the invariants by the Weyl group of G λ1 × G λ2 , we get an isomorphism HomR¯

λ1 λ2

¯ 0 1 2 , W ¯ 0 1 2) ' e (◦ W Uλ1 λ2 . λ λ λ λ

e ∈ e Let R Uλ1 λ2 be the element corresponding to the isomorphism of U-modules ∼ 0 0 ◦W ¯ 1 2 → ¯ 1 2 which is the identity on the highest weight vectors. Claim (iv) W λ λ λ λ es 1 s 2 intertwines the specialized modules whenever it follows from claim (iii) since R is defined and invertible. − Claim (v) is immediate since R2nρ is a sum of monomials in the elements xir + xi,−r , i ∈ I , r ≥ 1. Appendix Let us check that the operators introduced in Section 6 still satisfy the Drinfeld relations. As indicated in Remark 6.3, it is sufficient to check [16, (1.2.8) and (1.2.10)]. By [16, Section 10.2], the proof of the first relation is reduced to the equality (q −1 Vl 2 /Vl 3 )

f l− 4

α λ

⊗ (q −1 Vk3 /Vk4 )

= (q −1 Vl 2 /Vl 3 )

f l− 3

α λ

f k+ 3

α λ

⊗ xαl−4 λ ⊗ xαk+ 3λ

⊗ (q −1 Vk3 /Vk4 )

f k+ 2

α λ

l− ⊗ xαk+ 2 λ ⊗ xα3 λ ,

(A.1)

532


where α 2 , α 3 , α 4 ∈ Q + are such that α 4 = α 3 − αk , α 2 = α 3 + αl , and k 6= l. Then (A.1) follows from −

−

xαl−4 λ = (−1)nlk xαl−3 λ ⊗ (q −1 Vk3 /Vk4 )nlk , n+ kl

xαk+ 2 λ = (−1)

−n + kl

−1 V 2 /V 3 ) xαk+ 3 λ ⊗ (q l l

l− − f αl− 4 λ = f α 3 λ − n lk ,

k+ + f αk+ 2 λ = f α 3 λ + n kl ,

,

− and the identity n + kl = n lk . By [16, Section 10.3], the proof of the second relation is reduced to the equality

(q −1 Vl 3 /Vl 2 )akl ⊗ (q −1 Vk3 /Vk4 )

f k+ 2

α λ

⊗ (q −1 Vl 3 /Vl 2 )

= (−1)akl (q −1 Vk3 /Vk4 )akl ⊗ (q −1 Vl 3 /Vl 2 )

f l+ 4

α λ

f l+ 3

α λ

l+ ⊗ xαk+ 2 λ ⊗ xα3 λ

⊗ (q −1 Vk3 /Vk4 )

f k+ 3

α λ

⊗ xαl+4 λ ⊗ xαk+ 3λ, (A.2)

where α 2 , α 3 , α 4 ∈ Q + are such that α 2 = α 3 − αl , α 4 = α 3 − αk , and k 6 = l. Then (A.2) follows from +

+

n+ kl

n+ kl

xαl+4 λ = (−1)nlk xαl+3 λ ⊗ (q −1 Vk3 /Vk4 )nlk , xαk+ 2 λ = (−1)

−1 V 3 /V 2 ) xαk+ 3 λ ⊗ (q l l

,

l+ + f αl+ 4 λ = f α 3 λ − n lk ,

k+ + f αk+ 2 λ = f α 3 λ − n kl ,

+ and the identity n + kl + n lk = −alk .

Acknowledgment. We would like to thank the referee for numerous remarks on the first version of the paper. Note. While we were preparing this paper, M. Kashiwara mentioned to us that he has proved the conjecture in [1] by a different approach (via canonical bases). References [1]

[2] [3]

[4] [5]

T. AKASAKA and M. KASHIWARA, Finite-dimensional representations of quantum

affine algebras, Publ. Res. Inst. Math. Sci. 33 (1997), 839–867. MR 99d:17017 509, 527, 532 J. BECK, Braid group action and quantum affine algebras, Comm. Math. Phys. 165 (1994), 555–568. MR 95i:17011 510, 511 V. CHARI and A. PRESSLEY, Weyl modules for classical and quantum affine algebras, Represent. Theory 5 (2001), 191–223, http://ams.org/ert/home-2001.html MR 1 850 556 528 N. CHRISS and V. GINZBURG, Representation Theory and Complex Geometry, Birkhäuser, Boston, 1997. MR 98:22021 515, 516, 518, 519, 520, 521, 529, 531 I. DAMIANI, La R-matrice pour les algèbres quantiques de type affine non tordu, Ann. ´ Sci. Ecole Norm. Sup. (4) 31 (1998), 493–523. MR 99g:17027 529


[6]

[7]

[8] [9] [10]

[11] [12] [13] [14] [15] [16]

[17] [18]

533

I. DAMIANI and C. DE CONCINI, “Quantum groups and Poisson groups” in

Representations of Lie Groups and Quantum Groups (Trento, Italy, 1993), Pitman Res. Notes Math. Ser. 311, Longman Sci. Tech., Harlow, England, 1994, 1– 45. MR 97m:16070 529 V. GINZBURG, N. RESHETIKHIN, and E. VASSEROT, “Quantum groups and flag varieties” in Mathematical Aspects of Conformal and Topological Field Theories and Quantum Groups (South Hadley, Mass., 1992), Contemp. Math. 175 (1994), 101–130. MR 95j:17014 V. GINZBURG and E. VASSEROT, Langlands reciprocity for affine quantum groups of type An , Internat. Math. Res. Notices 1993, 67–85. MR 94j:17011 509 M. KASHIWARA, Crystal bases of modified quantized enveloping algebra, Duke Math. J. 73 (1994), 383–413. MR 95c:17024 528 S. M. KHOROSHKIN and V. N. TOLSTOY, “Twisting of quantum (super-)algebras” in Generalized Symmetries in Physics (Clausthal, Germany, 1993), World Sci., River Edge, N.J., 1994, 42–54. MR 98j:81157 529 S. Z. LEVENDORSKI˘I and YA. S. SOIBELMAN, Some applications of the quantum Weyl groups, J. Geom. Phys. 7 (1990), 241–254. MR 92g:17016 G. LUSZTIG, Introduction to Quantum Groups, Birkhäuser, Boston, 1994. MR 94m:17016 511 , Bases in equivariant K-theory, Represent. Theory 2 (1998), 298–369, http://ams.org/ert/home-1998.html MR 99i:19005 515 H. NAKAJIMA, Instantons on ALE spaces, quiver varieties, and Kac-Moody algebras, Duke Math. J. 76 (1994), 365–416. MR 95i:53051 509 , Quiver varieties and Kac-Moody algebras, Duke Math. J. 91 (1998), 515–560. MR 99b:17033 509, 514 , Quiver varieties and finite dimensional representations of quantum affine algebras, J. Amer. Math. Soc. 14 (2001), 145–238, http://ams.org/jams/2001-14-01 MR 1 808 477 509, 512, 513, 517, 518, 522, 524, 526, 527, 531, 532 M. VARAGNOLO and E. VASSEROT, On the K -theory of the cyclic quiver variety, Internat. Math. Res. Notices 1999, 1005–1028. MR 2000m:14011 509 E. VASSEROT, Affine quantum groups and equivariant K -theory, Transform. Groups 3 (1998), 269–299. MR 99j:19007 509

Varagnolo Département de Mathématiques, Université de Cergy-Pontoise, 2 avenue A. Chauvin, 95302 Cergy-Pontoise CEDEX, France; [email protected] Vasserot Département de Mathématiques, Université de Cergy-Pontoise, 2 avenue A. Chauvin, 95302 Cergy-Pontoise CEDEX, France; [email protected]


A DARBOUX THEOREM FOR HAMILTONIAN OPERATORS IN THE FORMAL CALCULUS OF VARIATIONS EZRA GETZLER

To Roger Richardson, in memoriam

Abstract We prove a formal Darboux-type theorem for Hamiltonian operators of hydrodynamic type, which arise as dispersionless limits of the Hamiltonian operators in the KdV and similar hierarchies. We prove that the Schouten Lie algebra is a formal differential graded Lie algebra, which allows us to obtain an analogue of the Darboux normal form in this context. We include an exposition of the formal deformation theory of differential graded Lie algebras g concentrated in degrees [−1, ∞); the formal deformations of g are parametrized by a 2-groupoid that we call the Deligne 2-groupoid of g, and quasiisomorphic differential graded Lie algebras have equivalent Deligne 2-groupoids. The Darboux theorem states that all symplectic structures on an affine space are locally isomorphic. Hamiltonian operators are a generalization of symplectic forms introduced by I. Gelfand and I. Dorfman [3] and are important in the study of integrable hierarchies such as the Korteweg–de Vries (KdV) and Kodomtsev-Petviashvili (KP) equations. It is natural to ask whether an analogue of the Darboux theorem holds for Hamiltonian operators. The problem is considerably simplified by restricting attention to formal deformations of a given Hamiltonian operator H . The study of the moduli space of deformations is then controlled by a differential graded (dg) Lie algebra, the Schouten Lie algebra, with differential ad(H ). The study of formal deformations is closely related to the problem of calculating the cohomology of this dg Lie algebra, which was posed by P. Olver [9]. DUKE MATHEMATICAL JOURNAL c 2002 Vol. 111, No. 3, Received 11 August 2000. Revision received 4 March 2001. 2000 Mathematics Subject Classification. Primary 32G34, 35Q53; Secondary 18D05, 55P62. Author’s work supported by the “Geometry Related to String Theory” project of the Research Institute for Mathematical Sciences and by National Science Foundation grant number DMS-9704320. 535

536

EZRA GETZLER

Let V be a finite-dimensional vector space with basis ea , and let η = ηab ea ⊗ eb be a nondegenerate bilinear form on V ∗ . In this paper we answer Olver’s question for the Hamiltonian operator of hydrodynamic type Hη = ηab ∂. We prove in Theorem 5.3 that the dg Lie algebra gη associated to Hη is formal; that is, it is homotopy equivalent to a dg Lie algebra with vanishing differential. (The proof consists of a series of reductions followed by an application of the Poincaré lemma in infinite dimensions.) The set of possible normal forms of a formal deformation of Hη is easy to calculate with this result in hand. B. Dubrovin and S. Novikov [2] showed that an operator of the form c ab ηab ∂ + Aab c t +B c ∗ is Hamiltonian if and only if [ea , eb ] = Aab c e is a Lie bracket on V with the metric ab η as a Killing form, and B ea ∧ eb is a two-cocycle for this Lie bracket; we show that these are precisely the normal forms that occur as deformations of H . All of our results are formulated in a global setting; the vector space V is replaced by a complex manifold X with flat contravariant metric η. (A contravariant metric is a nondegenerate symmetric bilinear form on the cotangent bundle.) We use a refined version of Gelfand and Dorfman’s generalization of the Schouten bracket to the formal calculus of variations introduced by V. Soloviev [10], which we explain in Section 4; his remarkable result, that the structure sheaf of the jet-space J∞ (X ) of a Poisson manifold carries a canonical Lie bracket, deserves to be more widely appreciated. In the first three sections, we give an exposition of the formal deformation theory of differential graded Lie algebras g concentrated in degrees [−1, ∞); the formal deformations of g are parametrized by a 2-groupoid that we call the Deligne 2groupoid of g, and quasi-isomorphic differential graded Lie algebras have equivalent Deligne 2-groupoids. Throughout this paper the summation convention is understood: indices a, b, . . . occurring once as superscript and once as subscript in a formula are to be summed over.

1. Poisson tensors on supermanifolds and the Schouten bracket In this section we recall the elements of the theory of Poisson supermanifolds. This theory differs a little from that of Poisson manifolds since the Poisson tensor on a supermanifold may have either even or odd parity. 1.1. Poisson tensors on supermanifolds Let Cm|n be the superspace with m even and n odd coordinates; if U is an open subset of Cm|n and |U | is the underlying open subset of Cm , we have O (U ) ∼ = O (|U |) ⊗ 3(Cn )∗ .

A DARBOUX THEOREM IN THE FORMAL CALCULUS OF VARIATIONS

537

Definition 1.1 A ν-Poisson tensor on U is a 2-tensor P = P ab ∂a ⊗ ∂b ∈ O (U ) ⊗ (Cm|n )⊗2 of total degree ν ∈ Z/2 (i.e., |P ab | = |a|+|b|+ν), such that P ba +(−1)|a||b|+ν P ab = 0 and X (−1)|b|(|a|+|c|+ν) P ba ∂a P cd = 0. (1.1) cycles in (b,c,d)

A ν-Poisson tensor P on U defines a Poisson bracket on O (U ) by the formula {u, v} = −(−1)(|b|+ν)|u| P ab ∂a u ∂b v. The symmetry of P is equivalent to skew-symmetry of the bracket {u, v} + (−1)(|u|+ν)(|v|+ν) {v, u} = 0, while (1.1) is equivalent to the Jacobi rule u, {v, w} − (−1)(|u|+ν)(|v|+ν) v, {u, w} = {u, v}, w . We conclude that the space of ν-Poisson tensors is invariant under change of coordinates; thus, we may define a ν-Poisson tensor on a complex supermanifold as a tensor that is a ν-Poisson tensor for some atlas. Definition 1.2 A holomorphic ν-Poisson supermanifold (X, P) is a complex supermanifold X together with a ν-Poisson tensor P on X . If the Poisson tensor is nondegenerate, we call (X, P) a holomorphic ν-symplectic supermanifold. 1.2. The Schouten bracket and supermanifolds If X is a manifold, let X be the 1-symplectic supermanifold obtained by forming the cotangent bundle T ∗X over X , which is a symplectic manifold; applying the functor 5, which reverses the parity of the fibres; and taking the underlying supermanifold. Let π : X → X be the projection. Let t a , 1 ≤ a ≤ m, be coordinates on an open subset of X , and let θa be the dual coordinates along the fibres of X ; let ∂a = ∂/∂t a and ∂ a = ∂/∂θa be the corresponding vector fields. The Poisson tensor (or, more accurately, the 1-Poisson tensor) of X is given by the formula P = ∂ a ⊗ ∂a + ∂a ⊗ ∂ a .

(1.2)

538

EZRA GETZLER

The sheaf π∗ OX is isomorphic to the graded sheaf 3 = 3TX of multivectors on X , and this isomorphism identifies the Poisson bracket on X with the Schouten bracket [−, −] on 3. The Poisson bracket on the Z/2-graded sheaf OX has odd degree, while we prefer to work with a Z-grading on 3 such that the Schouten bracket has degree zero. To this end, we define the degree p summand 3 p of 3 to be 3 p+1 TX . Taking this shift of degree into account, the formula for the Schouten bracket becomes [u, v] = (−1)|u| ∂a u ∂ a v − ∂ a u ∂a v. If Q ∈ 0(X, 31 ) = 0(X, 32 TX ), define an operation {u, v} Q on O by the formula {u, v} Q = [Q, u], v . In local coordinates Q = (1/2)Q ab θa θb , we have [Q, u] = Q ab θa ∂b u, and hence {u, v} Q = Q ab ∂a u ∂b u. If Q is a Poisson tensor, this is the Poisson bracket associated to Q. 1.1 The following conditions on a section Q of 31 = 32 TX are equivalent: (1) Q is a Poisson tensor on X ; (2) [Q, Q] = 0; (3) the operation δ Q = [Q, −] is a differential on the sheaf of graded Lie algebras 3; (4) the operation {u, v} Q = [δ Q u, v] on O is a Lie bracket. PROPOSITION

Proof In local coordinates, the formula [Q, Q] = 0 becomes equation (1.1) for the tensor Q on X ; thus, (1) and (2) are equivalent. The Jacobi rule for graded Lie algebras shows that 1 δ Q δ Q a = Q, [Q, a] = [Q, Q], a . 2 Thus, δ Q is a differential on 3 if and only if [Q, Q] = 0. The bracket {u, v} Q is skew-symmetric: {u, v} Q + {v, u} Q = [δ Q u, v] + [δ Q v, u] = δ Q [u, v] = 0. As for the Jacobi rule, we have u, {v, w} − v, {u, w} = δ Q u, [δ Q v, w] − δ Q v, [δ Q u, w] = [δ Q u, δ Q v], w = {u, v}, w − [δ Q δ Q u, v], w .


539

The anomalous term −(1/2)[[δ Q δ Q u, v], w] vanishes for all u, v, and w ∈ 0(U, O ) if and only if δ Q is a differential. Let (X, Q) be a Poisson manifold. We denote the sheaf of dg Lie algebras 3, with differential δ Q , by 3 Q . For example, if (X, Q) is a symplectic manifold, then the complex of sheaves underlying 3 Q is isomorphic to the de Rham complex, and the Poisson cohomology is isomorphic to the trivial sheaf C, with vanishing Lie bracket. 2. Graded Lie algebras and the Deligne 2-groupoid W. Goldman and J. Millson [4] have developed an approach to deformation theory based on a functor from nilpotent dg Lie algebras concentrated in degrees [0, ∞) to groupoids C (g), called the Deligne groupoid. The dg Lie algebra controlling the deformation theory of Poisson brackets is the Schouten Lie algebra, which is concentrated in degrees [−1, ∞); thus, the theory of the Deligne groupoid does not apply. It turns out that the deformation theory is best understood by means of a 2-groupoid, whose definition generalizes that of the Deligne groupoid.∗ In this section, all dg Lie algebras g are concentrated in degrees [−1, ∞). 2.1. The Deligne groupoid We now recall the definition of the Deligne groupoid. There is a sequence of elements Fn (x, y) of degree n in the free Lie algebra on two generators x and y such that if X and Y are elements of a nilpotent Lie algebra g of N steps, we have exp(X ) exp(Y ) = exp

N X

Fn (X, Y )

n=1

in the associated simply connected Lie group G; for example, F1 (X, Y ) = X + Y and F2 (X, Y ) = (1/2)[X, Y ]. We may identify the Lie group G with the manifold g with deformed product N X X ·Y = Fn (X, Y ). n=1

Denote the resulting functor from nilpotent Lie algebras to Lie groups by exp(g). Definition 2.1 If g is a dg Lie algebra, the set MC(g) of Maurer-Cartan elements of g is the inverse ∗ We

have learned that this 2-groupoid was proposed by P. Deligne in a letter to L. Breen (February 1994); it is also alluded to in M. Kontsevich [5, Section 3.3].

540

EZRA GETZLER

image Q −1 (0) ⊂ g1 of the quadratic map Q : g1 → g2 defined by the formula Q(A) = d A + (1/2)[A, A]. Thus, A is a Maurer-Cartan element if and only if the operator d A u = du + [A, u] is a differential on g. The subspace g0 of g is a nilpotent Lie algebra, and the group exp(g0 ) acts on g1 by the formula ∞ X ad(ξ )n (d A ξ ) exp(ξ ) · A = A − ; (2.1) (n + 1)! n=0

this is the affine action that corresponds to gauge transformations dexp(ξ )·A = Ad exp(ξ ) d A . Since Q(exp(ξ ) · A) = Q(A), this action preserves the subset MC(g) ⊂ g1 . Definition 2.2 The Deligne groupoid C (g) of g is the groupoid associated to the group action exp(g0 ) × MC(g) → MC(g). The sets of objects and morphisms of the Deligne groupoid are MC(g) and exp(g0 ) × MC(g); its source and target maps are s(exp(ξ ), A) = A and t (exp(ξ ), A) = exp(ξ ) · A, its identity is A 7→ (exp(0), A), and its composition is exp(η), exp(ξ ) · A · exp(ξ ), A = exp(η) exp(ξ ), A . The Deligne groupoid is a natural generalization of the Lie correspondence exp between nilpotent Lie algebras and simply connected nilpotent Lie groups, which is the case where g is concentrated in degree zero. Even if g is not nilpotent, we may consider the Deligne groupoid with coefficients in g ⊗ m, where m is a nilpotent commutative algebra. If G is a groupoid, let π0 (G ) be the set obtained by quotienting of the set of objects of G by the equivalence relation x ∼ y whenever there is a morphism between x and y. If g is a nilpotent dg Lie algebra, we write M (g) for π0 (C (g)). Much of deformation theory may be reformulated as the study of the sets M (g ⊗ m). The following result is proved by exactly the same method as in Goldman and Milsson [4, Theorem 2.4]. THEOREM 2.1 Let h = F 1 h ⊃ F 2 h ⊃ · · · and h˜ = F 1 h˜ ⊃ F 2 h˜ ⊃ · · · be filtered dg Lie algebras ˜ such that F N h and (i.e., d F i h ⊂ F i h and [F i h, F j h] ⊂ F i+ j h, and likewise for h)


541

F N h˜ vanish for sufficiently large N , and let f : h → h˜ be a morphism of filtered dg Lie algebras that induces weak equivalences of the associated chain complexes ˜ i+1 h. ˜ gri f : F i h/F i+1 h −→ F i h/F ˜ Then f induces a bijection M ( f ) : M (h) → M (h). 2.2. 2-groupoids The category of groupoids is a monoidal category, where G ⊗H is the product G ×H of the groupoids G and H . Definition 2.3 A 2-groupoid is a groupoid enriched over the monoidal category of groupoids. We see that a 2-groupoid G has a set G 0 of objects and, for each pair of objects x, y ∈ G 0 , a groupoid of morphisms G (x, y), and we see that there are product maps G (x, y) × G (y, x) −→ G (x, z)

(2.2)

satisfying the usual conditions of associativity for a category. The 2-morphisms of a 2-groupoid are the morphisms of the groupoids G (x, y). There are two compositions defined on the 2-morphisms: the horizontal composition of (2.2) and the vertical composition, which is composition inside the groupoid G (x, y). If G is a 2-groupoid, let π1 (G ) be the groupoid whose objects are those of G , and such that the set of morphisms π1 (G )(x, y) equals π0 (G (x, y)). Let π0 (G ) equal π0 (π1 (G )). If x is an object of G , let π1 (G , x) be the automorphism group π1 (G )(x, x), and let π2 (G , x) be the automorphism group of the identity of x in the groupoid G (x, x). The group π2 (G , x) is abelian for the same reason that π2 (X, x) is abelian for a topological space X : it carries two products, horizontal and vertical, satisfying (a ◦h b) ◦v (c ◦h d) = (a ◦v c) ◦h (b ◦v d). Definition 2.4 A weak equivalence ϕ : G → H of 2-groupoids is a homomorphism such that π0 (ϕ) is an isomorphism of sets and, for each object x ∈ G 0 , πi (ϕ, x) : πi (G , x) → πi H , ϕ(x) is an isomorphism of groups for all x ∈ G 0 and i = 1, 2.

542

EZRA GETZLER

With this notion of weak equivalence and suitable notions of cofibration and fibration, the category of 2-groupoids is a closed model category (see I. Moerdijk and J.-A. Svensson [7]). 2.3. The Deligne 2-groupoid We now show how the Deligne groupoid of a nilpotent dg Lie algebra g is the underlying groupoid of a 2-groupoid, which we denote by C (g); if g happens to vanish in degree −1, this 2-groupoid is identical to the Deligne groupoid of G . (Thus, the use of the same notation for the Deligne 2-groupoid and the Deligne groupoid should cause no difficulty.) Given an element A ∈ g1 , define a bracket {u, v} A on g−1 by the formula {u, v} A = [d A u, v].

(2.3)

The proof of the following proposition is the same as the proof of the equivalence of conditions (3) and (4) in Proposition 1.1. PROPOSITION 2.1 The bracket {u, v} A makes g−1 into a Lie algebra if and only if A ∈ MC(g).

If A ∈ MC(g), we denote the Lie algebra g−1 with bracket {u, v} A by g A . The nilpotence of g implies that g A is nilpotent. If u ∈ g A ∼ = g−1 , denote the corresponding element of the group exp(g A ) by exp A (u). Since d A {u, v} A = [d A u, d A v], the linear map d A : g A → g0 is a morphism of Lie algebras. Thus, the group exp(g A ) acts on exp(g0 ) by right translation: exp A (u) · exp(ξ ) = exp(ξ ) exp(d A u). Given a pair A, B of elements of MC(g), define C (g)(A, B) to be the groupoid associated to this group action. The set of 2-morphisms of C (g, m) may be identified with g−1 × g0 × MC(g); we denote its elements by (exp A (u), exp(ξ ), A), where u ∈ g−1 , ξ ∈ g0 , and A ∈ MC(g). The internal (or vertical) composition of 2-morphisms is given by the formula exp A (v), exp(ξ ) exp(d A u), A ◦v exp A (u), exp(ξ ), A = exp A (u) exp A (v), exp(ξ ), A . To complete the definition of the Deligne 2-groupoid, it remains to define the horizontal composition C (g)(B, C) × C (g)(A, B) −→ C (g)(A, C).

(2.4)


543

Given ξ ∈ g0 and A ∈ MC(g), there is an isomorphism of Lie algebras ead(ξ ) : gexp(ξ )·A −→ g A with inverse e− ad(ξ ) . Suppose that A ∈ MC(g) and ξ, η ∈ g0 , with B = exp(ξ ) · A and C = exp(η) · B. Then the horizontal composition of 2-morphisms is given by the formula expexp(ξ )·A (v), exp(η), exp(ξ ) · A ◦h exp A (u), exp(ξ ), A = exp A (e− ad(ξ ) v) exp A (u), exp(η) exp(ξ ), A . If g is a nilpotent dg Lie algebra and A ∈ MC(g), we write π1 (g, A) and π2 (g, A) for π1 (C (g), A) and π2 (C (g), A). THEOREM 2.2 Let g and g˜ be dg Lie algebras concentrated in degrees [−1, ∞), and let m be a nilpotent commutative algebra. A weak equivalence f : g → g˜ of dg Lie algebras induces a weak equivalence of 2-groupoids C ( f ⊗ m) : C (g ⊗ m) → C (˜g ⊗ m).

Proof By Theorem 2.1, a weak equivalence f : g → g˜ of dg Lie algebras induces a bijection M ( f ⊗ m) : M (g ⊗ m) → M (˜g ⊗ m); indeed, g ⊗ m is filtered by subspaces F i g ⊗ m = g ⊗ mi , and the same is true for g˜ . It remains to prove that f induces bijections πi (g ⊗ m, A) ∼ = πi (˜g ⊗ m, f (A)) for all A ∈ MC(g ⊗ m) and i ∈ {1, 2}. Given A ∈ MC(g ⊗ m), define a dg Lie algebra dA A (g ⊗ m) = 0 −→ (g ⊗ m) A −−→ ker(d A |g0 ⊗ m) −→ 0 , where (g ⊗ m) A is placed in degree zero. The construction A (g ⊗ m) behaves like a based loop space of g ⊗ m at A, in the sense that C (g ⊗ m)(A, A) ∼ (2.5) = C A (g ⊗ m) . To prove this we must first show that these groupoids have the same objects, that is, that exp(ξ ) · A = A if and only if d A ξ = 0. If exp(ξ ) · A = A, we see from (2.1) that dAξ = −

∞ X ad(ξ )n (d A ξ ) n=1

(n + 1)!

.

It follows, by induction on n, that d A ξ ∈ g1 ⊗ mn for all n > 0; hence d A ξ = 0. The remainder of the proof of (2.5) is straightforward.

544

EZRA GETZLER

That π1 ( f ⊗ m, A) : π1 (g ⊗ m, A) → π1 (˜g ⊗ m, f (A)) is a bijection now follows on applying Theorem 2.1 to the weak equivalence of filtered dg Lie algebras A ( f ⊗ m) : A (g ⊗ m) → f (A) (˜g ⊗ m). Finally, π2 ( f ⊗ m, A) : π2 (g ⊗ m, A) −→ π2 (˜g ⊗ m, f (A)) is a bijection since π2 (g ⊗ m, A) ∼ = H −1 (g ⊗ m, d A ). If g is a dg Lie algebra, its cohomology H (g) is a dg Lie algebra with vanishing differential. Definition 2.5 A dg Lie algebra g is formal if there exists a dg Lie algebra g˜ and weak equivalences of dg Lie algebras g˜ → g and g˜ → H (g). If g is formal, Theorem 2.2 implies that the 2-groupoids C (g, m) and C (H (g), m) are equivalent and hence that the 2-groupoid C (H (g), m) parametrizes normal forms for deformations of the differential on g. This motivates the following definition. Definition 2.6 A deformation problem is Darboux if it is controlled by a formal dg Lie algebra g. 3. Examples of Deligne 2-groupoids We now illustrate the Deligne 2-groupoid in two examples: the deformation theory of Poisson tensors, and a graded Lie algebra that occurs in the deformation theory of Hamiltonian operators of hydrodynamic type. 3.1. Deformation of Poisson tensors Let (X, Q) be a Poisson manifold, and let g be the dg Lie algebra g = 0(X, 3) with differential d Q . Given an integer n, let mn be the nilpotent ring ~ C[~]/(~n+1 ). The MaurerCartan elements of g ⊗ mn are the nth-order deformations Q=Q+

n X

~k Q k + O(~n+1 ),

Q k ∈ g1 ,

k=1

of the Poisson tensor such that [Q, Q] = O(~n+1 ). The Lie algebra g0 ⊗ mn may be identified with the Lie algebra of formal vector fields X =

n X k=1

~k ξk + O(~n+1 ),

ξk ∈ 0(X, 31 ),


545

and exp(g0 ⊗ mn ) with the group of formal diffeomorphisms; thus, M (g ⊗ mn ) is the set of equivalence classes of nth-order deformations Q of the Poisson bracket Q modulo formal diffeomorphisms. For i = 1, 2, we have πi (g ⊗ mn , Q) ∼ = exp H 1−i 0(X, 3) ⊗ mn , δQ ; in particular, π2 (g ⊗ mn , Q) is the space of Casimirs of Q. The deformation theory of an affine symplectic manifold (V, Q) is Darboux in the sense of Definition 2.6: its controlling dg Lie algebra 0(V, 3 Q ) has cohomology C[1] and hence is formal. In this way we recover a formal version of the usual Darboux theorem. From this example we see how powerful formality is: it allows the calculation of the homotopy type of the Deligne 2-groupoid (in this case, K (C, 2)) in a straightforward way. 3.2. A Deligne 2-groupoid associated to a Euclidean vector space We now consider the Deligne 2-groupoids of a class of graded Lie algebras associated to Euclidean vector spaces. If (V, η) is a Euclidean vector space, the odd superspace 5V ∗ is symplectic (i.e., zero-symplectic). If t a is a coordinate system on V (that is, a basis of V ∗ ) and θa is the dual coordinate system on 5V ∗ , the symplectic form on 5V ∗ equals ω = ηab dθa dθb . The Lie algebra h(V, η) of Hamiltonian vector fields on 5V ∗ is a Z-graded Lie algebra: the Poisson bracket has degree −2 (with respect to the degree in the generators θa of O5V ∗ = 3V ), so the Z-grading is defined by assigning to a Hamiltonian vector field its degree of homogeneity minus 1. Equivalently, this equals the degree of homogeneity of the corresponding Hamiltonian minus 2; thus, ( 3 p+2 V, p ≥ −1, h p (V, η) ∼ = 0, p < −1. Using the Hamiltonians to represent the corresponding Hamiltonian vector fields, the bracket of elements α ∈ h p (V, η) and β ∈ hq (V, η) is {α, β} = (−1) p+1 ηab ∂ a α ∂ b β. The graded vector space O [1] is a graded module for the graded Lie algebra h(V, η), with O [1] p ∼ = 3 p+1 V ; the action of α ∈ h p (V, η) on β˜ ∈ O [1] is given by the formula ˜ α · β = −ηab ∂ a α ∂ b β. The sign is explained by the fact that we consider the module O [1] and not O .

546

EZRA GETZLER

Let g(V, η) = O [1] o h(V, η) be the semidirect product of h(V, η) with the abelian graded Lie algebra O [1]; thus, g p (V, η) is isomorphic to 3 p+1 V ⊕ 3 p+2 V . We denote elements of g p (V, η) by (α, ˜ α), where α˜ ∈ O [1] p ∼ = 3 p+1 V and α ∈ h p (V, η) ∼ = 3 p+2 V . The Lie subalgebra g0 (V, η) ⊂ g(V, η) is isomorphic to iso(V, η), the Lie algebra of infinitesimal Euclidean transformations of V ; g(V, η) is an analogue of iso(V, η) in the graded world. Let m be a nilpotent commutative algebra. An element (α, ˜ α) of g1 (V, η) ⊗ m ∼ = (33 V ⊕ 32 V ) ⊗ m gives rise to a skew-symmetric operation on (C ⊕ V ∗ ) ⊗ m by the formula (a, x), (b, y) (α,α) = [(α, ˜ α), (a, x)], (b, y) . ˜ By Proposition 2.1, this is a Lie bracket if and only if (α, ˜ α) is a Maurer-Cartan element of g(V, η) ⊗ m. The homotopy group π2 (g(V, η), (α, ˜ α)) is the centre of the Lie algebra (g(V, η) ⊗ m)(α,α) . ˜ Given (α, ˜ α) ∈ MC(g(V, η) ⊗ m), the Lie algebra (g(V, η) ⊗ m)(α,α) is naturally ˜ isomorphic to the central extension of the Lie algebra (V ∗ ⊕ C) ⊗ m with bracket [x, y]α = [(0, α), (0, x)], (0, y) associated to the 2-cocyle α. ˜ This proves the following result. THEOREM 3.1 Let m be a commutative ring. The Maurer-Cartan elements of g ⊗ m are elements (α, ˜ α) ∈ (32 V ⊕ 33 V ) ⊗ m such that the bilinear operation [−, −]α on V ∗ ⊗ m defined by α is a Lie bracket, and α˜ is a 2-cocycle on the Lie algebra (V ∗ ⊗ m, [−, −]α ).

The inhomogeneous Euclidean group exp(iso(V, η) ⊗ m) is the semidirect product of the homogenous Euclidean group exp(so(V, η) ⊗ m) and the translation group V ⊗ m. The group exp(so(V, η) ⊗ m) acts on MC(g(V, η) ⊗ m) through its adjoint action on V ∗ ⊗ m, while V ⊗ m acts on MC(g(V, η) ⊗ m) by shifting the 2-cocycle α: ˜ if v ∈ V ⊗ m, v · (α, ˜ α) = α˜ + v(α), α , where v(α)(x, y) = v([x, y]α ). The quotient of MC(g(V, η) ⊗ m) by this group action is M (g(V, η) ⊗ m). The group π1 (g(V, η) ⊗ m, (α, ˜ α)) is the quotient of the subgroup of exp(iso(V, η)⊗m) consisting of automorphisms of the Lie algebra (g(V, η)⊗m)(α,α) ˜ by inner automorphisms.


547

4. Soloviev’s Lie bracket in the formal calculus of variations Let P be a Poisson tensor on an affine space V . Soloviev [10] has constructed a Lie bracket on the infinite jet space of V which prolongs the Poisson bracket of V . In this section we generalize Soloviev’s construction to Poisson supermanifolds. The main application we have in mind is to the 1-symplectic supermanifold X associated to a manifold X , whose Poisson algebra is the Schouten algebra of X . This case is far simpler than the general theory, and we have taken advantage of this at certain places in our exposition, where the general theory becomes a little complicated. However, just as in the case of Poisson manifolds, the general case may be reduced to the case X . 4.1. Higher Euler operators on supermanifolds Let Cm|n be a superspace with coordinates t a . Let |a| = |t a | equal 0 or 1 depending on whether t a is even or odd. If U is an open subset of Cm|n , let O (U ) be its (graded) ring of holomorphic functions. Let ∂a be the derivation ∂/∂t a : O (U ) → O (U ). Let O∞ (U ) be the graded commutative algebra O∞ (U ) = O (U )[tka | k > 0],

where |tka | = |a|. Let ∂k,a be the derivation ∂/∂tka : O∞ (U ) → O∞ (U ). We write t0a for the generators t a of O (U ) ⊂ O∞ (U ), and we write ∂0,a for the derivations ∂a . The algebra O∞ (U ) is the space of holomorphic functions on the supermanifold J∞ (U ) of infinite jets of curves in U ; such a jet may be parametrized by the formula t a (x) =

∞ X xk a t . k! k k=0

The derivation of O∞ (U ) representing differentiation with respect to x plays a fundamental role: it is given by the formula ∂=

∞ X

a tk+1 ∂k,a .

k=0

Let δk,a : O∞ (U ) → O∞ (U ) be the higher Euler operators of M. Kruskal, R. Miura, C. Gardner, and N. Zabusky [6], δk,a

∞ X = (−1)i

k+i i k ∂ ∂k+i,a ,

i=0

and let δk = dt a δk,a : O∞ (U ) −→ (Cm|n )∗ ⊗ O∞ (U )

548

EZRA GETZLER

be the total higher Euler operators. These are not derivations; indeed, they are infiniteorder differential operators. However, unlike the derivations ∂k,a , they have simple transformation properties under changes of coordinates. PROPOSITION 4.1 If f : U → V is a holomorphic map between open subsets of Cm|n , there is a unique homomorphism of algebras

f ∗ : O∞ (V ) −→ O∞ (U ) which extends the homomorphism f ∗ : O (V ) → O (U ) and satisfies ∂ · f ∗ = f ∗ · ∂. Let J = d f ∈ End(Cm|n ) ⊗ O (U ) be the Jacobian of f . For u ∈ O∞ (V ) and k ≥ 0, δk,a ( f ∗ u) = Jab f ∗ (δk,a u) ∈ O∞ (U ). Proof It suffices to define f ∗ on the generators xka of O∞ (V ) over O (V ). By the hypotheses on f ∗ , we have f ∗ tka = f ∗ ∂ k ta = ∂ k f ∗ ta , so the definition of f ∗ is forced. By induction on `, we see that ∂k,a · ∂ ` =

` X ` j

∂ `− j · ∂k− j,a .

j=0 ` `−k b Ja k ∂

It follows that ∂k,a f ∗ x`b =

∂k,a ( f ∗ u) =

and hence that, for u ∈ O∞ (V ),

∞ X ` `−k b Ja ) f ∗ (∂`,b u). k (∂ `=k

Thus, δk,a ( f ∗ u) =

X

=

X

=

X

(−1)i

k+i i k ∂ (∂k+i,a

(−1)i

k+i ` i k k+i ∂

f ∗ u)

i

(∂ `−k−i Jab ) f ∗ (∂`,b u)

i,`

(−1)i

k+i ` i `−k− j b Ja ) f ∗ (∂ j ∂`,b u) k k+i j (∂

i, j,`

=

X

(−1)i

i, j,`

`−k− j ` `−k `−k− j b Ja ) f ∗ (∂ j ∂`,b u). i− j k j (∂


549

The sum over i reduces to δ(i, j) δ(`, j + k), and the right-hand side reduces to Jab ( f ∗ δk,b u). Now suppose that f is a diffeomorphism, and define f ∗ = ( f −1 )∗ : O∞ (U ) → O∞ (V ). Since (g f )∗ = g∗ f ∗ , it follows that O∞ (U ) is a module over the pseudo(super)group of holomorphic diffeomorphisms between open subsets of Cm|n . Thus, the definition of the sheaf of graded commutative algebras O∞ extends to any (m|n)-dimensional complex supermanifold M, and by Proposition 4.1, the higher Euler operators extend as well: δ0 is a connection on the O -module O∞ , and the higher variational derivatives δk , k > 0, are sections of 1 ⊗O EndO (O∞ ). 4.2. Soloviev’s bracket Let P = P ab ∂a ⊗ ∂b be a ν-Poisson tensor on an open subset U of the superspace Cm|n . The following bracket on O∞ (U ) was introduced by Soloviev [10] (although he restricts attention to the case ν = 0): X {u, v} = − (−1)(|b|+ν)|u| ∂ k+` (P ab δk,a u δ`,b v). (4.1) k,`

It is obvious that this bracket extends the Poisson bracket on the subspace O (U ) of O∞ (U ). However, unlike the Poisson bracket on O (U ), Soloviev’s bracket does not act by derivations; this is a fundamental difference between the Hamiltonian formalisms for mechanics and field theory. It follows from Proposition 4.1 that the bracket (4.1) is invariant under changes of coordinate; hence the definition of the Soloviev bracket extends to the sheaf O∞ on a holomorphic ν-Poisson supermanifold (M, P). 4.1 We have ∂{u, v} = {u, ∂v}. LEMMA

Proof From the formula [∂k,a , ∂] =

( ∂k−1,a , k > 0, 0,

k = 0,

it follows that δ0,a ∂ = 0 and that δk,a ∂ = δk−1,a for k > 0; the lemma follows easily from this formula. Since we are interested only in the case where M is the 1-symplectic supermanifold X associated to a manifold X , it suffices for our purposes to extend Soloviev’s

550

EZRA GETZLER

proof that his bracket satisfies the Jacobi rule to ν-Poisson tensors P in the special case when their coefficients P ab are constant. The general case may be reduced to this one by expressing the Poisson bracket for a general ν-Poisson tensor in terms of the Schouten bracket. The first step in the proof is the following remarkable identity (see [10, Statement 6.1.1]). LEMMA 4.2 If the coefficients P ab are constant, then X {u, v} = − (−1)(|b|+ν)|u| P ab (∂ ` ∂k,a u) (∂ k ∂`,b v). k,`

Proof We have {u, v} = −

X

(−1)i+ j+(|b|+ν)|u|

k+i `+ j ab k+` i (∂ ∂k+i,a u)(∂ j ∂`+ j,b v) k ` P ∂

i, j,k,`

=−

X

(−1)i+ j+(|b|+ν)|u|

k ` k+`−i− j k−i `− j k+ p−i

i, j,k,`, p

× P ab (∂ k+ p ∂k,a u)(∂ `− p ∂`,b v). Since

n−i i k i (−1) k−i m−i

P

X

−

=

n−k m ,

(−1) j+(|b|+ν)|u|

this in turn equals

` `− j ab k+ p ∂k,a u) (∂ `− p ∂`,b v). `− j k+ p P (∂

j,k,`, p

The sum over j reduces to δ(`, k + p), and the lemma follows. We now apply the following lemma. LEMMA 4.3 Suppose that

{u, v}, w = α(u|v, w) − (−1)(|u|+ν)(|v|+ν) α(v|u, w), where {u, v} is an operation of degree ν satisfying {u, v} = −(−1)(|u|+ν)(|v|+ν) {v, u} and α(u|v, w) is an operation of degree zero satisfying α(u|v, w) = (−1)(|v|+ν)(|w|+ν) α(u|w, v). Then {u, v} is a graded Lie bracket.


551

A lengthy calculation shows that when P ab is constant, the hypotheses of this lemma hold for the bracket (4.1) on O∞ (U ), with α(u|v, w) given by the formula α u|v, w = −

X

(−1)|b| |u|+|d| |u|+(|d|+ν)(|v|+ν)+(a+b+ν)c

i, j,k,`, p,q j ` ab cd p q P P

(∂ j+`− p−q ∂k,c ∂i,a u) (∂ i+q ∂ j,b v) (∂ k+ p ∂`,d w).

Let P be the cokernel of the derivation ∂ : O∞ → O∞ , and denote the natural R projection from O∞ to P by u 7→ u d x. Lemma 4.1 implies that the Lie bracket {u, v} on O∞ induces a graded Lie bracket on P , given by the formula R R R R u d x, v d x = {u, v} d x = −(−1)(|b|+ν)|u| P ab δa u δb v d x. (4.2) 4.3. The Schouten bracket Let π : X → X be the projection from X to X , denote the sheaf π∗ O∞ on X by 3∞ , and denote its bracket by [u, v]∞ . The grading 3∞ is shifted by −1 in the same p way as the grading of 3: sections of 3∞ are those with p + 1 factors of θk,a . In a coordinate system of the form {t a , θa }, the Poisson tensor (1.2) is constant; applying Lemma 4.2, we obtain the following formula for the bracket on 3∞ : XR [u, v]∞ = (−1)|u| ∂ ` ∂ka u · ∂ k ∂`,a v − ∂ ` ∂k,a u · ∂ k ∂à v d x. (4.3) k,`

Note that the inclusion 3 ,→ 3∞ is a morphism of graded Lie algebras. In the special case where M equals X , the bracket (4.2) on P is the Schouten bracket of the formal calculus of variations, introduced by Gelfand and Dorfman [3] and Olver [9]. Denote the sheaf π∗ P on X by L , and denote its bracket by [[u, v]]. We grade L in the same way as the sheaves 3 and 3∞ . As a graded Lie algebra, L R is a quotient of 3∞ , and : 3∞ → L is a morphism of graded Lie algebras. The Schouten bracket is given by two rather different formulas, R [[u, v]] = (−1)|u| δ a u · δa v − δa u · δ a v d x XR = (−1)|u| ∂ ` ∂ka u · ∂ k ∂`,a v − ∂ ` ∂k,a u · ∂ k ∂à v d x, k,`

the first of which manifests the invariance of the bracket under coordinate transformations, while the second seems to be easier to apply in explicit calculations. 5. Hamiltonian manifolds The characterization of Hamiltonian operators via the Maurer-Cartan equation is due to Gelfand and Dorfman [3]. The following is a global form of their definition.

552

EZRA GETZLER

Definition 5.1 A Hamiltonian manifold (X, Q ) is a manifold X together with a section Q ∈ 0(X, L 1 ) satisfying the Maurer-Cartan equation [[Q , Q ]] = 0. The section Q is called a Hamiltonian operator. R A Hamiltonian operator has a canonical form Q = θa D ab θb d x, where D

ab

=

N X

Dkab ∂ k

k=0

is a formally skew-adjoint system of ordinary differential operators with coefficients in the sheaf O∞ . Formal skew-adjointness means that for every section u of the sheaf O∞ , N X Dkab (∂ k u) + (−∂)k (Dkba u) = 0. k=0

R For example, if X = C and Q = θ((1/8)∂ 3 + t∂)θ d x (the second Hamiltonian operator of the KdV hierarchy), the operator D equals (1/8)∂ 3 + t ∂ + (1/2)∂t. The analogue of Proposition 1.1 holds for Hamiltonian operators: Q ∈ 0(X, L 1 ) is a Hamiltonian operator if and only if the morphism of graded sheaves δQ = [[Q , −]] on L is a differential. Denote the sheaf of dg Lie algebras L with this differential by LQ ; it controls deformations of Q in the same way that the sheaf of dg Lie algebras 3 Q on a Poisson manifold controls deformations of the Poisson tensor Q. Definition 5.2 A Hamiltonian operator Q is Darboux if the sheaf of dg Lie algebras LQ is formal. 5.1. A resolution of L We now introduce a resolution L of the sheaf of graded Lie algebras L ; this resolution is a sheaf of Fock spaces. ˜ ∞ = 3∞ /(C · 1) be the quotient of 3∞ by its centre, and let L be the Let 3 ˜ ∞ → 3∞ ; in other words, L is isomorphic to the graded cone of the morphism ∂ : 3 ˜ ∞ [1], where O˜∞ [1] is a copy of 3 ˜ ∞ shifted in degree by −1. Denoting sheaf 3∞ ⊕ 3 ˜ elements of 3∞ [1] by εu, ˜ the differential equals D(u + εu) ˜ = ∂ u. ˜ Equipped with the bracket [u + εu, ˜ v + εv] ˜ ∞ = [u, v]∞ + ε [u, ˜ v]∞ + (−1)|u| [u, v] ˜∞ , L is a sheaf of dg Lie algebras.


553

THEOREM 5.1 R The morphism : L → L defined by the formula R R (u + εu) ˜ = u dx

is a weak equivalence of dg Lie algebras. Proof R It is clear that is compatible with the differential on L: R R D(u + εu) ˜ = (∂ u) ˜ d x = 0. R It is also easy to see that is a morphism of graded Lie algebras since R R R [u + εu, ˜ v + εv] ˜∞= (u + εu), ˜ (v + εv) ˜ . R It only remains to check that is a weak equivalence; this is a variant on the “exactness of the variational bicomplex.” We learned the idea used in the following proof from E. Frenkel. Let U be a connected open subset of Cm|n , and let u ∈ O∞ (U ). We must show that ∂u = 0 if and only if u is a multiple of 1. It is clear that this is so if u ∈ O (U ) since, in that case, ∂u = ∂t a ∂a u. The operators ∂ and ρ=

∞ X

k(k + 1) tka ∂k+1,a

k=0

generate an action of the Lie algebra sl(2) on O∞ (U ), whose Cartan subalgebra acts by the semisimple endomorphism H=

∞ X

k tka ∂k,a ,

k=0

with kernel O (U ). Suppose that ∂u = 0. Since ρ i u = 0 for i 0, we see that the irreducible sl(2)module spanned by u is finite-dimensional. Since a finite-dimensional representation of sl(2) on which H has no negative spectrum is trivial, we conclude that H u = 0; hence u lies in O (U ) ⊂ O∞ (U ) and, as we have seen, is a multiple of 1. 5.2. Ultralocal Hamiltonian operators Since the bracket on L prolongs the Schouten bracket, a Poisson tensor Q on X gives R rise to a Hamiltonian operator Q = Q d x. Such Hamiltonian operators are called ultralocal.

554

EZRA GETZLER

THEOREM 5.2 If Q is the Poisson tensor associated to a symplectic manifold (X, ω), the inclusion of sheaves of dg Lie algebras 3 Q ,→ LQ is a weak equivalence; in particular, the R Hamiltonian operator Q = Q d x is Darboux.

Proof We must show that if (X, Q) is a symplectic manifold, the inclusion (3• , δ Q ) ,→ (L• , D + δ Q ) of sheaves of dg Lie algebras is a weak equivalence. By the Darboux theorem (in its original sense!), it suffices to consider a convex subset U of C2` with its standard symplectic structure, and Poisson tensor Q=

` X

θa θa+` .

a=1

Let δQ be the differential associated to the Maurer-Cartan element Q=

` X

θ0,a θ0,a+`

a=1

of L(U ); it is given by the formula δQ =

` ∞ X X

(θk,a ∂k,a+` − θk,a+` ∂k,a ).

k=0 a=1

Clearly, the dg Lie algebra LQ (U ) = (L(U ), D + δQ ) is a resolution of (L , δR Q d x ). The complex LQ (U ) is isomorphic to the cone of the morphism ˜ • J∞ (U ) −→ • J∞ (U ) , ∂: ˜ • (J∞ (U )) where • (J∞ (U )) is the de Rham complex of the jet space J∞ (U ) and is its quotient by the constant functions. To see this, one identifies θk,a with dtka+` and θk,a+` with −dtka . Theorem 5.2 now follows from the de Rham theorem for J∞ (U ).

If the Poisson tensor Q is not symplectic, the inclusion 3 Q ,→ LQ is not a weak equivalence; this is obvious if the Poisson tensor vanishes, and the general case may be inferred from this one.


555

5.3. Hamiltonian manifolds of hydrodynamic type Let η be a flat contravariant metric on M, with coefficients ηab = η(dt a , dt b ). Dubrovin and Novikov [2] associate to η a Hamiltonian operator Hη ; in flat coordinates (those for which the coefficients ηab are constant), it is given by the formula Hη =

1 R ab η θa ∂θb d x. 2

The differential dη = [[Hη , −]] on L is given by the formula X R R dη u d x = − ηab θk+1,a ∂k,b u d x, k

and the resulting sheaf of dg Lie algebras is denoted Lη . We may now state the main result of this paper. Let g(X, η) be the sheaf of graded Lie algebras on X whose stalk at x ∈ X is the graded Lie algebra g(Tx∗ X, η) introduced in Section 3.2. Let τ0 : g(U, η)x → 3∞ (U ) be the operation that substitutes θ0a for θ a . THEOREM 5.3 The morphism σ : g(X, η) ,→ Lη defined by the formula R R σ (α, ˜ α) = τ0 (α) ˜ d x + ηab t0a ∂0b τ0 (α) d x

is a weak equivalence of sheaves of dg Lie algebras. In particular, hydrodynamic Hamiltonian operators are Darboux, and σ induces a weak equivalence of sheaves of Deligne 2-groupoids C (σ ) : C g(X, η) ' C (Lη ). 5.4. Lifting Hamiltonian operators to L The proof of Theorem 5.2 is based on the idea of lifting the Hamiltonian operator R Q = Q d x to a Maurer-Cartan element of L. This may be generalized as follows. Definition 5.3 R A lift of a Hamiltonian manifold (X, Q ) is a section Q of L1 with Q = Q d x and which satisfies the Maurer-Cartan equation DQ + (1/2)[Q, Q]∞ = 0. If Q is a lift of a Hamiltonian operator Q , there is a weak equivalence of sheaves of dg Lie algebras R : L, D + δQ −→ L , δQ , where δQ is the differential δQ u = [Q, u]∞ on L. Let us give some explicit examples of lifts. As we have already observed, an R ultralocal Hamiltonian operator Q d x has the lift Q. A hydrodynamic Hamiltonian

556

EZRA GETZLER

R operator (1/2) ηab θa ∂θb d x has the lift (1/2)ηab θa θ1,b . Since a manifold X with flat contravariant metric η has an atlas whose charts are flat and whose transition functions are inhomogeneous orthogonal transformations, these lifts patch together to give a lift of Hη over all of X . For a less trivial example, the second Hamiltonian operator of the KdV hierarchy, R Q = θ ((1/8)∂ 3 + t ∂)θ d x, has a family of lifts (cf. L. Dickey [1]) Q=

1 1 θ θ3 + tθθ1 + a ∂(θθ2 ) + ε θ θ1 θ2 , 8 8

a ∈ C.

PROPOSITION 5.1 Every Hamiltonian manifold (X, Q ) that is Stein has a lift Q.

Proof R Lifts Q = u + εu˜ of Q are characterized by the equations Q = u d x and 1 [u, u]∞ = [u, u] ˜ ∞ = 0. 2 R Let u be a section of 31∞ such that Q = u d x; there are no obstructions to the R existence of u, because X is Stein. Since [[Q , Q ]] = [u, u]∞ d x = 0, we see that there is a section u˜ of 32∞ such that ∂ u˜ + (1/2)[u, u]∞ = 0; again, there are no obstructions to the existence of u. ˜ Taking the bracket of this equation with u, we see that 1 [u, ∂ u] ˜ ∞+ u, [u, u]∞ ∞ = 0. 2 But [u, [u, u]∞ ]∞ vanishes by the Jacobi rule, while [u, ∂ u] ˜ ∞ = ∂[u, u] ˜ ∞ . By Theorem 5.1, we conclude that [u, u] ˜ ∞ = 0. ∂ u˜ +

6. Proof of Theorem 5.3 The differential of L(U ) associated to the lift (1/2)ηab θa θ1,b of the hydrodynamic R Hamiltonian operator (1/2) ηab θa ∂θb d x equals hh 1 ii 1 dη = ηab θa θ1,b , − = −d + ∂ · d0 , 2 2 P∞ ab P∞ ab where d = k=0 η θk+1,a ∂k,b and d0 = k=0 η θk,a ∂k,b . LEMMA 6.1 Let η be a constant metric on Cn , and let U be a convex subset of Cn containing zero. The map of graded vector spaces τ : g(U, η)0 = g(T0 U, η0 ) → Lη (U ), defined on g p (U, η)0 by the formula 1 τ (α, ˜ α) = τ0 (α) ˜ + ηab t0a ∂0b − ε p τ0 (α), 2


557

is a morphism of dg Lie algebras. Proof (1) The map τ is a morphism of complexes (i.e., (D + dη ) · τ = 0). Let (α, ˜ α) be an element of g p (U, η)0 . It is obvious that (D + dη )τ (α, ˜ 0) = (D + dη )τ0 (α) ˜ =0 since Dτ0 (α) ˜ and dη τ0 (α) ˜ both vanish. As for (D + dη )τ (0, α), we have 1 D(ηab t0a ∂0b )α˜ = dη − ε p α˜ = 0 2 and

1 1 1 dη (ηab t0a ∂0b )α˜ + Dl − ε p α˜ = p ∂ α˜ − p ∂ α˜ = 0. 2 2 2 (2) The map τ preserves the Lie bracket. If α ∈ h p (U, η)0 , we have ˜ β) = τ − ηab ∂ a α ∂ b β, ˜ (−1) p+1 ηab ∂ a α ∂ b β τ (0, α), (β, ˜ + (−1) p+1 = −τ0 (ηab ∂ a α ∂ b β) 1 × ηab ηcd t0c ∂0d − ε ( p + q) τ0 (∂ a α ∂ b β) 2 ˜ + (−1) p+1 = −ηab ∂0a τ0 (α) ∂0b τ0 (β) 1 × ηab ηcd t0c ∂0d − ε ( p + q) ∂0a τ0 (α) ∂0b τ0 (β). 2

On the other hand, h 1 ˜ β) = ηab t a ∂ b − ε p τ0 (α), τ0 (β) ˜ τ (0, α), τ (β, 0 0 ∞ 2 i 1 + ηcd t0c ∂0d − ε q τ0 (β) ∞ 2 ˜ + ηcd t c ∂ d τ0 (β) = ηab t0a ∂0b τ0 (α), τ0 (β) 0 0 ∞ 1 − ε (−1) p q ηab t0a ∂0b τ0 (α), τ0 (β) ∞ 2 + p [τ0 (α), ηcd t0c ∂0d τ0 (β)]∞ ˜ + (−1) p ηab ηcd t a ∂ c ∂ b τ0 (α) ∂ d τ0 (β) = −ηab ∂0b τ0 (α) ∂0a τ0 (β) 0 0 0 0 − ηab ηcd t0c ∂0b τ0 (α) ∂0a ∂0d τ0 (β) 1 + ε (−1) p q ηab ∂0b τ0 (α) ∂0a τ0 (β) 2 − (−1) p+1 p ηcd ∂0c τ0 (α) ∂0d τ0 (β) .

558

EZRA GETZLER

˜ β)] = [τ (0, α), τ (β, ˜ β)]∞ . Finally, it From these formulas we see that τ [(0, α), (β, ˜ ˜ 0) commute is clear that [τ (α, ˜ 0), τ (β, 0)]∞ = 0, as they must since (α, ˜ 0) and (β, in g(U, η)0 . The operations ι(u + εu) ˜ = u˜ and ε(u + εu) ˜ = εu on L(U ) satisfy the canonical graded commutation relations [ε, ι] = 1; using ι, the differential of L may be written D = ι∂. 6.2 The morphism T = 1 + (1/2)εd0 (of complexes, not of dg Lie algebras) induces an isomorphism of complexes T : Lη (U ) −→ L(U ), D − d . LEMMA

Proof We must show that T(D + dη ) − (D − d)T vanishes. Rewriting using the operators ε and ι, and taking into account that the operators d and d0 graded commute with ι and ε and that [∂, d0 ] = 0, we see that this equals 1 1 1 1 + εd0 ι∂ − d + ∂d0 − (ι∂ − d) 1 + εd0 2 2 2 1 1 1 = ∂ d0 − [ε, ι]d0 + ε[d0 , d0 ] − ε[d0 , d], 2 4 2 which vanishes since [d, d0 ] = [d0 , d0 ] = 0 and [ε, ι] = 1. LEMMA 6.3 The morphism of complexes T · τ : g(U, η)0 → (L(U ), D − d) has the formula T · τ (α, ˜ α) = τ0 (α) ˜ + ε τ0 (α).

Proof We have 1 1 T · τ (α, ˜ α) = 1 + ε d0 · τ0 (α) ˜ + ηab t0a ∂0b τ0 (α) − ε p τ0 (α) 2 2 1 1 a b a b = τ0 (α) ˜ + ηab t0 ∂0 τ0 (α) + ε d0 ηab t0 ∂0 τ0 (α) − ε p τ0 (α). 2 2 The formula follows since d0 (ηab t0a ∂0b τ0 (α)) = ( p + 2)τ0 (α)). Theorem 5.3 is now a consequence of the following lemma. 6.4 The morphism T · τ : g(U, η)0 → (L(U ), D − d) is a weak equivalence. LEMMA


559

Proof There is a short exact sequence of complexes ˜ ∞ (U )[1], −d −→ 0 0 −→ 3∞ (U ), −d −→ L(U ), D − d −→ 3 and hence, for p ≥ −1, a long exact sequence δ ˜ ∞ (U ), d −→ · · · −→ H p−1 L(U ), D − d −→ H p 3 H p 3∞ (U ), d −→ · · · . There is an isomorphism between the complex (3∞ (U ), d) and the de Rham complex • (J∞ (U ), 3Cn )[1], obtained by mapping θk+1,a to ηab dtkb and θ0,a to ˜ ∞ (U ), d) is isomorphic to the the basis vector θa of Cn . Likewise, the complex (3 • n ˜ reduced de Rham complex (J∞ (U ), 3C )[1]. The Poincaré lemma for J∞ (U ) shows that T · τ induces isomorphisms between ˜ ∞ (U ), d) and the group 3 p+1 Cn . The comthe groups H p (3∞ (U ), d) and H p (3 ˜ ∞ (U ), d) → H p (3∞ (U ), d) position of T · τ with the boundary map δ : H p (3 p+1 n vanishes: if α ∈ 3 C , we have δ · T · τ (α) = D ε τ0 (α) = ∂τ0 (α) = 0. We conclude that there is a short exact sequence 0 −→ 3 p+1 Cn −→ H p L(U ), D − d −→ 3 p+2 Cn −→ 0 and hence that T · τ is indeed an isomorphism onto the cohomology of D − d. Acknowledgments. This paper contributes to an area of mathematics in which my teacher Roger Richardson was a pioneer (see [8]). He first introduced me to the beautiful applications of algebra in differential geometry. Youjin Zhang brought the problem of proving a Darboux theorem for Hamiltonian operators to my attention. I thank him, B. Dubrovin, E. Frenkel, and P. Olver for useful discussions. I am especially grateful to K. Saito for helping make my visit to the Research Institute for Mathematical Sciences (RIMS) so enjoyable. References [1]

L. A. DICKEY, “Poisson brackets with divergence terms in field theories: Three

[2]

examples” in Higher Homotopy Structures in Topology and Mathematical Physics (Poughkeepsie, N. Y., 1996), Contemp. Math. 227, Amer. Math. Soc., Providence, 1999, 67–78. MR 99k:58065 556 B. A. DUBROVIN and S. P. NOVIKOV, Hydrodynamics of weakly deformed soliton lattices: Differential geometry and Hamiltonian theory, Uspekhi Mat. Nauk 44, no. 6 (1989), 29–98, 203; English translation in Russian Math. Surveys 44, no. 6 (1989), 35–124. MR 91g:58109 536, 555

560

EZRA GETZLER

[3]

I. M. GELFAND and I. JA. DORFMAN, Schouten bracket and Hamiltonian operators,

[4]

W. M. GOLDMAN and J. J. MILLSON, The deformation theory of representations of

Funktsional. Anal. i Prilozhen. 14 (1980), 71–74. MR 82e:58039 535, 551

[5] [6]

[7] [8]

[9]

[10]

´ fundamental groups of compact Kähler manifolds, Inst. Hautes Etudes Sci. Publ. Math. 67 (1988), 43–96. MR 90b:32041 539, 540 M. KONTSEVICH, Deformation quantization of Poisson manifolds, I, preprint, arXiv:q-alg/9709040 539 M. D. KRUSKAL, R. M. MIURA, C. S. GARDNER, and N. J. ZABUSKY, Korteweg– de Vries equation and generalizations, V: Uniqueness and nonexistence of polynomial conservation laws, J. Mathematical Phys. 11 (1970), 952–960. MR 42:6410 547 I. MOERDIJK and J.-A. SVENSSON, Algebraic classification of equivariant homotopy 2-types, I, J. Pure Appl. Algebra 89 (1993), 187–216. MR 94j:55013 542 A. NIJENHUIS and R. W. RICHARDSON JR., Cohomology and deformations of algebraic structures, Bull. Amer. Math. Soc. 70 (1964), 406–411. MR 31:2299 559 P. OLVER, “Hamiltonian perturbation theory and water waves” in Fluids and Plasmas: Geometry and Dynamics (Boulder, 1983), Contemp. Math. 28, Amer. Math. Soc., Providence, 1984, 231–249. MR 85i:58047 535, 551 V. O. SOLOVIEV, Boundary values as Hamiltonian variables, I: New Poisson brackets, J. Math. Phys. 34 (1993), 5747–5769. MR 94i:58072 536, 547, 549, 550

Research Institute for Mathematical Sciences, Kyoto University, Kyoto 606-8502, Japan; current: Department of Mathematics, Northwestern University, Evanston, Illinois 60208, USA; [email protected]


FUNDAMENTAL SOLUTIONS FOR THE TRICOMI OPERATOR, II J. BARROS-NETO and I. M. GELFAND

Abstract In this paper we explicitly calculate fundamental solutions for the Tricomi operator, relative to an arbitrary point in the plane, and show that all such fundamental solutions originate from the hypergeometric function F(1/6, 1/6; 1; ζ ) that is obtained when we look for homogeneous solutions to the reduced hyperbolic Tricomi equation. 1. Introduction The Tricomi operator T =y

∂2 ∂2 + 2, 2 ∂x ∂y

(1.1)

one of the simplest examples of a partial differential operator of mixed type, is (i) elliptic in the upper half-plane (y > 0), (ii) parabolic along the x-axis (y = 0), and (iii) hyperbolic in the lower half-plane (y < 0). Our aim is to obtain explicit solutions in the sense of distributions or generalized functions of the equation T E = δ(x − x0 , y − y0 ),

(1.2)

where δ(x − x0 , y − y0 ) is the Dirac function at (x0 , y0 ), an arbitrary point in the plane. A solution E of (1.2) is said to be a fundamental solution relative to point (x0 , y0 ). In a previous paper (see [2]) we considered the case when x0 = y0 = 0 and proved the existence of two remarkable fundamental solutions that clearly reflect the fact that the operator changes type across the x-axis. In this paper we study the general case when x0 = y0 6= 0 and compare our results to those of [2]. For the sake of completeness and in order to make the reading of this paper independent of that of [2], we briefly review the contents of that paper. It is known that the equation 9x 2 + 4y 3 = 0 defines the two characteristics for the Tricomi operator that DUKE MATHEMATICAL JOURNAL c 2002 Vol. 111, No. 3, Received 5 February 2001. 2000 Mathematics Subject Classification. Primary 35M10; Secondary 35M08.

561

562

BARROS-NETO AND GELFAND

emanate from the origin. These characteristics, which are tangent to the y-axis at the origin, divide the plane into two disjoint regions, D+ and D− (see Figure 1), defined as D+ = (x, y) ∈ R2 : 9x 2 + 4y 3 > 0 , the region “outside” the characteristics, and D− = (x, y) ∈ R2 : 9x 2 + 4y 3 < 0 , the region “inside” the characteristics.

D+

D−

Figure 1

The first fundamental solution is defined by ( C+ (9x 2 + 4y 3 )−1/6 F+ (x, y) = 0 with C+ = −

in D+ , elsewhere

1 1 1 F , ; 1; 1 , 6 6 21/3 · 31/2

and the second one is defined by ( C− |9x 2 + 4y 3 |−1/6 F− (x, y) = 0 with C− =

1

2

F 1/3

in D− , elsewhere

1 1 , ; 1; 1 . 6 6

(1.3)

(1.4)

(1.5)

(1.6)

FUNDAMENTAL SOLUTIONS FOR THE TRICOMI OPERATOR, II

563

In the expressions of both C+ and C− , the constant F

1 1 0(2/3) , ; 1; 1 = 2 6 6 0 (5/6)

is the value of the hypergeometric function F(1/6, 1/6; 1; ζ ) at ζ = 1. We remark that in [2] the constants C+ and C− were denoted, respectively, by −

0(1/6) 3 · 22/3 π 1/2 0(2/3)

and

30(4/3) . 22/3 π 1/2 0(5/6)

At that time we were unaware of the role played by the hypergeometric function F(1/6, 1/6; 1; ζ ) in the study of fundamental solutions for the Tricomi operator. We also observe that, according to formula (1.5), a perturbation at the origin spreads out to the entire region D− , inside the characteristics. On the other hand, according to formula (1.3), the same perturbation spreads out to the whole elliptic region (y > 0) and also to the hyperbolic region (y < 0) outside the characteristics. Since the Tricomi operator is invariant under translations along the x-axis, the same phenomena take place for fundamental solutions that correspond to the Dirac measure δ concentrated at an arbitrary point (a, 0) on the x-axis. It is well known that the Tricomi operator describes the transition from subsonic flow (elliptic region) to supersonic flow (hyperbolic region). In the extensive literature on the Tricomi operator, one finds in the works of several authors, among others, S. Agmon [1], K. Friedrichs [6], I. Gelfand (unpublished), P. Germain and R. Bader [8], L. Landau and E. Lifschitz [9], J. Leray [11], and C. Morawetz [12], a large number of examples of solutions to different problems which show the interaction between these two regions. It seems that the two distributions F+ (x, y) and F− (x, y) are the simplest of such examples. We now return to equation (1.2). In view of the invariance of the Tricomi operator under translations parallel to the x-axis, the problem of solving that equation is equivalent to the problem of solving T E = δ(x, y − b),

(1.7)

where b is an arbitrary real number and δ(x, y − b) denotes the Dirac measure concentrated at point (0, b). Consider the case when b < 0, that is, when point (0, b) is situated in the hyperbolic region. We introduce the characteristic coordinates ` = 3x + 2(−y)3/2

and

m = 3x − 2(−y)3/2 ,

(1.8)

set `0 = 2(−b)3/2 , and let a > 0 be such that 3a = 2(−b)3/2 . Note that (`0 , −`0 ) represents point (0, b) in characteristic coordinates. As shown in Figure 2, two char-

564


y

DI I (−a, 0)

(a, 0)

DI I

DI I

x

(0, b) DI V

DI I I DI

Figure 2

acteristics with equations 3(x − a) + 2(−y)3/2 = 0

and

3(x + a) − 2(−y)3/2 = 0

(1.9)

pass through point (0, b). In characteristic coordinates, these are the two half-lines ` = `0 ,

−∞ < m ≤ `0 ,

and

m = −`0 ,

−`0 ≤ ` < +∞,

(1.10)

that originate from point (`0 , −`0 ). Characteristics (1.9) meet the x-axis, respectively, at points (a, 0) and (−a, 0). Two new characteristics originate from these two points, namely, 3(x − a) − 2(−y)3/2 = 0

and

3(x + a) + 2(−y)3/2 = 0,

(1.11)

which we call reflected characteristics. The corresponding equations in characteristic coordinates are m = `0 ,

`0 ≤ ` < +∞

and

` = −`0 ,

−∞ < m ≤ −`0 .

(1.12)

The two characteristics through point (0, b) and the reflected characteristics divide the plane into four disjoint regions denoted by D I , D I I , D I I I , and D I V and illustrated in Figure 2.


565

In this paper we show the existence of four fundamental solutions, each supported by the closure of the corresponding region. Of these solutions, only two have physical meaning: the one defined in region D I and the one defined in region D I I . This situation is similar to what happens in the case of the wave operator in two dimensions, where the two relevant fundamental solutions are the ones supported by the forward and backward light cones. As we are going to see, it is the hypergeometric function F(1/6, 1/6; 1; ζ ) that plays a crucial role in defining these four fundamental solutions. The plan of this paper is as follows. We prove in Section 2 that the function 1 1 (` − `0 )(m + `0 ) E(`, m; `0 , −`0 ) = (` + `0 )−1/6 (`0 − m)−1/6 F , ; 1; 6 6 (` + `0 )(m − `0 ) (1.13) is a solution of Th u = 0, where Th denotes the reduced hyperbolic Tricomi equation (2.1). After replacing ` and m in (1.13) by their expressions in (1.8), we obtain the following solution of T u = 0 : −1/6 E(x, y; 0, b) = eiπ/6 9(x 2 − a 2 ) + 4y 3 − 12a(−y)3/2 1 1 9(x 2 − a 2 ) + 4y 3 + 12a(−y)3/2 × F , ; 1; . (1.14) 6 6 9(x 2 − a 2 ) + 4y 3 − 12a(−y)3/2 This is the function that generates the four fundamental solutions relative to point (0, b). In Section 3 we define the distribution E I (x, y; 0, b) as the restriction of E(x, y; 0, b), after multiplication by a suitable constant, to region D I and show that E I is a fundamental solution of the Tricomi operator. Region D I is entirely contained in the hyperbolic region where it is natural to use characteristic coordinates. Theorem 3.1 is then proved via integration by parts and by using the results of Proposition 2.1. As a consequence, we show that, as (0, b) tends to (0, 0), E I (x, y; 0, b) tends, in the sense of distributions, to F− (x, y), the fundamental solution defined by (1.5). The fundamental solutions supported by the closure of regions D I I I and D I V are defined and studied in Section 4. In both regions it is necessary to take into account the singularities of E(x, y; 0, b) along the reflected characteristics. The results of Proposition 4.1, which describe the asymptotic behavior of the hypergeometric function F(1/6, 1/6; 1; ζ ) as |ζ | → +∞, are then needed in the proof of Theorem 4.1. We note that, as b → 0, both fundamental solutions E I I I and E I V tend to zero. In Section 5 we define E I I (x, y; 0, b) as the restriction of E(x, y; 0, b), after multiplication by a suitable constant, to D I I and show that T E I I = δ(0, b). Since E I I is complex valued and the Tricomi operator has real coefficients, both its complex conjugate and real part are also fundamental solutions relative to point (0, b). The

566


imaginary part of E I I is then a solution of the homogeneous equation T u = 0, and we call it a Tricomi harmonic function. Contrary to what is proved in Corollary 3.1, it is not true that the fundamental solution F+ (x, y) defined by (1.3) is the limit of E I I (x, y; 0, b), or its real part, as (0, b) → (0, 0). It is necessary to take a suitable linear combination of E I I and its complex conjugate in order to achieve this result. As a final remark, we mention that Leray described in [11] a general method, based upon the theory of analytic functions of several complex variables, for finding fundamental solutions for a class of hyperbolic linear differential operators with analytic coefficients. In particular, he showed how his method could be used to obtain, in the hyperbolic region, a fundamental solution for the Tricomi operator relative to a point (0, b). He also produced an explicit formula for the fundamental solution in terms of the hypergeometric function F(1/6, 1/6; 1; ζ ). Our method is simpler, more direct, and gives us global fundamental solutions that clearly reflect the change of type of the Tricomi operator across the x-axis. 2. A special solution to Th u = 0 In characteristic coordinates (1.8), the Tricomi operator T becomes Th =

∂2 1/6 ∂ ∂ − − , ∂`∂m ` − m ∂` ∂m

(2.1)

and we call Th the reduced hyperbolic form of T . We now look for homogeneous solutions of the equation Th u = 0. Every homogeneous function of ` and m of degree λ, a complex number, can be written as u(`, m) = `λ φ(t), where φ is a function of a single variable t = m/`. Direct substitution into (2.1) shows that φ(t) must be a solution of the hypergeometric equation t (1 − t)φ 00 (t) +

7 i λ −λ − − λ t φ 0 (t) + φ(t) = 0. 6 6 6

h 5

As a solution of this equation, we choose the hypergeometric function F(−λ, 1/6; 5/6 − λ; t) extended, by analytic continuation, to the whole complex plane C minus the cut [1, ∞). Since we are looking for fundamental solutions to the Tricomi operator, we take for λ the value −1/6, as previously indicated in our joint paper [2]. Thus u(`, m) = `−1/6 F is a solution of Th u = 0.

1 1 m , ; 1; 6 6 `


567

Now let (`0 , m 0 ) be an arbitrary point in R2 , and consider the change of variables `→

` − m0 , ` − `0

m→

m − m0 . m − `0

After unenlightening calculations, one can show that the function 1 1 (` − `0 )(m − m 0 ) E(`, m; `0 , m 0 ) = (` − m 0 )−1/6 (`0 − m)−1/6 F , ; 1; 6 6 (` − m 0 )(m − `0 ) (2.2) is also a solution of the same equation. This is the special solution to Th u = 0 that we are looking for. Consider the adjoint equation Th∗ v =

∂ 2v 1/6 ∂v ∂v 1/3 v = 0. + − − ∂`∂m l − m ∂` ∂m (` − m)2

(2.3)

One can see that if v is a solution of equation (2.3), then u = (` − m)−1/3 v is a solution to Th u = 0. Since E(`, m; `0 , m 0 ) satisfies the equation Th u = 0, it follows that the function R(`, m; `0 , m 0 ) = (` − m)1/3 E(`, m; `0 , m 0 )

(2.4)

is a solution of the adjoint equation (2.3). 2.1 R(`, m; `0 , m 0 ) is the unique solution of Th∗ v = 0 that satisfies the following conditions: 1/6 (i) R` = R along the line m = m 0 ; `−m −1/6 (ii) Rm = R along the line ` = `0 ; `−m (iii) R(`0 , m 0 ; `0 , m 0 ) = 1. PROPOSITION

Proof Clearly, conditions (i), (ii), and (iii) imply uniqueness for R. As m = m 0 , the argument of the hypergeometric function in (2.2) equals zero, and so F(1/6, 1/6; 1; 0) = 1. Thus, along the line m = m 0 , we have ` − m 1/6 Z ` 0 R(`, m 0 ; `0 , m 0 ) = = exp a(t) dt , (2.5) `0 − m 0 `0 where a(t) = (1/6)/(t − m 0 ). Therefore R` = ((1/6)/(` − m))R along m = m 0 . In the same manner, one can see that Rm = ((−1/6)/(` − m))R along the line ` = `0 . Finally, it is clear that R(`0 , m 0 ; `0 , m 0 ) = 1.

568


Remark 1 The function R(`, m; `0 , m 0 ) is the Riemann function of the operator Th relative to point (`0 , m 0 ) (see [3]). If we go back to the notations in the introduction, with (0, b) and b < 0, and let `0 = 2(−b)3/2 , then m 0 = −`0 and from equation (2.2) we obtain equation (1.13), which, for further reference, we write as follows: 1 1 E(`, m; `0 , −`0 ) = (` + `0 )−1/6 (`0 − m)−1/6 F , ; 1; ζ , (2.6) 6 6 where

(` − `0 )(m + `0 ) . (2.7) (` + `0 )(m − `0 ) Similarly, after replacing ` and m by their expressions in (1.8), we obtain function (1.14), which we rewrite as 1 1 −1/6 E(x, y; 0, b) = eiπ/6 9(x 2 − a 2 ) + 4y 3 − 12a(−y)3/2 × F , ; 1; ζ , (2.8) 6 6 ζ =

where ζ =

9(x 2 − a 2 ) + 4y 3 + 12a(−y)3/2 . 9(x 2 − a 2 ) + 4y 3 − 12a(−y)3/2

(2.9)

We wish to analyze this function. Denote by ra the reflected characteristic 3(x − a) − 2(−y)3/2 = 0 at (a, 0) and by r−a the reflected characteristic 3(x + a) + 2(−y)3/2 = 0 at (−a, 0). PROPOSITION 2.2 The function E(x, y; 0, b), a solution of T u = 0, is well defined in the region R2 \(ra ∪ r−a ).

Proof Let z = 3(x + a) + 2(−y)3/2 3(x − a) − 2(−y)3/2 = 9(x 2 − a 2 ) + 4y 3 − 12a(−y)3/2 be the denominator of ζ in expression (2.9). For y ≤ 0, z = 0 on ra ∪r−a , it is negative in the region inside these characteristics, and it is positive in the region outside them. For y > 0, z is a complex number. It is possible to choose the argument of z so that it varies continuously in region R2 \(ra ∪ r−a ). Indeed, define φ : R2 \(ra ∪ r−a ) → S1 , the unit circumference, by φ(x, y) =

9(x 2 − a 2 ) + 4y 3 − 12a(−y)3/2 , ρ


569

where ρ = 9(x 2 − a 2 ) + 4y 3 − 12a(−y)3/2 . Since R2 \(ra ∪ r−a ) is contractible, φ lifts to R; that is, there exists a continuous function θ (x, y) defined on R2 \(ra ∪ r−a ) so that 9(x 2 − a 2 ) + 4y 3 − 12a(−y)3/2 = ρe−iθ(x,y) . For y < 0, we may take θ(x, y) to equal zero outside the reflected characteristics and to equal π inside them. It then follows that −1/6 z −1/6 = 9(x 2 − a 2 ) + 4y 3 − 12a(−y)3/2 as well as F

1 1 9(x 2 − a 2 ) + 4y 3 + 12a(−y)3/2 , ; 1; 6 6 9(x 2 − a 2 ) + 4y 3 − 12a(−y)3/2

are well defined in region R2 \(ra ∪ r−a ). The following remarks are in order: (1) For y = 0, we have 1 1 E(x, 0; 0, b) = F , , 1; 1 |x 2 − a 2 |−1/6 , ∀x ∈ (−a, a), 6 6 and 1 1 E(x, 0; 0, b) = eiπ/6 F , , 1; 1 (x 2 − a 2 )−1/6 , ∀x ∈ / (−a, a). 6 6 (2) For y > 0, we have z¯ = ρei2θ(x,y) , so |ζ | = 1. z It is known (see [5]) that the hypergeometric function F(1/6, 1/6; 1; ζ ) is then given by its hypergeometric series and that this series is absolutely convergent in the closed disk |ζ | ≤ 1. Along the characteristics ζ =

(3)

3(x − a) + 2(−y)3/2 = 0

and

3(x + a) − 2(−y)3/2 = 0

through point (0, b), function E(x, y; 0, b) is equal to E = 2−2/3 (by)−1/4 , (4)

and so it has a singularity of order −1/4 at y = 0. Along both reflected characteristics r−a and ra , function E(x, y; 0, b) has a logarithmic singularity. This follows from Proposition 4.1 in Section 4, which describes the asymptotic behavior of the hypergeometric function F(1/6, 1/6; 1; ζ ) as |ζ | → +∞.

570


3. The fundamental solution in region D I Let E(x, y; 0, b) be the function defined by expressions (2.8) and (2.9), and define ( 1 E(x, y; 0, b) in D I , 21/3 E I (x, y; 0, b) = (3.1) 0 elsewhere. Since E(x, y; 0, b) is C ∞ in D I and bounded on the boundary of D I , it follows that E I (x, y; 0, b) is a locally integrable function and defines a distribution whose support is the closure of D I . 3.1 E I (x, y; 0, b) is a fundamental solution for the Tricomi operator T relative to point (0, b). THEOREM

Proof We must show that hE I , T ϕi = ϕ(0, b),

∀ϕ ∈ Cc∞ (R2− ).

Since E(x, y; 0, b) is locally integrable in R2 , this is equivalent to showing that Z Z E I (x, y, ; 0, b)T ϕ(x, y) d x dy = ϕ(0, b), ∀φ ∈ Cc∞ (R2− ). (3.2) R2−

By introducing the characteristic coordinates ` and m, noting that in the new variables the Tricomi operator (1.1) becomes y

h ∂2 1/6 ∂ ∂ i ∂2 ∂2 2/3 2 2/3 − − + = −2 3 (` − m) ∂`∂m l − m ∂` ∂m ∂x2 ∂ y2

and that the Jacobian of the transformation is ∂(x, y) 1 = 1/3 2 , ∂(`, m) 2 3 (` − m)1/3 we can see that the integral in (3.2) is equal to Z −`0Z ∞ 1/6 1/6 − (` − m)1/3 E(`, m; `0 , −`0 ) φ`m − φ` + φm d` dm, `−m `−m −∞ `0 where φ(`, m) is ϕ(x, y) in characteristic coordinates (see Figure 3). We denote the last integral by I and write it as Z −`0 Z ∞ 1/6 1/6 φ` + φm d` dm (3.3) I =− R(`, m; `0 , −`0 ) φ`m − `−m `−m −∞ `0


571

m y (−a, 0)

(a, 0) x

`0 , −`0

(0, b)

`

DI

DI

` = `0 Figure 3

after recalling that (` − m)1/3 E(`, m; `0 , −`0 ) = R(`, m; `0 , −`0 ). Next we perform several integrations by parts and take into account the properties of the function R(`, m; `0 , m 0 ) with m 0 = −`0 , as stated in Proposition 2.1. First, consider the term in (3.3) that contains φ`m . Integrating by parts first with respect to m, say, and then with respect to `, one obtains Z

−`0

Z

∞

− −∞

`0

Rφ`m d` dm = φ(`0 , −`0 ) + +

1 6 Z

Z

−`0

Z

∞ `0

R(`, −`0 ; `0 , −`0 ) φ(`, −`0 ) d` ` + `0

R(`0 , m; `0 , −`0 ) φ(`0 , m) dm `0 − m

−∞ −`0 Z ∞

− −∞

1 6

`0

R`m φ d` dm.

(3.4)

Second, integrate by parts, relative to `, the term that contains φ` in (3.3), and obtain Z Z Z 1 −`0 ∞ R 1 −`0 R(`0 , m; `0 , −`0 ) φ` d` dm = − φ(`0 , m) dm 6 −∞ `0 ` − m 6 −∞ `0 − m Z Z 1 −`0 ∞ ∂ R − φ d` dm. (3.5) 6 −∞ `0 ∂` ` − m Finally, integrate by parts, relative to m, the term that contains φm in (3.3), and

572


get 1 − 6

Z

−`0

−∞

Z

∞

`0

R(`, −`0 ; `0 , −`0 ) φ(`, −`0 ) d` ` + `0 `0 Z Z 1 −`0 ∞ ∂ R φ d` dm. (3.6) + 6 −∞ `0 ∂m ` − m

R 1 = φm d` dm = − `−m 6

Z

∞

By adding (3.4), (3.5), and (3.6), we obtain I = φ(`0 , −`0 ), and this completes the proof. As a consequence of Theorem 3.1, we obtain the following result. COROLLARY 3.1 As (0, b) → (0, 0), the fundamental solution (3.1) converges, in the sense of distributions, to F− (x, y), the fundamental solution defined by (1.5) and (1.6).

Proof First, note that whenever Re(c − a − b) > 0, the value F(a, b; c; 1) is finite and we have 0(c)0(c − a − b) F(a, b; c; 1) = . 0(c − a)0(c − b) Thus F(1/6, 1/6; 1; 1) is well defined. As (0, b) → (0, 0), then (`0 , −`0 ) → (0, 0), and so the limit of E(`, m; `0 , −`0 ) defined by (2.6) and (2.7) is (−`m)−1/6 F

1 1 , ; 1; 1 . 6 6

On the other hand, at the limit, region D I coincides with D− , the region inside the two characteristics that emanate from the origin, where `m = 9x 2 + 4y 3 < 0. Remarks. (1) In [2], the multiplicative constant appearing in the definition of F− (x, y) was given by 30(4/3) C− = 2/3 1/2 . 2 π 0(5/6)

(2)

It is a matter of verification that this constant coincides with the one given by formula (1.6). Theorem 3.1 could have been proved in a different way, namely, by using Green’s formula (4.5) and by integrating along a suitable contour, as we did in [2]. This method, which is explained in the following section, is more adequate in proving existence of the fundamental solutions supported by the closure of regions D I I , D I I I , and D I V .


573

4. Fundamental solutions in regions D I I I and D I V For reasons of symmetry, it is enough to consider only region D I I I . This is the region bounded by part of the characteristic 3(x + a) − 2(−y)3/2 = 0, part of the characteristic 3(x − a) + 2(−y)3/2 = 0, and the reflected characteristic 3(x − a) − 2(−y)3/2 = 0 (see Figure 2). As we already pointed out, the function E(x, y; 0, b) is singular along the reflected characteristic 3(x − a) − 2(−y)3/2 = 0. The nature of the singularity depends on the behavior of the hypergeometric function F(1/6, 1/6; 1; ζ ), where ζ = (` − `0 )(m + `0 )/(` + `0 )(m − `0 ), along the characteristic m = `0 . This is revealed by the following result on the analytic continuation of the hypergeometric series F(a, a; c; ζ ) (see A. Erdélyi, W. Magnus, F. Oberhettinger, and F. Tricomi [5]). The same result also gives us the asymptotic behavior of the analytic extension as |ζ | → ∞. PROPOSITION 4.1 For ζ ∈ C with |arg(−ζ )| < π, we have

F(a, a; c; ζ ) = (−ζ )−a −ζ )u(ζ ) + v(ζ ) , where

(4.1)

∞

u(ζ ) =

X (a)n (1 − c + a)n 0(c) ζ −n 0(a)0(c − a) (n!)2

(4.2)

n=0

with (a)n = 0(a + n)/ 0(a), and ∞

v(ζ ) =

X (a)n (1 − c + a)n 0(c) h n ζ −n 0(a)0(c − a) (n!)2

(4.3)

n=0

with h n = 29(1 + n) − 9(a + n) − 9(c − a − n), 9(ζ ) = 0 0 (ζ )/ 0(ζ ). Moreover, series u(ζ ) and v(ζ ) both converge for |ζ | > 1. From this proposition it follows, as we show in the proof of Theorem 4.1, that the singularity of E(x, y; 0, b) along the reflected characteristics is logarithmic. We may now define ( 1 − 21/3 E(x, y; 0, b) in D I I I , E I I I (x, y; 0, b) = (4.4) 0 elsewhere, a distribution whose support is the closure of D I I I . We then have the following result. 4.1 E I I I (x, y; 0, b) is a fundamental solution for the Tricomi operator T relative to (0, b). THEOREM

574


Proof (1) We use Green’s formula for the Tricomi operator (see [2]): ZZ Z (E T ϕ − ϕ T E) d x dy = E(yϕx dy − ϕ y d x) − ϕ(y E x dy − E y d x), D

C

(4.5) where D is a bounded domain with smooth boundary C. If T E = 0 on D, then the last formula becomes ZZ Z E T ϕ d x dy = E(yϕx dy − ϕ y d x) − ϕ(y E x dy − E y d x) (4.6) D

C

and the contour integral may be used to evaluate the double integral. Now throughout this section the domain D = D I I I lies entirely in the hyperbolic region, and so it is more convenient to express the contour integral on the right-hand side of (4.6) in terms of characteristic coordinates. One can check that, in these coordinates, we have yϕx dy − ϕ y d x = A(` − m)1/3 ψ` d` − A(` − m)1/3 ψm dm, where ψ(`, m) = ϕ

` + m ` − m 2/3 ,− 6 4

and A = 1/22/3 . Similarly, y E x dy − E y d x = A(` − m)1/3 E ` d` − A(` − m)1/3 E m dm. Thus the contour integral in (4.6), denoted by IC , can be written as Z Z 1/3 IC = A (` − m) (Eψ` − ψ E ` ) d` − A (` − m)1/3 (Eψm − ψ E m ) dm. (4.7) C

C

(2) We must show that ZZ hE, T ϕi = lim E T ϕ d x dy = −21/3 ϕ(0, b), →0

D

∀ϕ ∈ Cc∞ (R2 ),

(4.8)

where D is the intersection of an open disk that contains the support of ϕ and the region, defined in characteristic coordinates, as follows: D I I I, = (`, m) : ` > `0 + , −`0 + < m < `0 − . Since ϕ ≡ 0 on the boundary of the disk, it follows from Green’s formula that we can replace the double integral in (4.8) by an integral of the form (4.7), where the contour C consists of the oriented line segments A¯ , B¯ , and C¯ shown in Figure 4.

FUNDAMENTAL SOLUTIONS FOR THE TRICOMI OPERATOR, II y

575

m

m = `0 − (−a, 0)

(a, 0)

B x

A `

C

m = −`0 +

(`0 , −`0 ) B

(0, b)

D I I I, A C

Figure 4

We start by evaluating the simplest of these integrals, namely, the ones along B¯ and C¯ . (3) The integral along B¯ . Denote by I B¯ the integral along the line segment ` = `0 + with m varying from `0 − to −`0 + . Taking into account the orientation of B¯ , we obtain from formula (4.7), Z `0 − I B¯ = A (`0 + − m)1/3 E(`0 + , m; `0 , −`0 )ψm (`0 + , m) dm −`0 + Z `0 −

−A

(`0 + − m)1/3 E m (`0 + , m; `0 , −`0 )ψ(`0 + , m) dm, (4.9)

−`0 +

where A = 1/22/3 . Integrating by parts the first integral in (4.9), we get `0 − I B¯ = A(`0 + − m)1/3 E(`0 + , m; `0 , −`0 )ψ(`0 + , m) −`0 + Z `0 − A (`0 + − m)−2/3 E(`0 + , m; `0 , −`0 )ψ(`0 + , m) dm − 3 −`0 + Z `0 − − 2A (`0 + − m)1/3 E m (`0 + , m; `0 , −`0 )ψ(`0 + , m) dm. −`0 +

(4.10) From equation (2.4) it follows that 1 Rm = − (` − m)−2/3 E(`, m; `0 , −`0 ) + (` − m)1/3 E m (`, m; `0 , −`0 ). 3

576


Substituting into (4.10), we obtain `0 − I B¯ = A(`0 + − m)1/3 E(`0 + , m; `0 , −`0 )ψ(`0 + , m) −`0 + Z `0 − h 1 R(`0 + , m; `0 , −`0 ) i Rm (`0 + , m; `0 , −`0 ) + − 2A 6 (`0 + − m) −`0 + × ψ(`0 + , m) dm.

(4.11)

Now, as → 0, the last integral tends to zero because, by Proposition 2.1, Rm = −R/6(` − m) along the line ` = `0 . On the other hand, the first term in (4.11), namely, `0 − A(`0 + − m)1/3 E(`0 + , m; `0 , −`0 )ψ(`0 + , m)

−`0 +

= A(2)1/3 E(`0 + , `0 − ; `0 , −`0 )ψ(`0 + , `0 − ) − A(2`0 )1/3 E(`0 + , −`0 + ; `0 , −`0 )ψ(`0 + , −`0 + ), tends to −A(2`0 )1/3 E(`0 , −`0 ; `0 , −`0 )ψ(`0 , −`0 ), as → 0. Therefore lim I B¯ =

→0

−1 ψ(`0 , −`0 ) 22/3

(4.12)

because E(`0 , −`0 ; `0 , −`0 ) = (2`0 )−1/3 and A = 1/22/3 . (4) The integral along C¯ . Denote by IC¯ the integral along the line m = −`0 + with `0 + < ` < +∞. From formula (4.7) we now obtain Z +∞ IC¯ = A (` + `0 − )1/3 E(`, −`0 + ; `0 , −`0 )ψ` (`, −`0 + ) d` `0 + Z +∞

−A

`0 +

(` + `0 − )1/3 E ` (`, −`0 + ; `0 , −`0 )ψ(`, −`0 + ) d`. (4.13)

As before, integrate the first integral by parts, use the formula R` =

1 (` − m)−2/3 E(`, m; `0 , −`0 ) + (` − m)1/3 E ` (`, m; `0 , −`0 ) 3

to substitute for E ` in the second integral, and rewrite (4.13) as IC¯ = −A(2`0 )1/3 E(`0 + , −`0 , −; `0 , −`0 )ψ(`0 + , −`0 + ) Z +∞ h 1 R(`, −`0 + ; `0 , −`0 ) i − 2A R` (`, −`0 + ; `0 , −`0 ) − 6 (` + `0 − ) `0 + × ψ(`, −`0 − ) d`.


577

As before, we obtain lim IC¯ =

→0

−1 ψ(`0 , −`0 ). 22/3

(4.14)

(5) The integral along A¯ . In the case when m = `0 − , ` varies from +∞ to `0 + , and the integral to be considered is Z +∞ I A¯ = −A (` − `0 + )1/3 E(`, `0 − ; `0 , −`0 )ψ` (`, `0 − ) d` `0 + +∞

Z +A

`0 +

(` − `0 + )1/3 E ` (`, `0 − ; `0 , −`0 )ψ(`, `0 − ) d`.

(4.15)

Our aim is to show that lim I A¯ = 0.

(4.16)

→0

Once this is done, by adding (4.12), (4.14), and (4.16), we obtain (4.8) and the theorem will be proved. As we did in the previous two cases, we integrate the first integral in (4.15) by parts and get I A¯ = A(2)1/3 E(`0 + , `0 − ; `0 , −`0 )ψ(`0 + , `0 − ) Z A +∞ + (` − `0 + )−2/3 E(`, `0 − ; `0 , −`0 )ψ(`, `0 − ) d` 3 `0 + Z +∞ + 2A (` − `0 + )1/3 E ` (`, `0 − ; `0 , −`0 )ψ(`, `0 − ) d`. `0 +

(4.17)

In order to prove (4.16), we now must take into account the asymptotic behavior of both E(`, m; `0 , −`0 ) and E ` (`, m; `0 , −`0 ) for m near `0 and, more specifically, the behavior of the hypergeometric function F(1/6, 1/6; 1; ζ ) and its derivative 1 1 1 7 7 F 0 , ; 1; ζ = F , ; 2; ζ , (4.18) 6 6 36 6 6 as ζ → ∞. To see this, we start by expressing (4.17) in terms of F(1/6, 1/6; 1; ζ ) and its derivative. In order to simplify notation, from now on we write F(ζ ) = F(1/6, 1/6; 1; ζ ) and G(ζ ) = F(7/6, 7/6; 2; ζ ). In the case we are considering, m = `0 − , so in view of formula (2.7), we set ζ =

σ () (2`0 − )(` − `0 ) = −(` + `0 ) −

with σ () =

(2`0 − )(` − `0 ) . (` + `0 )

Also, recalling formula (2.6), we have E(`, `0 − ; `0 , −`0 ) = −1/6 (` + `0 )−1/6 F

σ () −

,

(4.19)

578


and so

E ` (`, `0 − ; `0 , −`0 ) =−

σ () −7/6 ` (2` − ) σ () −1/6 0 0 (` + `0 )−7/6 F − (` + `0 )−13/6 G 6 − 18 − (4.20)

in view of (4.18) and taking into account the fact that dζ 2`0 (2`0 − ) = . d` −(` + `0 )2 By substituting (4.19) and (4.20) in (4.17) and combining the resulting integrals, we obtain − 2` 0 I A¯ = A(2)1/3 (2`0 + )−1/6 −1/6 F ψ(`0 + , `0 − ) 2`0 + Z +∞ A (` − `0 + )−2/3 σ () 1/6 + (2`0 − )5/6 1/6 (` + ` ) 3 0 `0 + (` − `0 ) σ () ×F ψ(`, `0 − ) d` − Z +∞ A (` − `0 + )1/3 σ () 7/6 − `0 (2`0 − )−1/6 7/6 (` + ` ) 9 0 `0 + (` − `0 ) σ () ψ(`, `0 − ) d`. (4.21) ×G − As → 0, the first term on the right-hand side of (4.21) clearly tends to zero. We must prove that the difference of the two integrals in (4.21) also tends to zero as → 0. For this, according to formulas (4.1), (4.2), and (4.3), we have σ () 1/6 σ () σ () σ () σ () F = log u +v − − − and

σ () 7/6 σ () σ () σ () σ () G = log U +V . − − − Using these two expressions, we combine the two integrals in (4.21) and write them

FUNDAMENTAL SOLUTIONS FOR THE TRICOMI OPERATOR, II (1)

(2)

(3)

579

as a sum I A¯ + I A¯ − ·I A¯ , where (1)

I A¯ =

(2)

I A¯ =

Z +∞ (` − `0 + )−2/3 A (2`0 − )5/6 1/6 (` + ` ) 3 0 `0 + (` − `0 ) σ () ψ(`, `0 − ) d` × (σ () u − Z +∞ A (` − `0 + )1/3 − `0 (2`0 − )−1/6 7/6 (` + ` ) 9 0 `0 + (` − `0 ) σ () × (σ () U ψ(`, `0 − ) d`, −

(4.22)

(` − `0 + )−2/3 σ () v ψ(`, `0 − ) d` 1/6 (` + ` ) − 0 `0 + (` − `0 ) Z +∞ σ () A (` − `0 + )1/3 −1/6 − `0 (2`0 − ) ψ(`, `0 − ) d`, V 7/6 (` + ` ) 9 − 0 `0 + (` − `0 ) (4.23) A (2`0 − )5/6 3

Z

+∞

and (3)

I A¯ =

(` − `0 + )−2/3 σ () ψ(`, `0 − ) d` u 1/6 (` + ` ) − 0 `0 + (` − `0 ) Z +∞ σ () A (` − `0 + )1/3 − `0 (2`0 − )−1/6 U ψ(`, `0 − ) d`. 7/6 (` + ` ) 9 − 0 `0 + (` − `0 ) (4.24) A (2`0 − )5/6 3

Z

+∞

LEMMA 4.1 (i) Each of the integrals I A¯ , 1 ≤ i ≤ 3, is O( 1/6 ).

Proof (1) Consider first the integral I A¯ . Since the series in (4.2) converge for large values of |ζ |, we may write u

σ () −

= u0 +

σ () u˜ σ () −

and

U

σ () −

= U0 +

where u˜ and U˜ are bounded functions for small values of .

˜ σ () U , σ () −

580


By substituting these two expressions in (4.22), rewrite it as a sum of three terms: (1) I A¯

=

(` − `0 + )−2/3 (σ () ψ(`, `0 − ) d` 1/6 3 (` + `0 ) `0 + (` − `0 ) Z +∞ A (` − `0 + )1/3 − `0 (2`0 − )−1/6 U0 7/6 (` + ` ) 9 0 ` + (` − `0 ) o0 × (σ () ψ(`, `0 − ) d` Z +∞ A (` − `0 + )−2/3 5/6 + (2`0 − ) 1/6 (` + ` ) 3 0 `0 + (` − `0 ) σ () u˜ ψ(`, `0 − ) d` × (σ () σ () − Z +∞ A (` − `0 + )1/3 − `0 (2`0 − )−1/6 7/6 (` + ` ) 9 0 `0 + (` − `0 ) σ () × (σ () U˜ ψ(`, `0 − ) d`. (4.25) σ () − nA

(2`0 − )

5/6

Z

+∞

u0

The first term (inside the brackets) in (4.25) tends to zero as → 0. Indeed, its limit as → 0 is Z h A(2` )5/6 u A`0 (2`0 )−1/6 U0 i ∞ (` − `0 )−5/6 2`0 (` − `0 ) 0 0 − ( ψ(`, `0 ) d`. 3 9 ` + `0 ` + `0 `0 The integral on the right-hand side is finite. On the other hand, u0 =

1 0(1/6)0(5/6)

and

U0 =

1 6 = 0(5/6)0(7/6) 0(1/6)0(5/6)

are the constant terms of the series u(ζ ) and U (ζ ) given by (4.2). It is a matter of verification that the quantity inside the brackets equals zero. Next, consider the second term in (4.25). After the change of variables ` − `0 = t, one can see that the absolute value of that term is bounded above by Z +∞ C 1/6 (t + 1)−2/3 t −7/6 dt 1

with C a constant independent of . With a similar calculation, one can show that the third term in (4.25) is bounded above by Z +∞ 1/6 C (t + 1)1/3 t −13/6 dt 1 (1)

with C another constant. Therefore I A¯ = O( 1/6 ).

FUNDAMENTAL SOLUTIONS FOR THE TRICOMI OPERATOR, II (3)

581

(1)

Expression (4.24) for I A¯ , similar to that of I A¯ , is even simpler because of the absence of the factor σ ()). Thus, with a similar proof, we also conclude that (3) (2) (3) I A¯ = O( 1/6 ). Finally, the integral I A¯ given by (4.23) is analogous to I A¯ with u and U replaced by v and V the power series defined by (4.3). The proof of Lemma 4.1 is then complete. Recalling expression (4.22) for I A¯ , one sees that Lemma 4.1 implies (4.16), which completes the proof of Theorem 4.1. 5. Fundamental solutions in region D I I In the same manner as in the previous sections, define the distribution ( 1 E(x, y; 0, b) in D I I , E I I (x, y; 0, b) = 21/3 0 elsewhere,

(5.1)

whose support is the closure of region D I I . Our aim is to prove the following result. 5.1 E I I (x, y; 0, b) is a fundamental solution for the Tricomi operator relative to point (0, b). THEOREM

Proof The proof is, with few modifications, analogous to that of Theorem 4.1. As before, it suffices to show that ZZ hE, T ϕi = lim E T ϕ d x dy = 21/3 ϕ(0, b), ∀ϕ ∈ Cc∞ (R2 ), (5.2) →0

D

where the domain of integration D is defined as follows. Let D be an open disk centered at the origin and with radius sufficiently large so that it contains points (−a, 0), (a, 0), and (0, b) and the the support of ϕ. Let D I I, be the region obtained from D I I by shifting it by along the characteristics and by removing two small half-disks centered at points (−a, 0), (a, 0), as shown in Figure 5. D is then the intersection of D and D I I, . By virtue of Green’s formula (4.6), the double integral in expression (5.2) is to be replaced by a contour integral along the oriented paths A0 , γ0 , B0 , B , γ , and A . The integration along A is similar to the one along A¯ calculated in Section 4, and one can see that its limit, as → 0, is zero. The same is true for the integral along A0 . The integral along B , like the integral along B¯ (see Section 4), tends to ψ(`0 , −`0 )/22/3 as → 0. The same thing happens with the integral along B0 . The

582

BARROS-NETO AND GELFAND m

D γ0 A0

γ B0

B

`

A

m = `0 +

m = −`0 + ` = −`0

m = `0

Figure 5

sum of these two limits is then 21/3 ψ(`0 , −`0 ), which is equal to the right-hand side of (5.2). Recall that ψ(`, m) = ϕ (` + m)/6, −((` − m)/4)2/3 . To complete the proof, one has to show that the limits, as → 0, of the integrals along γ0 and γ are both zero. Since these two contours lie in the elliptic region of the Tricomi operator, it is more convenient to use the reduced elliptic form of the Tricomi operator, namely, ∂2 ∂2 1 ∂ Te = + + , (5.3) 2 2 3s ∂s ∂x ∂s which one obtains from equation (1.1) via the change of variables x = x and s = 2y 3/2 /3, and the corresponding Green’s formula in the variables x and s. It is then a matter of verification, which we leave to the reader, that along both contours the integrands remain bounded and, consequently, both integrals along γ0 and γ tend to zero. The proof is then complete. We observe that, by virtue of Proposition 2.2, the fundamental solution E I I (x, y; 0, b) is complex valued in the upper half-plane (y > 0) plus the region in the lower halfplane outside the reflected characteristics, and it is real valued in the triangle with vertices (−a, 0), (a, 0), and (0, b). Since the Tricomi operator has real coefficients, the complex conjugate E¯ I I (x, y; 0, b) and the real part of E I I (x, y; 0, b) are fundamental solutions for T .


583

Contrary to what happened in Corollary 3.1 of Section 4, where we showed how the limit of E I (x, y; 0, b), as (0, b) → (0, 0), tends to the fundamental solution F− (x, y) given by formulas (1.5) and (1.6), neither of the fundamental solutions just obtained tends to the fundamental solution F+ (x, y) given by formulas (1.3) and (1.4). For this to happen, we need a particular linear combination of E I I and E¯ I I . Let λ and µ be such that ( 1 , λe(iπ )/6 + µe−((iπ )/6) = − 31/2 λ + µ = 1. Then the following result holds. COROLLARY 5.1 ] Let E I I = λE I I + µ E¯ I I . Then

]

E I I is a fundamental solution relative to (0, b) which converges, in the sense of distributions, to F+ (x, y) as (0, b) → (0, 0).

Acknowledgments. We would like to thank Abbas Bahri, Fernando Cardoso, and Vladimir Retakh for several helpful discussions. References [1]

[2] [3] [4]

[5]

[6] [7] [8]

[9]

S. AGMON, “Boundary value problems for equations of mixed type” in Convegno

Internazionale sulle Equazioni Lineari alle Derivate Parziali (Trieste, 1954), Edizioni Cremonese, Rome, 1955, 54–68. MR 17:859a 563 J. BARROS-NETO and I. M. GELFAND, Fundamental solutions for the Tricomi operator, Duke Math. J. 98 (1999), 465–483. MR 2000e:35162 561, 563, 566, 572, 574 R. COURANT and D. HILBERT, Methods of Mathematical Physics, Vol. I, Interscience, New York, 1953. MR 16:426a 568 G. DARBOUX, Leçons sur la théorie générale des surfaces, Vol. II, 2d ed., reprint, ´ Jacques Gabay, Sceaux, France, 1993. Grands Class. Gauthier-Villars, Ed. MR 97c:01046a ´ A. ERDELYI, W. MAGNUS, F. OBERHETTINGER, and F. G. TRICOMI, Higher Transcendental Functions, Vols. I, II, McGraw-Hill, New York, 1953, MR 15:419i; Vol. III, 1955. MR 16:586c 569, 573 K. O. FRIEDRICHS, Symmetric positive linear differential equations, Comm. Pure Appl. Math. 11 (1958), 333–418. MR 20:7147 563 I. M. GELFAND and G. E. SHILOV, Generalized Functions, Vol. I: Properties and Operations, Academic Press, New York, 1964. MR 29:3869 P. GERMAIN and R. BADER, Solutions e´ lémentaires de certaines e´ quations aux dérivées partielles du type mixte, Bull. Soc. Math. France 81 (1953), 145–174. MR 15:432b 563 L. D. LANDAU and E. M. LIFSCHITZ, Fluid Mechanics, Pergamon, London, 1959. MR 21:6839 563

584

[10] [11]

[12] [13] [14]


J. LERAY, La solution unitaire d’un opérateur différentiel linéaire (Problème de

Cauchy, II), Bull. Soc. Math. France 86 (1958), 75–96. MR 22:12309 , Un prolongementa de la transformation de Laplace qui transforme la solution unitaires d’un opérateur hyperbolique en sa solution e´ lémentaire (Problème de Cauchy, IV), Bull. Soc. Math. France 90 (1962), 39–156. MR 26:1625 563, 566 C. S. MORAWETZ, A weak solution for a system of equations of elliptic-hyperbolic type, Comm. Pure Appl. Math. 11 (1958), 315–331. MR 20:3375 563 L. SCHWARTZ, Théorie des distributions, Vol I, Hermann, Paris, 1950, MR 12:31d; Vol. II, 1951. MR 12:833d F. TRICOMI, Sulle equazioni lineari alle derivate parziali di secondo ordine di tipo misto, Atti Accad. Naz. Lincei Cl. Sci. Fis. Mat. Natur. Rend. Lincei (5) 14 (1923), 134–247.

Barros-Neto Rutgers University, Hill Center, Piscataway, New Jersey 08954-8019, USA; [email protected] Gelfand Rutgers University, Hill Center, Piscataway, New Jersey 08954-8019, USA; [email protected]

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

Recommend Documents