Lie Theory: Unitary Representations and Compactifications of Symmetric Spaces (Progress in Mathematics)

Lie Theory Unitary Representations and Compactifications of Symmetric Spaces Jean-Philippe Anker Bent Orsted Editors B...

Author: Jean-Philippe Anker | Bent Orsted

22 downloads 509 Views 2MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Lie Theory Unitary Representations and Compactifications of Symmetric Spaces

Jean-Philippe Anker Bent Orsted Editors

Birkhäuser Boston • Basel • Berlin

Jean-Philippe Anker Laboratoire de Mathématiques (MAPMO) Université d’Orléans F-45067 Orléans Cedex 2 France

Bent Orsted Department of Mathematics and Computer Science University of Southern Denmark DK-5230 Odense M Denmark

AMS Subject Classifications: Primary: 53C35, 22E40, 22E46; Secondary: 32M15, 31C35, 32J05, 20G30, 43A85, 53C50, 53D20 Library of Congress Cataloging-in-Publication Data Lie theory : unitary representations and compactifications of symmetric spaces / Jean-Philippe Anker, Bent Orsted, editors. p. cm. – (Progress in mathematics ; v. 229) Includes bibliographical references. ISBN 0-8176-3526-2 (alk. paper) 1. Symmetric spaces. 2. Lie theory. I. Anker, Jean-Philippe. II. Orsted, Bent. III. Progress in mathematics (Boston, Mass.) ; v. 229. QA649.L5 2004 516.3’62–dc22

ISBN 0-8176-3526-2

2004045091

Printed on acid-free paper.

c 2005 Birkhäuser Boston

All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Birkhäuser Boston, c/o Springer Science+Business Media Inc., Rights and Permissions, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed in the United States of America. 987654321 www.birkhauser.com

SPIN 10944370

(TXQ/HP)

Contents

Preface Introduction to Symmetric Spaces and Their Compactifications Lizhen Ji 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 The Poincaré disc D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 The bidisc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Symmetric spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

ix 1 1 2 24 32 66

Compactifications of Symmetric and Locally Symmetric Spaces Armand Borel and Lizhen Ji 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Geometry of symmetric spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Satake and Furstenberg compactifications . . . . . . . . . . . . . . . . . . . . . . . SF 5 Alternative constructions of X max . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Martin compactifications and Karpelevic compactification . . . . . . . . . 7 Geometry of locally symmetric spaces . . . . . . . . . . . . . . . . . . . . . . . . . 8 Compactifications of locally symmetric spaces . . . . . . . . . . . . . . . . . . 9 Satake compactifications of \X . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Compactifications in which all open orbits are isomorphic . . . . . . . . . 11 Real forms and real points of complex orbits . . . . . . . . . . . . . . . . . . . . 12 The wonderful compactification of G c /K c and its real points . . . . . . 13 The Oshima–Sekiguchi compactification of G/K . . . . . . . . . . . . . . . . 14 The wonderful compactification of G c /Hc and its real points . . . . . . 15 Appendix: Galois cohomology and real forms . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

69 69 70 74 79 86 92 99 102 109 113 118 121 125 128 133 135

Restrictions of Unitary Representations of Real Reductive Groups T. Kobayashi 139 1 Reductive Lie groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 1.1 Smallest objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

vi

Contents

2

3

4

1.2 General linear group G L(N , R) . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Cartan decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Reductive Lie groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5 Examples of reductive Lie groups . . . . . . . . . . . . . . . . . . . . . . . 1.6 Inclusions of groups and restrictions of representations . . . . . Unitary representations and admissible representations . . . . . . . . . . . 2.1 Continuous representations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Unitary representations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Admissible restrictions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.1 (Unitarily) discretely decomposable representations . . . . . . . 2.4.2 Admissible representations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.3 Gelfand–Piateski-Shapiro’s theorem . . . . . . . . . . . . . . . . . . . . . 2.4.4 Admissible restrictions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.5 Chain rule of admissible restrictions . . . . . . . . . . . . . . . . . . . . . 2.4.6 Harish-Chandra’s admissibility theorem . . . . . . . . . . . . . . . . . 2.4.7 Further readings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . S L(2, R) and Branching Laws . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Branching Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.1 Direct integral of Hilbert spaces . . . . . . . . . . . . . . . . . . . . . . . . 3.1.2 Irreducible decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.4 Branching problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Unitary dual of S L(2, R) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 S L(2, R)-action on P1 C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 Unitary principal series representations . . . . . . . . . . . . . . . . . . 3.2.3 Holomorphic discrete series representations . . . . . . . . . . . . . . 3.2.4 Restriction and proof for irreducibility . . . . . . . . . . . . . . . . . . . 3.3 Branching laws of S L(2, R) . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 Subgroups of S L(2, R) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.2 Branching laws G ↓ K , A, N . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.3 Restriction of holomorphic discrete series . . . . . . . . . . . . . . . . 3.3.4 Irreducibility of πn+ (n = 2, 3, 4, . . . ) . . . . . . . . . . . . . . . . . . . 3.4 ⊗-product representations of S L(2, R) . . . . . . . . . . . . . . . . . . 3.4.1 Tensor product representations . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.2 πλ ⊗ πλ (principal series) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.3 πm+ ⊗ πn+ (holomorphic discrete series) . . . . . . . . . . . . . . . . . . (g, K )-modules and infinitesimal discrete decomposition . . . . . . . . . . 4.1 Category of (g, K )-modules . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.1 K -finite vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.2 Underlying (g, K )-modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.3 (g, K )-modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.4 Infinitesimally unitary representations . . . . . . . . . . . . . . . . . . . 4.1.5 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.6 For further reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

140 140 141 142 144 145 145 145 145 146 146 146 146 147 147 147 148 148 148 148 149 150 150 150 150 151 151 152 153 153 153 154 155 155 155 156 157 158 158 158 158 159 159 160 160

Contents

4.2 4.2.1 4.2.2 4.2.3 4.2.4 4.2.5 4.2.6

5

6

Infinitesimal discrete decomposition . . . . . . . . . . . . . . . . . . . . Wiener subspace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Discretely decomposable modules . . . . . . . . . . . . . . . . . . . . . . Infinitesimally discretely decomposable representations . . . . Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Unitary case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Infinitesimal discrete decomposability ⇒ discrete decomposability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.7 K -admissibility ⇒ infinitesimally discrete decomposability 4.2.8 For further reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Algebraic theory of discretely decomposable restrictions . . . . . . . . . . 5.1 Associated varieties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1.1 Graded modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1.2 Associated varieties of g-modules . . . . . . . . . . . . . . . . . . . . . . . 5.1.3 Associated varieties of G-representations . . . . . . . . . . . . . . . . 5.1.4 Nilpotent cone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Restrictions and associated varieties . . . . . . . . . . . . . . . . . . . . . 5.2.1 Associated varieties of irreducible summands . . . . . . . . . . . . . 5.2.2 G : compact case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.3 Criterion for infinitesimally discretely decomposable restrictions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.1 Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.2 K C -orbits on NpC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.3 Interpretations from representation theory . . . . . . . . . . . . . . . . 5.3.4 Case (G, G 1 ) = (U (2, 2), Sp(1, 1)) (essentially, (S O(4, 2), S O(4, 1))) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.5 Case (G, G 2 ) = (U (2, 2), U (2, 1) × U (1)) . . . . . . . . . . . . . . 5.3.6 Case (G, G 3 ) = (U (2, 2), U (2) × U (2)) . . . . . . . . . . . . . . . . Admissible restriction and microlocal analysis . . . . . . . . . . . . . . . . . . . 6.1 Hyperfunction characters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.1 Finite group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.2 Dirac’s delta function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.3 Schwartz’s distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.4 Sato’s hyperfunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.5 Distributions or hyperfunctions . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.6 Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Asymptotic K -support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.1 Asymptotic cone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.2 Cartan–Weyl highest weight theory . . . . . . . . . . . . . . . . . . . . . 6.2.3 Asymptotic K -support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.4 Examples from S L(2, R) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Criterion for admissible restriction . . . . . . . . . . . . . . . . . . . . . . 6.3.1 The closed cone C K (K ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.2 Symmetric pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

vii

160 160 161 161 162 162 163 163 163 163 164 164 164 165 166 166 167 168 168 168 169 169 171 172 173 174 174 174 175 175 175 176 176 177 178 178 179 179 180 180 180 180

viii

Contents

6.3.3 Criterion for K -admissibility . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.4 Sufficient condition for G -admissibility . . . . . . . . . . . . . . . . . 6.3.5 Remark . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Application of symplectic geometry . . . . . . . . . . . . . . . . . . . . . 6.4.1 Hamiltonian action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.2 Affine varieties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.3 Cotangent bundle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Discretely decomposable restriction of Aq(λ) . . . . . . . . . . . . . . . . . . . 7.1 Elliptic orbits and geometric quantization . . . . . . . . . . . . . . . . 7.1.1 Elliptic orbits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.2 Complex structure on an elliptic orbit . . . . . . . . . . . . . . . . . . . 7.1.3 Elliptic coadjoint orbit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.4 Geometric quantization a` la Schmid–Wong . . . . . . . . . . . . . . . 7.1.5 Borel–Weil–Bott theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.6 Discrete series representations . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.7 Harish-Chandra’s discrete series representations . . . . . . . . . . 7.1.8 Discrete series representations for symmetric spaces . . . . . . . 7.2 Restriction of (λ) attached to elliptic orbits . . . . . . . . . . . . . 7.2.1 Asymptotic K -support, associated variety . . . . . . . . . . . . . . . . 7.2.2 Restriction to a symmetric subgroup . . . . . . . . . . . . . . . . . . . . 7.3 U (2, 2) ↓ Sp(1, 1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.1 Non-holomorphic discrete series representations for U (2, 2) 7.3.2 Criterion for K -admissibility for U (2, 2) ↓ Sp(1, 1) . . . . . . 7.3.3 Associated variety of (λ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Applications of branching problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1 Understanding representations via restrictions . . . . . . . . . . . . 8.1.1 Analysis and Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1.2 Cartan–Weyl highest weight theory, revisited . . . . . . . . . . . . . 8.1.3 Vogan’s minimal K -type theory . . . . . . . . . . . . . . . . . . . . . . . . 8.1.4 Restrictions to noncompact groups . . . . . . . . . . . . . . . . . . . . . . 8.2 Construction of representations of subgroups . . . . . . . . . . . . . 8.2.1 Finite-dimensional representations . . . . . . . . . . . . . . . . . . . . . . 8.2.2 Highest weight modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.3 Small representations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3 Branching problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4 Global analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4.1 Global analysis and restriction of representations . . . . . . . . . . 8.4.2 Discrete series and admissible representations . . . . . . . . . . . . 8.5 Discrete groups and restriction of unitary representations . . . 8.5.1 Matsushima–Murakami’s formula . . . . . . . . . . . . . . . . . . . . . . 8.5.2 Vanishing theorem for modular varieties . . . . . . . . . . . . . . . . . 8.5.3 Clifford–Klein problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

181 181 182 182 182 183 183 184 184 184 185 186 187 188 188 189 189 190 190 190 191 191 192 192 194 194 194 194 195 195 196 196 196 197 198 198 198 199 200 200 201 202 202

Preface

Unitary Representations and Compactifications is the second volume in a threevolume series: Lie Theory, aimed at presenting the central role played by semisimple Lie groups in much of currently active mathematics. These groups, and their algebraic analogues over other fields than the reals, are of fundamental importance in geometry, number theory, and mathematical physics. In this volume by A. Borel, L. Ji, and T. Kobayashi, the focus is on two fundamental questions in the theory of semisimple Lie groups, namely (1) the geometry of Riemannian symmetric spaces and their compactifications, and (2) branching laws for unitary representations, i.e., restricting unitary representations to (typically, but not exclusively, symmetric) subgroups and decomposing the ensuing representations into irreducibles. Both topics are important for their connections to several other mathematical fields. L. Ji presents a very thorough introduction to symmetric spaces and their compactifications, using carefully selected examples to illustrate the general theory to follow. This chapter contains a discussion of Satake and Furstenberg boundaries, and a survey of the geometry of Riemannian symmetric spaces in general, including orthogonal involutive Lie algebras, and root structure. It serves as a good background for the next chapter by A. Borel and L. Ji entitled Compactifications of Symmetric and Locally Symmetric Spaces, which is an authoritative treatment of the various types of compactifications that are useful for the study of symmetric and locally symmetric spaces, namely: (a) the geodesic compactification, (b) the Satake compactification, (c) the Furstenberg compactification, (d) the Martin compactification, and (e) the Karpelevic compactification. Corresponding to arithmetic subgroups locally symmetric spaces are introduced together with their geodesic compactifications, and the Borel–Serre compactifications. Other compactifications are constructed as well, and the main point is that these are important tools in studying both the cohomology of the locally symmetric spaces and the cohomology of the arithmetic subgroups. In the last part of the work we find constructions of Oshima, DeConcini, Procesi, and Melrose — motivated in part by other questions, namely complex and projective geometry, and partial differential equations — demonstrating the wide applicability of techniques of compactification.

x

Preface

T. Kobayashi treats the subject of Restrictions of Unitary Representations of Real Reductive Groups. This chapter provides the necessary background for understanding a currently active area of representation theory for Lie groups, namely that of branching laws. It gives many concrete examples, and introduces important concepts from modern representation theory, such as Harish-Chandra modules, associated varieties, microlocal analysis, derived functor modules, and geometric quantization, thus serving this purpose as well. The key definition is that of admissibility, meaning that the restriction decomposes discretely (i.e., with discrete spectrum of the subgroup) with finite multiplicity of each summand. This is both analytically and algebraically a very interesting situation, and it is studied in detail in this work. Several criteria are found for this to happen, and some surprising connections to other phenomena are discussed at the end, namely the existence of discrete series representations on affine symmetric spaces, the vanishing of modular symbols, and the existence of Clifford–Klein forms of symmetric spaces of non-Riemannian type; this gives a natural generalized setting for the questions discussed by Borel and Ji, and one might expect this to become an active research area. This volume is well suited for graduate students in semisimple Lie theory and neighboring fields, and also for researchers who wish to learn about some current core areas and applications of semisimple Lie theory. Prerequisites are limited to basic familiarity with semisimple Lie groups and symmetric spaces, and also basic representation theory of Lie groups. One may find these in for example Helgason’s book, Differential Geometry and Symmetric Spaces, and in Knapp’s book Representation Theory of Semisimple Groups. An overview based on examples, as well as Lie Groups Beyond an Introduction. Jean-Philippe Anker Bent Orsted Editors November 2004

Introduction to Symmetric Spaces and Their Compactifications Lizhen Ji∗ Department of Mathematics, University of Michigan, Ann Arbor, MI 48109 [email protected]

1 Introduction Symmetric spaces form an important class of Riemannian manifolds and arise in many fields through their relations to Lie groups. In this chapter we give a selfcontained introduction to symmetric spaces and study their compactifications using two examples, the Poincaré disc and the bidisc. The Poincaré disc is the most basic symmetric space and typical of symmetric spaces of rank 1, while the bidisc exhibits many of the difficulties of higher rank symmetric spaces. For compactifications of general symmetric and locally symmetric spaces, see [BJ4]. This chapter is organized as follows. In §2, we study the example of the Poincaré disc and examine many different constructions of compactifications all of which coincide with the closed disc. In §3, we study the example of the bidisc, which is one of the basic examples of higher rank symmetric spaces. The purpose of this section is to point out differences and difficulties which arise in carrying out these constructions in the higher rank case. In §4, we give an introduction to symmetric spaces, covering various geometric and group-theoretic aspects. The role of Lie groups and Lie algebras is emphasized here. Acknowledgments and further notes. This chapter is based on lectures given at the University of Hong Kong in the summers of 1998 and 1999 and prepared for the European Summer School in Group Theory at CIRM, Marseille-Luminy, July 2001. I appreciated the invitations and hospitality of A. Borel and N. Mok in Hong Kong and J.-P. Anker and P. Torasso for their invitation to the summer school. Section 4 makes reference to [Bo2] and is useful for understanding this section. I would also like to thank J. Lott for helpful comments on earlier drafts of this chapter. ∗ Partially Supported by NSF grants and an Alfred P. Sloan Research Fellowship.

2

L. Ji

2 The Poincaré disc D Let X be a complete Riemannian manifold. For every point x ∈ X , there is a local symmetry i x defined on normal neighborhoods of x such that i x reverses the geodesics passing through x. Clearly, i x satisfies i x2 = id, and x is an isolated fixed point of i x . Definition 2.1. A Riemannian manifold X is called a locally symmetric space if for every point x ∈ X , the local symmetry i x is a local isometry, and called a symmetric space if it is locally symmetric and every local isometry i x extends to a global isometry of X . It follows easily from the definition that the Euclidean space Rn and the unit sphere S n are symmetric spaces. Another example is the unit disc with the Poincaré metric |dz|2 D = {z ∈ C | |z| < 1}, ds 2 = . (1 − |z|2 )2 ab Lemma 2.2. The group SU (1, 1) = { ¯ | a, b ∈ C, |a|2 − |b|2 = 1} acts b a¯ isometrically and transitively on D by setting az + b ab z= . ¯ + a¯ b¯ a¯ bz The stabilizer of the origin z = 0 is the subgroup a0 U (1) = | a ∈ C, |a|2 = 1 , 0 a¯ and hence D ∼ = SU (1, 1)/U (1) under the map z = g · 0 → gU (1). Proof. This follows by a direct computation.

Lemma 2.3. The Poincaré disc is a symmetric space. Proof. The symmetry at the origin is given by z → −z and is defined globally on D. It clearly preserves the length element ds 2 and is an isometry. By the isometric, transitive action of SU (1, 1), the symmetries at other points are also global isometries. The spaces Rn , S n , D represent the three types of symmetric spaces: flat (or Euclidean), compact, and noncompact. In this chapter we are mainly concerned with symmetric spaces of noncompact type, which are simply connected and negatively curved. The Poincaré disc is the most basic example of a symmetric space of noncompact type. As mentioned earlier, one purpose of this chapter is to study compactifications of symmetric spaces of noncompact type. In this section we examine many different constructions of the same compactification of D. It can be shown that generalizations

Introduction to Symmetric Spaces and Their Compactifications

3

of these methods to higher rank symmetric spaces lead to different compactifications (see [BJ4] and [GJT]). By definition, D → C, and the closure of D is the closed unit disc D = D ∪ S1.

(1)

Alternatively, let P1 be the complex projective line, P1 = C∪{∞}. Then D is also the closure of D in P1 . Since P1 is compact, the closure of D is automatically compact. One important reason for completing C to P1 is to extend the SU (1, 1)-action on D to D. Recall that P1 = {(z 1 , z 2 )}/ ∼, where (z 1 , z 2 ) = 0, (z 1 , z 2 ) ∼ (w1 , w2 ) if z 1 w2 = z 2 w1 , i.e., z 1 , z 2 are the homogeneous coordinates of P1 . The group SU (1, 1) acts on P1 by ab az 1 + bz 2 z1 = . z2 ba bz 1 + az 2 Proposition 2.4. The embedding D → P1 is SU (1, 1)-equivariant, and the SU (1, 1)-action on D extends to a continuous action on D. Proof. Identify P1 with C ∪ {∞}. Then it can be checked easily that the SU (1, 1)action on P1 has three orbits: D, the unit circle S 1 , and the exterior of the unit circle. Since z ∈ D is identified with (z, 1) ∈ P1 , it is clear that the isometric action of SU (1, 1) on D is the restriction of the SU (1, 1)-action on P1 , and hence extends continuously to D = D ∪ S 1 . We remark that SU (1, 1) does not preserve the subspace C of P1 , and hence the extension of the embedding D → C to D → P1 is natural from the point view of the group action. For a symmetric space X , let G be the identity component of its isometry group I s(X ). It follows easily from the definition that G acts transitively on X (see Lemma 4.2 below). Let x0 ∈ X be a basepoint and K the stabilizer of x0 in G. Then X = G/K . The above proposition suggests the following approach to compactifications of X . Approach 2.5. For a symmetric space X = G/K , find a compact G-space Z , and a basepoint z 0 ∈ Z whose stabilizer is K and hence a G-equivariant embedding i : X → Z ,

gK → gz 0 .

Then the closure of i(X ) in Z gives a compactification X , and the G-action on X extends continuously to X . This procedure depends crucially on finding the ambient G-space Z and the equivariant embedding X → Z . In the above example of X = D, C is not a SU (1, 1)-space, and we need to complete C to P1 in order to get an ambient SU (1, 1)-space. Once such a compactification of X is constructed, an important problem is to understand it intrinsically, i.e., to relate it to compactifications constructed using the following method.

4

L. Ji

Approach 2.6. Define the boundary points and the topology of the compactification using geometric properties of X and group theoretical properties of G. By the geometric properties of X , we mean structures of geodesics and convergence of sequences of points in X in terms of distance together with other geometric properties; and by the group-theoretical properties, we mean parabolic subgroups of G and their induced decompositions of X , which describe the structure at infinity of X. A natural question for compactifications constructed by both approaches is to ask what properties the compactified spaces should have. Structures such as the group action should be preserved. Symmetric spaces are real analytic manifolds. One can ask whether the analytic structure can be extended to the compactifications, for example, whether the compactified spaces are real analytic manifolds with boundary, or corners, and if they are, how they can be embedded into closed analytic manifolds. When X = D, D is a real-analytic manifold with boundary. Besides the above embedding of D into P1 as a real analytic submanifold with boundary through the embedding of C, it can also be embedded into the sphere S 2 = P1 by viewing S 2 as the doubling of D across the boundary S 1 . Similar results hold for nontrivial compactifications of symmetric spaces of rank 1. On the other hand, for symmetric spaces of higher rank, the situation is more complicated. None of the compactifications is a manifold with boundary, and only some compactifications are real analytic manifolds with corners. The doubling process is also more involved. The general doubling process, called self-gluing, is described in [BJ1]. We now try to find more compactifications of D using these two general approaches. A particularly elegant one is the Satake compactification. Let H2 = {x + i y|y > 0} be the upper half plane with the metric hyperbolic 2 |dz| 1 1 i ∈ S L(2, C). ds 2 = . Then D is isometric to H2 . In fact, let c = √ y2 2 i 1 Then a map c : D → H2 is given by the Cayley transformation: w ∈ D → z = cw =

w+i w+i = (−i) ∈ H2 . iw + 1 w−i

The Lie group G = S L(2, R) acts isometrically and transitively on H2 : az + b ab ·z = cd cz + d cos θ − sin θ and the stabilizer in G of i is S O(2, R) = | 0 ≤ θ ≤ 2π , and sin θ cos θ hence H2 ∼ = S L(2, R)/S O(2, R)

under the map z = g · i → g S O(2, R).

This identification can be used to identify H2 with the space P2 of positive definite symmetric 2 × 2 matrices of determinant 1. In fact, S L(2, R) acts on P2 as follows. For A ∈ P2 , g ∈ S L(2, R), the action is given by


5

g · A = g Ag t . Using the spectral theorem for symmetric matrices, it can be shown that S L(2, R) acts transitively on P2 . When A = I d, the stabilizer is S O(2, R) = {g ∈ S L(2, R) | g · g t = I d}. Therefore, P2 ∼ = S L(2, R)/S O(2, R), which in turn implies that

H2 ∼ = P2 .

In fact, the map z = g · i → gg t gives an S L(2, R)-equivariant bijection, as stated in the next result. Proposition 2.7. The map H2 → P2 is given explicitly by Im z + (Re z)2 (Im z)−1 (Im z)−1 Re z . z ∈ H2 → (Im z)−1 Re z (Im z)−1 The quadratic form associated with the matrix is (Im z)−1 (zu + v)(zu + v), where u, v ∈ R are the variables of the quadratic form. a 0 1b Proof. Write z = · i = b + ia 2 where b = Re z, a 2 = Im z. Then 01 0 a −1 the image of z in P2 is the matrix a 0 a 0 1b 10 , 01 b1 0 a −1 0 a −1 which can be easily computed to be equal to the one in the proposition.

To apply the above approach to compactify H2 , the problem is that P2 is noncompact; for example, a sequence of positive definite matrices could converge to a semi-definite one. We need to find a compact S L(2, R)-space containing P2 . Let S2 be the space of 2 × 2 real symmetric matrices, a 3-dimensional real vector space. P(S2 ) the real projective space associated with S2 , i.e., P(S2 ) = S 2 −{0}/R× . Then P(S2 ) is a compact S L(2, R)-space since S L(2, R) acts on S2 and hence on P(S2 ) by g · A = g Ag t . For any point A ∈ S2 , its image in P(S2 ) is denoted by [A]. Lemma 2.8. The map P2 → P(S2 ), A → [A], is a S L(2, R)-equivariant embedding.

6

L. Ji

Proof. Since elements in P2 have determinant 1, this map is injective. It is clear that the topology of P(S2 ) restricts to the topology of P2 and the map is equivariant with respect to the S L(2, R)-action, and hence the map is an embedding. By composing with the identification H2 ∼ = P2 , we get an embedding i S : H2 → S(P2 ). Definition 2.9. The closure of i S (H2 ) in P(S2 ) is the Satake compactification of H2 , S

denoted by H2 . As mentioned earlier, D is homeomorphic to H2 under the Cayley transform. S

Therefore, the compactification H2 defines a compactification of D, also called the S Satake compactification and denoted by D . For the purpose of generalizing this method to all symmetric spaces X , we work out the composition of the identifications D ∼ = P2 , and the embedding = H2 and H2 ∼ 2 i S : H → P(S2 ). 1 1i Proposition 2.10. Let c = √ ∈ S L(2, C). Then the conjugation by c de2 i 1 fines an isomorphism i c : SU (1, 1) → S L(2, R),

g → cgc−1 ,

and the composed map D → P2 is given by g · 0 = gU (1) → i c (g)i c (g)t ∈ P2 . Proof. Under the identifications D ∼ = SU (1, 1)/U (1) and H2 ∼ = S L(2, R)/S O(2), 2 the Cayley transform D → H is given by gU (1) → i c (g)S O(2). Since the map H2 = S L(2, R)/S O(2) → P2 is given by gS O(2) → gg t ,

the proposition follows. We emphasize that the map i c : SU (1, 1) → S L(2, R)

gives a faithful irreducible representation of SU (1, 1) on R2 , and such a representation is crucial to mapping D into the space of positive definite matrices P2 and hence the embedding D → P(S2 ). S

Proposition 2.11. The SU (1, 1)-action on D extends continuously to D .


7

Proof. The group SU (1, 1) acts on P(S2 ) by g · A = i c (g)A i c (g)t ,

g ∈ SU (1, 1), A ∈ P(S2 ).

Since the embedding D → P(S2 ) is equivariant with respect to SU (1, 1), the exS tension of the SU (1, 1)-action to D is clear by Approach 2.5. Proposition 2.12. The identity map on D extends continuously to a SU (1, 1)S equivariant homeomorphism D → D . We prove this proposition by determining the compactification H2 of H2 corresponding to D and showing that H2 is S L(2, R)-equivariantly homeomorphic to S

H2 . Let H2 = H2 ∪ (R ∪ {∞}) be the compactification with the following topology: 1. A sequence z n ∈ H2 converges to ∞ if and only if |z n | → ∞. 2. For a point x∞ ∈ R, a sequence z n ∈ H2 converges to x∞ if and only if z n → x∞ in C. (This is the 1-point compactification of the closed upper half plane.) From this definition, it is clear that H2 is the closure of H2 in P1 via the embedding through C. Lemma 2.13. The Cayley transform c : D → H2 extends to a SU (1, 1)-equivariant homeomorphism c : D → H2 if SU (1, 1) acts on H2 through the representation i c : SU (1, 1) → S L(2, R). Proof. This follows from the fact that c ∈ S L(2, C), S L(2, C) acts continuously on P1 = C ∪ {∞}, and D, H2 are closures of D, H2 in P1 . Proposition 2.14. The identity map on H2 extends to a S L(2, R)-equivariant homeoS

morphism H2 → H2 . Proof. For a sequence z n ∈ H2 , by Proposition 2.7, the image of z n in the projective space P(S2 ) is (Im z n )2 + Re(z n )2 Re(z n ) (Im z n )−1 1 Re(z n ) 2 2 (Im z n ) + Re(z n ) Re(z n ) = . Re(z n ) 1 10 If |z n | → +∞, then the limit is . On the other hand, if |z n | is bounded and 00 Im z n → 0, then the image converges if and only if Re(z to some point 2 n ) converges 2 x x x∞ x ∞ x∞ , and the limit is . Clearly, the matrix ∞ ∞ uniquely determines x∞ 1 x∞ 1

8

L. Ji

x∞ . This shows that a sequence z n in H2 converges in H2 if and only if it converges S

S

in H2 . Therefore, the identity map on H2 extends to a homeomorphism H2 → H2 . The S L(2, R)-equivariance follows from the continuity of these actions. Remark 2.15. In the embedding D → P(S2 ), every boundary point is given by a semipositive matrix. S L(2, R) acts on P(S2 ) and has 3-orbits: (a) the positive definite matrices; (b) semipositive matrices; (c) nonsingular, indefinite matrices (i.e., of signature (1, 1)). A double covering of the (c) orbit is a pseudo-Riemannian symmetric space S L(2, R)/S O(1, 1). Hence this embedding D → P(S2 ) is different from the earlier embedding D → P1 , where P1 is the doubling of D across the boundary. In fact, the embedding D → P1 is related to the Oshima compactification in [Os1] [Os2], while the embedding D → P(S2 ) and the decomposition of P(S2 ) into the three orbits are closely related to the Oshima–Sekiguchi compactification [OS] and the real locus of the wonderful compactification of complex symmetric varieties defined by De Concini and Procesi in [DP]. These problems are studied in more detail in [BJ3]. Remark 2.16. H2 is not dense in P(S2 ) with respect to the usual topology, but it is dense with respect to the Zariski topology. If in the G-equivariant embedding X → Z , the ambient space Z is an algebraic variety, then the Zariski closure of X in Z is an algebraic subvariety and defines a completion of X . This completion should be related to the real locus of compactifications of the complexified symmetric variety X C . Remark 2.17. Let Pn be the space of n × n positive definite matrices of determinant 1. Then X = S L(n, R)/S O(n, R) can be identified with Pn , and the above approach to compactify H2 ∼ = P2 by embedding P2 into the projective space P(S2 ) can be directly applied to define a compactification of X = S L(n, R)/S O(n, R) Pn , which is one of the many Satake compactifications of X . Next we discuss the Furstenberg compactification of D. The difference from the Satake compactification is the choice of the ambient space Z and the embedding D → Z . Recall D = D ∪ S 1 , S 1 being the boundary. Consider the Brownian motion in D. Almost every Brownian path converges to a point on the boundary S 1 . Starting from every interior point z, there is an exit measure in the boundary S 1 , denoted by dµz , such that for any subset E ⊂ S 1 , the probability that a Brownian path starting from z will exit at some point in E is equal to µz (E) = E dµz . The exit measure can be defined more explicitly using the boundary values of harmonic functions. Consider the boundary value problem on D: u = 0 in D, u= f on ∂ D = S 1 , f ∈ C 0 (S 1 ). This problem is uniquely solvable. For any z ∈ D, the map f → u(z) is a continuous linear functional on C 0 (S1 ) and hence there exists a measure dµz on the boundary, the harmonic measure of z, S 1 such that


9

u(z) =

f (ξ )dµz (ξ ).

S1

In fact, the measure dµz is given explicitly by dµz (ξ ) =

1 − |z|2 dξ, |z − ξ |2

where dξ is the Haar measure on S 1 of total measure 1. From the formula, it is clear that when z is close to a boundary point ξ0 ∈ S 1 , the measure is concentrated near ξ0 , i.e., most Brownian paths starting from z exit near the boundary near ξ0 . Let M1 (S 1 ) be the space of probability measures on S 1 , i.e., positive measures on S 1 of total measure 1. Since S 1 is compact, M1 (S 1 ) is compact with respect to the weak ∗-topology, i.e., for a sequence dµn in M1 (S 1 ), dµn converges to a probability measure dµ0 if and only if for all f ∈ C 0 (S 1 ), f dµn → f dµ0 . S1

S1

The group SU (1, 1) acts on D and leaves S 1 invariant, and hence SU (1, 1) acts on M1 (S 1 ) by (g · µ)(E) = µ(g −1 E), where g ∈ SU (1, 1), µ ∈ M1 (S 1 ), E ⊂ S 1 . Therefore M1 (S 1 ) is a compact SU (1, 1)-space. Proposition 2.18. The map i F : D → M1 (S 1 ), z → dµz , is a SU (1, 1)equivariant embedding. Proof. Since u = 1 is a harmonic function with boundary value f = 1, 1= dµz (ξ ) = µz (S 1 ), S1

and hence dµz ∈ M1 (S 1 ). Since SU (1, 1) acts on D by conformal maps and hence preserves harmonicity, for any g ∈ SU (1, 1) and any harmonic function u with boundary value f , w(z) = u(g · z) is also harmonic on D and satisfies the boundary condition w(ξ ) = f (g · ξ ) on S 1 . Hence u(g · z) = w(z) = f (g · ξ )dµz (ξ ) S 1 = f (ξ )dµz (g −1 ξ ) S1

On the other hand, u(g · z) =

S1

f (ξ )dµgz (ξ ).

By the arbitrary choice of f ∈ C 0 (S 1 ), this implies that

10

L. Ji

dµgz (ξ ) = dµz (g −1 ξ ). Since the right hand side is exactly the SU (1, 1)-action on M1 (S 1 ), this proves that the map D → M1 (S 1 ) is SU (1, 1)-equivariant. 1−|z|2 dξ, it is clear that the map i F is injective. It From the formula dµz (ξ ) = |z−ξ |2 can be checked easily that a sequence z n ∈ D converges to z 0 ∈ D if and only if dµz n converges to dµz 0 in M1 (S 1 ). Therefore, i F is a SU (1, 1)-equivariant embedding. Remark 2.19. Another way to define the map i F is as follows. The subgroup K = U (1) acts transitively on S 1 and hence there is a unique K -invariant probability measure on S 1 , which is clearly dξ , if S 1 is identified with R/Z = {e2πiξ | ξ ∈ [0, 1]}. This invariant measure is the measure dµ0 (ξ ) for the origin. For any point gK ∈ D, define a measure on S 1 by g · dµ0 (ξ ). The above discussions show that g · dµ0 (ξ ) = dµg·0 (ξ ) and the stabilizer of dµ0 (ξ ) is K . Therefore, the map gK → g · dµ0 (ξ ) gives an embedding SU (1, 1)/U (1) → M1 (S 1 ). The closure of i F (D) in M1 (S 1 ) is called the Furstenberg compactification of F D, denoted by D . Proposition 2.20. The identity map on D extends to a SU (1, 1)-equivariant homeF omorphism D → D . Proof. For any sequence z n ∈ D, if z n converges to a boundary point ξ0 in D, 1−|z n |2 then dµz n (ξ ) = |z dξ converges to the delta measure supported at ξ0 , denoted −ξ |2 n

by δξ0 in M1 (S 1 ). This implies that the identity map on D extends to a continuous F

map D → D . Since for ξ1 = ξ2 , dµξ1 = dµξ2 , this map is injective and hence is a F

homeomorphism because both D and D are compact and Hausdorff. The SU (1, 1)equivariance follows from the continuity of the actions. Remark 2.21. This method works for general symmetric spaces X = G/K . The crucial point is to find analogues of S 1 , called the Furstenberg boundaries of X , which are compact G-spaces and on which K acts transitively so that the space of probability measures on them is a compact G-ambient space and contains a unique K -invariant measure. For G = SU (1, 1), the Furstenberg boundary is unique and equal to S 1 . For general X , there are more than one Furstenberg boundaries, and hence more than one Furstenberg compactifications of X . We can get two more compactifications of D by choosing different compact ambient spaces Z . Let S(D, 0) be the space of closed subspaces of D containing the origin 0, which are orbits of subgroups of G. The space S(D, 0) is given the topology of Gromov–Hausdorff convergence of pointed spaces, i.e., for any sequence Yn in S(D, 0), Yn converges to Y∞ if and only if for any R > 0 and the ball B(R, 0)


11

with respect to the Poincaré metric, the Hausdorff distance between Yn ∩ B(R, 0) and Y∞ ∩ B(R, 0) converges to zero. Clearly SU (1, 1) acts on S(D, 0). It is a known fact that S(D, 0) is a compact SU (1, 1)-space. For every point z ∈ D, there exists a unique circle C z in D containing 0 and z as the end points of a diameter of this small circle, i.e., the center of the disc inside the circle is the middle point of the geodesic segment between 0 and z. Lemma 2.22. The map i : D → S(D, 0),

z → C z

is a SU (1, 1)-equivariant embedding. Proof. Since z can be uniquely determined by C z , the map is injective and SU (1, 1)equivariant. It is also clear that a sequence z n in D converges to z ∞ ∈ D if and only if the circle C z n converges to C z ∞ in S(D, 0). Since the map is SU (1, 1)-equivariant, it is an SU (1, 1)-embedding. The closure of i(D) in S(D, 0) gives a compactification of D, called the subset ss compactification and denoted by D . Clearly, the SU (1, 1)-action on D extends ss continuously to D . Proposition 2.23. The identity map on D extends to a continuous SU (1, 1)-equiss variant homeomorphism D → D . Proof. For any ξ ∈ S 1 , let z n be a sequence in D converging to ξ in D. Then the circle C z n converges to the circle Cξ (or rather Cξ − {ξ }) passing through the origin and tangent to ξ in D. With respect to the Poincaré metric, this circle Cξ is a horocircle associated to ξ . This implies that the identity map on D extends to a ss SU (1, 1)-equivariant continuous map D → D . Since the horocircle Cξ uniquely determines the point ξ in S 1 , this map is an embedding. A closely related compactification is obtained by choosing Z to be the space S(SU (1, 1)) of closed subgroups in SU (1, 1). Clearly SU (1, 1) acts on S(SU (1, 1)) by conjugation. It is a known fact that S(SU (1, 1)) is a compact space. In fact, in [Bou, Chap. 8, §5], two topologies are given and both are the same as the topology of pointed Gromov–Hausdorff convergence mentioned earlier, where the distinguished point is the identity element in SU (1, 1). We can define a map D → S(SU (1, 1)) using the identification D ∼ = SU (1, 1)/U (1). Specifically, for any point z = gU (1), define the subgroup to be gU (1)g −1 . Proposition 2.24. The map D → S(SU (1, 1)),

gU (1) → gU (1)g −1

is a SU (1, 1)-equivariant embedding. Proof. Since the normalizer of U (1) in SU (1, 1) is equal to U (1), the map

12

L. Ji

D → S(SU (1, 1)), gU (1) → gU (1)g −1 is injective. Clearly, if a sequence gn U (1) converges to g∞ U (1) in D, then −1 . Conversely, suppose that g U (1)g −1 congn U (1)gn−1 converges to g∞ U (1)g∞ n n −1 verges to g∞ U (1)g∞ . Since gn U (1)gn−1 is bounded, gn is also bounded and hence , g U (1)g −1 = has accumulation points. For any accumulation point g∞ ∞ ∞ −1 U (1) = g U (1). g∞ U (1)g∞ . Since the normalizer of U (1) is equal to itself, g∞ ∞ This implies that gn U (1) converges to g∞ U (1). The closure of D in S(SU (1, 1)) is called the subgroup compactification and sb sb denoted by D . Clearly, the SU (1, 1)-action on D extends continuously to D . Proposition 2.25. The identity map on D extends to a SU (1, 1)-equivariant homeosb morphism D → D . Proof. To prove this proposition, we use the identification D = SU (1, 1)/U (1) ∼ = H2 = S L(2, R)/S O(2). Then it is equivalent to proving that the map H2 = S L(2, R)/S O(2) → S(S L(2, R)),

gS O(2) → gS O(2)g −1

extends to a S L(2, R)-equivariant homeomorphism. Due to the S L(2, R)-equivalence and the Cartan decomposition S L(2, R) = S O(2) A S O(2), where

H2 = S O(2) A · i,

a 0 + A= |a∈R , 0 a −1

we first consider sequences z n = gn i in H2 , where an 0 gn = . 0 an−1 Assume that an → +∞. Then z n converges to ∞ in H2 . We claim that gn S O(2)gn−1 converges to the subgroup N M, where N is the subgroup of unipotent upper triangular matrices, 1b N= |b∈R 01 10 and M is the center of S L(2, R) consisting of two elements ± . In fact, 01 −1 an 0 cos θ sin θ cos θ an2 sin θ an 0 . = − sin θ cos θ 0 an 0 an−1 −an−2 sin θ cos θ


13

Since an → +∞, this matrix has a limit if and only if an2 sin θ has a finite limit, in particular, sin θ → 0, i.e., either θ → 0 or θ → π. Then it is clear that the limit of an U (1)an−1 is equal to N ∪ −N = N M. The cases when z n converges to other boundary points can be proved similarly. In the above five compactifications of D, we followed the first approach (2.5) and embedded D into compact ambient spaces. In the rest of this section, we follow the second approach (2.6) and construct several compactifications intrinsically, exploring the geometric and group-theoretical properties of D. The intrinsic approach might be more complicated in the definition of the compactifications, but the topology is often more geometric and easier to understand. It will be seen that many problems on symmetric spaces naturally lead to such compactifications. We start with a geometric compactification D ∪ D(∞), the conic (or more appropriately the geodesic) compactification. The Poincaré disc D is a complete, simply connected Riemannian manifold of curvature −1. We consider the set of all unit speed, directed geodesics in D, denoted by {γ }. Two geodesics γ1 , γ2 are defined to be equivalent if lim d(γ1 (t), γ2 (t)) < +∞.

t→+∞

This clearly defines an equivalence relation on {γ }. Define D(∞) = {γ }/ ∼ to be the set of equivalence classes of geodesics. Lemma 2.26. Every equivalence class contains a unique geodesic passing through the origin, and hence D(∞) can be identified with the set of directions from the origin, i.e., the unit sphere S 1 in the tangent space to D at the origin. Proof. Each direction θ ∈ S 1 determines a unique geodesic γθ passing through the origin. It can be proved by comparison with the flat space R2 that when θ1 = θ2 , γθ1 ∼ γθ2 . To finish the proof, we need to show that every geodesic γ is equivariant to such a geodesic γθ . Let γn be the geodesic passing through the origin, γn (0) = 0, and the point γ (n). It can be shown that this sequence of geodesics γn converges to some geodesic passing through the origin and hence of the form γθ . Since D is simply connected and negatively curved, by comparison with the flat space R2 again, it can be shown that γ ∼ γθ . This shows that every equivalence class contains a unique geodesic passing through the origin. The space D(∞) is endowed the topology of S 1 , and the space D∪D(∞) is given the following topology. For an unbounded sequence z n ∈ D, and an equivalence class [γ ] ∈ D(∞), z n → [γ ] in D ∪ D(∞) if and only if the geodesic γn passing through 0 and z n converges to the unique geodesic in the class [γ ] passing through the origin. A neighborhood system of a point [γ ] ∈ D(∞) can be described as follows. Let γθ be the unique representative passing through 0. For any ε > 0, form a cone with vertex angle ε and based at 0. Denote this cone by C(γθ , ε). Then truncate the cone at distance T and get C(γθ , ε, T ) = C(γθ , ε)\B(T, γθ (0)),

14

L. Ji

where B(T, γθ (0)) is the geodesic ball of radius T and center γθ (0). Let Vε = {θ ∈ S 1 | d(θ , θ) < ε}, a neighborhood of θ in S 1 . For sequences ε j → 0 and T j → +∞, the family C(γθ , ε j , T j ) ∪ Vε j form a basis of neighborhood system of the point [γ ] in D ∪ D(∞). Due to this construction using cones, this topology on D ∪ D(∞) is called the conic topology. Therefore, the compactification D ∪ D(∞) is called the conic compactification in [GJT]. In the following, it is also called the geodesic compactification. In the above definition of topology or convergence of sequences, we have used the basepoint z = 0. The construction works for any other basepoint, and the compactifications for different basepoints are homeomorphic. This homeomorphism follows from the lemma below. Lemma 2.27. Let z 1 , z 2 be any two basepoints in D. For any unbounded sequence yn in D with respect to the Poincaré metric, the geodesic passing through z 1 and yn , denoted by z 1 yn , converges to a geodesic as n → +∞ if and only if the geodesic z 2 yn converges to a geodesic, and the two limit geodesics are equivalent. Proof. This lemma also follows from comparison with the flat space R2 .

Proposition 2.28. The isometric action of SU (1, 1) on D extends continuously on D ∪ D(∞). Proof. Since SU (1, 1) acts isometrically, SU (1, 1) preserves the equivalence relation and hence acts on D(∞). It can also be shown that for a sequence gn ∈ SU (1, 1) converging to g∞ and a sequence z n in D converging to [γ ] in D ∪ D(∞), gn z n con verges to g∞ [γ ]. Proposition 2.29. The identify map on D extends to a SU (1, 1)-equivariant map D → D ∪ D(∞). Proof. The geodesics in D with respect to the Poincaré metric are semicircles orthogonal to S 1 in C, and the geodesics passing through the origin are diameters of the unit disc. From this it is clear that if a sequence z n converges to a point θ ∈ S 1 in D, it converges to [γθ ] in D ∪ D(∞), where γθ is the unique geodesic passing through the origin in the direction of θ. Remark 2.30. The above construction D ∪ D(∞) works also for any Hadamard manifold, i.e., a non-positively curved, simply connected Riemannian manifold. In particular, for any symmetric space of noncompact type X , we get a geodesic compactification X ∪ X (∞). The boundary X (∞) is often called the sphere at infinity. This geodesic compactification plays an important role in studying manifolds of nonpositive curvature [BGS].


15

Remark 2.31. Both the simply connectedness and nonpositivity of curvature are used crucially in the construction of the geodesic compactification X ∪ X (∞). They are used to show that the exponential map at any basepoint is a diffeomorphism and also used in comparison with the Euclidean space Rn , n = dim X . Another consequence of these two conditions is that every geodesic is distance minimizing, and so points along a geodesic definitely go to infinity. This is not true for non-simply connected manifolds. For example, for a noncompact Riemannian surface \H2 , where ⊂ S L(2, R) is a discrete subgroup, there are many geodesics which are not bounded but do not go to infinity. To guarantee convergence of sequences of points along geodesics, for a nonsimply connected Riemannian manifold, we should only consider the set of distance minimizing rays (or eventually distance minimizing geodesics). Then a problem is that not every two points are connected by a distance minimizing ray. In a joint work with MacPherson [JM], we defined a geodesic compactification for a not necessarily simply connected, or non-positively curved manifolds under two assumptions: 1. Instead of a basepoint, we choose a base compact subset. Then every point can be connected to a point in the compact base subset by a distance minimizing ray. 2. If two points near infinity are close, then the rays connecting them are also close. These two conditions seem to be natural if one wants to define a compactification via geodesics. They are satisfied by locally symmetric spaces \X , and hence we get a geodesic compactification for \X . Remark 2.32. The procedure to construct the above intrinsic compactification D ∪ D(∞) can be summarized as follows: 1. Define ideal points to be added at infinity. 2. For each ideal point, describe the ways (or sequences) converging to this point, or the structure of infinity adapted to this point. We can also change the order slightly. 1. Specify ways of going to infinity or structures at infinity. 2. Add ideal points for each mode of going to infinity. The structure at infinity of X can be described by parabolic subgroups. Next we construct another compactification of D using proper parabolic subgroups. For this purpose, we use H2 instead of D. Then H2 (∞) = R ∪ {∞}. Recall that S L(2, R) acts continuously on H2 ∪ H2 (∞). For any point ξ ∈ H2 (∞), the stabilizer of ξ in S L(2, R) is a parabolic subgroup, and any parabolic subgroup of S L(2, R) is of this form. When ξ = {∞}, the parabolic subgroup P∞ consists of upper triangular matrices and admits the following Langlands decomposition, P∞ = N AM ∼ = N × A × M, where N is the subgroup of unipotent upper triangular matrices, A is the subgroup of diagonal matrices with positive entries, and M is the center of S L(2, R), which consists of two elements. Other parabolic subgroups are conjugates of P∞ . Each

16

L. Ji

parabolic subgroup P acts transitively on H2 , and its Langlands decomposition induces the horospherical decomposition H2 = N × A ∼ = R × R+ , t 0 + where we have identified A with R by → t. When P = P∞ , the orbits of 0 t −1 N × {a} are horizontal lines while the orbits of {n} × A are vertical lines in the upper half plane H2 . In terms of geodesics, this horospherical decomposition can be explained as follows. By definition, each parabolic subgroup P determines a unique equivalence class of geodesics [γ ], which is also denoted by [P]. The geodesics in [γ ] are disjoint and they give a disjoint decomposition of H2 . In fact, every point in H2 is connected to [γ ] at infinity through a unique geodesic. The orbits of {n} × A are geodesics in [γ ], and the orbits of N are horocircles for the point [γ ], in particular, N acts transitively on the set of geodesics in the equivalence class [γ ]. For any point z ∈ H2 , denote the coordinates in the decomposition H2 = N × A by (n, a), where n ∈ N , a ∈ A. For any t > 0 and a bounded set W ⊂ N , define a Siegel set associated with P by S P,t,W = {(n, a) ∈ N × A | n ∈ W, a > t}. Lemma 2.33. For any parabolic subgroup P and a sequence z j = (n j , a j ) in H2 , if a j → +∞ and n j is bounded, then z j converges to the boundary point [P] in H2 ∪ H2 (∞). Proof. Assume that P = P∞ . Then in terms of the coordinates of H2 , z j = x j + i y j , y j → +∞ and x j is bounded. It is clear that z j converges to [P] in H2 ∪ H2 (∞). The other cases follows from the S L(2, R)-equivariance. The converse of this lemma is not true, i.e., the Siegel sets are not neighborhoods in H2 of points at infinity. For example, for the parabolic subgroup P∞ , a sequence z j = x j + i y j with x j → ∞, y j → ∞, it is true that z j → {∞} in H2 ∪ H2 (∞), but z j does not satisfy the condition in the lemma. In fact, in the horospherical coordinates with respect to P∞ , z j = (n j , a j ), a j → ∞ and n j → ∞, and hence z j does not belong to any Siegel set S P,t,U . To describe neighborhoods of points in H2 in terms of parabolic subgroups, we need to generalize the Siegel sets. Fix a norm on sl(2, R) and hence a left invariant metric on S L(2, R). This induces a left invariant metric on N . Denote the ball in N with center the identity element and radius ε by B N (ε). For any ε > 0, t > 0, define a generalized Siegel set S P,t,ε = {(n, a) ∈ N × A | a > t, a −1 na ∈ B N (ε)}. Let C be a neighborhood of the identity element in S O(2). We are going to show that the closure of C S P,t,ε in H2 ∪ H2 (∞) is a neighborhood of [P]. For this purpose, we define another compactification of H2 . For each parabolic subgroup P, let [P] be an ideal point. Let (H2 ) = ∪ P [P], where P runs over all proper parabolic subgroups. Define a topology on H2 ∪ (H2 ) as follows.


17

1. (H2 ) can be canonically identified with H2 (∞) and hence with S 1 . Endow (H2 ) with the topology of S 1 . 2. A sequence z j in H2 converges to a point [P] if and only if z j can be written as z j = k j n j a j · i, where k j ∈ S O(2), n j ∈ N , a j ∈ A satisfy (1) k j → e, (2) a j → +∞, (3) a −1 j n j a j → e. These convergence sequences define a unique topology on H2 ∪ (H2 ). In fact, let C be a neighborhood of the identity element in S O(2). Then C(S P,t,ε ∪ [P]) is a neighborhood of [P] in H2 ∪ (H2 ). Since H2 = N A · i, the decomposition z j = k j n j a j · i is not unique, i.e., k j , n j , a j are not uniquely determined by z j . An immediate question is whether the topology on H2 ∪ (H2 ) is Hausdorff, i.e., every convergent sequence has a unique limit. Proposition 2.34. For two different parabolic subgroups P1 , P2 , and a compact neighborhood C of the identity element in S O(2) such that k P1 k −1 = P2 for all k ∈ C, if ε is sufficiently small and t sufficiently large, then k S P1 ,t,ε ∩ S P2 ,t,ε = ∅. Proof. We prove this by contradiction. If not, there exists a sequence z j ∈ k j S P1 ,t j ,ε j ∩ S P2 ,t j ,ε j for some k j ∈ C, t j → +∞, ε j → 0. By passing to a subsequence, we can −1 = P , by replacing P by k P k −1 , we can assume that k j → k∞ . Since k∞ P1 k∞ ∞ 1 ∞ 2 1 assume k j → e. We use the Satake compactification to get a contradiction. Since z j ∈ S P2 ,t j ,ε j , z j = n j a j , where n j ∈ N P2 , a j ∈ A P2 , a j → +∞, a −1 j n j a j → e. Let i S : H2 = S L(2, R)/S O(2) → P(S2 ) be the embedding for the Satake compactification. Assume the parabolic subgroup P2 to be equal to P∞ . Then a j is a diagonal matrix, tj 0 , aj = 0 t −1 j

and hence

t j → +∞,

2 tj 0 1 0 10 i S (a j · i) = [a j a j ] = = → . −4 −2 0 t 00 0 tj j

Since n j a j = a j n j with n j = a −1 j n j a j converging to the identity element,

18

L. Ji

i S (n j a j · i) = n j [a j a j ]n j → t

10 00

in P(S2 ). In general, if P2 is the parabolic subgroup associated with x2 ∈ R, the limit x22 x2 of i S (n j a j · i) will be . Similarly, using k −1 j z j ∈ S P1 ,t j ,ε j , we can show x2 1 that 2 x1 x1 i S (n j a j · i) → x1 1 in P(S2 ), where x1 is the point in R ∪ {∞} corresponding to [P1 ]. Since P1 = P2 , x1 = x2 , and these two limits are different and give a contradiction. This proves the proposition. Proposition 2.35. The compactification H2 ∪ (H2 ) is Hausdorff. Proof. For any two boundary points [P1 ], [P2 ], if t 0, ε is sufficiently small, and C is a sufficiently small neighborhood of the identity element in S O(2), then the neighborhoods C(S P1 ,t,ε ∪ [P1 ]), C(S P2 ,t,ε ∪ [P2 ]) are disjoint. In fact, C S P1 ,t,ε , C S P2 ,t,ε are open dense subsets of the neighborhoods. By the previous proposition, they are disjoint, and therefore the two neighborhoods are also disjoint. Proposition 2.36. The S L(2, R)-action on H2 extends to a continuous action on H2 ∪ (H2 ). Proof. It suffices to show that if a sequence z j ∈ H2 converges to [P], then for any g ∈ S L(2, R), gz j converges to [g Pg −1 ]. By definition, z j = n j a j ·i with a j ∈ A P , a j → +∞, n j ∈ N P , a −1 j n j a j → e. If g ∈ S O(2), then gz j = (gn j g −1 )(ga j g −1 ) · i, and gn j g −1 , ga j g −1 are horospherical coordinates of z j with respect to the parabolic subgroup g Pg −1 , and it is clear that gz j converges to [g Pg −1 ]. In general, write g = kan, where k ∈ S O(2), a ∈ A P , n ∈ N P . Then gz j = kn j a j · i, where n j = ann j a −1 , a j = aa j . Clearly, a j → +∞ and a j n j a j −1 → e, and hence gz j → [k Pk −1 ] = [g Pg −1 ].

Proposition 2.37. The identity map on H2 extends to a continuous S L(2, R)-equivariant homeomorphism H2 ∪ (H2 ) → H2 ∪ H2 (∞). Proof. It suffices to show that if a sequence z j ∈ H2 converges to [P] in H2 ∪ (H2 ), it also converges to [P] in H2 ∪ H2 (∞). By definition, z j = k j n j a j · i, k j → e, a j ∈ A P , a j → e, n j ∈ N P , a −1 j n j a j → e. It can be shown easily that a j · i → [P] in H2 ∪ H2 (∞). Similarly, k j a j · i also converges to [P]. Since d(k j n j a j · i, k j a j ·) = d(a j n j a −1 j · i, i) → 0,


19

where d(·, ·) is the invariant Riemannian distance on H2 , k j n j a j · i also converges to [P] in H2 ∪ H2 (∞). This proposition implies that for any point [P] ∈ H2 (∞), C(S P,t,ε ∪ [P]) is a neighborhood of [P] in H2 ∪ H2 (∞). Remark 2.38. In the construction of the compactification H2 ∪ (H2 ), Siegel sets of parabolic subgroups and their separation properties play an important role. One feature of this approach is that it parallels compactifications of locally symmetric spaces. In fact, if ⊂ S L(2, Q) is an arithmetic subgroup, then \H2 is a noncompact Riemann surface of finite area and can be compactified by adding finitely many points, which correspond bijectively to the -equivalence classes of rational parabolic subgroups. Another feature is that by changing the ideal boundary sets associated with each parabolic subgroup, we can construct different compactifications. See [BJ4] for such constructions for general X . Next we present yet another compactification of D using the Cartan decomposition of SU (1, 1). Let cosh t sinh t A= |t ∈R sinh t cosh t be a Cartan subgroup of SU (1, 1). Clearly, A ∼ = R. Then SU (1, 1) = U (1)AU (1). Applied to D, it gives the polar decomposition of D: D = U (1)A · 0 = U (1)γ0 = U (1) · R, where γ0 = A · 0 is the geodesic passing through 0 determined by A. Geometrically, the orbit of A in D is a geodesic γ0 in D with respect to the Poincaré metric passing through the origin, and D is obtained by rotating γ0 around the origin, i.e., D is union of these geodesics passing through the origin. Note these geodesics are not disjoint. In fact, for any two such geodesics, the only intersection point is the origin. Since U (1) is compact, we only need to compactify γ and extend the U (1)action. Equivariantly, we need to compactify all the geodesics passing through the origin and put a suitable topology on the union of these compactified geodesics. One natural compactification of R is R ∪ {±∞} = [−∞, +∞] ∼ = [−1, +1]. Therefore, each geodesic γ is compactified by adding two ideal points γ (−∞), γ (+∞). Define D ∪ ∗ (D) = D ∪γ {γ (−∞) ∪ γ (+∞)} = D ∪ {γ (+∞)}, γ

where γ runs over all geodesics passing through the origin 0. For two geodesics γ , γ tracing out the same curve but with different orientations, γ (−∞) (γ (+∞))

20

L. Ji

is identified with γ (+∞) (γ (−∞)respectively). Let M be the center of SU (1, 1) 10 which consists of two elements ± . Then M acts as identity on D and hence 01 fixes every geodesics passing through the origin, and U (1)/M acts freely and transitively on the set of geodesics passing through the origin. We extend this action of U (1)/M to the boundary ∗ (D) by g · γ (+∞) = (g · γ )(+∞),

g · γ (−∞) = (g · γ )(−∞),

where g ∈ U (1)/M, and g · γ is the geodesic obtained from γ under the action of U (1)/M. Define a topology on D ∪ ∗ (D) as follows. Let γ0 = A · 0 be the fixed geodesic as above. A sequence z j in D converges to a boundary point γ (+∞) if z j = k j γ0 (t j ), where k j ∈ U (1)/M converges to k and kγ0 = γ , and t j → +∞. Proposition 2.39. The space D ∪ ∗ (D) is a compactification of D, and the identity map on D extends to a U (1)-equivariant homeomorphism D∪∗ (D) → D∪ D(∞). Proof. Since [−∞, +∞] and U (1) are compact, D ∪ ∗ (D) is compact. Clearly, D ∪ ∗ (D) contains D as a dense open set and is hence a compactification. To prove the proposition, it suffices to show that any sequence in D which converges in D ∪ ∗ (D) also converges in D ∪ D(∞). For any convergent sequence g j in U (1), any geodesic γ and a sequence t j → +∞, γ (t j ) converges to [γ ] in D ∪ D(∞). Due to the continuous action of U (1), g j γ (t j ) converges to g · [γ ] in D ∪ D(∞). The homeomorphism D ∪ ∗ (D) → D ∪ D(∞) is clearly U (1)-equivariant. The above compactification D ∪ ∗ (D) can also be characterized by the following properties: 1. For any geodesic γ passing through the origin, its closure γ is γ (−∞) ∪ γ ∪ γ (+∞) ∼ = [−∞, +∞]. 2. For any two different geodesics γ1 , γ2 , γ1 ∩ γ2 = γ1 ∩ γ2 = {0}. Remark 2.40. The generalization of D ∪ ∗ (D) to higher rank spaces X is called the dual cell compactification X ∪ ∗ (X ) in [GJT]. In the case of D, a maximal flat subspace containing the origin 0 is a geodesic γ ∼ = (−∞, +∞), and the Weyl chamber decomposition is given by (−∞, +∞) = (−∞, 0) ∪ {0} ∪ (0, +∞). The dual simplex for this polyhedral decomposition is γ (−∞), γ (+∞), which is the boundary for γ . The basic idea of the dual cell compactification is to add the dual simplex to the infinity of each flat and glue all these compactifications together. In the higher rank case the intersection pattern of flats is more complicated. Remark 2.41. One problem with D ∪ ∗ (D) is that the continuous extension of SU (1, 1) to the boundary is not obvious, since we only use the geodesics passing through the origin. The above constructions of compactifications of D are either geometric or group theoretic. We now discuss a natural compactification from potential theory, the Martin compactification.


21

As mentioned earlier, given any function f ∈ C 0 (S 1 ), there is a unique solution of the boundary value problem: u = 0 in D,

u = f on S 1 ,

where can be either the Euclidean Laplacian or the Laplacian for the hyperbolic metric. In fact, the solution u is given by u(z) =

S1

f (ξ )

1 − |z|2 dξ = |z − ξ |2

S1

1 − |z|2 f (ξ )dξ. |z − ξ |2

In particular, if f ≥ 0, we get a positive harmonic function u on D. If f (ξ )dξ is replaced by any positive measure dµ(ξ ) on S 1 , the above formula also gives a positive harmonic function u, u(z) =

S1

1 − |z|2 dµ(ξ ). |z − ξ |2

For example, if dµ is the delta measure supported at ξ0 , the harmonic function is u(z) =

1 − |z|2 , |z − ξ0 |2

the Poisson kernel at ξ0 . Proposition 2.42. Every positive harmonic function u on D is of this form for some positive measure dµ, and the map dµ → u gives an one-to-one correspondence between the set of positive measures on S 1 and the set of positive harmonic functions on D. This proposition follows from identification of the Martin compactification of D. We give a brief summary of the Martin compactification of a complete Riemannian manifold in this section. See [Ta2] for more motivations and detailed definition. Let (X, g) be a noncompact complete Riemannian manifold. Let be the Laplace–Beltrami operator associated with g, = −

∂ 1 ∂ ( det gi j g i j ), ∂x j det gi j i, j ∂ xi

where ds 2 = gi j d xi d x j , (g i j ) = (gi j )−1 . Then is a nonnegative operator. For n ∂ 2 example, when X = Rn with the Euclidean metric, = − i=1 2. ∂ xi

For any λ ∈ R, let Cλ (X ) be the convex cone of positive solutions of u = λu on X . When λ = 0, Cλ (X ) is the set of positive harmonic functions. One problem in potential theory is to study the structure of the convex cone Cλ (X ). The first question is when Cλ (X ) is nonempty. The Laplace operator defines a unique selfadjoint operator on L 2 (X ). Since ≥ 0, its spectrum Spec() ⊂ [0, +∞). Denote the bottom of Spec() by

22

L. Ji

λ0 = λ0 (X ) = inf{λ | λ ∈ Spec()}, which can be also defined directly by λ0 (X ) =

inf ∞

ϕ∈C0 (X )

X | ϕ| 2 X |ϕ|

2

.

By definition, λ0 (X ) ≥ 0. For symmetric spaces of noncompact type, λ0 (X ) > 0. For example, for X = D, λ0 = 1. The following proposition is due to Cheng–Yau and Sullivan (see [Su]), and the proof is a variant of a proof by Varoupoulos, which appeared in [Ta1]. Proposition 2.43. The cone Cλ (X ) is nonempty if and only if λ ≤ λ0 (X ). Proof. If λ < λ0 , we can use the positivity of the Green function G λ (x, y) of − λ to get a positive solution of u = λu. More precisely, for a sequence of points y j in X going to infinity, we can extract a convergent subsequence of the normalized G (x,y ) functions G λλ(x0 ,yjj ) using the Harnack inequality. Since G λ (x, y) = λG λ (x, y), the limit function u satisfies u = λu, and u(x0 ) = 1 and hence is positive. By taking a sequence λ j → λ0 , λ j < λ0 , we can use positive solutions u j of u j = λ j u j to extract a convergent subsequence of {u j } and hence a positive solution of u = λ0 u, u(x0 ) = 1. Conversely, if u is a positive solution of u = λu, we show that λ ≤ λ0 . Define ˜ by setting a new operator 1 ˜ = (uϕ). ϕ u ˜ is symmetric with respect to the measure u 2 d x. The bottom of the It is clear that ˜ of ˜ with respect to the measure u 2 d x is easily seen to be λ0 (X ). spectrum λ0 () (Recall that for a matrix, eigenvalues are invariant under conjugation). ˜ in a different way. On the other hand, we can compute λ0 () 1 2 ˜ (uϕ)ϕud x (ϕ, ϕ)u 2 d x = (uϕ) · ϕu d x = u X X = (u · ϕ − 2u · ϕ + uϕ)ϕud x X = λu 2 ϕ 2 − 2(u · ϕ)uϕ + (ϕ)u 2 ϕd x X = λu 2 ϕ 2 − 2(u · ϕ)ϕu + ϕ · (ϕ · u 2 + (2u)ϕu)d x X = λu 2 ϕ 2 + |ϕ|2 u 2 d x X ≥λ u 2 ϕ 2 = λ(ϕ, ϕ)u 2 d x X

and hence


˜ = λ0 () = λ0 ()

23

˜ ϕ)u 2 d x (ϕ, ≥ λ. (X ) (ϕ, ϕ)u 2 d x

inf ∞

ϕ∈C0

This completes the proof of the proposition.

The convex cone Cλ (X ) is uniquely determined by its extremal elements, where a function u is extremal if it is not a nontrivial linear combination of two functions in Cλ (X ), which is equivalent to saying that u is minimal, i.e., for any v ∈ Cλ (X ), if v ≤ u, then v = cu for some constant c ≤ 1. In fact, any function in Cλ (X ) is a linear combination of extremal functions. A natural problem is to determine the set of minimal (extremal) functions in Cλ (X ) intrinsically from X . This problem is solved by the Martin compactification. For each λ ≤ λ0 (X ), there is a Martin compactification X ∪ ∂λ X . Let G λ (x, y) be the Green function for the operator − λ. For λ < λ0 (X ), − λ is invertible and G λ (x, y) exists. For λ0 = λ0 (X ), if G λ0 (x, y) does not exist, then the cone Cλ (X ) is one dimensional, and the Martin compactification is defined to be the one point compactification X ∪ ∂λ0 X = X ∪ {∞}. Suppose that λ ≤ λ0 (X ) and the Green function G λ (x, y) exists. Fix a basepoint x0 . Let K λ (x, y) =

G λ (x, y) G λ (x0 , y)

be the normalized Green function, i.e., for every fixed y, K λ (x0 , y) = 1, for x = y, K λ (x, y) = 0, K λ (x, y) → 0 as x → ∞. Then the Martin compactification X ∪ ∂λ X is characterized by the following conditions: 1. For every x ∈ X , the function y → K λ (x, y) extends continuously to X ∪ ∂λ X . 2. The extended functions K λ (·, ξ ), ξ ∈ ∂λ X , separate the points of the boundary ∂λ X . From this characterization, it is clear that the Martin compactification is determined by the asymptotic behaviors of the Green function at infinity. For each ξ ∈ ∂λ X , let K λ (x, ξ ) be the extended function. Then K λ (·, ξ ) ∈ Cλ (X ) and K λ (x0 , ξ ) = 1. For different choices of the basepoint, the Martin compactifications are canonically homeomorphic, though the Martin kernel functions will change by multiplicative constants. A basic result of [Ma] (see also [Ta2]) is the following representation. Proposition 2.44. 1. The collection of the boundary functions K λ (·, ξ ), ξ ∈ ∂λ X , generate the cone Cλ (X ), i.e., for any u ∈ Cλ (X ), there exists a nonnegative measure dµ on ∂λ X such that u(x) = K λ (x, ξ )dµ(ξ ). ∂λ X

2. Every minimal function in Cλ (X ) is a multiple of some K λ (·, ξ ). Let ∂λ,min X be the subset of ∂λ X consisting of points ξ such that K λ (·, ξ ) are minimal. Then for every function u ∈ Cλ (X ), there exists a unique measure dµ on ∂λ,min X such that

24

L. Ji

u(x) =

∂λ,min X

K λ (x, ξ )dµ(ξ ).

There exists an extensive literature on the determination of the Martin compactification of Riemannian manifolds. See [AS], [An] and [GJT] and their references. Proposition 2.45. For any λ ≤ λ0 (D), the Martin compactification D ∪ ∂λ D is the same as D, i.e., the identity map on D extends to a homeomorphism D → D ∪ ∂λ D. Under this identification, for any ξ = eiθ ∈ S 1 , the Martin kernel function for the basepoint x0 are given by K λ (x, ξ ) = (

1 − |x|2 1 +√1−λ )2 . |x − ξ |2

For λ = 0, the above proposition immediately implies Proposition 2.42. The above proposition is a special case of the result of Anderson–Schoen [AS] since the curvature of D is equal to −1. On the other hand, the precise asymptotics of the Green function of the Poincaré disc can be determined (see [AJ] for example), and the Martin compactification of D can be determined easily from this. Summary. We have defined many compactifications of D from different points of view. They all turn out to be the same compactification. One basic reason is that maximal totally geodesic flat submanifolds of D are of dimension 1. A consequence is that all geodesics in D are the same, i.e., SU (1, 1) acts transitively on the set of geodesics and hence on the sphere at infinity. In this sense, it is reasonable to add one point to every equivalence class of geodesics [γ ]. On the other hand, if the rank of a symmetric space X is greater than 1, G does not act transitively on X (∞), and there are different types of geodesics. Then the geodesic compactification X ∪ X (∞) may not be the only natural compactification since we can assign different boundary sets for different equivalence classes of geodesics. The classification of geodesics into different types is basically the Tits building of X and its realization in terms of geodesics.

3 The bidisc An important invariant of symmetric spaces is the rank, which is the maximal dimension of flat totally geodesic submanifolds. The rank of the Poincaré disc is equal to 1, and the rank of the bidisc is equal to 2. The bidisc D × D is one of the simplest examples of higher rank symmetric spaces. In this section, we show that generalizations to D × D of some compactifications of D in the previous section lead to different compactifications and also point out some difficulties in the higher rank case. Though the geometry of D × D is relatively simple, many difficulties of the higher rank spaces are already present. Let γ ∼ = R be a geodesic in D. Then γ × γ ∼ = R2 is a flat in D × D. Therefore, D × D is of rank two. Since D = SU (1, 1)/U (1),


25

D × D = SU (1, 1) × SU (1, 1)/U (1) × U (1). The embedding D × D → P1 × P1 is SU (1, 1) × SU (1, 1)-equivariant, and the closure D × D = D × D of D × D in P1 × P1 is a compactification on which the natural action of SU (1, 1) × SU (1, 1) extends continuously. Remark 3.1. D × D is a reducible symmetric space, and this compactification D × D preserves the product structure, i.e., it is the product of compactifications of the irreducible factors. It will be seen below that many other compactifications do not preserve the product structure. Remark 3.2. This method also works for all bounded symmetric domains in Cn , where a domain is called symmetric if for every point x ∈ , there exists a holomorphic automorphism T such that T 2 = id and x is an isolated fixed point of T . Among many embeddings in Cn , every bounded symmetric domain has a canonical embedding such that the holomorphic automorphisms of extend to a compactification of Cn . The compactification of Cn is called the compact dual and denoted by ∧ . When = D × D, ∧ = P1 × P1 . Let G be the automorphism group of . Then G also acts on ∧ , and the embedding → ∧ is G-equivariant. The closure of in ∧ gives a G-compactification, called the Baily–Borel compactification. For a general symmetric space of noncompact type X , the basic steps for constructing the Satake compactifications of X are as follows: 1. Compactify the special symmetric space Pn = S L(n, R)/S O(n). 2. Embed X into Pn as a totally geodesic submanifold of Pn and take the closure of X in the compactification of Pn . Let G be the identity component of the isometry group of X . Since X is a totally geodesic submanifold of Pn , the isometric action of G on X extends to Pn and hence the compactification of Pn defined in step 1, and the closure of X is a G-compactification. We carry out these steps for the case X = D × D. The first step was mentioned in Remark 2.17. Briefly, let Sn be the vector space of n ×n symmetric matrices, and P(Sn ) the real projective space. Then Pn is embedded into P(Sn ) by gS O(n) → [gg t ], where [gg t ] is the image of the matrix gg t in P(Sn ). Since S L(n, R) also acts on P(Sn ) by g · [A] = [g Ag t ], the embedding Pn → P(Sn ) is S L(n, R)-equivariant. The second step involves the following result of Satake [Sa1]. Proposition 3.3. Let X = G/K be a symmetric space of noncompact type. If τ : G → S L(n, R) is a faithful representation satisfying τ (θ (g)) = (τ (g)t )−1 , where θ is the Cartan involution of G associated with K , then the map i τ : X → Pn ,

gK → gg t

embeds X as a totally geodesic submanifold of Pn .

26

L. Ji

Briefly, the condition τ (θ (g)) = (τ (g)t )−1 implies that K is mapped into S O(n). Then the map gK → gg t is well-defined. Since the representation τ is faithful, this map is an embedding. The above condition again implies that the image i τ (X ) is a totally geodesic submanifold of Pn . The closure of i τ (X ) in the compactification of Pn constructed in step (1), or rather in P(Sn ), is called the Satake compactification of X associated with the repS resentation τ , denoted by X τ . Remark 3.4. This method is simple and elegant. It reduces compactifications of general symmetric spaces to the special case Pn = S L(n, R)/S O(n). It can be used to explain the canonical embedding of bounded symmetric domains mentioned earlier in Remark 3.2. On the other hand, it is not easy to see which sequences are convergent and what limit points are added at infinity. For many purposes, it is important to understand relations between the boundary points and parabolic subgroups S of G. These questions and the dependence of X τ on τ will be discussed later. We apply the above procedure to X = D × D using the identification with H2 × H2 . Then X = S L(2, R) × S L(2, R)/S O(2) × S O(2). Let τ1 , τ2 be two copies of the standard representation on R2 , i.e., τ1 , τ2 are the identify map. Then the tensor product τ1 ⊗ τ2 is a representation of G = S L(2, R) × S L(2, R): τ : G → S L(R2 ⊗ R2 ) = S L(4, R),

(g1 , g2 ) → g1 ⊗ g2 .

Lemma 3.5. The representation τ : S L(2, R) × S L(2, R) → S L(4, R) is faithful and maps K = S O(2) × S O(2) into S O(4), and the embedding i τ : G/K → P4 is given by (g1 , g2 )K → g1 g1t ⊗ g2 g2t . Proof. Since the standard representation of S L(2, R) is faithful, τ is also faithful. In the lemma, we have identified P2 ⊗ P2 with P4 . S

Proposition 3.6. For the representation τ , H2 × H2 τ = H2 × H2 . Proof. It follows from the fact that for two sequences An , Bn of 2 × 2 symmetric matrices of determinant one, their tensor product An ⊗ Bn converges in P(S4 ) if and only if both factors An , Bn converges in P(S2 ). The product structure of D × D is preserved by this Satake compactification. In fact, the same is true for general reducible symmetric spaces. As mentioned in the previous section, the closure of a maximal flat totally geodesic submanifold, i.e., a flat, plays an important role in compactifications of S X . For X = D × D, a flat is R2 ∼ = (−1, 1) × (−1, 1), and its closure in X τ is homeomorphic to [−1, 1] × [−1, 1]. The boundary is a simplicial complex dual to


27

the decomposition of R2 into coordinate quadrants, which is also the Weyl chamber decomposition of the Cartan subalgebra a = R2 of the group S L(2, R) × S L(2, R). For the Furstenberg compactification, the procedure is the same except for the choice of the space of probability measures. In the unit disc D, we embedded D into the space of probability measures on the unit circle S 1 . The reason for choosing the unit circle is that the boundary values of harmonic functions on the unit disc are defined on the unit circle. For the bidisc X = D × D, we also use the space of probably measures on a suitable boundary. Such boundaries are called the Furstenberg boundaries. In the compactification D × D = D × D, the ideal boundary consists of the union S1 × S1 ∪ S1 × D ∪ D × S1. It turns out that there is a unique Furstenberg boundary consisting of corner points S 1 × S 1 , the so-called distinguished boundary. One explanation for the choice of this distinguished subset is an answer to the following question. Suppose u is a harmonic function on D × D and extends to a continuous function on D × D. The question is which part of the boundary values determines u. Proposition 3.7. If u is a bounded harmonic function on D × D, then u is strongly harmonic, i.e., u(x, y) is harmonic in each variable x and y separately. This result is due to Furstenberg [Fu] and follows from the identification of the Poisson boundary of D × D with S 1 × S 1 . Briefly, bounded harmonic functions correspond to bounded measurable functions on the Poisson boundary; Furstenberg showed that the Poisson boundary has the product structure, and hence u(x, y) is the product of two harmonic functions f (x) and g(y). Remark 3.8. The boundedness assumption on u in the proposition is crucial. Otherwise it is not true. For example, for some λ = 0, let f, g ∈ C ∞ (D) be two functions satisfying f = λ f , g = −λg. Then u(x, y) = f (x)g(y) is harmonic but not strongly harmonic on D × D. Proposition 3.9. If u is a harmonic function on D × D and extends continuously to D × D, then u is uniquely determined by its values on the distinguished boundary S1 × S1. Proof. By continuity, for any ξ1 , ξ2 ∈ S 1 , both u(ξ1 , ·) and u(·, ξ2 ) are harmonic functions on D and hence determined by their boundary values on S 1 . By iteration, for any (z 1 , z 2 ) ∈ D × D, the value u(z 1 , z 2 ) is determined by the boundary values u(ξ1 , z 2 ), which in turn are determined by the boundary values u(ξ1 , ξ2 ), (ξ1 , ξ2 ) ∈ S1 × S1. The dependence of u on its boundary values can be made precise by iteration of the Poisson formula for the disc D. Let f denote the restriction of u to the distinguished boundary S 1 × S 1 . For any point x = (z 1 , z 2 ) ∈ D × D, let

28

L. Ji

dµx =

1 − |z 1 |2 1 − |z 2 |2 · dξ1 dξ2 , |z 1 − ξ1 |2 |z 2 − ξ2 |2

a product measure on S 1 × S 1 . Then u(x) = f (ξ1 , ξ2 )dµx (ξ1 , ξ2 ).

(*)

S 1 ×S 1

Proposition 3.10. The map D × D → M1 (S 1 × S 1 ), x → dµx , is a S L(2, R) × S L(2, R)-equivariant embedding, where M1 (S 1 × S 1 ) is the space of probability measures on S 1 × S 1 . Proof. It follows from the explicit formula of the measure dµx that this map is injective. The equivariance also follows from the fact that S L(2, R) × S L(2, R) preserves the space of harmonic functions as in Proposition 2.18. Definition 3.11. The closure of D × D in M1 (S 1 × S 1 ) is the Furstenberg compactF ification, denoted by D × D . Proposition 3.12. The identity map on D × D extends to a SU (1, 1) × SU (1, 1)F equivariant homeomorphism D × D → D × D . Proof. It is clear from the formula for dµx that a sequence xn ∈ D × D converges in D × D if and only if the sequence of measures dµxn converges in M1 (S 1 × S 1 ). Remark 3.13. The measure dµx can also be interpreted in terms of the Brownian motion. The Brownian motion, associated to the invariant Laplacian on D × D associated to the invariant Laplacian exits almost surely on the distinguished boundary S 1 × S 1 , and when the Brownian motion starts at x, the exit measure is given by dµx . Remark 3.14. For general symmetric spaces, one difficulty in constructing the Furstenberg compactifications is to identify a space replacing S 1 × S 1 and probability measures dµx on it so that the above equation (∗) still holds. A space with such a family of probability measures is called the Poisson boundary. One reason of this difficulty is that it is not easy to embed X into some simple, natural space, and get some “distinguished” boundary to represent all bounded harmonic functions. Instead, in [Fu], the procedure is as follows: do not treat the Furstenberg boundary as a subset of the boundary of some compactification, but study them as natural homogeneous spaces. From this brief description, it is conceivable that the Furstenberg compactifications are more complicated in definition than the Satake compactifications. On the other hand, each has its own advantage. For general symmetric spaces, they will define the same compactifications. Next we discuss the conic (or geodesic) compactification of D× D. It will be seen below that this compactification is quite different from the earlier compactifications.


29

As mentioned earlier in Remark 2.30, the geodesic compactification X ∪ X (∞) can be defined for all simply connected, nonpositively Riemannian manifold. Therefore, D × D ∪ (D × D)(∞) is defined, and the boundary (D × D)(∞) is homeomorphic to S 3 . S F To compare with D × D, D × D τ and D × D , we study the closure of flats in D × D ∪ (D × D)(∞). A flat A in D × D has dimension 2 and its closure in D × D ∪ (D × D)(∞) is A ∪ A(∞). If A is identified with R2 , then A(∞) = S 1 , and the topology of R2 ∪ S 1 is given as follows: a sequence yn ∈ R2 converges to a unit vector V ∈ S 1 if and only if yn /||yn || → V . Geometrically, the closure of A in D × D ∪ (D × D)(∞) is a closed disc, while its closure in D × D is a square. This suggests the following result. Proposition 3.15. The identity map on D × D does not extend to a homeomorphism D× D → D× D∪(D× D)(∞). Therefore, these two compactifications are different. Proof. If the identity map extends on the whole space, it does so on a flat. By the above discussions, the closures of a flat in these two compactifications are different, the extension on the flat is impossible. In the case of X = D, the stabilizers of points in D(∞) are parabolic subgroups, and the correspondence from points in D(∞) to parabolic subgroups is one-to-one. Furthermore, all parabolic subgroups are conjugate. These results do not hold in the higher rank case. Let γ1 , γ2 be two geodesics in D. For any a, b ≥ 0 satisfying a 2 + b2 = 1, γ (t) = (γ1 (at), γ2 (bt)) defines a unit speed geodesic in D × D. Let P1 be the stabilizer of [γ1 ] in SU (1, 1), and P2 the stabilizer of [γ2 ]. Proposition 3.16. When a, b > 0, the stabilizer of [γ ] ∈ (D× D)(∞) in SU (1, 1)× SU (1, 1) is equal to P1 × P2 . On the other hand, when a = 0, the stabilizer of [γ ] is SU (1, 1) × P2 ; and when b = 0, the stabilizer of [γ ] is equal to P1 × SU (1, 1). Proof. For any g = (g1 , g2 ), g · γ (t) = (g1 γ1 (at), g2 γ2 (bt)), d(g · γ (t), γ (t))2 = d(g1 γ1 (at), γ1 (at))2 + d(g2 γ2 (bt), γ2 (bt))2 . Then the proposition follows easily from the definition of P1 , P2 .

The above proposition shows that not all parabolic subgroups are conjugate, i.e., there are different types of parabolic subgroups. Since for a, b > 0, when the ration a/b changes, the point [γ ] in (D × D)(∞) changes also and traces out a 1-simplex, but its stabilizer is always P1 × P2 . Therefore the correspondence between points in (D × D)(∞) and parabolic subgroups is not one-to-one. Another difference from the case of D is that for any two geodesics γ1 , γ2 in D, γ1 is equivalent to γ2 , i.e., limt→+∞ sup d(γ1 (t), γ2 (t)) < +∞ if and only if

30

L. Ji

for suitable parametrization, limt→+∞ d(γ1 (t), γ2 (t)) = 0. This conclusion does not hold for D × D. In fact, there are many parallel geodesics, i.e., γ1 , γ2 such that d(γ1 (t), γ2 (t)) is equal to a constant. Remark 3.17. The correspondence between the points of X (∞) and parabolic subgroups of G can naturally be explained using the spherical Tits building, and the structure of parallel geodesics by the Langlands decompositions. To compare compactifications, we introduce the following definition. 1

2

1

Definition 3.18. 1. Let X , X be two compactifications of X . X is said to dom2 1 2 inate X if the identity map on X extends to a continuous map from X to X , which is automatically surjective. 3 1 2 2. A third compactification X is called a common quotient of X and X if it is 1 2 dominated by both X and X : 3 1 2 3. A compactification X is called a common refinement of X and X if it domi1 2 nates both X and X : 1

2

Lemma 3.19. Given any two compactifications X , X , there exist a unique greatest common quotient (GC Q) which dominates all other common quotients, denoted by 1 2 X ∧ X , and a unique smallest common refinement (LC R) which is dominated by 1 2 all other common refinements, denoted by X ∨ X . Proof. The existence follows from Zorn’s lemma. In fact, the set of compactifications 1 2 of X which are common quotients of X and X is partially ordered. It can be shown that every chain has a maximal element, and Zorn’s lemma implies the existence of a maximal common quotient. The existence of LCR can be proved similarly. See [J2, Lemma 6.1.2] for more details. S

Proposition 3.20. For X = D × D, the GC Q of the Satake compactification X τ S and the conic compactification, X ∧ (X ∪ X (∞)), is the one point compactification X ∪ {∞}. Proof. For any two unit speed geodesics γ1 (t), γ2 (t) in D and any two positive numbers a, b satisfying a 2 + b2 = 1, γ (t) = (γ1 (at), γ2 (bt)) is a unit geodesic in S D × D. Identifying D × D τ with D × D, the geodesic γ (t) converges to ([γ1 ], [γ2 ]) S in D × D τ as t → +∞, and the limit is independent of a, b. On the other hand, the limit of γ (t) in D × D ∪ (D × D)(∞) depends on the ratio b/a. This implies that any common quotient of the closure of the flat spanned by γ1 , γ2 is the one point S compactification, and so is X ∧ (X ∪ X (∞)). S

Remark 3.21. This proposition shows that the two compactifications X and X ∪ X (∞) are completely different when X = D × D. The same conclusion holds for all symmetric spaces of rank at least 2. We also comment that the conic compactification does not respect the product structure.

Introduction to Symmetric Spaces and Their Compactifications S

31

S

The GCQ of X τ and X ∪ X (∞) is trivial, but the LCR X ∨ (X ∪ X (∞)) is not. In fact, it is related to the Martin compactification. As pointed out earlier, λ0 (D) = 14 , and hence λ0 (X ) = 12 for X = D × D. Proposition 3.22. For X = D × D, the Martin compactifications are determined as follows. S

1. For λ = λ0 (X ), X ∪ ∂λ X is isomorphic to the Satake compactification X . S 2. For λ < λ0 (X ), X ∪ ∂λ X is isomorphic to LCR X ∨ (X ∪ X (∞)). The Martin kernel functions K λ (x, ξ ) are products of the Martin kernels and the spherical functions of the disc D. This result holds also for other symmetric spaces X when the maximal Satake compactification is used, and is one of the main results in [GJT]. For X = D × D, Part (1) is due to Guivarch–Taylor [GT], and Part (2) due to Giulini–Woess [GW]. The closure of a flat A in X ∪ ∂λ X for λ < λ0 (X ) for X = D × D and its relation to the closures in the Satake and the conic compactifications is given in the following figure.

The Furstenberg boundary S 1 × S 1 can be identified with the following points in the Martin boundary, the middle points of the arcs.

32

L. Ji

A consequence of the identification of the Martin compactification gives the asymptotic behavior of the Brownian motion. Almost surely, a Brownian path starting from the origin will converge along a ray to such a distinguished middle point. A similar result holds for the general symmetric space and is due to Malliavin– Malliavin [MM]. For the Poincaré disc D, we discussed several more approaches to compactification. We will not construct all such compactifications for the bidisc. For compactifications of general symmetric spaces, see [BJ4].

4 Symmetric spaces In this section, we give a crash course on symmetric spaces. Most of the results presented here can be found in [Bo2] and [He], but we would like to establish directly from basic definitions relations between geometry of symmetric spaces and Lie group theory. An outline of this section is as follows: 1. Establish a correspondence between Riemannian symmetric spaces and orthogonal involutive Lie algebras. 2. Classify orthogonal involutive Lie algebras into three types. 3. Show that every noncompact simple Lie algebra admits a unique structure as an orthogonal involutive Lie algebra up to conjugation, and hence every noncompact simple Lie group has a unique quotient which is a Riemannian symmetric space. 4. Understand the curvature and geodesic submanifolds of a symmetric space in terms of Lie group theory. 5. Show that symmetric spaces of compact type are compact. 6. Show that symmetric spaces of noncompact type are noncompact. Let X be a complete Riemannian manifold. As defined in §2, X is a symmetric space if every point x ∈ X is an isolated fixed point of an involutive global isometry i x of X , which restricts to the local geodesic symmetry at x. Let G = I s 0 (X ) be the identity component of the isometry group of X .


33

Proposition 4.1. For any Riemannian manifold X , the group I s(X ) is a locally compact topological group. If X is a symmetric space, I s(X ) is a Lie group. The proof of this proposition is long. For details, see [He]. Lemma 4.2. For a symmetric space X , the isometry group I s(X ) acts transitively on X . In fact, the identity component G also acts transitively on X . Proof. For any two distinct points p, q, let γ (t) be a geodesic segment connecting them, γ (0) = p, γ (t0 ) = q. Let r = γ ( t20 ) be the middle point, and sr be the geodesic symmetry at r . Then sr ∈ I s(X ), and sr ( p) = q. This proves that I s(X ) acts transitively on X . To prove the second statement, define Tt = sγ ( t ) sγ (0) , t ∈ R. It can be checked 2 easily that Tt is an one-parameter subgroup in G and hence contained in G, and Tt (γ (0)) = γ (t), in particular Tt0 ( p) = q. This implies that G acts transitively on X . For each geodesic γ (t) in X , the isometries Tt induce translations on γ (t) and are called transvections on γ . The transvections for all geodesics generate a closed subgroup of G, which is proper if and only if X contains a nontrivial Euclidean (or flat) factor. In the following, we can replace the group G, cf. [He], by the subgroup generated by the transvections. From now on, we will assume that X is a symmetric space unless otherwise indicated. Let x0 ∈ X be a basepoint, and K its stabilizer in G. Then K is a compact subgroup of G. In fact, it can also be identified with a subgroup of O(Tx0 X ), the orthogonal subgroup of the tangent space with respect to the Riemannian metric. Since G acts transitively on X , G/K ∼ = X,

gK → gx0 .

A natural question is the following: Question 4.3. What kinds of pairs (G, K ) arise from symmetric spaces? Proposition 4.4. If G is a Lie group and K a compact subgroup, then the homogeneous space G/K admits a G-invariant Riemannian metric. Proof. Let g, k be the Lie algebras of G, K respectively. Then the tangent space of G/K at the identity coset can be identified with g/k. Since K is compact, there exists a K -invariant norm on g/k. Under the left translation, this norm defines a left G-invariant Riemannian metric on G/K . This G-invariant metric is not necessarily symmetric as defined before Proposition 4.1, and we need to characterize the symmetries of G/K in terms of the pair (G, K ). Proposition 4.5. Let X be a symmetric space, G the identity component of the isometry group and K the stabilizer of a basepoint x 0 . Let s0 be the geodesic symmetry at x0 . Then σ : G → G, g → s0 gs0 is an involutive automorphism, i.e., σ 2 = I d, σ = I d, such that K lies between the closed subgroup G σ of fixed points of σ and

34

L. Ji

the identity component G σ,0 of G σ . Furthermore, K does not contain any nontrivial normal subgroup of G. Proof. Since s0 ∈ I s(X ) and s0−1 = s0 , it is clear that σ is an involutive automorphism of I s(X ) and preserves the identity component G. For any k ∈ K , we claim that σ (k) = k, i.e., s0 ks0 = k. In fact, s0 ks0 fixes x0 . Since the differential of s0 at x0 is equal to minus the identity, the differential of s0 ks0 at x0 is equal to the differential of k. Since an isometry is determined by its differential at one point, this implies that s0 ks0 = k and hence k ∈ G σ . On the other hand, for any X in the Lie algebra of G σ , (dσ )X = X , where dσ is the differential of σ . This implies that σ (exp t X ) = exp t X and s0 exp t X = exp t X s0 for t ∈ R, and hence s0 exp t X x0 = exp t X x0 . Since x0 is an isolated fixed point of s0 , exp t X x0 = x0 for all t ∈ R, and hence exp t X ∈ K . This implies that G σ,0 ⊂ K . The last statement of the proposition follows from the following lemma. Lemma 4.6. The action of G on a homogeneous space G/K is effective if and only if K does not contain any nontrivial normal subgroup of G. Proof. We first show that any normal subgroup H of G contained in K acts as the identity on G/K . In fact, for h ∈ H , and any point gK ∈ G/K , h(gK ) = gh K = gK , where h ∈ H . On the other hand, if the G-action is not effective, let H be the subgroup of elements in G that act as the identity on G/K . Since H fixes the basepoint K in G/K , H ⊂ K . Next we show that H is a normal subgroup of G. For any h ∈ H and g ∈ G, h(gK ) = gK , and hence g −1 hg ∈ K . We claim that g −1 hg acts as the identity on G/K . In fact, for any m ∈ G and the point m K in G/K , g −1 hgm K = m((gm)−1 h(gm)K ) = m K , since (gm)−1 h(gm) ∈ K as observed earlier. The above lemma says that any normal subgroup H of G contained in K can be divided out on the right and hence acts trivially. Proposition 4.5 can be expressed in terms of Lie algebras. Proposition 4.7. Let g, k be the Lie algebras of G, K . Then k = {Y ∈ g | dσ Y = Y }. Let p = {Y ∈ g | dσ Y = −Y }. Then g = k ⊕ p, and ad(k) acts on g and preserves this decomposition, [k, k] ⊂ k, [k, p] ⊂ p. Furthermore, k does not contain any non-zero ideal of g.


35

Proposition 4.8. Let X = G/K , and π : G → X, g → gx0 be the natural projection map. Then dπe maps k to zero and p bijectively to Tx0 X , where e is the identity element. If Y ∈ p, then the geodesic passing through x0 with tangent vector dπe Y is given by exp tY x0 , and multiplication by exp tY on the cosets in G/K gives the isometry of X and induces translation along the geodesic exp tY x0 . Proof. Since X = G/K and p ∼ = g/k, it is clear that dπe (k) = 0 and dπe defines an isomorphism p → Tx0 X . For any Y ∈ p, let γ (t) be the unique unit speed geodesic such that γ (0) = x0 , γ (0) = Y . Define Tt = sγ ( t ) sγ (0) , t ∈ R. Then Tt Ts = Tt+s , and Tt defines an one2 parameter subgroup in G. Hence there exists Z ∈ g such that Tt = exp t Z . Since σ Tt = T−t , it follows that dσ Z = −Z , and hence Z ∈ p. By definition, exp t Z x0 = Tt x0 = γ (t)x0 = exp tY x0 . This implies that Z = Y , Tt = exp tY , and γ (t) = exp tY x0 .

Definition 4.9. An involutive Lie algebra is a pair (g, σ ) consisting of a real Lie algebra g and an involutive automorphism σ on g. Let g = k ⊕ p be decomposition by the ±1-eigenspaces of σ as above. It is called reduced if k does not contain any non-zero ideal of g. The restriction of ad(k) on p is called the isotropy representation of k on p. Definition 4.10. An orthogonal involutive Lie algebra is an involutive Lie algebra (g, σ ) which admits a positive non-degenerate quadratic form on g which is invariant under σ and ad(k). Proposition 4.11. For a Riemannian symmetric space X = G/K as above, let σ be the involution on G defined by σ (g) = s0 gs0 . Denote its differential on g by σ as well. Then (g, σ ) is an orthogonal involutive Lie algebra. Proof. We only need to find an invariant positive definite quadratic form on g. Let be the inner product on p induced from the Riemannian metric when p is identified with Tx0 X . Let < ·, · > be an inner product on k invariant under ad(k). For Y ∈ g, write Y = U + V , where U ∈ k, V ∈ p. Define < Y, Y >=< U, U > + < V, V > . It can be checked easily that this defines a positive definite quadratic form on g invariant under σ and ad(k). We now reverse the process and construct symmetric spaces from orthogonal involutive Lie algebras. Definition 4.12. Let G be a connected Lie group and K a closed subgroup. The pair (G, K ) is called a symmetric pair if there exists an involutive automorphism σ of G such that G σ,0 ⊂ K ⊂ G σ . If, in addition, the image AdG (K ) under the map AdG : G → G L(g) is compact, (G, K ) is called a Riemannian symmetric pair.

36

L. Ji

Every symmetric pair (G, K ) defines an involutive Lie algebra (g, σ ). If AdG (K ) is compact, p admits AdG (K )-invariant inner products, which in turn define a Ginvariant Riemannian metrics on G/K . Proposition 4.13. Let (G, K ) be a Riemannian symmetric pair. For every G-invariant Riemannian metric, the Riemannian manifold G/K is a globally Riemannian symmetric space. The geodesic symmetry s0 at the identity coset x0 = K ∈ G/K satisfies s0 ◦ π = π ◦ σ, where π : G → G/K . Proof. Denote the identity coset K in X = G/K by x0 . We define an involution s0 of X by setting s0 (gK ) = σ (g)K . Clearly x0 is an isolated fixed point of s0 . We need to show that s0 is an isometry. Since the metric on G/K is left Ginvariant, we claim that it suffices to show that for any g ∈ G, s0 ◦ L g = L σ (g) ◦ s0 ,

(∗)

where L g is the left multiplication on X by g. In fact, let < , >0 be the inner product at the basepoint x0 . Then the inner product at gx0 is L ∗g−1 < , >0 , and the inner product at s0 (g)x0 is L ∗σ (g−1 ) < , >0 , and s0 being an isometry means that s0∗ L ∗σ (g−1 ) < , >0 = L ∗g−1 < , >0 , which is equivalent to s0∗ L ∗σ (g−1 ) < , >0 = L ∗g−1 s0∗ < , >0 . The latter follows from the claim (*) when g is replaced by g −1 . To prove the claim, we note that for any hx0 ∈ X , L σ (g) ◦ s0 (hx0 ) = L σ (g) (σ (h)x0 ) = σ (g)σ (h)x0 . On the other hand, s0 ◦ L g (hx0 ) = s0 (ghx0 ) = σ (gh)x0 = σ (g)σ (h)x0 . This proves the claim.

We note that any involutive isometry with x0 as an isolated fixed point is the geodesic symmetry at x0 , and hence s0 defined above is the geodesic symmetry at x0 . Since G acts transitively and isometrically, the geodesic symmetry at other points are also isometric, and the proposition is proved.


37

Proposition 4.14. For any orthogonal involutive Lie algebra (g, σ ), let g = k + p be the decomposition given by the eigenspaces of σ . Let G be the simply connected Lie group whose Lie algebra is equal to g, and K the connected subgroup corresponding to k. Then G/K with the invariant metric induced from the inner product on g is a Riemannian symmetric space. Proof. Since G is simply connected, the automorphism σ on g lifts to an involutive automorphism on G. Since K is connected, σ fixes K . In fact, K is equal to the identity component G σ,0 , and (G, K ) is a symmetric pair. Let < , > be the positive definite quadratic form on g invariant under σ and ad(k). Under the identification p∼ = Tx0 G/K , where x0 = K ∈ G/K , the restriction of < , > to p defines a Ad(K )invariant inner product on Tx0 G/K , which extends to a left G-invariant metric on G/K . By the previous proposition, G/K is a Riemannian symmetric space. Propositions 4.11 and 4.14 establish a correspondence between symmetric spaces and orthogonal involutive Lie algebras, and hence the study of symmetric spaces is reduced to the study of involutive Lie algebras. Orthogonal involutive Lie algebras: properties and classifications into types Next we summarize several facts about involutive Lie algebras and classifications of symmetric spaces into the three types: flat, compact and noncompact types. Definition 4.15. 1. A Lie algebra k is called compact if it is the Lie algebra of a compact Lie group. 2. The group Adgk is the connected subgroup of the adjoint subgroup Adg corresponding to the Lie subalgebra adgk. Proposition 4.16. The following conditions on an involutive Lie algebra (g, σ ) are equivalent: 1. The involutive Lie algebra (g, σ ) is orthogonal. 2. k is a compact Lie subalgebra of g, and the isotropy representation of k on p leaves invariant a positive definite quadratic form. 3. Adgk is a compact subgroup of Adg. Lemma 4.17. The following conditions on a Lie algebra g are equivalent: 1. g is the Lie algebra of a compact Lie group. 2. Adg is compact. 3. Adg leaves invariant a positive definite quadratic form on g. Proof. (1) ⇒ (2) follows from the fact that if G is a connected Lie group whose Lie algebra is equal to g, then there exists a surjective homomorphism G → Adg. (2) ⇒ (3) follows from the fact a finite dimensional representation space of a compact Lie group admits an inner product invariant under the Lie group. To show that (3) ⇒ (1), we first prove that (3) implies that g = z(g) + [g, g] and [g, g] is semisimple and compact, where z(g) is the center of g.

38

L. Ji

Let Q be the positive definite quadratic form on g invariant under Adg. Since z(g) is an ideal of g and Q is invariant under g, the orthogonal complement, denoted by g , of z(g) in g with respect to Q is also an ideal in g. The Killing form B of g is the restriction of the Killing form B of g. Since Q is invariant under Ad(g), there exists a basis X 1 , · · · , X n of g such that Q(X ) =

n i=1

xi2 ,

X=

n

xi X i ∈ g.

i=1

With respect to this basis, for each X ∈ g, ad(X ) is represented by a skew symmetric matrix. Write ad X = (ai j (X )), i, j = 1, · · · , n. Then ai j (X ) = −a ji (X ), and ai j (X )2 ≤ 0. B(X, X ) = tr (ad X ad X ) = − i, j

This implies that B(X, X ) = 0 if and only if ad(X ) = 0, i.e., X ∈ z(g), and hence B is negative definite on g . This in turn implies that g is semisimple and compact. In fact, the compactness follows from the fact that Adg is a closed subgroup of the compact group O(B ). Since g = z(g) + g , [g, g] = [g , g ] = g is semisimple and Adg is compact. To prove (1), let Z = S 1 × · · · × S 1 , where the number of copies of S 1 is equal to dim z(g). Then the compact Lie group Z × Adg has Lie algebra equal to g. Proof of Proposition 4.16 (1) ⇒ (2). Assume that (g, σ ) is orthogonal. Let < , > be the k-invariant positive definite quadratic form on g. Then its restriction to k is positive definite and invariant under Adk. By the previous lemma, k is a compact Lie algebra. The restriction of < , > to p is also positive definite and invariant under ad(k). (2) ⇒ (3). Combining k-invariant positive definite forms on k and p, we get a kinvariant positive definite form on g. Denote the orthogonal group of g with respect to this form by O(g). Then we have an embedding Adgk → O(g). We need to show that Adgk is a closed subgroup. The involutive automorphism σ extends to an automorphism of Adg, and Adgk is the identity component of the fixed subgroup of σ . Since Adg is closed, Adgk is also closed. (3) ⇒ (1). Take a positive definite quadratic form on g invariant under σ . Then averaging via integration over the compact group Adgk gives an Adgk-invariant positive definite quadratic form on g which is also invariant under σ . In the proof of Lemma 4.17, we have used an important class of Lie algebras. Definition 4.18. A Lie algebra g is called reductive if its adjoint representation is fully reducible, i.e., for any ideal a in g, there exists a complementary ideal b such that a + b = g. Proposition 4.19. A Lie algebra g is reductive if and only if g = z(g) + [g, g] and [g, g] is semisimple.


39

Proof. Suppose that g is reductive. Let a be the direct sum of minimal ideals of g, i.e., each of which does not contain a smaller nonzero ideal. We claim that a = g. If not, a is a proper ideal, and there exists a nontrivial ideal b such that a + b = g. Taking a nonzero ideal in b of smallest dimension and adding it to a gives a contradiction. Write g = a = a1 ⊕ · · · a j ⊕ a j+1 ⊕ ak , where a1 , · · · , a j are abelian and one dimensional, and a j+1 , · · · , ak are nonabelian and simple. Then z(g) = a1 ⊕ · · · ⊕ a j , [g, g] = a j+1 ⊕ · · · ak , which is semisimple, and g = z(g) + [g, g]. On the other hand, assume that g = z(g) + [g, g] and [g, g] is semisimple. Since abelian and semisimple Lie algebras are fully reducible and hence reductive, it is clear that g is reductive. Proposition 4.20. Let g be a real Lie algebra of matrices over R or C. If g is closed t under the conjugate transpose Y → Y ∗ = Y , then g is reductive. Proof. For matrices X, Y , define < X, Y >= Re T r (X Y ∗ ). This defines a positive definite quadratic form on g. For any ideal a ⊂ g, we need to get a complementary ideal. Let a⊥ be its orthogonal complement with respect to < , >. We claim that a⊥ is an ideal and hence the desired complementary ideal. For any X ∈ a⊥ , Y ∈ g, Z ∈ a, < [X, Y ], Z > = Re T r (X Y Z ∗ − Y X Z ∗ ) = −Re T r (X Z ∗ Y − X Y Z ∗ ) = −Re T r (X (Y ∗ Z )∗ − X (Z Y ∗ )∗ ) = −Re T r (X (Y ∗ Z − Z Y ∗ )∗ ) = − < X, [Y ∗ , Z ] >= 0. In the last equation, we used the fact that a is an ideal and Y ∗ ∈ g. Since Z ∈ a is arbitrary, [X, Y ] ∈ a⊥ . In the above proposition, the converse is also true with respect to a suitable basis. An immediate corollary is that many classical Lie algebras are reductive. For example, u(n) = {X ∈ gl(n, C) | X + X ∗ = 0} is clearly closed under X → X ∗ and hence reductive. Remark 4.21. The class of reductive groups is important because it is closed under induction when passing to the Levi factors of parabolic subgroups, which plays an important role in compactifications of symmetric spaces.

40

L. Ji

To classify orthogonal involutive Lie algebras, we need to decompose them into irreducible summands. Lemma 4.22. Let (g, σ ) be an involutive Lie algebra and z(p) the centralizer of p in g. Then z(p) ∩ k is the greatest ideal of g contained in k. Proof. We first show that z(p) ∩ k is an ideal of g. Since [z(p) ∩ k, g] = [z(p) ∩ k, k] + [z(p) ∩ k, p] = [z(p) ∩ k, k] and [z(p) ∩ k, k] ⊂ k, it suffices to show that [z(p) ∩ k, k] ⊂ z(p). But this follows from the Jacobi identity [[z(p) ∩ k, k], p] = [z(p) ∩ k, [k, p]] + [z(p) ∩ k, p] ⊂ [z(p), p] + [0, k] = 0. To show that it is the maximal ideal in k, let I ⊂ k be an ideal of g. Then [I, p] ⊂ I ⊂ k. On the other hand, [I, p] ⊂ [k, p] ⊂ p, and hence [I, p] ⊂ k ∩ p = 0. Therefore, I ⊂ z(p) ∩ k. Corollary 4.23. An involutive Lie algebra (g, σ ) is reduced if and only if the isotropy representation of k on p is faithful. Corollary 4.24. For any involutive Lie algebra (g, σ ), (g/z(p) ∩ k, σ ) is a reduced involutive Lie algebra. Lemma 4.25. If an orthogonal involutive Lie algebra (g, σ ) is reduced, the restriction of the Killing form of g to k is negative definite. Proof. Since adgk leaves invariant a positive definite quadratic form on g, for any X ∈ k, adg X is semisimple with purely imaginary eigenvalues iλ1 , · · · , iλn ∈ iR. This implies that n B(X, X ) = (iλ j )2 ≤ 0, j=1

and the equality holds if and only if adg X = 0, or X ∈ z(g). Since (g, σ ) is reduced, k does not contain any nonzero element in z(g). This proves that the Killing form is negative definite on k. Proposition 4.26. Let (g, σ ) be a reduced orthogonal involutive Lie algebra and z(p) the centralizer of p in g. Then z(p) is an abelian ideal of g contained in p, and g is semisimple if and only if z(p) is trivial. Proof. Clearly z(p) is a subalgebra of g. Since p is stable under σ , z(p) is also stable under σ . This implies that z(p) = z(p) ∩ k ⊕ z(p) ∩ p.


41

Since (g, σ ) is reduced, by Lemma 4.22, z(p) ∩ k = 0, and hence z(p) ⊂ p. This implies that [z(p), z(p)] = 0, i.e., z(p) is abelian. Next we show that z(p) is an ideal of g. It suffices to show that [k, z(p)] ⊂ z(p). Since [k, p] ⊂ p, by the Jacobi identity, [p, [k, z(p)]] = [[p, k], z(p)] + [k, [p, z(p)]] ⊂ [p, z(p)] + [k, 0] = 0, and hence [k, z(p)] ⊂ z(p). This proves the first statement. If g is semisimple, the ideal z(p) has to be trivial. On the other hand, suppose that z(p) = 0. Let a be the largest subspace where the Killing form is zero, i.e., a = {X ∈ g | B(X, Y ) = 0, Y ∈ g}. Since the Killing form is invariant under σ , a is stable under σ , and hence a = a ∩ k ⊕ a ∩ p. Since the restriction of the Killing form of g to k is the Killing form of k, by Lemma 4.25, a ∩ k = 0. Then a ⊂ p, and [a, p] ⊂ a ∩ [p, p] ⊂ a ∩ k = 0, and hence a ⊂ z(p). This implies that a = 0. Therefore, g is semisimple. Definition 4.27. A reduced orthogonal Lie algebra (g, σ ) is maximal if for any reduced orthogonal involutive Lie algebra (g , σ ) such that g ⊃ g, σ restricts to σ on g and p = p, the equality g = g holds. Proposition 4.28. Let (g, σ ) be a reduced semisimple orthogonal involutive semisimple Lie algebra. Then it is maximal and k = [p, p]. Proof. Let (g , σ ) ⊃ (g, σ ) be a reduced orthogonal involutive Lie algebra such that σ restricts to σ and p = p. Let m = p + [p, p]. We claim that m is an ideal of g . In fact, [m, k ] = [p, k ] + [[p, p], k ] ⊂ p + [[p, k ], p] ⊂ p + [p , p ] = p + [p, p] = m,

[m, p ] = [p, p] + [[p, p], p] ⊂ [p, p] + [k, p] ⊂ [p, p] + k = m. Since g is semisimple, m is also semisimple and is a direct summand of g . Let b be the complementary ideal in g , g = m ⊕ b. We claim that b = zg (m). In fact, for any X ∈ b, [X, m] ⊂ b. Since m is an ideal, [X, m] ⊂ m. This implies that [X, m] = 0 and X ∈ zg (m). On the other hand, m is semisimple, if X ∈ zg (m), then X ∈ b. This proves the claim. Since m is stable under σ , zg (m) is also stable under σ , and hence zg (m) = zg (m) ∩ p + zg (m) ∩ k . Since g is semisimple, by the previous proposition, zg (m) ∩ p ⊂ zg(p) = 0. Since g is reduced, zg (m) ∩ k is an ideal contained in k and hence equal to zero. This implies that zg (m) = 0, and g = m. Since m ⊂ g, we get g = g and k = [p, p].

42

L. Ji

Definition 4.29. An orthogonal involutive Lie algebra (g, σ ) is called irreducible if the isotropy (or adjoint) representation of k on p is irreducible, and called flat if [p, p] = 0. Proposition 4.30. If (g, σ ) is irreducible, then k is a maximal subalgebra of g. Proof. If m is an algebra of g containing k, then m = k ⊕ (m ∩ p). In fact, for any X ∈ m, write X = Y + Z , where Y ∈ k, Z ∈ p. Then Z = X − Y ∈ m also. Since (m ∩ p) is invariant under k, by the irreducibility assumption, either m ∩ p = 0 or m ∩ p = p, which implies either m = k or m = g. Proposition 4.31. If (g, σ ) is irreducible and Q a positive definite quadratic form on p which is invariant under k, then Q is equal to a multiple of the Killing form B. Proof. Define A : p → p by B(X, Y ) = Q(AX, Y ),

X, Y ∈ p.

Since B is invariant under k, A is equivariant with respect to the isotropy representation of ad(k). By the irreducibility assumption, A is a multiple of the identity map, and hence Q is a multiple of B. Theorem 4.32. Let (g, σ ) be a reduced orthogonal involutive Lie algebra. Then (g, σ ) is direct sum of a flat reduced orthogonal involutive Lie algebra (g0 , σ0 ) and irreducible reduced semisimple involutive Lie algebras (g j , σ j ), j = 1, · · · , a, g = g0 ⊕ g1 ⊕ · · · ⊕ ga . This decomposition is unique up to the order of the factors, and z(p) = p0 . Proof. Let Q be a positive definite quadratic form on p invariant under k. The idea is to decompose p under the isotropy representation of k, p = p0 ⊕ · · · ⊕ pa . For each p j , define k j = [p j , p j ], g j = k j + p j , to obtain the decomposition of g. Specifically, let A : p → p be the linear transformation defined by Q(AX, Y ) = B(X, Y ) as in the proof of the previous proposition. Since Q, B are symmetric, Q(AX, Y ) = Q(X, AY ). With respect to an orthonormal basis of p, A is represented by a symmetric matrix, and hence A is diagonalizable. Let q j be the eigenspace of A of eigenvalue c j , j = 0, 1, · · · , where c0 = 0 and c j = ck if j = k. Then we have


B(q j , q j ) = Q(qi , q j ) = 0,

43

j = k,

B(q0 , p) = 0, B|q j = c j Q|q j . Since B, Q are invariant under k, A commutes with ad(k), and hence each q j is invariant under k. For j ≥ 1, decompose q j into irreducible orthogonal subspaces: q j = ⊕k p jk , and p jk are perpendicular to each other with respect to Q. Reindex the subspaces p jk by p j for j ≥ 1, we get a decomposition p = ⊕aj=1 p j ⊕ q0 . We claim that [p j , pk ] = 0,

j = k,

[q0 , p j ] = 0, [q0 , q0 ] = 0. This claim follows from the fact that if u, v are two subspaces of p such that B(u, v) = 0 and [k, v] ⊂ v, then [u, v] = 0. In fact, for any U ∈ u, V ∈ v, then W = [U, V ] ∈ k, and hence [V, W ] ∈ v. This implies that B(W, W ) = B([U, V ], W ) = B(U, [V, W ]) = 0. By Lemma 4.25, B is negative definite on k, and hence W = 0. Define g j = p j + [p j , p j ], j ≥ 1. Then it can be checked easily that 1. 2. 3. 4. 5.

[g j , gk ] = 0 if j = k, [q0 , g j ] = 0 for j ≥ 1, [k, g j ] ⊂ g j , [p, g j ] = [p j , g j ] ⊂ g j , g j is an ideal of g.

We claim that g j is semisimple for all j ≥ 1. The restriction of the Killing form B of g gives the Killing form of g j . Since Q(p j , k) = 0, B(p j , k) = 0. By the choice of p j and Proposition 4.31, B is nondegenerate on ⊕p j , and by Lemma 4.25, nondegenerate on k. Combined with B(p j , pk ) = 0 for j = k, we conclude that the restrictions B|p j , B|[p j ,pk ] are nondegenerate. Therefore, B|g j is non-degenerate and g j is semisimple. Let m = g1 + · · · + ga . Since m is semisimple, by the proof of Proposition 4.28,

44

L. Ji

g = m + zg(m). Let g0 = zg(m). Then

g = g0 + g1 + · · · + ga .

We claim that g0 is flat. Clearly, q0 ⊂ g0 ∩ p. Since p = q0 ⊕ ⊕ j p j and the p j , j ≥ 1, are semisimple, zg(m) ∩ p j = 0, and hence g0 ∩ p ⊂ q0 . This implies that p0 = g0 ∩ p = q0 , [p0 , p0 ] ⊂ [q0 , p] = [q0 , p j ] = 0. j

Since g j are semisimple, it is clear that z(p) = p0 . Since p j , j ≥ 1, are irreducible summands of the representation of k on the complement of p0 in p with respect to Q, and k j = [p j , p j ], g j = k j + p j , we have the uniqueness of the above decomposition of g. Corollary 4.33. 1. If g is simple, then (g, σ ) is irreducible and k is a maximal subalgebra of g, and hence the normalizer n(k) of k is equal to k. 2. If g is semisimple, n(k) = k. Corollary 4.34. Let (g, σ ) be an irreducible, reduced, orthogonal involutive Lie algebra, Q a positive definite quadratic form on p invariant under k, c the constant such that B|p = cQ. Then there are three possibilities: 1. c = 0, and (g, σ ) is flat and g has no semisimple ideal. 2. c > 0, g is simple and noncompact and k is a maximal compact subalgebra. 3. c < 0, g is compact and either g is simple or g = g1 × g1 , where g1 is simple and σ (X, Y ) = (Y, X ), where X, Y ∈ g1 . Proof. If c = 0, then g = g0 in the notation of the previous proposition, and case (1) is reduced to showing that if (g, σ ) is an irreducible flat reduced orthogonal involutive Lie algebra, then g does not contain any semisimple ideal. In fact, let a be a semisimple ideal of g. We show that a = 0 in three steps: (1) a ∩ p is an ideal of g, (2) a ∩ p is equal to 0, (3) a ⊂ z(p) and a = 0. For step (1), let X ∈ a ∩ p. Then for any Y ∈ k, [X, Y ] ∈ [p, k] ⊂ p, and hence [X, Y ] ∈ a ∩ p. Since g is flat, [p, p] = 0, and so for any Y ∈ p, [X, Y ] ⊂ [p, p] = 0. For step (2), we note that since a ∩ p is also semisimple, [p, p] = 0 implies a ∩ p = 0. For step (3), we note that [a, p] ⊂ a, [a, p] ⊂ [k + p, p] ⊂ p, and hence [a, p] ⊂ a ∩ p = 0 by step (2). This implies that a ⊂ z(p). Since g is reduced, z(p) ⊂ p and hence a ⊂ p. The semisimplicity of a contradicts [p, p] = 0. Assume that c = 0. Then g is semisimple. Since g is semisimple, by Lemma 4.25, B|k is negative definite. If c < 0, then B is negative definite on g, and hence


45

by Lemma 4.17 g is compact. If c > 0, then B is indefinite and nondegenerate, and hence g is noncompact. We need to show that if g is not simple, then it is of the form in case (3). Decompose g into simple ideals. Then σ permutes these ideals. Since g is irreducible and σ is an involution, there exist two simple ideals g1 , g2 such that g = g1 + g2 , σ (g1 ) = g2 . Then k = {(X, σ (X )) | X ∈ g1 } ∼ = g1 . Since k is compact, g1 and hence g is compact. This is exactly the second case in case (3). Definition 4.35. For an irreducible reduced orthogonal involutive Lie algebra (g, σ ), let c be the constant such that B|p = cQ as in the previous proposition. It is called of compact type if c < 0, and of noncompact type if c > 0, and flat if c = 0. Definition 4.36. A Riemannian symmetric pair (G, K ), or equivalently the symmetric space G/K , is called of compact type, noncompact type, or flat if the corresponding reduced orthogonal involutive Lie algebra (g, σ ) is a product of irreducible orthogonal involutive Lie algebras of compact, noncompact type, flats respectively. Orthogonal involutive Lie algebras: Existence The above discussions show that study of symmetric spaces is reduced to study of orthogonal involutive Lie algebras. Then a natural problem is the following one. Question 4.37. Given a Lie algebra g, when does it admit an orthogonal involutive Lie algebra structure? If g is abelian, by defining σ = −1, (g, σ ) is an orthogonal involutive Lie algebra. More generally, if g contains an abelian ideal p and a subalgebra k such that Adgk is compact and by taking Q to be any positive definite quadratic form on g g = p ⊕ k, then (g, σ ) can clearly be given an involutive orthogonal Lie algebra structure. For example, for g = o(n) × Rn , where o(n) acts on Rn by multiplication, we can take p = Rn , and k = o(n). The involutive Lie algebra (g, σ ) is flat but g is not abelian. Proposition 4.38. If g is reductive and (g, s) is reduced and flat, then g is abelian. Proof. Since g is reductive, [g, g] is semisimple and g = z(g) + [g, g]. Since g is reduced and flat, by Corollary 4.34, g does not contain any semisimple ideal. This implies that g = z(g) is abelian. Corollary 4.39. Let g be reductive, and (g, σ ) be a reduced orthogonal involutive Lie algebra. Then (g, σ ) = (g0 , σ0 ) + (g1 , σ1 ) + · · · + (gn , σn ), where g0 is abelian, and for j ≥ 1, g j is semisimple.

46

L. Ji

After the flat case is taken care of, the question becomes: Which semisimple Lie algebras have orthogonal involutive Lie algebra structure and what are their types? The answer to this question depends on the existence of compact real forms of complex Lie algebras and Cartan involution. First we notice a duality between the compact and noncompact type. Let (g, σ ) be an involutive Lie algebra, and g = k + p the decomposition according to the eigenspaces of σ . Let gC = g ⊗ C, the complexification of g. Let p = ip, and g = k + p = k + ip. Then it can be checked easily that g is a real algebra and a real form of g ⊗ C. Define σ on g by σ |k = 1 and σ |p = −1. Then (g , σ ) is an involutive Lie algebra. Similarly, the positive definite quadratic form Q on p induces a positive definite quadratic form Q on p , Q (i X, iY ) = Q(X, Y ),

X ∈ p.

Lemma 4.40. Let (g, σ ) be a reduced, irreducible orthogonal involutive Lie algebra. Then (g, σ ) is of compact type if and only if (g , σ ) is of noncompact type. Proof. We need to compute the Killing form on g and g . Let B be the Killing form of gC as a complex Lie algebra. We claim that B restricts to the Killing form of g. In fact, let X 1 , · · · , X n be a basis of g over R. Then they also form a basis of gC over C. For any X, Y ∈ g, ad X ad Y maps g to g and ig to ig. This implies immediately that the Killing form B(X, Y ) = T r (ad X ad Y ) restricts to the Killing form of g. Similarly, since g is a real form of gC , B|g is also the Killing form of g . Now for X ∈ p, i X ∈ p , and B(i X, i X ) = −B(X, X ), and hence if B = cQ on p, then B = −cQ on p . This implies that (g, σ ) is of compact type if and only if (g , σ ) is of noncompact type. The above lemma shows that to study irreducible reduced orthogonal involutive Lie algebras (g, σ ), it suffices to study those of noncompact type. In this case, g is a simple noncompact real algebra. On the other hand, for the compact type, g could be simple or not. Partly for this reason, we concentrate on the noncompact case. Theorem 4.41. For every noncompact simple real Lie algebra g, there exists a unique orthogonal involutive Lie algebra structure on g up to conjugation by elements in Adg. As a corollary, we obtain that every semisimple Lie algebra whose simple factors are noncompact admits a unique structure of orthogonal involutive Lie algebra up to conjugation and scaling on the simple factors. We need some preparation before proving the theorem. Definition 4.42. Let g be a real semisimple Lie algebra. An involution θ of g is called a Cartan involution if the bilinear form


Bθ (X, Y ) = −B(X, θ Y ),

47

X, Y ∈ g,

is positive definite, where B is the Killing form of g. Lemma 4.43. For any simple noncompact Lie algebra g, there is an one-to-one correspondence between the set of Cartan involutions and the set of orthogonal involutive Lie algebras on g. Proof. Given a Cartan involution θ, we claim that (g, θ) has an orthogonal involutive Lie algebra structure. Let k = {X ∈ g | θ (X ) = X }, and p = {X ∈ g | θ(X ) = −X }. It suffices to show that Bθ is invariant under θ and adk. For X, Y ∈ g, Bθ (θ X, θ Y ) = −B(θ X, Y ) = −B(X, θ Y ) = Bθ (X, Y ). This implies that k and p are orthogonal to each other with respect to Bθ . Since Bθ = −B on k and Bθ = B on p, it is clear that Bθ is invariant under adk. On the other hand, for any orthogonal involutive structure (g, σ ) on g. Let Q be the positive definite quadratic form on g invariant under k and σ . Since g is noncompact, B|p = cQ|p for some positive constant c. Define θ = σ . Then it can be checked easily that θ is a Cartan involution. The above proposition reduces the problem to the existence and uniqueness of Cartan involutions. Let gC be the complexification of g, and σ be the conjugation on gC with respect to g, i.e., for X + iY ∈ gC , where X, Y ∈ g, σ (X + iY ) = X − iY . Suppose there is a compact real form u of gC stable under σ . Then σ restricts to an involution on u. Let u = k + p ,

σ |k = 1, σ |p = −1.

Then every vector in k is real with respect to the real form g of gC , and hence k ⊂ g. Since σ |p = −1, every vector in p is purely imaginary with respect to g, and hence p = ip ⊂ g. We claim that g = k ⊕ p. In fact, k = u ∩ g and p = iu ∩ g, and hence k ∩ p = 0, because u ∩ iu = 0. Since k ⊕ p ⊂ g and dim k + dim p = dim u = dim g, the claim follows. Define θ : g → g by θ |k = 1 and θ|p = −1. Lemma 4.44. The transformation θ : g → g is a Cartan involution. Proof. Clearly θ 2 = id. We need to check that θ is a homomorphism. For X 1 , X 2 ∈ k, Y1 , Y2 ∈ p, [X 1 + Y1 , X 2 + Y2 ] = [X 1 , X 2 ] + [Y1 , Y2 ] + [X 1 , Y2 ] + [Y1 , X 2 ], and θ ([X 1 + Y1 , X 2 + Y2 ]) = [X 1 , X 2 ] + [Y1 , Y2 ] − [X 1 , Y2 ] − [Y1 , X 2 ]. On the other hand,

48

L. Ji

[θ (X 1 + Y1 ), θ (X 2 + Y2 )] = [X 1 − Y1 , X 2 − Y2 ] = [X 1 , X 2 ] + [Y1 , Y2 ] − [X 1 , Y2 ] − [Y1 , X 2 ]. This shows that for any X, Y ∈ g, θ ([X, Y ]) = [θ(X ), θ (Y )], and hence θ is an involution. Since u is compact, B|i p < 0 and hence B|p > 0. Similarly, B|k < 0. This implies that Bθ is positive definite on g. Proposition 4.45. Let g be a real semisimple Lie algebra, and gC its complexification. Denote the conjugation of gC with respect to g by σ . Then there is an one-to-one correspondence between Cartan involutions of g and real compact forms of gC which are stable under σ . Proof. The previous lemma shows that any σ -invariant compact real form u of gC defines a Cartan involution of g. On the other hand, if θ is a Cartan involution of g, let k = {X ∈ g | θ (X ) = X }, p = {X ∈ g | θ (X ) = −X }. Define u = k + ip. Clearly, u is a real form of gC . By Proposition 4.40, u is a compact real form of gC . Compact real forms of semisimple complex Lie algebras The above proposition shows that the existence of Cartan involutions on g is equivalent to the existence of compact real forms of gC with respect to the real structure induced by g. We solve this problem in two steps: 1. Existence of compact real forms of gc with respect to some real structure. 2. Existence of compact real forms stable under the conjugation σ induced by g. Let gC be a complex semisimple Lie algebra, h ⊂ gC a Cartan subalgebra, i.e., a maximal abelian subgroup consisting of semisimple elements. Let (gC , h) be the set of roots of gC with respect to h. For any α ∈ (gC , h), let gα be the root space. Define g0 = {X ∈ g | [H, X ] = 0, H ∈ h}. Proposition 4.46. 1. gC = g0 + α∈ gα . 2. For every α ∈ , dim gα = 1. 3. Let B be the Killing form. Then for any X α ∈ gα , X −α ∈ g−α , [X α , X −α ] = B(X α , X −α )Hα , where Hα is the unique vector in h defined by α(H ) = B(H, Hα ) for all H ∈ h. This is a basic result for complex semisimple Lie algebras, see [He]. Proposition 4.47. There exist root vectors X α ∈ gα , α ∈ , satisfying the following conditions:


49

1. [X α , X −α ] = Hα . 2. [X α , X β ] = Nα,β X α+β if α + β ∈ , and zero otherwise; and the constants 2 = 1 q(1 + p)|α|2 , Nα,β satisfy two conditions: (a) N−α,−β = −Nα,β , (b) Nα,β 2 where p, q are integers such that α + kβ ∈ if and only if k ∈ [− p, q], in particular Nα,β are real. This proposition allows us to construct real forms. Definition 4.48. Let h0 be the real subspace of h where all the roots take real values. It can be shown that the subspace h0 is a real form of h, and a real basis of h0 is also a complex basis of h. Definition 4.49. A basis of gC consisting of a (real) basis of h0 and the root vectors in Proposition 4.47 is called a Weyl basis. The point of choosing the Weyl basis is that all the structural constants Nα,β are real, and a real form of g can therefore be constructed. The basic steps of the proof of Proposition 4.47 is as follows: (1) Study general properties of structural constants, (2) Modify the root vectors. To prove it, we need some preparations. For each pair of roots α, −α ∈ , fix E α ∈ gα , E −α ∈ g−α such that B(E α , E −α ) = 1. Then [E α , E −α ] = B(E α , E −α )Hα = Hα . For α, β ∈ , if α + β ∈ , define cα,β by [E α , E β ] = cα,β E α+β , otherwise, define cα,β = 0. Proposition 4.50. For any α, β ∈ with α + β ∈ , let p, q be integers such that β + kα ∈ if and only if k ∈ [− p, q]. Then 1 cα,β c−α,−β = − q(1 + p)|α|2 . 2 We need two lemmas to prove this proposition. Lemma 4.51. For α, β, γ ∈ , if α + β + γ = 0, then cα,β = cβ,γ = cγ ,α . Proof. By the Jacobi identity, [[E α , E β ], E γ ] + [[E β , E γ ], E α ] + [[E γ , E α ], E β ] = 0, cα,β [E α+β , E γ ] + cβ,γ [E β+γ , E α ] + cγ ,α [E γ +α , E β ] = 0, cα,β Hγ + cβ,γ Hα + cγ ,α Hβ = 0. Then the lemma follows from Hγ = −Hα − Hβ . Lemma 4.52. For the p, q defined above, [E −α , [E α , E β ]] =

1 q(1 + p)|α|2 E β . 2

50

L. Ji

Proof. Let e, f, h be the generator of sl(2, C) satisfying [e, f ] = h, [h, e] = 2e, [h, f ] = −2 f . Identify the span of Hα , E α , E −α with sl(2, C) by e=

2 Eα , |α|2

h=

2 Hα , |α|2

f = E −α .

Then the equation in the lemma is equivalent to [ f, [e, E β ]] = q(1 + p)E β . Let g = ⊕k∈[− p,q] gβ+kα . Then sl(2, C) acts on g and the representation is irreducible. The highest weight is E β+qα . The lemma follows from the following fact about representation of sl(2, C): For each integer m ≥ 1, there exists a unique irreducible complex representation π of sl(2, C) of dimension m which admits a basis v0 , · · · , vm−1 such that (1) π(h)vi = (m − 1 − 2i)vi , (2) π(e)v0 = 0, (3) π( f )vi = vi+1 , where vm is defined to be zero, (4) π(e)vi = i(m − i)vi−1 . In particular, v0 is the highest weight vector. In fact, E β is a multiple of (ad f )q E β+qα and belongs to the span of vq . By assuming E β = vq and E β−α = vq+1 , we obtain (ad e)E β = q( p + q + 1 − q)E β−α , (ad f ad e)E β = q( p + 1)E β .

This proves the lemma. Proof of Proposition 4.50. By Lemma 4.52, 1 q(1 + p)|α|2 E β , 2 1 c−α,α+β cα,β E β = q(1 + p)|α|2 E β , 2

[E −α , [E α , E β ]] =

and hence c−α,α+β cα,β =

1 q(1 + p)|α|2 . 2

By Lemma 4.51, c−α,α+β = c−β,−α = −c−α,−β , and hence

1 c−α,−β cα,β = − q(1 + p)|α|2 . 2 This proves Proposition 4.50.

To prove Proposition 4.47, we need to modify E α such that c−α,−β = −cα,β . Suppose there exists an automorphism ϕ˜ of g such that ϕ| ˜ k = −1 and ϕ(E ˜ α) = −E −α , then ˜ β )] = [E −α , E −β ], ˜ α ), (E ϕ([E ˜ α , E β ]) = [ϕ(E and hence c−α,−β = −cα,β . For this purpose, we need the following result.


51

Proposition 4.53. Let gC , gC be two complex semisimple Lie algebras with Cartan subalgebras h, h and root systems , respectively. Let ϕ : h → h be an isomorphism such that its dual ϕ ∗ : h ∗ → h∗ satisfies ϕ ∗ ( ) = . For α ∈ , write (ϕ ∗ )−1 (α) = α . Fix a set of simple roots α1 , · · · , αl in . For each j, choose nonzero vectors E α j ∈ gα j , E α j ∈ gα . Then there exists a unique Lie j

algebra isomorphism

ϕ˜ : gC → gC such that ϕ| ˜ h = ϕ and ϕ(E ˜ α j ) = E α j for j = 1, · · · , l.

Briefly, for each α j , there is a canonical choice of H j and Fα j ∈ gα j such that H j , E α j , Fα j span a Lie subalgebra isomorphic to sl(2, C). Such a basis of g is called a Serre basis. By mapping the Serre basis of g to the Serre basis of g , we get an isomorphism ϕ˜ : gC → gC . Proof of Proposition 4.47. Let ϕ : h → h,

H → −H.

Then ϕ maps to . Fix any nonzero vectors of gα for simple roots α and their negatives −α. By the previous proposition, we obtain an extension ϕ˜ : gC → gC . For the vectors E α ∈ gα chosen earlier satisfying B(E α , E −α ) = 1, there exist constants cα , α ∈ , such that ϕ(E ˜ α ) = c−α E −α . Since B(ϕ(E ˜ α ), ϕ(E ˜ −α )) = B(E α , E −α ), we get c−α cα = 1. Choose aα , α ∈ , such that aα a−α = 1, aα2 = −cα . For example, if cα = r eiθ , √ θ+π 1 θ+π r > 0, θ ∈ R, take aα = r e 2 i and a−α = −r − 2 e− 2 i . Define X α = aα E α . Then [X α , X −α ] = aα a−α [E α , E −α ] = [E α , E −α ] = Hα , −1 ϕ(X ˜ α ) = aα ϕ(E ˜ α ) = aα c−α E −α = a−α c−α E −α = −X −α .

Let Nα,β be the structural constants, [X α , X β ] = Nα,β X α+β .

52

L. Ji

As discussed before Proposition 4.53, for such choice of X α , Nα,β = −N−α,−β . Then Proposition 4.50 shows that 2 Nα,β =

1 q(1 + p)|α|2 . 2

This completes the proof of Proposition 4.47. For the root vectors X α above in Proposition 4.47, define g = h0 + ⊕α∈ RX α ,

i.e., g is a real subspace spanned by a Weyl basis. Then g is a real form of the complex Lie algebra gC , and g is called the split (or normal) real form of gC . Define u = ih0 + R(X α − X −α ) + iR(X α + X −α ). α∈

α∈

Proposition 4.54. The real subalgebra u is a real compact form of the complex Lie algebra gC . Proof. It is clear that u ⊕ iu = gC . To show that u is a real form of gC , we need to show that u is a Lie algebra, i.e., closed under the Lie bracket. For any i H ∈ ih0 , [i H, X α − X −α ] = α(H )i(X α + X −α ),

[i H, i(X α + X −α )] = −α(H )(X α − X −α ),

and hence [ih0 , u] ⊂ u. For any α, β ∈ , [(X α − X −α ), (X β − X −β )] = Nα,β X α+β + N−α,−β X −α−β − N−α,β X −α+β − Nα,−β X α−β = Nα,β (X α+β − X −α−β ) − N−α,β (X −α+β − X α−β ) ∈ u; in the last equation, we have used the equality −Nα,β = N−α,−β . Similarly we can check that u is closed under the bracket for other generators, and hence u is a real Lie algebras. Next we show that u is a compact subalgebra. It suffices to show that the Killing form B of u is negative definite. Since g is semisimple, B|h0 > 0 and hence B|i h0 < 0. Since B(gα , gβ ) = 0 if α + β = 0, it implies that for linearly independent α, β, B(X α − X −α , X β − X −β ) = 0, B(X α − X −α , i(X β + X −β )) = 0,


53

B(i(X α + X −α ), i(X β + X −β )) = 0. On the other hand, B(X α − X −α , X α − X −α ) = −2B(X α , X −α ) = −2, B(i(X α + X −α ), i(X α + X −α )) = −2B(X α , X −α ) = −2. It follows that B is negative definite on u.

For the purpose of constructing orthogonal involutive Lie algebra structures on g, we need a compact real form of gC with respect to the real structure induced by g, but the real form u constructed above is not necessarily such a form. Theorem 4.55. Let gC = g ⊗ C, and u a real compact form. Then there exists ϕ ∈ AdgC such that ϕuϕ −1 is a real compact form stable under the conjugation with respect to g. We need the following two results to prove this theorem. Proposition 4.56. Let g be a real semisimple Lie algebra, θ a Cartan involution, σ any involution. Then there exists ϕ ∈ Adg such that the new Cartan involution ϕθ ϕ −1 commutes with σ . Proof. Let ω = σ θ. Then ω is an automorphism of g, and hence B(ωX, θ Y ) = B(X, ω−1 θ Y ) = B(X, θωY ), Bθ (ωX, Y ) = Bθ (X, ωY ). Since Bθ is positive definite, this implies that ω is a selfadjoint automorphism with respect to the inner product defined by Bθ , and hence ρ = ω2 is positive definite. In a suitable basis, ρ is diagonalizable with positive eigenvalues, and hence for any r ∈ R, ρ r is defined and belongs to Adg. Since ρθ = ω2 θ = σ θ σ = θ ω−2 = θρ −1 , for any r ∈ R, we have

ρ r θ = θρ −r .

1

Let ϕ = ρ 4 . We claim that (ϕθ ϕ −1 )σ = σ (ϕθ ϕ −1 ). 1

1

In fact, since ρ 4 θ = θρ − 4 , we have 1

1

1

(ϕθ ϕ −1 )σ = ρ 4 θρ − 4 σ = ρ 2 θσ 1

1

1

= ρ 2 ω−1 = ρ − 2 ρω−1 = ωρ − 2 , and similarly, 1

1

1

1

σ (ϕθ ϕ −1 ) = σρ 4 θρ − 4 = σ θρ − 2 = ωρ − 2 .

54

L. Ji

Proposition 4.57. Let gC be a complex semisimple Lie algebra, and u ⊂ gC be a compact real form, τ the conjugation of gC with respect to u. If gC is regarded as a real Lie algebra, then τ is a cartan involution of gC . Proof. Let B be the Killing form of gC as a complex semisimple Lie algebra, and B R the Killing form of gC as a real semisimple Lie algebra. Then B R = 2ReB, and we need to check BτR > 0. For any nonzero Z ∈ gC , write Z = X + iY , where X, Y ∈ u. Let Bu be the Killing form of u. Then Bu is negative definite, and Re B(Z , τ Z ) = Re B(X + iY, X − iY ) = B(X, X ) + B(Y, Y ) = Bu(X, X ) + Bu(Y, Y ) < 0, and hence

BτR (Z , Z ) = −B R (Z , τ Z ) = −2ReB(Z , τ Z ) > 0.

This proves that τ is Cartan involution of gC . Proof of Theorem 4.55. Let τ be the conjugation of gC with respect to u and σ the conjugation of gC with respect to g. By Proposition 4.57, τ is a Cartan involution. By Proposition 4.56, there exists ϕ ∈ AdgC such that θ = ϕτ ϕ −1 commutes with σ . θ = ϕ(gτ ) = ϕ(u) is a real compact form of g . Since θ, σ commute, gθ Clearly, gC C C C is stable under σ . Proposition 4.58. For any semisimple Lie algebra g, any two Cartan involutions of g are conjugate under Adg. Proof. By Proposition 4.56, we can assume that θ, θ commute, and hence they have joint eigenspaces. It suffices to show that for any X ∈ g, if θ X = X , then θ X = X . If not, X belongs to the eigenspace of θ of eigenvalue −1, i.e., θ X = −X . Then Bθ (X, X ) = −B(X, θ X ) = −B(X, X ) = B(X, θ X ) = −Bθ (X, X ). Since both Bθ , Bθ are positive definite, this is a contradiction. Corollary 4.59. For a complex semisimple Lie algebra gC treated as a real Lie algebra, the only Cartan involutions are those obtained from real compact forms, and they are all conjugate under AdgC . Proof of Theorem 4.41. Now we combine the above results to prove Theorem 4.41, i.e., for any real simple noncompact Lie algebra g, there exists a unique orthogonal involutive Lie algebra structure up to conjugation by Adg. By Lemma 4.2, it is reduced to Cartan involutions of g. By Proposition 4.45, the problem is equivalent to real compact forms of gC stable under conjugation with respect to g. By Theorems 4.47 and 4.55, there exist such real compact forms, and hence g admits Cartan involutions. By Proposition 4.58, all the Cartan involutions of g are conjugate under Adg. This completes the proof of Theorem 4.41.


55

Geometry of symmetric spaces Let (G, K ) be a Riemannian symmetric pair. Then its involutive Lie algebra (g, σ ) is orthogonal. Let g = k + p be the decomposition by the eigenspaces of σ . Let Q be a positive definite quadratic form on p invariant under σ and k. Let B be the Killing form of g. We may assume Q = −B|p if g is of compact type, and Q = B|p if g is of noncompact type. Let X = G/K . Denote the identity coset K by x0 . Then Tx0 X ∼ = p, and Q defines a G-invariant Riemannian metric on X = G/K . By Proposition 4.13, X is a symmetric space. Proposition 4.60. If X is of compact type, then its sectional curvature is positive; and if X is of noncompact type, then its sectional curvature is negative; if X is of flat type, then its sectional curvature is zero. We need the following curvature formula to prove this proposition. Lemma 4.61. Let M = G/K be a symmetric space. For any X, Y, Z ∈ p = Tx0 M, the sectional curvature K 0 of M at x0 satisfies K 0 (X, Y )Z = −[[X, Y ], Z ]. Proof. For each Y ∈ p, exp tY , t ∈ R, is the family of isometries sliding along the geodesic etY x0 , i.e., the transvections along the geodesic. The induced vector field on the symmetric space X is also denoted Y . Let L Y be the Lie derivative with respect to the field Y , and DY the covariant differentiation along Y . Since detY is the parallel transport on the geodesic etY x0 , for any Z ∈ p, DY Z |x0 = L Y Z |x0 = [Y, Z ]x0 . By definition, K 0 (X, Y )Z = (D X DY Z − DY D X Z − D[X,Y ] Z )|x0 = (L X DY Z − L Y D X Z − D[X,Y ] Z )|x0 . Since et X is an isometry, it commutes with the covariant differentiation, et X (DY Z ) = Det X (Y ) (et X (Z )). Taking derivatives at t = 0, we get L X (DY Z ) = D L X (Y ) (Z ) + DY (L X (Z )) = D[X,Y ] (Z ) + L Y (L X (Z )). Similarly, L Y (D X Z ) = D L Y (X ) (Z ) + D X (L Y Z ) = D[Y,X ] (Z ) + L X (L Y Z ). Substituting these formulas into the expression for K 0 (X, Y )Z , we obtain

56

L. Ji

K 0 (X, Y )Z = (D[X,Y ] (Z ) + L Y (L X (Z )) − D[Y,X ] (Z ) − L X (L Y Z ) − D[X,Y ] Z )|x0 = (L Y L X − L X L Y (Z ) − D[X,Y ] Z )|x0 = (L [Y,X ] Z − D[Y,X ] (Z ))|x0 . Since X, Y ∈ p, [X, Y ] ∈ k, the value of the vector field of [X, Y ] evaluated at x0 equals zero and hence D[Y,X ] (Z )|x0 = 0. This implies that K 0 (X, Y )Z = L [Y,X ] Z = [[Y, X ], Z ] = −[[X, Y ], Z ].

Proof of Proposition 4.60. For X, Y ∈ p, the sectional curvature of the plane spanned by X, Y is given by Q(K 0 (X, Y )X, Y ) Q(X, X )Q(Y, Y ) − Q(X, Y )2 Q([[X, Y ], X ], Y ) = . Q(X, X )Q(Y, Y ) − Q(X, Y )2

R(X, Y ) = −

If (g, σ ) is of flat type, then [X, Y ] = 0 and hence R(X, Y ) = 0. If (g, σ ) is of compact type, then Q(X, Y ) = −B(X, Y ), and B([[X, Y ], X ], Y ) Q(X, X )Q(Y, Y ) − Q(X, Y )2 B([X, Y ], [X, Y ]) =− Q(X, X )Q(Y, Y ) − Q(X, Y )2 ≥ 0.

R(X, Y ) = −

The last inequality holds since [X, Y ] ∈ k and B is negative definite on k by Lemma 4.25. If (g, σ ) is of noncompact type, then Q(X, Y ) = B(X, Y ). It can be shown similarly that R(X, Y ) ≤ 0. An important way to understand the geometry of symmetric spaces X is by examining its submanifolds. Definition 4.62. A connected submanifold S of X is called totally geodesic if for every point p ∈ S and every vector V ∈ T p S, the unique geodesic γ in X with γ (0) = p, γ (0) = V is contained in S.


57

Proposition 4.63. For a symmetric space X , any totally geodesic submanifold S of X is also a symmetric space. Proof. For any point p ∈ S, the geodesic symmetry i p on X preserves the geodesics in S and leaves S invariant, and hence defines an isometry on S. For a totally geodesic submanifold S of X = G/K , define G(S) = {g ∈ G | g(S) = S}. Lemma 4.64. Suppose that the basepoint x0 = K in X = G/K belongs to S. Then S = G(S)/K ∩ G(S). Proof. Since transvections along geodesics in S belong to G(S), G(S) acts transitively on S, and the lemma follows. Lemma 4.65. Assume that the basepoint x0 = K of X is contained in S. Let σ be the involution of G associated with the basepoint x0 as in Proposition 4.5. Then the subgroup G(S) is stable under σ . Proof. For any g ∈ G(S), we need to show that σ g(S) ⊂ S. Since g(S) = S and the symmetry s0 preserves all the geodesics passing through x0 and hence leaves S invariant, σ (g)(S) = s0 gs0 (S) = s0 g(S) = s0 (S) = S. Lemma 4.66. For any subgroup G of G stable under σ , the orbit G x0 in X is a totally geodesic submanifold. Proof. Let g be the Lie algebra of G . Since G is σ -stable, g is also stable under σ . Then (g , σ |g ) is an orthogonal involutive Lie algebra. Let g = g0 ⊕ · · · ⊕ ga be the decomposition into irreducible summands in Theorem 4.32. Then g is sum of some of the summands, and the tangent space of G x0 at x0 can be identified with g ∩ p. Let s0 be the geodesic symmetry of X at x0 . For any Y ∈ g ∩ p, the geodesic exp tY x0 has tangent vector Y at x0 and clearly belongs to the orbit G x0 . By the transitive, isometric action of G , the same result holds at other points in G x0 . This implies that G x0 is totally geodesic. Corollary 4.67. There is a one-to-one correspondence between the set of totally geodesic submanifolds containing the basepoint x0 and the set of σ -stable subgroups. Proposition 4.68. Let X = G /K and X = G/K be two symmetric spaces, and a morphism ϕ : G → G satisfying ϕ(G ) ∩ K = ϕ(K ). Then the map i ϕ : X = G /K → X = G/K ,

g K → ϕ(g )K ,

is a totally geodesic embedding if and only if ϕ satisfies ϕσ (g ) = σ ϕ(g ),

g ∈ G ,

where σ , σ are the involutions on G , G for the identity coset.

58

L. Ji

Proof. The assumption ϕ(G )∩ K = ϕ(K ) implies that the map i ϕ is an embedding. Let g = k + p and g = k + p be the decomposition of the Lie algebras according to the eigenspaces of the involutions. Then the condition ϕσ (g ) = σ ϕ(g ) holds if and only if the differential of ϕ induces a morphism of involutive Lie algebras dϕ : (g , σ ) → (g, σ ), which implies that p is mapped injectively into p. Then the proposition follows from the previous two lemmas 4.66, 4.65. Remark 4.69. Totally geodesic submanifolds are usually treated using triple Lie systems (see [He] for example). The above treatment is more convenient for the embedding needed for the Satake compactifications. Proposition 4.70. For any abelian subalgebra a contained in p, let A = exp a be the corresponding subgroup of G. Then the orbit Ax0 is a flat totally geodesic submanifold in X . Conversely, any flat totally geodesic submanifold of X passing through the basepoint x0 is of this form. Proof. Let σ be the involution on G as defined in Lemma 4.65. Since a ⊂ p, it can be seen easily that A is stable under σ . By Lemma 4.66, Ax0 is a totally geodesic submanifold. Since A is abelian, the curvature formula in Proposition 4.60 shows that Ax0 is flat. Conversely, for any flat totally geodesic submanifold F in X , let G = G(F) in Lemma 4.65. Then G is stable under σ . Let g = k + p be the decomposition according to σ . Then the curvature formula in Proposition 4.60 shows that [p , p ] = 0. Let a = p . Then a is an abelian subalgebra contained in p. Let A = exp a. Then Ax0 is a flat subspace contained in F of the same dimension as F. This implies that Ax0 = F. Definition 4.71. For a symmetric space X of either compact or noncompact type, the maximal dimension of flat totally geodesic submanifolds in X is called the rank of X , denoted by r k(X ). As mentioned earlier, the rank of the Poincaré disc D is equal to 1, while the rank of the bidisc D × D is equal to 2. Proposition 4.70 immediately the following. Lemma 4.72. For a symmetric space X = G/K , the rank of X is equal to the maximal dimension of abelian subalgebras in p. For example, when G = S L(n, R), the rank of X is equal to n − 1. Definition 4.73. Let (g, σ ) be an orthogonal involutive Lie algebra. A maximal abelian subalgebra a contained in p is called a Cartan subalgebra of the symmetric pair (g, σ ). Proposition 4.74. For any reduced semisimple orthogonal involutive Lie algebra (g, σ ), any two Cartan subalgebras are conjugate under Adk, and hence they are of the same dimension.


59

Before we prove the proposition, we need some preliminaries. Definition 4.75. An element H ∈ p is called regular if the centralizer of H in p is a Cartan subalgebra. Lemma 4.76. Let (g, σ ) be a semisimple orthogonal involutive Lie algebra, and a a Cartan subalgebra. Then a contains regular elements. Proof. Since every element of a is semisimple, g = g0 + gα . α∈(g,a)

Take H ∈ a such that α(H ) = 0 for all α. We claim that z(H ) ∩ p = a and hence H is regular. In fact, by the choice of H , z(H ) = g0 , which is also equal to z(a). Since a is maximal in p, z(a) ∩ p = a. This implies the claim. Proof of Proposition 4.74. Let a1 , a2 be two Cartan subalgebras of (g, σ ). Pick two regular elements X 1 , X 2 in a1 , a2 respectively. Consider the real valued function k → B(ad(k)X 1 , X 2 ) on Adk. Since Adk is compact, this function has an extremal value at a point q ∈ K . Let Y = ad(q)(X 1 ). Then for every Z ∈ k, the function f (t) = B(ad(et Z ) Y, X 2 ) has an extremum at t = 0, and hence, B(Z , [X 2 , Y ]) = −B([Z , Y ], X 2 ) = 0. Since [X 2 , Y ] ∈ k and B is negative definite on k by Lemma 4.25, it follows that [X 2 , Y ] = 0, and Y ∈ z(X 2 ) ∩ p = a2 , z(Y ) ∩ p ⊃ a2 . Since a1 = z(a1 ) ∩ p, ad(q)a1 = z(ad(q)X 1 )∩p = z(Y )∩p ⊃ a2 . By definition, a2 is maximal abelian and ad(q)(a1 ) is abelian. This implies that a2 = ad(q)(a1 ). Corollary 4.77. For X = G/K , r k(X ) is equal to the dimension of any Cartan subalgebra of (g, σ ). Corollary 4.78. For any orthogonal involutive Lie algebra (g, σ ), let a be a Cartan algebra. Then p = ∪k ad(k)a, exp p = ∪k Ad(k)A, where k ∈ Adk, A = exp a. Proof. Since every element in p belongs to an abelian subalgebra contained in p, it is also contained in a maximal abelian subalgebra, which is of the form ad(k)a by Proposition 4.74.

60

L. Ji

Proposition 4.79. Let X = G/K be a symmetric space, and (g, σ ) the associated orthogonal involutive Lie algebra. Then G = exp pK . Proof. For any g ∈ G, the point gx0 is contained in a geodesic passing through the basepoint x0 . Therefore, there exists an element Y ∈ p such that gx0 = exp Y x0 . This implies that exp(−Y )gx0 = x0 , and hence exp(−Y )g ∈ K , i.e., g ∈ exp(Y )K . While the decomposition G = P K suggests that X = G/K ∼ = exp p, it is not obvious that the map exp p × K → exp pK given by multiplication is a diffeomorphism. Lemma 4.80. In the above notation, exp p is a closed subset of G. Proof. By Corollary 4.78, the map Adk × a → p, (k, H ) → ad(k)(H ) is surjective. Since exp a is an abelian subgroup of G and a is a maximal abelian subalgebra, exp a = exp a and hence is closed. Since Adk is compact and acts continuously, the image exp p is also closed. Define an operation ∗ on G by g ∗ = σ (g −1 ). Then for x, y ∈ G, x ∗∗ = x, (x y)∗ = y ∗ x ∗ , e∗ = e and x x ∗ = e if x ∈ K . Proposition 4.81. Define a G-action on G by (x, g) → xgx ∗ . Then the G-orbit through e can be identified with exp p, i.e., exp p = {x x ∗ | x ∈ G}. If K = G σ , which holds when G σ is connected, then G/K ∼ = exp p. Proof. For any exp Y ∈ exp p, take x = exp 12 Y . Then 1 1 x x ∗ = exp Y exp σ (Y ) = exp Y. 2 2 This implies that exp p ⊂ G · e. Conversely, by Proposition 4.79, any x ∈ G can be written as x = exp(Y )k, where Y ∈ p, k ∈ K . Then x x ∗ = exp(Y )kk ∗ exp Y = exp(2Y ). This proves that G · e = exp p. The stabilizer of e consists of x such that x x ∗ = e and is equal to K when K = G σ . This implies that the map G/K → exp p, x K → x x ∗ is a bijective map. Since both spaces are analytic, they are diffeomorphic. Symmetric spaces of compact type. We will study briefly symmetric spaces of compact type in this section. The first natural question is whether symmetric spaces of compact type are compact. The answer is positive and there are two approaches. One is quicker and uses some basic results from differential geometry.


61

Proposition 4.82. If X = G/K is of compact type, then the Ricci curvature is strictly positive. In particular, X has finite diameter and hence compact. Furthermore, π1 (X ) is finite. The second statement follows from the first and the following Bonnet-Myers Theorem in differential geometry. See [CE, Ch. 1, §9] for a proof of this result. Proposition 4.83. For any complete Riemannian manifold M of dimension n, if the Ricci curvature Ric(X, X ) ≥ (n − 1)R, R > 0, for all unit vectors X , then the diameter of M is less than or equal to π R −1/2 , and hence M is compact. In fact, by applying the Bonnet-Meyers Theorem to the universal covering space X˜ , we conclude that X˜ is compact and π1 (X ) is finite. Proof of Proposition 4.82. By definition, the Ricci curvature Ric(X, X ) = =

n i=1 n

< K (X i , X )X, X i > < −[[X i , X ], X ], X i >

i=1

=

n −1 B([X i , X ], [X i , X ]) , c i=1 < X, X >< X i , X i > − < X, X i >2

where < , > is the inner product on Tx0 = p, c is the positive constant such that B = −c |p on p, and X 1 , · · · , X n are an orthonormal basis of p. Since (g, σ ) is non-flat, there exists X i such that [X i , X ] = 0. Since B is negative definite on k, this implies that Ric(X, X ) > 0. There is another purely group theoretical approach to prove the compactness of X . Assume that (g, σ ) is irreducible and of compact type. There are two cases to consider: 1. g is a simple compact real Lie algebra, and σ an involution. 2. g = g1 + g1 , where g1 is a simple compact real algebra, and σ (X, Y ) = (Y, X ) for X, Y ∈ g1 . Then the compactness of X follows from the next result. Proposition 4.84. The simply connected Lie group G of any semisimple compact Lie algebra g is compact. This result is due to Weyl, and we outline a proof. By Corollary 4.78 and Proposition 4.79, G = ∪k∈Ad kk AK , where A = exp a, a is a Cartan subalgebra. Since K and Adk are compact, it suffices to show that A is compact, which is equivalent to show that the kernel of the map exp : a → A contains a lattice in a. Let (g, σ ) be an orthogonal involutive compact semisimple Lie algebra, a a Cartan subalgebra, (g, a) the associated set of roots.

62

L. Ji

Lemma 4.85. The roots in (g, a) take imaginary values on a. Proof. Let (g , σ ) be the noncompact dual of (g, σ ). Then ia is a Cartan subalgebra of g . For any Y ∈ p , adY acting on g is symmetric with respect to the positive definite quadratic form Bσ (U, V ) = −B(U, σ V ) and hence has real eigenvalues. This implies that elements of ad(a) have imaginary eigenvalues. Write the roots in (g, a) in the form iα. For each root iα, let h α ∈ a be the unique element determined by < H, h α >= α(H ), H ∈ a. Proposition 4.86. The compact Lie algebra g is a direct sum of g0 = z(a) and of two dimensional subspaces q invariant under ada and σ . Let q be such a subspace and ±iα its roots. Then g = q + Rh α is a simple subalgebra of g. For suitable u ∈ k ∩ q, v ∈ p ∩ q, and for all h ∈ a, we have [h, u] = α(h)v, [h, v] = −α(h)u, [u, v] = h α B(u, u), and hence

4π h α

belongs to the kernel of the exponential map exp : g → G.

Proof. First we use the existence of such subspaces q and their basis u, v and prove 4π that h α belongs to the kernel. Let h ∗α =

2 hα . < α, α >

Then 2 α(h α )v = 2v, < α, α > 2 [h ∗α , v] = − α(h α )u = −2u, < α, α >

[h ∗α , u] =

and

This implies that

⎛

⎞ 0 20 adh ∗α |g = ⎝−2 0 0⎠ . 0 00 ⎛

⎞ cos 2t sin 2t 0 Ad exp th ∗α = ⎝− sin 2t cos 2t 0⎠ , 0 0 1

and hence exp π h ∗α belongs to the center of the subgroup G of G generated by the subalgebra g . Since g is isomorphic to su(2), G has center of order at most 4π 2. It follows that exp 2π h ∗α = id, i.e., h α belongs to the kernel of the map 4π exp : g → G. Clearly, for α ∈ (g, a), the vectors h α generate a lattice of a. Next we produce the two dimensional subspaces q. Let g = g0 + giα iα∈(g,a)


63

be the root space decomposition. For X ∈ giα , let u = X + σ (X ),

v = i(X − σ (X )).

Then it can be checked easily that for h ∈ a, [h, u] = α(h)v, [h, v] = −α(h)u, [u, v] = h α B(u, v). For example, [h, u] = [h, X + σ (X )] = iα(h)X − iα(h)σ (X ) = α(h)i(X − σ (X )) = α(h)v. Since u ∈ g and α(h) ∈ R, v ∈ g, and u, v, h α span a simple subalgebra. By choosing X from a basis of giα for positive roots, we get the desired decomposition. Using the above results about A, we can derive several results about geometry of X . Since A is a compact abelian Lie group, it is isomorphic to a compact torus (S 1 )r = (R/Z)r . The dimension r of a or A is called the rank of X . Corollary 4.87. Every symmetric space of compact type X is the union of compact torus (S 1 )r . Proof. It follows from the decomposition X = ∪k∈Ad(k) k Ak −1 x0 and the fact that for any k ∈ Adk, k Ak −1 is a compact torus and k Ak −1 x0 ∼ = k Ak −1 . Corollary 4.88. If X is a rank 1 symmetric space of compact type, then all its geodesics are simple and closed and have the same length. Proof. Due to the homogeneity, we only consider geodesics passing through the basepoint x0 . For any such geodesic γ , γ (0) = x0 , the tangent vector γ (0) spans a Cartan subalgebra a in p. This implies that the geodesic γ is the orbit Ax0 , a simple closed curve. Since all Cartan subalgebras are conjugate, all these geodesics have the same length. On the other hand, for higher rank symmetric spaces of compact type, there are many non-closed or simple geodesics. The above proposition can be rephrased in terms of geodesic flow. For any complete Riemannian manifold M, let S(M) be its unit sphere bundle. Then there is a geodesic flow gt on S(M) defined as follows. For any geodesic γ and s ∈ R, (γ (s), γ (s)) ∈ S(M), define gt (γ (s), γ (s)) = (γ (s + t), γ (s + t)). Corollary 4.89. For a rank one symmetric space X of compact type, its geodesic flow gt is periodic, with period equal to the common length of all geodesics. Symmetric spaces of noncompact type. As in the case of symmetric spaces of compact type, a natural question is whether a symmetric space of noncompact type is noncompact.

64

L. Ji

Proposition 4.90. Let X = G/K be a symmetric space of noncompact type, and g = k + p the Cartan decomposition. Then the map K × p → G,

(k, X ) → k exp X

is a diffeomorphism. In particular, G/K ∼ = p and hence diffeomorphic to a vector space. Proof. By Proposition 4.79, G = K exp p. We need to show that the map K × p → G, (k, X ) → k exp X is injective. Since Bσ (X, Y ) = −B(X, σ Y ) is positive definite, then with respect to an orthonormal basis for Bσ , Adk is orthogonal for k ∈ K and ad(X ) is symmetric for X ∈ p. This implies that under the map Ad : G → G L(g), the subgroup Ad(K ) is contained in S O(n) and Ad(exp p) consists of positive definite symmetric matrices. Suppose that (k, X ), (k , X ) ∈ K × p are mapped to the same point g in G,

g = ke X = k e X . Then

Ad(g)t Ad(g) = Ad(e2X ) = Ad(e2X ), and hence

Ad(e X ) = Ad(e X ) =

Ad(g t g).

Since Ad is injective on exp p, this implies that X = X , k = k and completes the proof. As a corollary of Theorem 4.41 and the above proposition, we have the following result. Proposition 4.91. For any connected noncompact simple Lie group G, there is a (Riemannian) symmetric space G/K of noncompact type, which is unique in the sense that K is unique up to conjugation by elements in G. To prove this proposition, we need the following result ([Bo2, Theorem 4.1]). Lemma 4.92. Let G/K be a symmetric space of noncompact type, then the normalizer N (K ) of K is equal to K . Proof. By Proposition 4.90, G = exp pK ∼ = exp p × K , and hence it suffices to prove that for p = exp X ∈ exp p, if p ∈ N (K ), then p = e. In fact, for any k ∈ K , pkp −1 = k ∈ K . Then k p = pk = k(k −1 pk). Since k −1 pk ∈ exp p, by the uniqueness of the decomposition G = exp pK again, k = exp(X )k exp(−X ) = k, and hence X centralizes k. Since k is a maximal subalgebra (Proposition 4.28), X ∈ k and hence X = 0, p = id.


65

Proof of Proposition 4.91. Let g be the Lie algebra of G, and G˜ the universal covering space of G, i.e., the simply connected Lie group whose Lie algebra is equal to g. By Theorem 4.41, g admits a unique structure (g, σ ) of orthogonal involutive Lie algebra. Let g = k + p be the decomposition according to the eigenspaces of σ . Let K˜ and K be the subgroups of G˜ and G corresponding to k respectively. The ˜ and (G, ˜ K˜ ) is a Riemannian symmetric pair involution σ lifts to an involution on G, of noncompact type. Since the kernel N of the map G˜ → G is contained in K˜ , ˜ and hence the involution of G˜ induces an involution on G = G/N , and (G, K ) is ˜ a Riemannian symmetric pair, where K = K /N . In fact, since N is a discrete and ˜ it belongs to the centralizer of K˜ and hence to the normalizer normal subgroup of G, ˜ of K . By Lemma 4.92, N belongs to to K˜ . The orthogonal involutive Lie algebra of (G, K ) is (g, σ ). Then Proposition 4.13 shows that G/K admits a structure of Riemannian symmetric space. The uniqueness of such a structure follows from the uniqueness in Theorem 4.41. Corollary 4.93. For any connected noncompact semisimple Lie group G, there exist quotient spaces G/K which are symmetric spaces. Proof. Let g be the Lie algebra of G and g = g 1 ⊕ · · · gn be the decomposition into simple ideals. By reordering them if necessary, we can assume that there exists m ≤ n such that g1 , · · · , gm are noncompact while gm+1 , · · · , gn are compact. Then Theorem 4.41 shows that there is an orthogonal involutive structure (g, σ ) on g such that gm+1 , · · · , gn ⊂ k, and such a structure is unique up to scaling of the positive definite quadratic form on each factor. The algebra (g, σ ) is not necessarily reduced. In fact, if m < n, (g, σ ) is not reduced, and its reduction is (g1 ⊕ · · · ⊕ gm , σ ). Let K be the connected subgroup of G with Lie algebra equal to k. Then similar arguments to the proof of Proposition 4.91 shows that G/K is a Riemannian symmetric space. Remark 4.94. It can be shown that if the center of G is finite, then the subgroup K in the above proposition is a maximal compact subgroup, and all maximal compact subgroups of G are conjugate. In fact, by [Bo2, Theorem 4.6], Adg(K ) is maximal compact in Adg. Since the center of G is finite, K is also compact and maximal. Therefore, for any connected noncompact semisimple Lie group G with finite center, there is a unique symmetric space, up to scaling of the metric on each irreducible factor, which can be identified with the space of maximal compact subgroups of G. The same conclusion holds for semisimple Lie groups with finitely many connected components and finite center. Such nonconnected Lie groups arise from the real locus of connected semisimple linear algebraic groups defined over R. For more information about the geometry of symmetric spaces of noncompact type and their relations to parabolic subgroups, see [BJ4].

66

L. Ji

References [AJ]

J. P. Anker, L. Ji, Sharp bounds on the heat kernel and Green function estimates on noncompact symmetric spaces, GAFA 9 (1999), 1035–1091. [An] A. Ancona, Negatively curved manifolds, elliptic operators, and the Martin boundary, Ann. of Math. 125 (1987), 495–536. [AS] M. Anderson, R. Schoen, Positive harmonic functions on complete manifolds of negative curvature, Ann. of Math. 121 (1985), 429–461. [AMRT] A. Ash, D. Mumford, M. Rapaport, Y. Tai, Smooth compactifications of locally symmetric varieties, Math Sci Press, Brookline, Mass, 1975. [BB] W. L. Baily, A. Borel, Compactification of arithmetic quotients of bounded symmetric domains, Ann. of Math. 84 (1966), 442–528. [BGS] W. Ballmann, M. Gromov, V. Schroeder, Manifolds of nonpositive curvature, Progress in Math., Vol. 61, Birkhäuser, Boston, 1985. [Bo1] A. Borel, Introduction aux groupes arithmétiques, Hermann, Paris, 1969. [Bo2] A. Borel, Semisimple Groups and Riemannian Symmetric Spaces, Texts & Readings in Math. Vol. 16, Hindustan Book Agency, New Delhi, 1998. [BJ1] A. Borel, L. Ji, Compactifications of locally symmetric spaces, to appear in J. Diff. Geom. [BJ2] A. Borel, L. Ji, Compactifications of symmetric spaces, to appear in J. Diff. Geom. [BJ3] A. Borel, L. Ji, Compactifications of symmetric and locally symmetric spaces, Math. Research Letters 9 (2002), 725–739. [BJ4] A. Borel, L. Ji, Compactifications of Symmetric and Locally Symmetric Spaces, in this volume. [BS] A. Borel, J. P. Serre, Corners and Arithmetic Groups, Comment. Math. Helv. 48 (1973), 436–491. ´ ements de Mathématique, Livre VI, Intégration, Hermann, Paris, [Bou] N. Bourbaki, El´ 1963. [Br] K. Brown, Buildings, Springer-Verlag, New York, 1989. [BuS] K. Burns, R. Spatzier, On the topological Tits buildings and their classifications, IHES 65 (1987), 5–34. [CE] J. Cheeger, D. Ebin, Comparison Theorems in Riemannian Geometry, NorthHolland Publishing Co., 1975. [Fu] H. Furstenberg, A Poisson formula for semi-simple Lie groups, Ann. of Math. 72 (1963), 335–386. [GJT] Y. Guivarc’h, L. Ji, J. C. Taylor, Compactifications of Symmetric Spaces, Progress in Math., Vol. 156, Birkhäuser, Boston, 1998. [GT] Y. Guivarc’h, J. C. Taylor, The Martin compactification of the polydisc at the bottom of the positive spectrum, Colloquium Math LX/LXI (1990), 537–546. [GW] S. Giulini, W. Woess, The Martin compactification of the cartesian product of two hyperbolic spaces, J. für die Reine u. angew. Math., 444 (1993), 17–28. [He] S. Helgason, Differential Geometry, Lie Groups, and Symmetric Spaces, Academic Press, New York, 1978. [J1] L. Ji, Satake and Martin compactifications of symmetric spaces are topological balls, Math. Res Letters, 4 (1997), 79–89. [J2] L. Ji, The greatest common quotient of Borel–Serre and the Toroidal Compactifications, Geometric and Functional Analysis, 8 (1998), 978–1015. [JM] L. Ji, R. MacPherson, Geometry of compactifications of locally symmetric spaces, Ann. Inst. Fourier, 52 (2002), 457–559.

Introduction to Symmetric Spaces and Their Compactifications [Ka]

[MM]

[Ma] [Mo] [Os1] [Os2]

[OSS] [Sa1] [Sa2] [Su] [Ta1]

[Ta2] [Zu1] [Zu2]

67

F. I. Karpelevic, The geometry of geodesics and the eigenfunctions of the Beltrami– Laplace operator on symmetric spaces, Trans. Moscow Math. Soc., 14 (1965), 51– 199. M. P. Malliavin, P. Malliavin, Factorisation et lois limites de la diffusion horizontale audessus d’un espace Riemannien symétrique, Lecture Notes in Math., Vol. 404, 1974, Springer-Verlag. R. S. Martin, Minimal positive harmonic functions, Trans. Amer. Math. Soc., 49 (1941), 137–172. C. C. Moore, Compactifications of symmetric spaces I, Amer. J. Math., 86 (1964), 201–218. T. Oshima, A realization of Riemannian symmetric spaces, J. Math. Soc. Japan, 30 (1978), 117–132. T. Oshima, A realization of semisimple symmetric spaces and construction of boundary value maps, in Adv. Studies in Pure Math. Vol. 14, Representations of Lie groups, 1986, pp. 603–650. T. Oshima, J. Sekiguchi, Eigenspaces of invariant differential operators on an affine symmetric spaces, Invent. Math., 57 (1980), 1–81. I. Satake, On representations and compactifications of symmetric spaces, Ann. of Math. 71 (1960), 77–110. I. Satake, On compactifications of the quotient spaces for arithmetically defined discontinuous groups, Ann. of Math. 72 (1960), 555–580. D. Sullivan, Related aspects of positivity in Riemannian geometry, J. Diff. Geom., 25 (1987), 327–308. J. C. Taylor, The Martin compactification of a symmetric space of non-compact type at the bottom of the positive spectrum: An introduction, in Potential Theory, M. Kishi, ed. Walter de Gruyter & Co., Berlin, 1991, pp. 127–139. J. C. Taylor, Martin compactification, in Topics in Probability and Lie groups: Boundary Theory, J. C. Taylor, ed., in CRM-AMS, 2001, pp. 153–202. S. Zucker, L 2 cohomology of warped products and arithmetic groups, Invent. Math., 70 (1982), 169–218. S. Zucker, Satake compactifications, Comment. Math. Helv. 58 (1983), 312–343.

Compactifications of Symmetric and Locally Symmetric Spaces Armand Borel and Lizhen Ji∗ School of Mathematics, Institute for Advanced Study, Princeton, NJ 08540 Department of Mathematics, University of Michigan, Ann Arbor, MI 48109 [email protected]

1 Introduction Symmetric and locally symmetric spaces arise in different contexts. Many natural spaces of this type are not compact and, for many applications, it is necessary to compactify them. This has been carried out in a number of ways, from different points of view, and has led to a variety of compactifications. In this chapter we try to survey these compactifications and also to indicate a procedure which allows one to give a unified approach to many of them, and introduce some new ones. We assume some familiarity with the theory of semisimple Lie groups (mostly real or complex, sometimes algebraic defined over Q), although a number of basic definitions and results will be recalled as needed. For basic results on Lie groups, see [He] and [Kn]. Beyond this, we have tried to make this chapter self-contained as far as stating results, but most proofs are omitted. A more detailed treatment of part of this material may be found in [BJ1], [BJ2]. It is planned to give a systematic exposition in a monograph in preparation. This work is of course a collaborative effort. The material is based in part on the lectures that were given at the European Summer School in Group Theory at CIRM, Marseille-Luminy, July 2001. However §§3–8 (resp. §§9–15) were lectured upon and written up by the second- (resp. first-) named author, who, accordingly, must bear the primary responsibility for them. We wish to thank the organizers, Jean-Philippe Anker and Pierre Torasso, for their invitation and for their excellent coordination of all aspects of the meeting. We also wish to thank an anonymous referee for helpful comments. ∗ Partially Supported by NSF grants and an Alfred P. Sloan Research Fellowship.

70

A. Borel and L. Ji

2 Summary This chapter is divided into three main parts as follows. I. Compactifications in which the Riemannian symmetric space is open dense. II. Compactifications of an arithmetic quotient in which the latter is open dense. III. Smooth compactifications of Riemannian or semisimple symmetric spaces in which the space is open, not dense. We now summarize the contents of each. In Part I (Sections 3–6), G denotes a semisimple linear Lie group with finitely many connected components, K is a maximal compact subgroup of G, and X = G/K is the quotient space. Endowed with an invariant Riemannian metric, X is a negatively curved Riemannian manifold, diffeomorphic to Euclidean space, which is symmetric (any point is the isolated fixed point of an involutive isometry, necessarily unique). Conversely, any Riemannian manifold M with these properties which has no Euclidean factor arises as such a quotient, where G may be taken to be a subgroup of finite index in the group I s(M) of isometries of M. The most basic example is the upper half-plane, where G = SL2 (R), K = SO2

X = {z ∈ C | Im z > 0},

with G acting on X by setting az + b ab g.z = .z = cd cz + d

(g ∈ G, z ∈ X ).

It is isomorphic to the open unit disc D = {w ∈ C, |w| < 1} on which ab 2 2 G = SU(1, 1) = | a, b ∈ C, |a| − |b| = 1 b¯ a¯

(1)

(2)

(3)

acts as in (2), the isotropy group of the origin being the subgroup of diagonal matrices. It admits in particular two important generalizations: the space Pn = SLn (R)/SOn of positive definite quadratic forms of determinant one on Rn and the Siegel upper half-space Sp2n (R)/Un = {Z ∈ Mn (C) | Z = t Z , Im Z > 0}, where

0 In . J= −In 0

(4)

Sp2n (R) = {g ∈ SL2n (R) | g.J g = J }, t

(5)

Five types of compactifications and their interrelations will be discussed. a) The geodesic compactification X ∪ X (∞), where X (∞) represents equivalence classes of geodesics. The action of G extends continuously to X ∪ X (∞). The isotropy groups of the points at infinity, (i.e., in X (∞)), are the proper parabolic subgroups of G, and X (∞), which is diffeomorphic to Sn−1 (n = dim X ), has in

Compactifications of Symmetric and Locally Symmetric Spaces

71

a natural way a simplicial structure related to the Tits building of proper parabolic subgroups of G. b) The Satake compactifications. This construction is a first step towards compactifications of arithmetic quotients \X of X (see below). Starting from an irreducible representation (τ, V ) of G, one defines a G-equivariant embedding of X into the real projective space P H(V ) of positive definite hermitian forms on V , and takes the closure X¯ τS of the image of X . It is the union of X and of finitely many orbits of G. Each is fibered over a quotient G/P of G (P parabolic), with fibres that are symmetric spaces X P of certain semisimple subgroups of P called the boundary components. Up to a G-isomorphism, the compactification X¯ τ depends only on the face of a positive Weyl chamber containing the highest weight µτ of the representation. If µσ is in the closure of the face containing µτ , there is a natural G-map of X¯ τ onto X¯ σ . In particular, if µτ is regular, X¯ τ dominates all the other Satake S of X . compactifications and is called the maximal Satake compactification X¯ max c) The Furstenberg compactifications. The incentive here is to generalize to X the representations of bounded harmonic functions on the unit disc D by the Poisson integral of L ∞ -functions on its boundary, the unit circle. Fix a proper parabolic subgroup P of G which does not contain any simple factor of G. The space M1 (G/P) of probability measures on the compact quotient space G/P, endowed with the weak topology, is a compact G-space. By assigning to x ∈ X the unique element of M1 (G/P) invariant under the isotropy group K x of x, one defines an embedding of X into M1 (G/P), and its closure is a Furstenberg compactification of X . It depends on P only up to conjugacy. The parametrization of the G-orbits in the Furstenberg compactifications is the same as that of the Satake compactifications and corresponding compactifications are naturally G-isomorphic. d) The Martin compactifications. The previous compactification yields a representation of bounded harmonic functions by means of generalized Poisson integrals over the quotient G/P for a minimal P. The Martin compactifications arise out of the study of positive harmonic functions or, more generally, of positive eigenfunctions of the Laplace–Beltrami operator . While it can be defined for any connected complete noncompact Riemannian manifold M, we restrict ourselves here to the case of the symmetric spaces. For λ ∈ R, let Cλ∞ (X ) = {u ∈ C ∞ (X ), u = λu | u ≥ 0}.

(6)

This is a convex cone admitting a compact homothety section. It is nonempty if and only if λ ≤ λ0 , where λ0 is the bottom of the spectrum of . Consideration of its extremal elements leads to a compactification X¯ λM = X ∪ ∂λ X , and a representation S . of u ∈ Cλ (X ) as a Poisson-type integral over ∂λ X . If λ = λ0 , then X¯ λM = X¯ max If λ < λ0 , it admits a continuous G-map onto X ∪ X (∞), and the fibres over the boundary are maximal Satake compactifications of symmetric spaces X P associated to parabolic subgroups of G. e) The Karpelevic compactification is a refinement of the geodesic compactification. To the equivalence relation on geodesic half-rays defining X (∞) is added another equivalence relation ∼N . The quotient of the set of geodesics with respect to

72

A. Borel and L. Ji

both sits accordingly over the geodesic compactification, and the fibres of this map can be described (Section 6). In Part II (Sections 7–9), G is the group of real points G(R) of a connected algebraic semisimple group G defined over Q, assumed to have strictly positive Qrank (so that G is not compact) and, for simplicity, to be Q-simple, and ⊂ G(Q) is an arithmetic subgroup (i.e., if G is identified with a Q-subgroup of some GL N (C), it is a subgroup commensurable with G(Q) ∩ GL N (Z)). In this part we are concerned with compactifications of \X or of \G. Reduction theory is required here, i.e., the construction of fundamental sets for the action of on X or G. The pertinent facts, reviewed in Section 7, involve in particular Siegel sets and their separation properties. In Section 8 two types of compactifications of \X are considered. The first is an analogue of the geodesic compactification, using so-called (EDM) geodesics, i.e., geodesics γ (t), t ∈ R, which become distance minimizing between γ (t1 ) and γ (t2 ) if t1 > t2 0. The second is the Borel–Serre (BS) compactification \ X¯ B S , which starts with a partial compactification X¯ B S of X , which is obtained by adding at infinity subspaces e(P) parametrized by proper parabolic subgroups defined over Q. This BS partial compactification is operated upon continuously and properly by , and the B S-compactification of \X is the quotient \ X¯ B S . It is the union of \X and of finitely many quotients e (P) = ( ∩ P)\e(P). A key property is that if is torsion free, it has the same homotopy type as \X ; hence it is a compact Eilenberg–MacLane space K (, 1) for . It has proved to be a fundamental tool for studying the cohomology of . The space e(P) is a product of a symmetric space by the unipotent radical N P of P. By collapsing the unipotent radicals, one gets the reductive Borel–Serre partial compactification X¯ R B S of X . Its quotient by is the reductive Borel–Serre compactification \ X¯ R B S of \X . There is a natural quotient map \ X¯ B S → \ X¯ R B S , whose fibres are the compact nilmanifolds ( ∩ N P )\N P . This compactification was introduced by S. Zucker to study the L 2 -cohomology of \X . Similar constructions are also made for G and \G. The third type of compactification is defined for \G when is maximal among discrete subgroups (any is contained in one). By associating to x ∈ G the conjugate xx −1 of by x, one defines a G-equivariant embedding of \G into the space S(G) of closed subgroups of G, which is a compact G-space. The compactification sb \G in question is then the closure in S(G) of the image of \G. The boundary subgroups are described. The identity map on \G extends to a continuous map of sb \G¯ R B S onto \G . A variation of the B S-construction allows one to describe a procedure which yields most of the compactifications considered in Sections 3 to 8. Section 9 is devoted to the Satake compactifications of \X and to the extension of his procedure to slightly more general cases. It is a generalization of the compactifications of the quotient of the upper half-plane by a discrete subgroup with cofinite area, a study going back to the 19th century. One starts from a Satake compactification X¯ µ of X , which is the union of X and boundary components parametrized


73

by the parabolic subgroups of G. The strategy is to isolate some of those rational boundary components, and try to show that the quotient by of the union X ∗ of X and those components, endowed with a suitable topology, is a compact Hausdorff space. Results and problems pertaining to rational boundary components and cases in which this procedure could be carried out are surveyed. A very important case is when X is a bounded symmetric domain (9.5). In this case there is a Satake compactification in which all boundary components are themselves bounded symmetric domains. And, the quotient \X ∗ also admits the structure of a normal projective variety inducing the given complex structure on each rational boundary component. In Part III (Sections 10–15), except for one remark about \X (10.5), we go back to symmetric spaces and let G, K , X be as in Part I, but also include the case of semisimple symmetric spaces G/H , where H is the fixed point set of an involution of G. The compactifications are smooth, and contain the given symmetric space as an open, but nondense orbit. The first examples were given by T. Oshima [Os1] and Oshima–Sekiguchi [OSS]. Their motivation was to supply a framework for the solution of a problem/conjecture of S. Helgason about joint eigenfunctions of invariant differential operators on X (later on generalized to G/H ). Three constructions will be outlined: that of [Os1], [Os2], [OSS], the gluing of copies of a manifold with corners, and the consideration of real points of the wonderful compactification of C. DeConcini and C. Procesi of G c /K c or G c /Hc (the subscript c indicates complexification) [CP]. Section 10 first describes a variation on a special case of a construction of R. Melrose (see [BJ1]) to get a smooth compact manifold by gluing to one another copies S , after having exhibited its of a given manifold with corners. This is applied to X¯ max structure of manifold with corners (10.3). It yields a compact smooth manifold containing 2 copies of X ( being the rank of X ), whose complement is the union of smooth hypersurfaces with transversal intersections, altogether 3 orbits. An earlier construction of Oshima [Os1] giving a similar space is also described (10.4). Oshima has also carried this out for G/H [Os2]. For reasons of exposition, it is taken up later (14.6). A third procedure is to start from the wonderful compactification X¯ cW of X c = G c /K c . This is a smooth projective variety in which the complement of X c is the union of smooth divisors with normal crossings. It is defined over R, and therefore its real points form a smooth real projective variety on which G acts. The set of real points of a G c -orbit is the union of finitely many orbits of G, which can be described using Galois cohomology. In particular the orbits of G in X c (R) are open, isomorphic to quotients G/K ε , where K ε are certain real forms of K c . Section 11 is devoted to the construction of the K ε and describes the structure of the real points of the orbits of G c in X¯ cW . Section 12 outlines first the construction of X¯ cW , sketches a proof of its properties, and applies the results of Section 11 to the description of its real points. Some of these results are also extended to the case of G/H but, at this point, are not as complete. Section 13 first summarizes the construction of [OsS]. It provides a compact analytic manifold X¯ O S on which G operates. The open orbits are not exactly the

74

A. Borel and L. Ji

G/K ε but rather some finite coverings of them. It turns out that there is a free action of the Weyl group W of the symmetric pair (G, K ) on X¯ O S such that the quotient is isomorphic to X¯ cW (R). Section 14 extends some of these results to G/H , and describes the construction of [Os2]. Finally, Section 15 gives some indications on the Galois cohomology used implicitly in Section 11.

I. Compactifications of Riemannian Symmetric Spaces In this part we assume that G is a connected semisimple Lie group with trivial center, K ⊂ G a maximal compact subgroup, and X = G/K a symmetric space of noncompact type. We discuss compactifications of X in which X is open and dense.

3 Geometry of symmetric spaces In this section we review some basic geometric properties of X and explain how parabolic subgroups arise from these geometric considerations. See [BG5] for the proof of Proposition 3.1 and discussions about geodesics below. Proposition 3.1 The symmetric space X is simply connected, complete and negatively curved, and hence diffeomorphic to the Euclidean space. We consider the set of all unit speed, directed geodesics in X . Two such geodesics γ1 (t), γ2 (t), t ∈ R are called equivalent if lim sup d(γ1 (t), γ2 (t)) < +∞. t→+∞

Denote by X (∞) the set of equivalence classes of geodesics. When X = D, the closure of D in C is the closed unit disc. Every geodesic in D is a semicircle perpendicular to the unit circle or a diameter, and hence every directed geodesic converges to a unique point on the unit sphere. Clearly every point on the unit sphere is the limit of such a geodesic. This implies the following. Lemma 3.2 For X = D, X (∞) can be canonically identified with S1 . Let x0 be a basepoint in X , and Sn−1 the unit sphere in the tangent space Tx0 X at x0 , where n = dim X . Proposition 3.3 The set of equivalence classes of geodesics X (∞) can be canonically identified with Sn−1 . More precisely, each equivalence class contains a unique geodesic passing through x 0 and then its direction at x 0 gives the unit vector in Sn−1 . Proof. First we observe that any two distinct geodesics passing through x0 are not equivalent. In fact, since X is simply connected and negatively curved, the separation of the two geodesics is greater than the separation in the Euclidean space, and hence they are not equivalent. This proves uniqueness.


75

For any geodesic γ (t), consider the sequence of geodesics γn from x0 to γ (n). By comparison with the Euclidean space, it can be shown that γn converges to a geodesic which clearly passes through x0 and is equivalent to γ . In view of the above proposition, X (∞) is called the sphere at infinity. The space X can be compactified by adding X (∞) at infinity. Specifically, X ∪X (∞) admits the following topology. Equivalence classes in X (∞) are denoted by [γ ]. An unbounded sequence y j in X converges to a point [γ ] ∈ X (∞) if the sequence of geodesics connecting x0 and yn converges to a geodesic in [γ ]. Lemma 3.4 The space X ∪ X (∞) with the above topology is compact and Hausdorff, and the compactification X ∪ X (∞) is independent of the choice of the basepoint. Proof. Since every geodesic through x0 is determined by its direction, it can be shown that every unbounded sequence y j in X has a convergent subsequence in X ∪ X (∞). This proves compactness. The Hausdorff property follows from the uniqueness of the limit of a convergent sequence. The independence on the choice of basepoint follows again from comparison with the Euclidean space. The compactification X ∪ X (∞) is called the geodesic (or conic) compactification. It can be shown that X ∪ X (∞) is homeomorphic to the closed unit ball in Tx0 X . Neighborhoods of boundary points are given by truncated cones and the topology is called conic topology. Recall that G is the identity component of I s(X ) and hence acts isometrically on X . See [He]. Proposition 3.5 The isometric action of G on X extends to a continuous action on X ∪ X (∞). Proof. Since G acts isometrically on X , G preserves the equivalence relation on geodesics, and hence the G-action extends to X ∪ X (∞). Since the topology does not depend on the choice of basepoint, the G-action is continuous. To understand properties of X ∪ X (∞) as a G-space, we need to describe the isotropy subgroups of all points and their G-orbits. For points in X , the isotropy subgroups in G are maximal compact subgroups, and every maximal compact subgroup fixes a unique point in X . This correspondence between maximal compact subgroups of G and interior points is bijective. Proposition 3.6 For every point [γ ] ∈ X (∞), the isotropy subgroup in G is a (proper) parabolic subgroup, and every (proper) parabolic subgroup of G is the isotropy subgroup of some point in X (∞). One proof is given in [GJT, Proposition 3.8]. This proposition can also be taken as a definition of (proper) parabolic subgroups. The group G is also a parabolic subgroup. In the following, by parabolic subgroups we often only refer to proper ones. Unlike the case for interior points in X , the correspondence between parabolic subgroups and points in X (∞) is not bijective unless the rank of X is equal to 1. We consider two examples. For X = H2 , the upper half-plane, we can take G = SL(2, R), K = SO2 , H2 = G/K . The sphere at infinity H2 (∞) = R ∪ {i∞}.

76

A. Borel and L. Ji

The isotropy subgroup of the point i∞ is a b P= | a = 0, b ∈ R . 0 a −1 G acts transitively on H2 (∞), and the isotropy groups of other boundary points are conjugates of P. Since H2 (∞) = G/P, it can be shown that there is a one-toone correspondence between H2 (∞) and the set of parabolic subgroups. The same conclusion holds for all X of rank 1. On the other hand, the correspondence is more complicated in the higher rank case. Consider X = D × D. Let γ1 , γ2 be two unit speed geodesics in D. For any θ ∈ [0, π2 ], define γθ (t) = (γ1 (t cos θ ), γ2 (t sin θ )). Then γθ is a unit speed geodesic in X . Lemma 3.7 For any two different θ1 , θ2 ∈ (0, π2 ), γθ1 is not equivalent to γθ2 , but their boundary points [γθ1 ], [γθ2 ] in X (∞) have the same isotropy subgroup in G. Proof. The first statement is clear. Let G = SU(1, 1) × SU(1, 1), K = U(1) × U(1). Then X = D × D = G/K . Let P j be the isotropy subgroup of [γ j ] in SU(1, 1). It can be shown that for any θ ∈ (0, π2 ), the isotropy subgroup of [γθ ] is equal to P1 × P2 . The lemma shows that all the points in the open one-dimensional simplex {[γθ ] | θ ∈ (0, π2 )} correspond to the same parabolic subgroup. When θ = 0, the isotropy subgroup of [γθ ] is equal to P1 × SU(1, 1); when θ = π2 , the isotropy subgroup is equal to SU(1, 1) × P2 . The parabolic subgroups P1 × P2 are minimal, while P1 × SU(1, 1), SU(1, 1) × P2 are maximal. The two points (or zero-dimensional simplices) [γ0 ], [γ π2 ] are the vertices of the previous one-dimensional simplex. These discussions give the following correspondence between parabolic subgroups and points in X (∞). Proposition 3.8 For X = D × D, the sphere at infinity X (∞) can be decomposed into 1-dimensional open and 0-dimensional simplices such that points in each simplex have the same parabolic subgroup as the isotropy subgroup. The correspondence between the simplices and parabolic subgroups is inclusion-reversing, i.e., for each parabolic subgroup P, let σ P be the corresponding simplex; then for a pair of parabolic subgroups P1 , P2 , P1 ⊂ P2 if and only if σ P1 contains σ P2 as a simplicial face, and maximal parabolic subgroups correspond to vertices. Corollary 3.9 For X = D × D, G does not act transitively on X (∞). In fact, there are infinitely many G-orbits that are parametrized by the closed 1-simplex [γθ ], θ ∈ [0, π2 ].


77

The above proposition shows that X (∞) is the underlying space of an infinite simplicial complex. In fact, this complex is the spherical Tits building of G. The above proposition holds for general X and gives a geometric construction of the spherical Tits building. See [GJT]. Proposition 3.10 Let X be a symmetric space of rank r . Then X (∞) admits the structure of an infinite simplicial complex such that the top simplices have dimension equal to r − 1, and points in each (open) simplex have the same parabolic subgroup as an isotropy subgroup. The correspondence between the simplices and parabolic subgroups is inclusion-reversing as in the previous proposition, and maximal parabolic subgroups correspond to vertices. For more details about the spherical Tits building and parabolic subgroups, see [GJT, Chap. 3]. It will be shown later that boundaries of compactifications of X are often cell complexes related to the spherical Tits building. As the example of X = D × D shows, there are infinitely many G-orbits in X (∞) when the rank of X is greater than 1. In order to refine the geodesic compactification X ∪ X (∞) and to construct other compactifications, we need to understand the internal structure of each equivalence class [γ ]. This analysis naturally leads to the Langlands decomposition of parabolic subgroups. N

Define a refined equivalence relation ∼ as follows: two geodesics γ1 , γ2 are N

γ1 ∼ γ2 , N -related if limt→+∞ d(γ1 (t), γ2 ) = 0, where d(γ1 (t), γ2 ) = infs∈R d(γ1 (t), γ2 (s)). Clearly N -related geodesics are equivalent. For each equivaN

N

lence class [γ ], there is an associated quotient [γ ]/∼. We consider the quotient [γ ]/∼ N

for several examples. When X = D, the Poincaré disc, [γ ]/ ∼ has only one point since two geodesics are equivalent if and only if they are N -related. On the other hand, for X = D × D and γ (t) = (z, γ2 (t)), where z ∈ D and γ2 (t) a geodesic in D, N

[γ ]/∼ can be identified with D. In fact, for different choices of z ∈ D, the geodesics (z, γ2 (t)) are equivalent but not N -related. For γθ (t) = (γ1 (t cos θ), γ2 (t sin θ)), N

θ ∈ (0, π2 ), it can be shown that [γθ ]/∼ consists of only one point. N

These discussions suggest that the size of [γ ]/∼ depends on the parabolic subgroup P which stabilizes the point [γ ] in X (∞). In fact, it can be explained by the Langlands decomposition of the parabolic subgroup P. See [GJT]. Let P = NP AP MP be the Langlands decomposition with respect to the basepoint x0 = K ∈ G/K , where N P is the unipotent radical, A P M P the Levi factor stable under the Cartan involution associated with K , A P the split component. By definition, g ∈ P if and only if g[γ ] = [γ ]; equivalently, lim sup d(gγ (t), γ (t)) < +∞. t→+∞

Choose the unique representative γ in [γ ] passing through x0 , γ (0) = x0 . Then there is a one-parameter subgroup exp RH in A P such that γ (t) = exp t H · x0 . For

78

A. Borel and L. Ji

g ∈ NP ,

lim d(gγ (t), γ (t)) = 0,

t→+∞

and for g ∈ M P , the distance d(gγ (t), γ (t)) is a constant independent of t. N

When dim A P = 1, [γ ]/∼ can be identified with M P x0 = M P /(M P ∩ K ). The subgroup M P ∩ K is a maximal compact subgroup of M P , and X P = M P /(M P ∩ K ) is a symmetric space of lower dimension, called the boundary symmetric space asN

sociated with P. When dim A P > 1, [γ ]/∼ can be identified with X P × Rdim A P −1 , where Rdim A P −1 is the orthogonal complement of H in a P . Since G = P K , the subgroup P acts transitively on X , and the Langlands decomposition of P induces the horospherical decomposition of X : X = NP × AP × X P , which is an analytic diffeomorphism, in particular, every point x ∈ X can be written uniquely as namx0 , where n ∈ N P , a ∈ A P , mx0 ∈ X P . In the case X = H2 , these are the usual horospherical coordinates x, y for z = x + i y ∈ H2 . As mentioned in Proposition 3.10, X (∞) is an infinite simplicial complex with one (open) simplex for each parabolic subgroup P. This simplex can be described by the Langlands decomposition; i.e., α ∈ (p, a P ) if and only if {Y ∈ n P | [H, Y ] = α(H )Y, H ∈ a P } = 0. Note that (p, a P ) is not a root system. When P is minimal, (p, a P ) is the set of positive roots of a root system in a suitable ordering. Let (p, a P ) be the set of roots of a P acting on n P , and a+ P = {H ∈ a P | α(H ) > 0, α ∈ (p, a P )}, + a+ P (∞) = {H ∈ a P | ||H || = 1}.

Then a+ P (∞) is the simplex for P in the Tits building. In this chapter, we will emphasize the role of the horospherical decomposition in understanding compactifications of X , i.e., how sequences of interior points converge to boundary points relative to the horospherical decomposition. The Langlands and horospherical decompositions are generalizations of the Iwasawa decomposition. Another important decomposition is the Cartan decomposition of G or the polar decomposition of X . Specifically, let g be the Lie algebra of G, let g = k + p be the Cartan decomposition, i.e., the decomposition into the eigenspaces of the Cartan involution θ. Let a ⊂ p be a maximal abelian subalgebra, called a Cartan subalgebra of the symmetric pair (G, K ). Then A = exp a is called a Cartan subgroup of the pair (G, K ), and Ax0 = exp ax0 is a maximal flat totally geodesic submanifold in X , which is simply called flat. The converse is also true, i.e., every flat passing through x0 is of the form exp a x0 for some a . Since all Cartan subalgebras of the symmetric pair (G, K ) are conjugate under K and every element of G is contained in some Cartan subgroup, we have the Cartan decomposition G = K AK , and


79

X = K Ax0 = ∪k∈K exp(Ad(k)a)x0 . Let a+ be a positive Weyl chamber in a and let a+ be the closure in a. Then the Weyl group acts on a with a+ as a fundamental domain and hence the the above decomposition of X can be refined to X = K exp a+ · x0 . Since K is compact, to compactify X it is essential to understand compactifications of the flat exp ax0 . In summary, we will look at compactifications of X from two points of view: 1. their relations to parabolic subgroups, the horospherical decomposition and the Tits building, and so on, 2. in terms of the closure of a flat and the Cartan decomposition. Both points of view are intrinsic and do not depend on embeddings of X into some compact spaces.

4 Satake and Furstenberg compactifications One common feature of the Satake and Furstenberg compactifications of X = G/K is that they are both obtained by embedding X equivariantly in a compact G-space. Given a compact G-space Z and a G-equivariant embedding X → Z , the closure of X in Z is a compactification X of X , which admits a continuous G-action. Advantages of this approach include that it is natural and direct and that the continuous extension of the G-action is automatic. On the other hand, it is not always easy to find such an ambient G-space, to identify the closure of a flat, and to interpret the boundary and topology in terms of parabolic subgroups. We start with one particular Satake compactification of Pn = S L(n, C)/SU (n), the space of positive definite Hermitian matrices of determinant 1. The basic observation, that when positive definite matrices degenerate they become semidefinite, suggests using all Hermitian matrices. Let Hn be the real vector space of all Hermitian n×n matrices, P(Hn ) = Hn /R× the associated real projective space. Clearly, S L(n, C) acts on Hn by g · A = g Ag ∗ and hence on P(Hn ), and so P(Hn ) is a compact S L(n, C)-space. Then A ∈ Pn → [A] = RA ∈ P(Hn ) gives a S L(n, C)-equivariant embedding. The closure of Pn in P(Hn ) is called the Satake compactification of Pn associated with the standard S representation id : S L(n, C) → S L(n, C), denoted by Pn . It will be seen below that Pn also admits other Satake compactifications. Some immediate questions include the following: Question 4.1 What are the boundary points? What is the orbital structure of G on the boundary? What are the relations of boundary points to the parabolic subgroups and to the closure of a flat?

80

A. Borel and L. Ji

We first determine the matrices which lie on the boundary. Intuitively, boundary points should consist of semidefinite ones. To be precise, we determine the limits of sequences of diagonal matrices. Any unimodular Hermitian matrix A can be written as A = B diag(d1 , . . . , dn )B ∗ , where B ∈ SU (n), d1 ≥ d2 ≥ · · · ≥ dn > 0, d1 · · · dn = 1. Let y j = diag(d1, j , · · · , dn, j ) be an unbounded sequence satisfying the above condition. Then d1, j → +∞ and dn, j → 0. Assume for all 1 ≤ k ≤ n, ak = lim j dk, j /d1, j exists. Let k0 be the largest integer such that ak0 > 0, i.e., a1 , . . . , ak0 are > 0, and ak0 +1 = · · · = an = 0. Then the image of y j in P(Hn ) is [y j ] = [diag(1, d2, j /d1, j , . . . , dn, j /d1, j )] → [diag(1, a2 , . . . , ak0 , 0, . . . , 0)]. By composing [diag(1, a2 , . . . , ak0 , 0, . . . , 0)] with B ∈ SU (n), we get all boundary points. Since every k0 between 1 and n − 1 can occur in the above limit, the S L(n, C)-orbit of Pk0 in P(Hn ) appears in the boundary, S

Pn = Pn ∪ GPk0 ,

1 ≤ k0 ≤ n − 1.

Since there is only one orbit isomorphic to GPk0 , this suggests that this compactifiS

cation Pn is one of the minimal Satake compactifications. For general symmetric spaces X , the Satake compactifications are constructed in two steps: 1. Embed X into some Pn as a totally geodesic submanifold. S 2. Take the closure of X in Pn , or equivalently in P(Hn ). The basic reference is [Sa1]. Step 1 is carried out by means of finite-dimensional faithful projective representations. For any irreducible faithful projective representation τ : G → P G L(n, C) satisfying τ (θ (g)) = (τ (g)∗ )−1 , where θ is the Cartan involution, define τ τ ∗ : X = G/K → Pn−1 = P G L(n, C)/PU (n),

gK → τ (g)τ (g)∗ .

The above assumption on τ implies that τ τ ∗ is well defined. The closure of τ τ ∗ (X ) in P(Hn ) is called the Satake compactification associated with the representation τ and will be denoted by X¯ τS . S

Question 4.2 What is the structure of X τ as a G-space? How does it depend on the representation τ ? S

S

As in the case of Pn , we analyze X τ through the Cartan decomposition. Let g = k + p be the Cartan decomposition, and let a ⊂ p be a maximal abelian subalgebra in p. Choose a suitable basis of Cn such that for all H ∈ a, τ (e H ) is a diagonal matrix:


81

τ (e H ) = diag(eµ1 (H ) , . . . , eµn (H ) ), where µ1 , . . . , µn are the weights of τ . Choose a positive Weyl chamber a+ . Then by the refined decomposition of X in Section 3, X = K exp a+ x0 . For a sequence H j ∈ a+ going to infinity, we consider the limit τ τ ∗ (exp H j ). When H j goes to infinity, there are several possibilities: either it goes through the interior and stays away from the walls, or it goes along some walls of the positive chamber. Let = (g, a) be the set of simple roots, and µτ the highest weight of the representation τ . Then other weights of τ are of the form µi = µτ − cα,i α, cα,i ≥ 0. α∈

Definition 4.3 The support supp (µi ) of a weight µi of τ is {α ∈ | cα,i > 0}. Proposition 4.4 [Sa1] A subset I of is the support of some weight µ of the irreducible representation τ if and only if I ∪ {µτ } is a connected subset, i.e., not the union of two orthogonal subsets with respect to the Killing form. Such subsets of are called µτ -connected. When µτ is generic, i.e., not contained in any wall of a∗+ , every subset of is µτ -connected. For any µτ -connected subset I , consider a sequence H j ∈ a+ such that 1. For all α ∈ \ I , α(H j ) → +∞. 2. For all α ∈ I , α(H j ) converges to a finite number. Let H∞ be the unique element in a+ such that for α ∈ \ I , α(H ) = 0, while for α ∈ I , α(H∞ ) = lim j→+∞ α(H j ). For simplicity, assume that µ1 , . . . , µk are all the weights whose supports are contained in I , and that µk+1 , . . . , µn are the other weights. Then τ τ ∗ (e H j ) = e2µτ (H j ) diag(e−2

α cα,1 α(H j )

, . . . , e−2

α cα,n α(H j )

),

and its image in P(Hn ) converges to [diag(e−2

α∈I cα,1 α(H∞ )

, . . . , e−2

α∈I cα,k α(H∞ )

, 0, . . . , 0)].

In general, for any subset J ⊂ and a sequence H j ∈ a+ satisfying the following: 1. for α ∈ \ J , α(H j ) → +∞, 2. for α ∈ J , α(H j ) converges to a finite number, let I be the largest µτ -connected subset contained in J . Let H∞ be the unique vector as above satisfying α(H∞ ) = lim α(H j ) for α ∈ I , and α(H∞ ) = 0 for α ∈ \ I.

82

A. Borel and L. Ji

Using similar arguments as above, we can prove the following result: Proposition 4.5 With the above assumption and notation, the sequence τ τ ∗ (e H j ) of S points in X τ converges and the limit only depends on the values lim α(H j ) for α ∈ I , independently of the values lim α(H j ) for α ∈ J \ I . Specifically, let µ1 , . . . , µk be all the weights supported on I as above. Then [τ τ ∗ (e H j )] → [diag(e−2

α∈I cα,1 α(H∞ )

, . . . , e−2

α∈I cα,k α(H∞ )

, 0, . . . , 0)].

S

This proposition suggests that X τ only depends on the collection of µτ -connected S subsets of . In fact, X τ admits a characterization in terms of the µτ -connected subsets (see [Sa1]). Let P0 be the minimal parabolic subgroup determined by the positive Weyl chamber, and P0,I , I ⊂ , the standard parabolic subgroups containing P0 (see [GJT]). Definition 4.6 A standard parabolic subgroup P0,I , I ⊂ , is called µτ -connected if I is µτ -connected. Conjugates of µτ -connected standard parabolic subgroups are also called µτ -connected. Given a µτ -connected parabolic subgroup P, consider the set of all (proper) parabolic subgroups Q which contains P as a maximal µτ -connected parabolic subgroup, i.e., P is a maximal among all the µτ -connected parabolic subgroups contained in Q. This set contains a unique maximal element, called the µτ -saturation of P. Given a µτ -saturated parabolic subgroup Q, its boundary symmetric space X Q admits a unique splitting X Q = X P(Q) × X P(Q)⊥ , where P(Q) is a maximal µτ -connected parabolic subgroup contained in Q, X P(Q) is the boundary symmetric space of P(Q), and X P(Q)⊥ is the boundary symmetric space of a complementary parabolic subgroup P(Q)⊥ contained in Q. Though P(Q) is not uniquely determined, the boundary space X P(Q) is a canonical subspace in X Q . When the representation τ is generic, i.e., µτ is contained in the positive chamber a+ , every subset I of is µτ -connected, and hence every parabolic subgroup P is µτ -connected, and the µτ -saturation of P is equal to itself. S The following characterization of X τ is given in [Sa1]. S

Proposition 4.7 The Satake compactification X τ is the unique Hausdorff G-compactification satisfying the following conditions: (1). For every µτ -connected parabolic subgroup P, the boundary component X P is S canonically embedded in X τ , and S Xτ = X ∪ X P(Q) . µτ −saturated Q


83

(2). For any µτ -saturated Q, g ∈ G, either g X P(Q) = X P(Q) or g X P(Q) ∩ X P(Q) = ∅, and the stabilizer of the boundary component X P(Q) , namely {g ∈ G | g X P(Q) = X P(Q) }, is equal to Q. (3). Given an unbounded sequence H j ∈ a+ , the points τ τ ∗ (e H j ) converge in S

X τ if and only if there exists a µτ -connected subset I of such that for α ∈ I , lim j→+∞ α(H j ) exists and is finite, and for any µτ -connected subset J strictly containing I , there exists at least one α ∈ J \ I , such that α(H j ) → +∞. The limit of τ τ ∗ (e H j ) belongs to X PI and only depends on the values lim j→+∞ α(H j ), α ∈ I . In (3), if H∞ is the vector in a such that α(H∞ ) = lim α(H j ) for α ∈ I , and α(H∞ ) = 0 for α ∈ \ I , then the limit is exp H∞ · x0 ∈ X PI . We apply this proposition to the Satake compactification of Pn associated with the standard representation computed earlier. The positive Weyl chamber a+ consists of matrices H = diag(t1 , . . . , tn ) satisfying t1 > t2 > . . . > tn , the highest weight is µτ (H ) = t1 , and the µτ -connected subsets are α1 , . . . , αk , k = 1, · · · , n, where α j = t j − t j+1 . The µτ -connected boundary spaces are exactly the Pk , k = 1, . . . , n − 1. S

Corollary 4.8 The Satake compactification X τ only depends on the Weyl chamber face which contains µτ as an interior point, and hence there are at most finitely many different Satake compactifications. S

When µτ is generic, X τ is called a maximal Satake compactification, denoted by S X max . S

Proposition 4.9 For any Satake compactification X τ , the identity map on X extends S S to a continuous, surjective map X max → X τ . In fact, the set of all Satake compactifications is partially ordered. S

S

Proposition 4.10 Given two Satake compactifications X τ1 , X τ2 , if µτ1 is more regular than µτ2 , i.e., the Weyl chamber face containing µτ1 as an interior point contains the face for µτ2 in its boundary, then the identity map on X extends to a continuous S S surjective map X τ1 → X τ2 . For the space X = S L(3, R)/S O(3), there are two possibilities, either µτ belongs to the interior of a+ or to its edges. In the former case, the compactification S is maximal, and in the latter case, it is minimal. The compactification Pn associated with the standard representation is a minimal one since the highest weight is contained in an edge. Proposition 4.7 describes clearly relations between the Satake compactifications and parabolic subgroups. As mentioned earlier, another point of view is to consider the closure of a flat ea x0 . It turns out to also have a simple characterization (see [J1]).

84

A. Borel and L. Ji S

Proposition 4.11 The closure of a flat ea x0 in X τ is canonically homeomorphic to the convex hull of the weights of the representation τ , or equivalently, to the convex hull of the Weyl group orbit of the highest weight µτ . S

Corollary 4.12 The Satake compactification X τ is a topological ball and hence contractible. Now we study the Furstenberg compactifications of X . We motivate this construction through the example of X = D, the Poincaré disc. The Dirichlet problem u = 0 in D u= f on ∂ D = S 1 , f ∈ C 0 (S 1 ), is solved by

u(z) =

S1

1 − |z|2 f (ξ )dξ, |z − ξ |2

for every point z ∈ D,

1−|z|2 dξ |z−ξ |2

is a probability measure on S 1 . Let

1−|z|2 S 1 |z−ξ |2 dξ , M1 (S 1 ) be the

where the total measure of dξ on S 1 is normalized to be 1. Since 1 = space of probability measures on S 1 . Then the map z →

1 − |z|2 dξ |z − ξ |2

gives an embedding i : D → M1 (S 1 ). Since M1 (S 1 ) is compact, the closure of D in M1 (S 1 ) is a compactification, called the Furstenberg compactification and denoted F by D . F

Lemma 4.14 The identity map on D extends to a homeomorphism D ∪ S 1 → D . Proof. It can be seen easily that for a sequence z j ∈ D converging to a boundary point 1−|z |2

ξ0 ∈ S 1 , the measures |z −ξj |2 dξ converge to the delta (or Dirac) measure supported j at ξ0 . This gives the homeomorphism. Recall that D = SU (1, 1)/U (1). To show that M1 (S 1 ) is a SU (1, 1)-space and the above embedding is SU (1, 1)-equivariant, we note that G = SU (1, 1) acts transitively on S 1 . Fix any point in S 1 . Its stabilizer in G is a parabolic subgroup P. Then S 1 = G/P. The measure dξ on S 1 is the unique probability measure invariant 1−|z|2 dξ is under K = U (1). It can be shown that for a point z = g · 0, the measure |z−ξ |2 equal to the measure g ∗ dξ . This fact suggests the following approach to compactify general symmetric spaces X = G/K . For any proper parabolic subgroup P, G/P is a nontrivial compact G-space, called a Furstenberg boundary. Let M1 (G/P) be the space of probability measures on G/P. Then M1 (G/P) is a compact G-space with respect to the weak *-topology, where a sequence of measures µ j converges to a measure µ if and only for any f ∈ C 0 (G/P), f dµ j → f du.


85

Since G = K P, the subgroup K acts transitively on G/P, and hence there exists a unique K -invariant probability measure dµ P in M1 (G/P). Define a map i P : X = G/K → M1 (G/P),

gK → g ∗ dµ P .

Since dµ P is K -invariant, this map is well defined and G-equivariant. Proposition 4.15 The map i P is an embedding if G is simple. In general, for a semisimple G, if P does not contain any simple factor of G, i P is an embedding. The idea of the proof is to use the Bruhat decomposition of G/P. In fact, G/P contains N P− as a dense open subset, where P − is the opposite parabolic subgroup of P. If g ∈ K , then g = k1 ak2 , where k1 , k2 ∈ K , a ∈ A, and a acts nontrivially on N P− . This implies injectivity. See [Mo1] or [GJT] for details. For the rest of this section, we assume that parabolic subgroups satisfy the condition in the above proposition. Definition 4.16 The closure of i P (X ) in M1 (G/P) is called the Furstenberg compactification associated with P or the Furstenberg boundary G/P and denoted by F X P. Clearly conjugate parabolic subgroups define isomorphic compactification (see [GJT]). When P is a minimal parabolic subgroup, G/P is a maximal Furstenberg F boundary, and X P is called the maximal Furstenberg compactification and denoted F by X max . F

F

Proposition 4.17 For two Furstenberg compactifications X P1 and X P2 , if P1 ⊂ P2 , F

F

then the identity map on X extends to a continuous map X P1 → X P2 . In particular, F

the maximal Furstenberg compactification X max dominates all other Furstenberg compactifications. Proof. When P1 ⊂ P2 , G/P1 projects surjectively to G/P2 , and the unique K invariant measure on G/P1 is pushed forward to the unique K -invariant measure on G/P2 . This implies that the map g ∗ dµ P1 → g ∗ dµ P2 is continuous and proves the proposition. Proposition 4.18 For a standard parabolic subgroup P = P0,I , the Furstenberg F S compactification X P is isomorphic to the Satake compactification X τ , where τ satisfies the condition that its highest weight µτ is contained in the interior of + a+ I = {H ∈ a | α(H ) = 0, α ∈ I }. Therefore, the Furstenberg compactificaF

S

tions X are the same as the Satake compactifications X . The maximal Satake– SF Furstenberg compactification will be denoted by X max . This proposition was proved by Moore [Mo1] using the axiomatic characterization of the Satake compactifications in Proposition 4.7. Remark 4.19 One nice feature of the Furstenberg compactifications is that the F Furstenberg boundary G/P sits canonically in X P . In fact, for every point ξ in G/P,

86

A. Borel and L. Ji

the Dirac measure δξ belongs to the boundary of X in M1 (G/P). It should be emF

phasized that the whole boundary ∂ X P contains G/P strictly when the rank of X is greater than 1. For a minimal parabolic subgroup P, G/P is the Poisson boundary of X in the sense that bounded harmonic functions on X are in one-to-one correspondence with the L ∞ -functions on G/P.

SF

5 Alternative constructions of X max In the previous section we studied general Satake–Furstenberg compactifications. Among all these compactifications, the maximal one plays a special role and occurs naturally in other contexts. Its relations to potential theory and the Martin compactification will be explained in detail in the next section. In this section, we give alternative constructions of the maximal Satake–Furstenberg compactification. The three alternative constructions can be briefly described as follows: 1. Assign boundary components (or faces) to parabolic subgroups and use reduction theory to construct the compactification. This is related to the Borel–Serre compactification of locally symmetric spaces in Section 8. 2. Start from compactifications of flats and glue these compactifications together. 3. Use the compact G-space S(G) of closed subgroups in G and an equivariant embedding X → S(G). SF

Besides these constructions of X , there are others. For example, the closure of X in the Oshima compactification, or the Oshima–Sekiguchi compactification is SF X , though X is not dense in them. They will be discussed in Part II. We start with the first construction. Borel and Serre defined a partial compactification X¯ B S of X in [BS] by assigning boundary faces to rational parabolic subgroups and defining convergence to ideal boundary points intrinsically. The generalization to symmetric spaces X uses all real parabolic subgroups. It will be seen later that this procedure can be used to construct many known compactifications. Specifically, the basic steps of this method consist of the following: 1. For each real parabolic subgroup P, assign a boundary face e(P). 2. Convergence of interior points to boundary points in e(P) is given in terms of the Langlands decomposition of P or the induced horospherical decomposition of X . 3. Show that the topology on X ∪ P e(P) is Hausdorff. Unlike the case of locally symmetric spaces \X , we need to put a topology on the set of boundary components e(P) or the set of real parabolic subgroups. Because of this, the Hausdorff property of the topology is more subtle to establish and can be proved by some stronger version of reduction theory for real parabolic subgroups. It will be seen below that this approach has several general features:


87

1. The extension of the G-action to the compactification is easy. 2. The use of reduction theory relates to compactifications of locally symmetric spaces. 3. It gives a uniform construction of compactifications of X by varying the choices of boundary faces e(P) and the collection of parabolic subgroups. Every real parabolic subgroup P admits a Langlands decomposition with respect to the basepoint x0 = K ∈ X = G/K : P = NP AP MP , where A P M P is the Levi component invariant under the Cartan involution associated with the maximal compact subgroup K . Let X P = M P /K ∩M P . Then the Langlands decomposition of P induces a horospherical decomposition of X , X = NP × AP × X P . For each P, define its boundary face e(P) = X P .

The topology on X ∪ P e(P) is given as follows. A sequence of points y j in X converges to a point z ∞ ∈ X P if y j can be written in the form y j = k j (n j , a j , z j ), where k j ∈ K , (n j , a j , z j ) ∈ N P × A P × X P satisfy 1. k j → e, the identity element, α 2. a j → ∞ away from the walls of exp a+ P , i.e., for all α ∈ (P, A P ), a j → +∞, 3. a −1 j n j a j → e.

For a pair of parabolic subgroups P1 ⊂ P2 , e(P1 ) appears as a boundary face of e(P2 ), and convergence of sequences of points in e(P2 ) to points in e(P1 ) can also be described similarly, although the topology of boundary faces needs to be taken into account. See [BJ2] for details. It is important to point out that the expression of y = k(n, a, z) is not unique, and hence an immediate question is whether each convergent sequence has a unique limit, i.e., whether the topology is Hausdorff. For this purpose, we need to recall the definition of a Siegel set and its generalizations. For bounded sets U ⊂ N P , V ⊂ X P , and A P,t = {a ∈ A P | a α > t, α ∈ (P, A P )},

t > 0,

the subset U × A P,t × V in X is called a Siegel set associated with the parabolic subgroup P. A key result in reduction theory for parabolic subgroups is the following strong separation property (see [BJ2]). Proposition 5.1 Let P1 , P2 be two different parabolic subgroups, and U1 ×A P1 ,t × V1 , U2 × A P2 ,t × V2 associated Siegel sets. Let C be a small neighborhood of e in K such that for all k ∈ C, k P1 k −1 = P2 . If t 0, then for all k ∈ C,

88

A. Borel and L. Ji

k(U1 × A P1 ,t × V1 ) ∩ (U2 × A P2 ,t × V2 ) = ∅. The Siegel sets are not large enough to form neighborhoods of boundary points. One consequence of the boundedness of U ⊂ N P is that for a sequence a j ∈ exp a+ P going to infinity away from the walls, a −1 j U a j → e. But there are unbounded se-

quences n j ∈ N P such that a −1 j n j a j → e. To overcome this difficulty, we need to generalize Siegel sets. Let d be a left invariant metric on N P and B N P (ε) a ball of radius ε with center e. For ε > 0, V ⊂ X P a bounded set, t > 0, define S P,ε,t,V = {(n, a, z) ∈ N P × A P × X P | a ∈ A P,t , z ∈ V, a −1 na ∈ B N P (ε)}. The discussion in the previous paragraph shows that every Siegel set is contained in one of these, and that a convergent sequence of points in X is also eventually contained in any such set. Proposition 5.2 For two different P1 , P2 and a small neighborhood C of e in K such that for k ∈ C, k P1 k −1 = P2 , then for ε 1, t 0, all k ∈ C, k S P1 ,ε,t,V1 ∩ S P2 ,ε,t,V2 = ∅. For the proof of this proposition and the earlier one, see [BJ2]. Corollary 5.3 The topology on X ∪ P e(P) is Hausdorff. Proposition 5.4 The G-action on X extends to a continuous action on X ∪

P

e(P).

Proof. For z ∈ X , g ∈ G, write g = kman, k ∈ K , m ∈ M P , a ∈ A P , n ∈ N P , and define the action on the boundary by g · z = k(mz) ∈ X k P . In this definition, the N P , A P factors are ignored. We note that although each of k, m is not uniquely determined by g, the product km is. It can be checked easily that g maps a convergent sequence to a convergent sequence and this G-action is continuous. Proposition 5.5 The compactification X ∪ P e(P) defined here is isomorphic to SF

X max . Proof. By the axiomatic characterization of the Satake compactifications, it is clear SF that X max has the same underlying space as X ∪ P e(P). We only need to SF show that a convergent sequence in X ∪ P e(P) also converges in X max . Let y j = k j (n j , a j , z j ), where k j ∈ K , n j ∈ N P , a j ∈ A P , z j ∈ X P , for some parabolic subgroup P, satisfy conditions: (1) k j → e, (2) z j → z ∞ , (3) a j → ∞ −1 through the positive chamber exp a+ P and away from the walls, (4) a j n j a j → e. Write z j = m j K for m j ∈ M P and assume that m j → m ∞ . Then


89

τ τ ∗ (y j ) = τ (k j )τ (n j )τ (a j )τ (m j )τ (m j )∗ τ (a j )∗ τ (n j )∗ τ (k j )∗ = τ (k j )τ (a j )τ (a j n j )τ (m j )τ (m j )∗ τ (a j n j )∗ τ (k j )∗ , aj ∗ where a j n j = a j n j a −1 j . Since k j → e and n j → e, the sequence τ τ (k j ) has the ∗ same limit as τ τ (a j m j ), which converges to z ∞ ∈ X P . Therefore, y j also conSF

verges to z ∞ in X max .

aj Remark 5.6. The condition n j → e in the definition of convergent sequences in X ∪ P e(P) was motivated by the computations just made. Next we describe the dual cell compactification in [GJT]. Recall that a flat in X is a maximal totally geodesic flat submanifold. For a maximal abelian subalgebra a in p, where g = k + p is the Cartan decomposition, exp ax0 is a flat passing through the basepoint x0 . The converse is also true, i.e., every flat passing through x0 is of this form. By the Cartan decomposition,

X = K exp ax0 = ∪k∈K Ad(k) exp ax0 , i.e., X is the union of flats passing through the basepoint x0 . One approach to compactify X is to compactify all such flats and then glue these compactifications together. S For X max , we know that the closure of a flat is homeomorphic to the convex hull of the Weyl group orbit of a generic highest weight µτ . But this convex hull depends on the choice of the highest weight µτ . To get a more intrinsic construction, we note that the boundary of the convex hull is a cell complex dual to the Weyl chamber decomposition of a. We start with polyhedral compactifications of the Euclidean space a. Definition 5.7. A polyhedral cone decomposition of a is a partition of a − {0} into convex polyhedral cones, i.e., cones of the form (∩mj=1 { j > 0}) ∩ (∩nj=m+1 { j = 0}), where j are linear functionals, and m > 0 but n could be equal to m. Each polyhedral cone here is proper, i.e., does not contain any nontrivial linear subspace. Let a(∞) be the unit sphere in a. Then a polyhedral cone decomposition determines a cell complex structure () on a(∞). In fact, for each cone C ∈ , let C(∞) = a(∞) ∩ C. Then C(∞) is a cell and a(∞) = C(∞). C∈

The cell complex () has a dual cell complex ∗ (), which can be realized ⊥ as follows. For each cone C ∈⊥ , let C be the largest linear subspace orthogonal ∗ to C. Let () = C∈ C with the following cell complex structure. For two cones C1 , C2 ∈ , C1⊥ is a face of C2⊥ if and only if C1 contains C2 as a face. It can be checked that ∗ () is a cell complex dual to ().

90

A. Borel and L. Ji

This realization of the dual cell complex ∗ () can be used to compactify a. For a cone C ∈ and a point H ∈ C ⊥ , a sequence y j ∈ a converges to H if and only if when y j = y j,C + y j,C ⊥ , where y j,C belongs to the linear span of C and y j,C ⊥ ∈ C ⊥ , then (1) y j,C → ∞ through the interior of C, i.e., y j,C belongs to C + V for every v ∈ C; (2) y j,C ⊥ → H . Definition 5.8 The compactification a ∪ ∗ () is called the dual cell compactification associated with the polyhedral cone decomposition . For each Cartan subalgebra a, the Weyl chambers and their faces give a polyhedral cone decomposition. This implies that the flat exp ax0 ∼ = a admits a dual cell compactification, denoted by a ∪ ∗ (a). For any two Cartan subalgebras a1 , a2 , their Weyl chamber decompositions coincide on the intersection a1 ∩ a2 and induce a polyhedral cone decomposition on the intersection. This implies that the compactifications a1 ∪ ∗ (a1 ), a2 ∪ ∗ (a2 ) are compatible, and the intersection (a1 ∪ ∗ (a1 )) ∩ (a2 ∪ ∗ (a2 )) is the dual cell compactification of a1 ∩ a2 . This shows that these dual cell compactifications of the flats passing the basepoint x0 can be glued together and define a compactification of X , called the dual cell compactification of X and denoted by X ∪ ∗ (X ). Proposition 5.9 The dual cell compactification X ∪ ∗ (X ) is isomorphic to the S maximal Satake compactification X max . S Proof. For each flat exp ax0 ∼ = a, its closure in X max is homeomorphic to the convex hull of the Weyl group orbit of a generic weight and hence homeomorphic to the dual S cell compactification. This homeomorphism between the closure of exp ax0 in X max and the dual cell compactification can also be checked directly from the axiomatic S characterization of X max . Then the rest of the proof follows from the decomposition X = ∪k∈K k exp ax0 . See [GJT] for details. The dual cell compactification X ∪ ∗ (X ) can be characterized by the following proposition (see [GJT] for proof).

Proposition 5.10 The dual cell compactification X ∪ ∗ (X ) is the unique Hausdorff compactification such that 1. the K -action on X extends to a continuous action on X ∪ ∗ (X ); 2. the closure of a flat exp ax0 is the dual cell compactification a ∪ ∗ (a); 3. for any two flats exp a1 x0 , exp a2 x0 , their closures satisfy exp a1 x0 ∩ exp a2 x0 = exp a1 x0 ∩ exp a2 x0 . Remark 5.11. The condition (3) in the above proposition is not automatic. We could have two disjoint sets whose closures have nonempty intersection. For example, the one point compactification does not satisfy this condition. There is one unsatisfactory point about X ∪ ∗ (X ). The continuous extension of the G-action to X ∪ ∗ (X ) is not obvious. One reason is that we only use flats passing through the basepoint x0 , and another is that the definition of the topology of X ∪ ∗ (X ) depends on the Cartan decomposition X = K exp ax0 . In [GJT], a


91

G-action on X ∪ ∗ (X ) is defined, but the continuity is not obvious and is proved through identification with other G-compactifications of X . This problem is one of the motivations for introducing the Borel–Serre approach earlier, i.e., giving an intrinsic direct construction. Proposition 5.12 The dual cell compactification X ∪ ∗ (X ) is homeomorphic to the compactification X ∪ P e(P), and hence the G-action on X extends to a continuous action on X ∪ ∗ (X ). Proof. It can be checked easily that a convergent sequence in X ∪ ∗ (X ) is also convergent in X ∪ P e(P). Since both compactifications have the same underlying space, they are homeomorphic, and hence the extension of the G-action on X ∪ ∗ P e(P) is transported to X ∪ (X ). For details, see [BJ1]. To connect to a compactification of \G to be discussed later, we consider yet SF another construction of X max , the third construction. Let S(G) be the space of closed subgroups in G. S(G) has a topology given as follows (see [Bou, Chap. 8, §5] for more details and a proof of the next lemma). For a subgroup H ∈ S(G), let C ⊂ G be a compact subset in G and U ⊂ G a neighborhood of the identity element. Define VH (C, U ) = {H ∈ S(G) | H ∩ C ⊂ U H, H ∩ C ⊂ U H }. The condition in the definition says that H, H are close to each other on any compact region C. Lemma 5.13 The space S(G) is a compact, Hausdorff G-space, where G acts by conjugation. Define a map X = G/K → S(G),

gK → gK g −1 .

Lemma 5.14 The map X → S(G) is a G-equivariant embedding. Proof. It follows from the fact that K is equal to its own normalizer. The closure of X in S(G) is a G-compactification, called the subgroup compactsb ification, denoted by X . SF

sb

Proposition 5.15 The identity map on X extends to a homeomorphism X max → X . For more details and proofs in the general case, see [GJT]. We illustrate this through the example of X = H2 = S L(2, R)/S O(2). Let P be the standard parabolic subgroup of upper triangular matrices. Then a 0 + AP = |a∈R . 0 a −1 aj 0 Consider a sequence z j = · i = a 2j · i, where a j → +∞. Then 0 a −1 j

92

A. Borel and L. Ji

−1 −1 aj 0 aj 0 aj 0 aj 0 cos θ sin θ K = | θ ∈ [0, 2π ] −1 0 a −1 0 a − sin θ cos θ 0 a 0 aj j j j

=

cos θ a 2j sin θ −a −2 j sin θ cos θ

| θ ∈ [0, 2π] .

For any sequence θ j , the limit a 2j sin θ j exists only if either θ j → 0 and a 2j θ j converges to a finite number, or θ j → π and a 2j (θ j − π ) converges to a finite number. This implies that the limit subgroup of this sequence in S(G) is equal to 1b ± | b ∈ R = NP MP . 01 In the general case of X = G/K , if a sequence y j in X converges to a point m ∞ K P ∈ X P = M P /K P , where K P = K ∩ P, the sequence of subgroups y j K y −1 j −1 in S(G) converges to m ∞ (N P M K P )m −1 ∞ = N P Mm ∞ K P m ∞ , where M is the centralizer of A = exp a in K , and m ∞ K P m −1 ∞ is the stabilizer of the point m ∞ K P in M P .

6 Martin compactifications and Karpelevic compactification In this section we review the Martin compactification of a complete Riemannian manifold and its determination in terms of other more geometric compactifications in the case of symmetric spaces. Then we summarize the Karpelevic compactification and show how the general Borel–Serre approach can be applied to construct all the previous compactifications by varying the choices of the boundary faces (or components). Let X be any complete, connected noncompact Riemannian manifold and its Laplacian–Beltrami operator. As mentioned earlier, one problem is to understand the structure of the set of harmonic functions. Clearly, constant functions are harmonic functions, and the following questions occur naturally. Question 6.1 1. Are there any nonconstant bounded harmonic functions? 2. How to parametrize bounded harmonic functions? 3. How to parametrize positive harmonic functions? To suggest answers to these questions, we start with the example of the Poincaré disc X = D. The Dirichlet problem u = 0 in D,

u = f on ∂ D = S1 , f ∈ C 1 (S1 ),

is solved by the Poisson integral formula


u(z) =

S1

93

1 − |z|2 f (ξ )dξ. |z − ξ |2

By taking f ∈ L ∞ (S1 ), we get all bounded harmonic functions on D using the Poisson integral formula. This gives a one-to-one correspondence between the set of bounded harmonic functions and L ∞ (S1 ). This means that S1 is the Poisson boundary of D. Replacing f (ξ )dξ by nonnegative measures on S1 , we get a one-to-one correspondence between the set of positive harmonic functions on D and the set of nonnegative measures on S1 , and hence S1 is also the Martin boundary of D. In this example, the Martin boundary coincides with the Poisson boundary. To construct the Poisson and Martin boundaries of general manifolds, we study a generalization of the above problems. For each λ ∈ R, define Cλ (X ) = {u ∈ C ∞ (X ) | u = λu, u > 0}. Lemma 6.2 For each λ, Cλ (X ) is a convex cone with a compact homothety section. Proof. It is clear that Cλ (X ) is a convex cone. For any basepoint, the subset of functions satisfying u(x0 ) = 1 forms a homothety section, denoted by Cλ,1 (X ). By the Harnack inequality, for any compact subset K ⊂ X , there is a constant M such that for all u ∈ Cλ,1 (X ), x ∈ K , u(x) ≤ M. This implies that every sequence of functions in Cλ,1 has a convergent subsequence. Since Cλ (X ) is a convex cone, a natural problem is to find extremal elements and to express other functions as linear combinations of them. This problem can be solved by the Martin compactification. Let 2 X | ϕ| λ0 (X ) = inf 2 ϕ∈C0∞ (X ) X |ϕ| be the bottom of the spectrum of X . Lemma 6.3 The cone Cλ (X ) is nonempty if and only if λ ≤ λ0 (X ). This result is due to Cheng–Yau, and Sullivan; see [Ta1] [J4] for a proof due to Varopoulos. For each λ ≤ λ0 (X ), Cλ (X ) is nonempty, and there is an associated Martin compactification of X , denoted by X ∪ ∂λ X . Let G λ (x, y) be the Green function of − λ, i.e., G λ (x, y) is a positive function such that G λ (x, y) = G λ (y, x), G λ (x, y)−λG λ (x, y) = δ y (x), and G λ (x, y) → 0 as y → ∞. Fix a basepoint x0 ∈ X , define the normalized Green function K λ (x, y) = G λ (x, y)/G λ (x0 , y). An unbounded sequence y j in X is called fundamental if K λ (x, y j ) converges uniformly over compact subsets to a function K λ (x, ξ ). Two fundamental sequences are called equivalent if the K λ (x, y) converge to the same function.

94

A. Borel and L. Ji

Define ∂λ X to be the set of equivalence classes of fundamental sequences. For each point ξ ∈ ∂λ X , there is a unique limit function K λ (x, ξ ). Clearly, K λ (x, ξ ) ∈ Cλ (X ). The Martin compactification of X is the space X ∪ ∂λ X with the following topology: an unbounded sequence y j in X converges to a point ξ ∈ ∂λ X if and only if K λ (x, y j ) converges uniformly to K λ (x, ξ ). This topology can be described explicitly in terms of a metric (see [GJT] or [Ta2]). Proposition 6.4 For every u ∈ Cλ (X ), there is a nonnegative measure dµ on ∂λ X such that u(x) = K λ (x, ξ )dµ. ∂λ X

The integral in the above proposition is similar to the Poisson integral formula, and called the Martin integral representation. In general, the measure dµ on ∂λ X is not unique. To get uniqueness, we need to restrict the support of the measure. A function u ∈ Cλ (X ) is called minimal if for any v ∈ Cλ (X ), v ≤ u implies that v = cu for some constant c, equivalently, v is an extremal element of Cλ (X ). The Martin integral representation implies that every minimal function is of the form K λ (x, ξ ). Let ∂λ,min X = {ξ ∈ ∂λ X | K λ (x, ξ ) is minimal}. Proposition 6.5 For any u ∈ Cλ (X ), there is a unique nonnegative measure dµ on ∂λ,min X such that u(x) = K λ (x, ξ )dµ(ξ ). ∂λ,min X

The unique measure in the above proposition dµ is called the representing measure of u and denoted by dµu . It satisfies the following monotonicity property: u ≤ v implies that dµu is absolutely continuous with respect to dµv . For λ = 0, the case of harmonic functions, let dµ0 be the representing measure of u = 1. Denote the support of dµ0 by . Then the monotonicity property implies the following correspondence. Proposition 6.6 The measure space (, dµ0 ) is the Poisson boundary of X , i.e., every bounded harmonic function u is of the form u(x) = K λ (x, ξ ) f (ξ )dµ0 ,

where f ∈ L ∞ (, dµ0 ) and is uniquely determined by u, and hence there is a oneto-one correspondence between the set of bounded harmonic functions on X and the space L ∞ (, dµ0 ). Another related application of the Martin compactification for λ = 0 is that almost surely, every Brownian path converges to a point in in the topology of X ∪ ∂0 X .


95

Although the Martin compactification solves the problems abstractly, a natural question is how to determine the Martin compactification and its minimal boundary. By definition, it depends on the asymptotic behavior of Green’s function at infinity. For symmetric spaces, we can study the Green function using the spherical Fourier transform. For the rest of this section, we assume that X is a symmetric space of noncompact type. Let h t (x, y) be the heat kernel of X , i.e., h t (x, y) is the unique function satisfying ∂ −h t (x, y) = h t (x, y), lim h t (x, y) = δ y (x). t→0 ∂t Then the Green function G λ (x, y) can be expressed by ∞ G λ (x, y) = h t (x, y)e−tλ dt. 0

Since G acts isometrically on X , for any g ∈ G, h t (gx, gy) = h t (x, y). Let x0 = K ∈ X = G/K be the basepoint. Then it is suffices to consider h t (e H x0 , x0 ), where H ∈ a+ . The spherical Fourier transform shows that 2 2 H h t (e x0 , x0 ) = const. e−(|ρ| +|| )t ϕ (e H )|c()|−2 d, a∗

where ρ is the half sum of the positive roots, with multiplicities, |ρ|2 = λ0 (X ), ϕ (x) is the spherical function, and c() is the Harish-Chandra c-function. In a joint work with Anker [AJ], we get sharp bounds on h t (e H x0 , x0 ) and asymptotics near infinity, and as a corollary, we got sharp bounds and asymptotics of the Green function G λ (x, y) at infinity. The sharp bounds were used in [GJT] to identify the Martin compactification of X . Proposition 6.7 If λ = λ0 (X ), the space X ∪ ∂λ X is isomorphic to the maximal SF Satake–Furstenberg compactification X max ; if λ < λ0 (X ), X ∪ ∂λ X is the least SF common refinement of X ∪ X (∞) and X max , i.e., the closure of X under the diagonal SF embedding into (X ∪ X (∞)) × X max . The Martin kernel functions K λ (x, ξ ) can be written down explicitly in terms of spherical functions of boundary symmetric spaces. For λ < λ0 , the identify map on X extends to a continuous map π : X ∪ ∂λ X → X ∪ X (∞), and the fibers over the boundary can be described explicitly. For every point z ∈ X (∞), let P be its stabilizer in G. Then P is a proper parabolic subgroup of G. Let X z = X P be the boundary symmetric space associated with P. Then the fiber π −1 (z) is equal to the maximal Satake–Furstenberg compactification of X z . This implies that SF X z max . X ∪ ∂λ X = X ∪ z∈X (∞)

96

A. Borel and L. Ji

In [Ka], Karpelevic defined a compactification of X using the two equivalence N

relations ∼ and ∼ on the set of geodesics. In fact, he used these relations to get the boundary symmetric spaces X z , z ∈ X (∞). The Karpelevic compactification is K defined inductively on the rank. When r k(X ) = 1, X = X ∪ X (∞). Let r = r k(X ) K and assume that for any symmetric space Y of rank less than r , Y has been defined. For any z ∈ X (∞), the boundary symmetric space X z is of rank less than r and K hence X z is defined. Let K K X =X∪ Xz . z∈X (∞) K

This only defines X as a set. It is helpful to compare this with the description of SF SF X ∪ ∂λ X = X ∪ z∈X (∞) X z given in the previous identity. Since X z does not dominate the geodesic compactification, the Martin compactification is not inductive. K In [GJT], an alternative, non-inductive definition of X is given, i.e., convergent sequences to all boundary points are given. The idea is to consider the closure of a flat exp a · x0 as in the dual cell compactification X ∪ ∗ (X ). K To motivate the definition of convergent sequences in X , we consider converS F gent sequences exp H j .x0 , H j ∈ a+ , in X max and X ∪ ∂λ X . SF

In X max , if there exists a proper subset I ⊂ (g, a) such that for α ∈ I , α(H j ) converges to a finite number lim α(H j ), while for α ∈ \ I , α(H j ) → +∞, then SF

e H j x0 converges in X max , and the limit point only depends on the values lim α(H j ), α ∈ I , but not on the relative sizes of the α(H j ), α ∈ \ I . In X ∪ ∂λ X , λ < λ0 (X ), e H j x0 converges if and only if there exists a proper subset I ⊂ such that for α ∈ I , lim α(H j ) exists and is finite, and for α ∈ \ I , H H α(H j ) → +∞; moreover, the direction ||H jj || converges to a vector lim j→+∞ ||H jj || , the limit point in X ∪ ∂λ X depends on both the values lim α(H j ), α ∈ I , and the H direction lim j→+∞ ||H jj || . K

In X , we refine the information contained in the direction

Hj ||H j || .

In fact, the

K

sequence · x0 converges in X if and only if H j satisfies the conditions for convergence in X ∪∂λ X ; furthermore, there is a partition of \ I , \ I = J1 ∪· · ·∪ Jk α(H ) such that for all α, β in the same subset Jm , lim j→+∞ β(H jj ) exists and is a finite eHj

positive number, while for α ∈ Jm 1 , β ∈ Jm 2 , m 1 < m 2 ,

α(H j ) β(H j )

→ 0. The limit of

K

e H j · x0 in X depends on all the above data. The above discussions suggest the following result. Proposition 6.8 The identity map on X extends to a continuous map X For λ < λ0 , this map is a homeomorphism if and only if r k(X ) ≤ 2.

K

→ X ∪∂λ X .

Assume that r k(X ) = 2. If I = ∅, the only partition of \ I is the trivial one; if I = ∅, admits 3 different partitions. In all these cases, it can be checked that a


97

K

sequence e H j x0 is convergent in X if and only if it is convergent in X ∪ ∂λ X . For K more details of X and the proof of this proposition, see [GJT]. As mentioned in Section 5, the Borel–Serre approach can be used to construct all the compactifications mentioned earlier, and different compactifications come from different choices of boundary faces. We start with the geodesic compactification X ∪ X (∞). For each proper parabolic subgroup P, let A P be the split component in the Langlands decomposition. P de+ + termines a positive chamber a+ P in a P . Let a P (∞) be the set of unit vectors in a P . Take e(P) = a+ P (∞). + Then endow X ∪ P a P (∞) with the following topology: A sequence y j = k j (n j , a j , z j ) ∈ X , where k j ∈ K , n j ∈ N P , a j ∈ A P , z j ∈ X P , converges to a point H∞ ∈ a+ P (∞) if k j → e, a j → ∞ in A+ P , and log a j /|| log a j || → H∞ , a −1 n a → e, j j j d(z j , x0 )/|| log a j || → 0. It can be shown that X P ∪ P e(P) is a compact, Hausdorff space and is homeomorphic to X ∪ X (∞). SF The maximal Satake–Furstenberg compactification X max has been constructed in this way in Section 4, and for each parabolic subgroup P, its boundary face is e(P) = X P . If λ < λ0 (X ), for each parabolic subgroup P, its boundary face in the Martin compactification X ∪ ∂λ X is 1. 2. 3. 4.

e(P) = a+ P (∞) × X P . On the space X ∪ P e(P), an unbounded sequence y j in X converges to a boundary point (H∞ , z ∞ ) ∈ e(P) if y j can be written as y j = k j (n j , a j , z j ), where k j ∈ K , n j ∈ N P , a j ∈ A P , z j ∈ X P satisfy conditions: 1. 2. 3. 4.

k j → e, a j → ∞ in A+ P , and log a j /|| log a j || → H∞ , n a → e, a −1 j j j z j → z∞.

It can be shown that this defines a Hausdorff topology on X ∪ P e(P), and the compactification is isomorphic to the Martin compactification X ∪ ∂λ X . K For the Karpelevic compactification X , the boundary face of a parabolic subgroup P is K

e(P) = a+ P (∞) × X P ,

98

A. Borel and L. Ji K

+ where a+ P (∞) is a suitable blowup of the closure a P (∞) in a P . Basically, it takes all possible partitions of into consideration as indicated before. For details see [BJ2]. In these compactifications, there is one boundary face for each proper parabolic S subgroup. On the other hand, for the nonmaximal Satake compactification X τ , only X P appears for µτ -connected parabolic subgroups. By attaching only X P for µτ S connected parabolic subgroups, we can construct X τ in the Borel–Serre way. See [BJ2] also for details.

Summary of relations between the compactifications of X . In Part I, we discussed five types of compactifications of X and their relations can be summarized in the following proposition. Proposition 6.9 1. When the rank of X is equal to 1, all the compactifications are isomorphic. S 2. The Satake compactifications X are isomorphic to the Furstenberg compactifiF cations X . 3. When the rank of X is greater than or equal to 2, the only common quotient of the geodesic compactification X ∪ X (∞) and the Satake–Furstenberg compactSF ifications X is the one point compactification. K 4. The Karpelevic compactification X dominates the Martin compactification X ∪ ∂λ X , the geodesic X ∪ X (∞), and the Satake–Furstenberg compactification SF X . 5. If λ = λ0 (X ), the Martin compactification X ∪∂λ X is isomorphic to the maximal SF Satake–Furstenberg compactification X max ; if λ < λ0 (X ), the Martin compactSF ification X ∪ ∂λ X is isomorphic to X ∪ X (∞) ∨ X max , the least common refinement of the geodesic compactification and the maximal Satake–Furstenberg compactification.

II. Compactifications of Locally Symmetric Spaces In this part we study compactifications of locally symmetric spaces \X . As in Part I, we limit ourselves to compact spaces in which \X is open and dense. The group G is assumed to be the group G(R) of real points of a connected affine algebraic groups G defined over Q, and K , X are as before. Briefly, the algebraic group G ⊂ G L(n, C) is a subvariety defined by polynomial equations with Q-coefficients such that the group operations are given by morphisms; the real locus G(R) consists of elements with real entries, and the rational locus G(Q) consists of elements with rational entries. Moreover, ⊂ G(Q) is an arithmetic subgroup (see below).


99

7 Geometry of locally symmetric spaces In this section we discuss the reduction theory of arithmetic groups and the geometry of locally symmetric spaces of finite volume. The most basic example is given by G = SL(2, R), K = SO2 , = SL(2, Z). Then X = SL(2, R)/SO2 can be identified with the upper half plane, and the quotient \X is the moduli space of elliptic curves as mentioned in Section 2. Another example is given by G = SL(n, R), K = SOn , and = SL(n, Z). In these two examples, is the set of integral points of linear algebraic groups. In fact, this procedure works in the general case: G can always be identified to a subgroup of some G L(V ), where V is a vector space over Q. Any basis of V over Q spans a lattice in V . Denote the stabilizer of in G L(V ) by G L (V ). A discrete subgroup of G(Q) is called an arithmetic subgroup if it is commensurable to G(Q) ∩ G L (V ), i.e., (G(Q) ∩ G L (V )) is of finite index in both and G(Q) ∩ G L (V ). It turns out that the class of arithmetic subgroups of G does not depend on the choices of the embedding into G L(V ) or of the rational basis of V . An important corollary of the reduction theory says that arithmetic subgroups are lattice subgroups. In the following we will be mainly concerned with arithmetic subgroups. A basic problem in reduction theory is to find a good fundamental domain ⊂ X for the -action on X . Definition 7.1 An open set in X is called a fundamental domain for if 1. is mapped injectively into \X . 2. is mapped onto \X , and the map is finite-to-one. Definition 7.2 A set of X is called a fundamental set for if 1. the interior of is dense in , 2. is mapped surjectively onto \X , and the map is finite-to-one. In the definition of fundamental sets, it is not required that or its interior be mapped injectively into \X . In the example of = S L(2, Z), X = H2 , the upper half-plane, a fundamental √ 3 1 1 2 set is given by = {x + i y ∈ H | x ∈ (− 2 , 2 ), y > 2 }, but a fundamental domain is given by the subset \ {x + i y ∈ H2 | x 2 + y 2 ≤ 1}. The map → S L(2, Z)\H2 is not injective, and the fibers of the map → S L(2, Z)\H2 consist of at most two points. set associated with the parabolic subgroup P∞ = The set here is a Siegel a b | a = 0, b ∈ R . 0 a −1 In the general case, Siegel sets are defined as follows. For every parabolic subgroup P defined over Q, called a rational parabolic subgroup, its real locus P = P(R) has a rational Langlands decomposition, which generalizes the decom a b position P = N P A P M P , where P = { | a ∈ R, a = 0, b ∈ R}, 0 a −1

100

A. Borel and L. Ji

a 1b 1 | a > 0}, M | b ∈ R}, A P = { = {± }. This P 01 01 0 a −1 rational Langlands decomposition is different from the real Langlands decomposition defined earlier. Let NP be the unipotent radical of P. Then LP = NP \P is the Levi quotient, a linear reductive algebraic group defined over Q. Let L P = LP (R) be the real locus of LP , AP the identity component of the real locus of the maximal Qsplit torus in LP , where a torus is Q-split if it is isomorphic to a product of C× over Q, and MP is the complement of AP such that L P = AP MP ∼ = AP × MP . Associated with the basepoint x0 = K ∈ X = G/K , there is a canonical lift i 0 : LP → P such that the image i 0 (LP ) is stable under the Cartan involution defined by K . It should be pointed out that i 0 (x)(LP ) is not necessarily defined over Q. Identify AP , MP with their lifts. Then P admits the rational Langlands decomposition NP = {

P = N P A P MP . To compare with the real Langlands decomposition P = N P A P M P , we note that AP ⊂ A P and that the quotient AP ⊆ A P is basically the part of A P which does not split over Q. Let K P = K ∩ MP , and X P = MP /K P . Recall that X P = M P /K P is a symmetric space of noncompact type. Then X P is a product of X P and of a Euclidean space, which is isomorphic to A P /AP . This rational Langlands decomposition of P also induces a horospherical decomposition of X X = N P × AP × X P . For bounded sets U ⊂ N P , V ⊂ X P , t > 0, the subset U × AP,t × V is called a Siegel set in X associated with the rational parabolic subgroup P. The classical reduction theory due to Borel and Harish-Chandra (see [Bo1]) can be stated as follows. Proposition 7.3 For any arithmetic subgroup , there are only finitely many conjugacy classes of rational parabolic subgroups of G. Let P1 , . . . , Pk be a set of representatives of these conjugacy classes. For each P j , there is a Siegel set U j × AP j ,t j × V j such that = ∪kj=1 U j × A P j ,t j × V j is a fundamental set of . The finiteness of the map → \X follows from the following Siegel property. Proposition 7.4 For any two rational parabolic subgroups P1 , P2 and associated Siegel sets U1 × AP1 ,t1 × V1 , U2 × AP2 ,t2 × V2 , the set {γ ∈ | γ (U1 × AP1 ,t1 × V1 ) ∩ U2 × AP2 ,t2 × V2 = ∅} is finite. To compare with earlier results for real parabolic subgroups in Proposition 5.1, we also state the separation property for Siegel sets. Proposition 7.5 Let P1 , P2 be two rational parabolic subgroups that are not conjugate under . When t1 , t2 0, for all γ ∈ , γ (U1 × AP1 ,t1 × V1 ) ∩ U2 × AP2 ,t2 × V2 = ∅.


101

To get a fundamental domain for and to better understand the geometry at infinity of \X , we need the precise reduction theory. To motivate this theory, we start with a noncompact Riemann surface \H2 with finitely many cusps. Its geometry can be described as follows. 1. The surface \H2 admits a disjoint decomposition into a compact region and cusp neighborhoods. 2. There is a one-to-one correspondence between the -conjugacy classes of rational parabolic subgroups and the cusps. For each cusp, its cusp neighborhood is the bijective image of a Siegel set. The generalization of this statement to \X is called precise reduction theory. For this purpose, we need to modify slightly the definition of Siegel sets. For any rational parabolic subgroup P, and T ∈ AP , define AP,T = {a ∈ AP | a α > T α , α ∈ (P, AP )}, where (P, AP ) is the set of rational roots of AP acting on N P . Equivalently, AP,T is the shifted positive Weyl chamber with vertex at T . Proposition 7.6 For any arithmetic subgroup , let P1 , . . . , Pk be a set of representatives of the -conjugacy classes of proper rational parabolic subgroups. For each P j , there is a Siegel set U j × AP,T j × V j such that U j × AP,T j × V j is mapped injectively into \X . Identify these Siegel sets with their images in \X . Then there is a compact subset 0 in \X such that \X admits the following disjoint decomposition: k \X = 0 ∪ U j × AP,T j × V j . j=1

Unlike the case of classical reduction theory in the previous proposition, the choices of T j , U j , V j and 0 are interrelated, and the T j have to be chosen to be sufficiently large, i.e., far into the positive chamber. For details see the references in [J3]. One application of the precise reduction theory describes the rough geometry of \X . Since U j , V j are bounded, the decomposition \X = 0 ∪

k

U j × AP j ,T j × V j

j=1

shows that \X is within bounded distance of the skeleton x0 ∪

k

AP j ,T j ,

j=1

where x0 is a basepoint in 0 . It turns out that the simplicial cones AP j ,T j can be put together into a metric cone over a finite simplicial complex. For example, in the case of a Riemann surface \H2 , this skeleton is a union of half-rays, one for each cusp.

102

A. Borel and L. Ji

In general, the rational Tits building Q (G) consists of simplices associated to rational parabolic subgroups, i.e., there is one simplex for each rational parabolic subgroup such that the maximal proper rational parabolic subgroups correspond to vertices, and the correspondence is inclusion-reversing. The Tits building Q (G) admits a metric such that acts isometrically on Q (G), the quotient \Q (G) is a finite simplicial complex, and every simplex is metrically a subset of a unit sphere. Lemma 7.7 With respect to the invariant metric on X , the set x 0 ∪ kj=1 AP j ,T j can be glued into a metric cone over \Q (G), which is denoted by C(\Q (G)). Proposition 7.8 The Hausdorff distance between \X and C(\Q (G)) is finite. One application of this result is a solution of a conjecture of Siegel’s. Let d X , d\X be the distance functions on X and \X , respectively. Denote the projection X → \X by π. Then the Siegel conjecture can be stated as follows. Corollary 7.9 For any rational parabolic subgroup P and a Siegel set U × AP,t × V with respect to P, there exists a constant c such that for any p, q ∈ U × AP,t × V , d X ( p, q) − c ≤ d\X (π( p), π(q)) ≤ d X ( p, q) + c. The idea of the proof is to replace points p, q by points on the skeleton C(\Q (G)). For details regarding the topics discussed here, see [J3].

8 Compactifications of locally symmetric spaces In this section we consider three types of compactifications of locally symmetric spaces \X : an analogue of the geodesic compactification, the Borel–Serre compactification, and an analogue of the Satake–Furstenberg compactification via embedding into compact ambient spaces. We first consider the analogue of the geodesic compactification for \X . Unlike the case of X , there are many unbounded geodesics which do not go to infinity. The correct class of geodesics consists of those that are eventually distance minimizing. Let γ (t) be a unit speed, directed geodesic in \X . It is called eventually distance minimizing (EDM) if d(γ (t1 ), γ (t2 )) = |t1 − t2 |, for t1 , t2 0. For such a geodesic γ (t), when t → +∞, γ (t) leaves every compact subset eventually. On the other hand, the converse is not true. It should also be pointed out that for an EDM geodesic γ (t), when t → −∞, γ (t) does not necessarily leave every compact subset. As in the case of X , we can also define an equivalence relation on the set of EDM geodesics: γ1 and γ2 are equivalent if lim supt→+∞ d(γ1 (t), γ2 (t)) < +∞. Let \X (∞) be the set of equivalence classes of EDM geodesics in \X . In [JM],


103

a compact Hausdorff topology is defined on \X ∪ \X (∞). This compactification is called the geodesic compactification of \X . This geodesic compactification \X ∪ \X (∞) has connections to the Martin compactification of \X and parametrization of the continuous spectrum. More refined relations between a subclass of EDM geodesics and the spectral measure of the continuous spectrum have been obtained in [JZ]. More precisely, a scattering geodesic in \X is a geodesic which is EDM in both directions, i.e., as t → ±∞. Though a scattering geodesic has infinite length, it has a normalized length, called the sojourn time, which measures basically the time it spends around the compact core of \X . The set of sojourn times of scattering geodesics forms a discrete subset of R. The generalized eigenfunctions of the continuous spectrum of \X are given by the Eisenstein series. The constant terms of the Eisenstein series can be described by the scattering matrices, and the spectral measure of the continuous spectrum can also be expressed in terms of the scattering matrices. For a locally symmetric space \X of Q-rank 1, i.e., for all proper rational parabolic subgroup P, dim AP = 1, it was shown in [JZ] that the set of frequencies of oscillation of the scattering matrices is contained in the set of sojourn times. This established a close relation between the continuous spectrum and the geometry at infinity. When the Q-rank of \X is greater than 1, scattering geodesics alone are not sufficient, and scattering flats also have to be considered (see [J5]). As mentioned earlier, a general approach to compactifications of X was motivated by the Borel–Serre compactification of \X in [BS]. For \X , the Borel–Serre approach modified in [BJ1] consists of the following steps: 1. For each rational parabolic subgroup P, choose a boundary face e(P). 2. Attach the boundary face e(P) at infinity of X using the horospherical decomposition of X = N P × AP × X P . 3. Form the completion (or partial compactification) X ∪ P e(P) and show that acts continuously on it. 4. Show that the quotient \X ∪ P e(P) is compact, Hausdorff. Besides giving the Borel–Serre compactification in [BS], this method also conBS structs other compactifications. For the Borel–Serre compactification \X , the boundary face e(P) is given by e(P) = N P × X P . BS = X ∪ P e(P) can be described in terms of convergent The topology of X sequences. An unbounded sequence y j in X converges to a point (n ∞ , z ∞ ) ∈ N P × X P = e(P) if and only if in the decomposition y j = (n j , a j , z j ) ∈ N P × AP × X P , the coordinates satisfy the following conditions: 1. n j → n ∞ in N P , 2. a j → ∞ through the positive chamber A+ P and away from the walls, 3. z j → z ∞ in X P .

104

A. Borel and L. Ji

When X = H2 , there is a one-to-one correspondence between the set of rational BS parabolic subgroups and the subset Q ∪ {∞} ⊂ X (∞), and X is obtained by adding one line R to every point in Q ∪ {∞}. For example, for the point i∞, the boundary face is R, and a sequence z j = x j + i y j converges to a point x∞ ∈ R if and only if x j → x∞ and y j → +∞. BS

Proposition 8.1 The G(Q)-action on X extends to a continuous action on X , in BS BS particular, acts continuously on X . Furthermore, acts properly on X with a compact quotient. The continuity of the G(Q)-action follows from the definition. In fact, it can BS be checked easily that if y j is a sequence in X converging to a point e(P) in X , −1 g ∈ G(Q), then gy j converges to a point in e(gPg ). The properness of the action follows from the finiteness property of Siegel sets, and the compactness of the BS quotient from the fact that the closure of a Siegel set in X is compact. BS The quotient \X is the Borel–Serre compactification of \X . We now study BS some properties of \X . For each rational parabolic subgroup P, let (P, AP ) be the set of roots of AP acting on N P , and α1 , . . . , αr the set of simple roots. Then AP ∼ = Rr>0 , Define

a → (a −α1 , . . . , a −αr ). AP ∼ = Rr≥0 .

Then AP is a manifold with one corner. In fact, the corner point corresponds to the infinity of the positive chamber A+ P. Proposition 8.2 For every rational parabolic subgroup P, the inclusion N P × AP × BS BS X P → X extends to an embedding N P × AP × X P → X . BS

The image of N P × AP × X P in X is called the corner associated with P and denoted by X (P). In terms of the boundary faces, X (P) = X ∪ e(Q). Q⊇P

Since AP is a real analytic manifold with corners, and X = N P × AP × X P is an analytic diffeomorphism, X (P) is a real analytic manifold with corners whose analytic structure is compatible with the interior analytic structure of X . Proposition 8.3 For a pair of rational parabolic subgroups P1 , P2 , if P1 ⊆ P2 , then X (P2 ) is naturally embedded in X (P1 ) as a real analytic submanifold with corners. Using the description of X (P j ) in terms of boundary faces, it is clear that X (P2 ) is naturally a subset of X (P1 ). The problem is to check the compatibility of their analytic structures. For proofs of these propositions, see [BJ1]. Corollary 8.4 For any two rational parabolic subgroups P, Q, the analytic structures of X (P) and X (Q) are compatible.


105

Proof. Let R be the smallest parabolic subgroup containing both P and Q. The subgroup R could be equal to G. In this case, set X (G) = X . Then it can be shown that X (P1 ) ∩ X (P2 ) = X (R), and the compatibility follows from the previous proposition. Proposition 8.5 X

BS

is a real analytic manifold with corners. BS

BS

Proof. The corners X (P) are open in X and cover X . Since their analytic strucBS tures are compatible, they define an analytic structure of X . BS

When is torsion free, the extended action of on X is not necessarily free. On the other hand, if is assumed to be neat (see [Bo1, §17] for definition), will BS act freely on X . It is proved in [Bo1, §17] that every arithmetic subgroup has a neat subgroup of finite index. Corollary 8.6 If is neat, \X

BS

is a real analytic manifold with corners.

BS

The compactification \X can be described more explicitly in terms of the boundary faces. For each rational parabolic subgroup P, let P = P(R), P = ∩ P, and N P = ∩ N P . Denote the image of P under the projection P = N P AP MP → MP by MP . When is neat, MP is torsion free. It can be shown that these induced subgroups fit into an exact sequence: 0 → N P → P → MP → 0. Hence the quotient P \e(P) is a fiber bundle over MP \X P with fiber N P \N P . Let P1 , . . . , Pk be a set of representatives of -conjugacy classes of proper rational parabolic subgroups. Then \X

BS

= \X ∪

k

P j \e(P j ).

j=1

In studying the L 2 -cohomology of \X , Zucker [Zu1] introduced a quotient of BS \X by collapsing the nilpotent fibers N P \N P in the boundary. This compactifiRBS cation is also called the reductive Borel–Serre compactification, denoted by \X and plays an important role in other contexts. Next we follow the Borel–Serre proRBS BS cedure and give a direct construction of \X independent of \X . For each rational parabolic subgroup P, define its boundary face to be e(P) ˆ = X P. RBS The topology on X = X ∪ P e(P) ˆ is given by: a unbounded sequence y j in X ˆ if and only if in the decomposition y j = (n j , a j , z j ), converges to a point z ∞ ∈ e(P) the coordinates satisfy 1. a j → ∞ through A+ P and away from the walls, ˆ = X P. 2. z j → z ∞ in e(P)

106

A. Borel and L. Ji

Unlike the case of X

BS

, there is no condition on the N P -factor. RBS

Proposition 8.7 The topology on X is Hausdorff, and the G(Q)-action on X RBS RBS extends continuously to X . The group acts continuously on X and the RBS is compact. quotient \X RBS

The group does not act properly on X . In fact, N P is infinite but acts as the identity on e(P). ˆ In terms of the parabolic subgroups P1 , . . . , Pk and their boundary faces, k RBS \X = \X ∪ MP j \X P j . j=1

Proposition 8.8 The identity map on \X extends to a continuous map \X RBS \X . The fiber in \X

BS

BS

→

over a boundary point in MP j \X P j is equal to N P j \N P j , RBS

and hence this compactification \X is the same as the one constructed by Zucker in [Zu1]. The Borel–Serre method can be applied to construct other compactifications such as the whole family of Satake compactifications of \X . For them, instead of the collection of all rational parabolic subgroups, we choose a subcollection. Remark. The geodesic compactification \X ∪ \X (∞) mentioned at the beginning of this section can also be constructed by this method and is called the Tits compactification in [JM]. Briefly, for any P, define its boundary component e(P) = a+ P (∞). The convergence of interior points is also given in terms of the Langlands decomposition. This method can also be applied to construct compactifications of \G. The space \G is more natural than \X . For example, \G is a homogeneous space, while \X is not; automorphic representations are defined on L 2 (\G). For some nonmaximal compact subgroup H , the quotient \G/H appears as the period domain in the theory of variation of Hodge structures, and compactifications of \G/H were sought after. If \G can be compactified so that the right K -action extends, the quotient by H gives a compactification of \G/H . We now construct the analogue of the Borel–Serre and reductive Borel–Serre compactifications of \G. For each rational parabolic subgroup P, the analogue of the horospherical decomposition for G is G = N P × AP × (MP K ). For G

BS

, the boundary face of a rational parabolic subgroup P is e(P) = N P × (MP K ),


and G

BS

=G∪

107

e(P).

P BS

The topology on G is defined by convergent sequences using the horospherical BS decomposition as in the case of X . Proposition 8.9 The group acts properly on G pact Hausdorff.

BS

and the quotient \G

BS

is com-

Proposition 8.10 The right K -action of \G extends to a continuous action on BS \G , but the right G-action does not extend. The statement that the right K -action does not extend to G can be seen from the example of SL(2, Z)\SL(2, R). For a rational parabolic subgroup P, a sequence y j = na j with a j → +∞ clearly converges to n ∈ e(P). For any n 0 ∈ N P , y j = −1 na j n 0 = n(a j n 0 a −1 j )a j . For suitable choices of a j and n 0 , the image of n(a j n 0 a j ) in N P \N P runs around the cusp and does not form a convergent sequence. Hence BS y j n 0 does not converge in \G . BS

Proposition 8.11 The quotient of \G by K on the right is equal to \X BS BS is neat, \G is a K -principal bundle over \X . For G

RBS

BS

. When

, the boundary face of a rational parabolic subgroup P is defined to be e(P) ˆ = MP K . RBS

is defined similarly in terms of convergent sequences except The topology on G that there is no condition on the N P -factor. RBS

is a Hausdorff space, and acts continuously with a comProposition 8.12 G RBS pact quotient \G . Let P1 , . . . , Pk be representatives of -conjugacy classes of rational parabolic subgroups as before. Then \G

RBS

= \G ∪

k

MP j \MP j K .

j=1

Proposition 8.13 The identity map on G extends to a continuous map G BS RBS and hence the compactification \G dominates \G . BS

The fibers of the map \G → \G MP j \MP j K are equal to N P j \N P j . An important difference between \G

BS

RBS

BS

→G

RBS

,

over the boundary component

and \G

RBS

is the following.

108

A. Borel and L. Ji

Proposition 8.14 The right G-action on \G extends to a continuous action, in particular the right K -action extends as well. The reason for the extension of the G-action is that the N P -component does not play any role in convergent sequences. See [BJ1] for details. Proposition 8.15 The projection map \G → \X extends to a continuous map RBS RBS RBS RBS \G → \X , and the quotient of \G by K is equal to \X . RBS

To explain the extension of the G-action onto \G , we construct another compactification of \G. The basic reference is [BJ1]. Suppose there is a compact G-space Z and a G-equivariant embedding i : \G → Z . Then the closure of i(\G) is a G-compactification. This procedure is similar to the one underlying the construction of the Satake and Furstenberg compactifications. Let S(G) be the space of closed subgroups of G. As explained in Section 4, it is a compact G-space. For any discrete subgroup , not necessarily arithmetic, there is a natural G-equivariant map i : \G → S(G), g → g −1 g. It is clear that when the normalizer N () of is equal to , the map i is injective. To get an embedding, we need more assumption. Proposition 8.16 When is an arithmetic subgroup and N () = , the map i is an embedding. The point is to show that a sequence g j in \G converges to g∞ if and only −1 g in S(G). The proof depends on reduction theory. if converges to g∞ ∞ See [BJ1] for details. Under the assumptions in the proposition, the closure i (\G) is a G-compactifisb cation, called the subgroup compactification, denoted by \G . Before proceeding, we give conditions under which N () = holds. g −1 j g j

Definition 8.17 A discrete subgroup of G is called maximal if it is not contained in any other discrete subgroup. Assume G is simple over Q. Then every arithmetic subgroup is contained in some maximal discrete subgroup as a subgroup of finite index. Lemma 8.18 If is arithmetic and maximal, then N () = , and hence the map i : \G → S(G) is an embedding. For G = SL(n, R), the subgroup = SL(n, Z) is a maximal arithmetic subgroup. Proposition 8.19 When is a maximal arithmetic subgroup, the identity map RBS sb on \G extends to a continuous map \G → \G . If for every rational


109

parabolic subgroup P, the induced subgroup M P is semisimple and has no simple RBS sb compact factor, and then the map \G → \G is finite to one. RBS

sb

Under some further conditions, \G is isomorphic to \G . For example, for G = S L(n, R) and = S L(n, Z), these conditions are satisfied and hence RBS ∼ sb \G = \G . See [BJ1] for more details. The basic idea of the proof of this proposition is to consider sequences g j ∈ G going to infinity through a Siegel set of a rational parabolic subgroup P. For simplicity, we consider the case g j = a j ∈ AP , a j → +∞ through A+ P and away from the walls. Write = γ P = P ∪ γ P . γ ∈ P \

γ ∈ P \,γ ∈ P

Lemma 8.20 As j → ∞, a −1 j P a j converges to N P MP in S(G). Since a j shrinks the N P -factor and commutes with MP and P is an extension of MP by N P , this lemma is intuitively clear. Lemma 8.21 As j → ∞, a −1 j ( γ ∈ P \,γ ∈ P γ P )a j diverges to infinity, i.e., has eventually empty intersection with every compact subset of G. The proof of this lemma depends crucially on the reduction theory for and is related to the proof of absolute convergence of Eisenstein series for large parameters. See [BJ1] for details. These two lemmas imply the following result. Proposition 8.22 As j → ∞, a −1 j a j converges to N P MP in S(G). For more general sequences g j = n j a j m j with n j ∈ N P bounded, a j ∈ AP going to infinity as above, and m j ∈ MP converging to a point m ∞ , the sequence −1 g −1 j g j converges to m ∞ N P MP m ∞ . This shows that a sequence g j ∈ \G which RBS

sb

converges in \G also converges in \G , and hence the identity map on \G RBS sb extends to a continuous map \G → \G .

9 Satake compactifications of \X The Satake compactifications of X were introduced by Satake as a tool to define compactifications of \X using a procedure generalizing the well-known method for Fuchsian groups, which goes back to the 19th century. We first review the latter briefly. 9.1. We assume that G = SL2 (R), K = SO2 and X is the upper half-plane, (or the open unit disc D) and is a subgroup of finite index of SL2 (Z). The construction of \X is divided into 5 steps. a) Consider X¯ as eitherX ∪ R ∪ ∞ or D¯ = {z ∈ C, |z| ≤ 1}.

110

A. Borel and L. Ji

b) For z ∈ ∂ X , let G z be its isotropy group in G. It is conjugate to the group of real unimodular upper triangular matrices. Its unipotent radical Nz is conjugate to the group of translations in the x-direction. The point z ∈ ∂ X¯ is -cuspidal or cuspidal if it satisfies the following equivalent conditions: ∩ G z is infinite ⇔ ∩ Nz is infinite ⇔ G z is defined over Q ⇔ z ∈ Q ∪ (∞). c) Form X ∗ = X ∪ {cuspidal points} and endow it with a suitable topology. The closure in X ∗ of a (well-chosen) fundamental set for is the union of and of finitely many cuspidal points. d) operates continuously on X ∗ and \X ∗ is compact Hausdorff, the union of \X and of finitely many cuspidal points. e) \X ∗ admits the structure of a projective curve. An embedding into some complex projective space is defined by means of holomorphic automorphic forms of some sufficiently high weight. [This construction can be carried out for any Fuchsian group of the first kind, (a discrete subgroup of SL2 (R) such that \G has finite volume, but is not compact) cf. e.g., [Bo2].] 9.2. In the general case, we start from a Satake compactification X¯ µ , where µ is a dominant weight. We use the notation of 4.7, except that we write µ for µτ and X¯ µ for X¯ τS . We have X¯ µ = X ∪ XP, (1) Pµ-connected

and the stabilizer N (X P ) of X P in G is the µ-saturation of P. In what follows we need to be more explicit. Assume P to be standard, hence P = PJ where J ⊂ is µ-connected. Let J = {β ∈ | β ⊥ J ∪ {µ}}

J˜ = J ∪ J .

(2)

The µ-saturation of PJ is PJ˜ . The group PJ˜ has the usual Langlands decomposition PJ˜ = M J˜ .A J˜ .N J˜ .

(3)

We can write M J˜ as an almost direct product: M J˜ = G(X J ).G(X J ).M(J ) .

(4)

Here X J = X PJ (resp, X J = X PJ ). G(X J ) (resp. G(X J ) ) is the connected semisimple group with Lie algebra spanned by the gβ , β ∈ [J ] (resp. β ∈ [J ]). Its symmetric space is X J (resp. X J ). Finally M(J ) is the smallest closed subgroup of Mφ such that M J˜ is generated by G(X J ).G(X J ) and M(J ) . It is compact. In analogy with 9.1(b), we have to define among the X P ’s the analogues of the -cuspidal points, which will be called the “rational boundary components”. The goal is then to prove that / X may be compactified by finitely many quotients of rational boundary components by groups of automorphisms of arithmetic type. In analogy with (b), a first natural requirement for a boundary component X P is (R1) ∩ Ru N (X P ) is cocompact in Ru N (X P ) ⇔ N (X P ) is defined over Q.


111

Let (X P ) be the group of automorphisms of X P induced by ∩ N (X P ). It is isomorphic to ∩ N (X P )/ ∩ Z(X P ). A second condition for X P to be rational is that (X P ) act properly on X P , and be in fact (for induction purposes) a group of arithmetic type. This is surely satisfied if Z(X P ) is defined over Q, but a slightly weaker condition also suffices, namely: (R2) Z(X P ) contains a cocompact closed subgroup Z normal in N (X P ) defined over Q. Indeed, assume (R1) and (R2) and moreover that P = PJ is standard. In the notation of (2), (3) the group Z contains G(X J ).A J˜ .N J˜ ; it is generated by that group and, possibly, a factor of M(J ) , so that PJ /Z is (up to a finite intersection) the product of G(X P ) and of a compact group (a quotient of M(J ) ). The group PJ˜ /Z is defined over Q and ¯ = ∩ PJ˜ / ∩ Z is an arithmetic subgroup. The group (X P ) is the image of ¯ in G(X P ) by a projection with compact kernel, hence it is of arithmetic type. 9.3. The strategy is to carry out the analogues of 9.1(c), (d), using the rational boundary components instead of cuspidal points. We now make a number of comments on the notion of rational boundary components and on the cases in which this program has been completed. In all these, (R1) implies (R2). We do not know whether this is always the case. (a) In [Sa 2], I. Satake handles a number of classical groups, using the reduction theory available to him. (b) A rational representation of G is strongly rational if it is defined over Q and if the highest weight line is defined over Q. This notion is assumed in [Bo0] but the terminology was introduced later in [BT]. (See 1.6 in [Bo0]. It is understood in that paper that rational representations always satisfy 1.6, which is equivalent to the later strongly rational.) If so, it is shown in [Bo0] that (R1) implies that Z(X P ) is defined over Q, in particular that (R2) is fulfilled. In that case (c) and (d) are valid. The analogue of X ∗ is the union X µ∗ of X and the rational boundary components and \X is the quotient \X µ∗ . (c) Note that if G(X J ) is reduced to the identity, then (R2) automatically follows from (R1), with Z equal to the radical of N (X P ). This is in particular the case if µ is a regular highest weight, or, equivalently, if X¯ µ is a maximal Satake compactification. It is not known whether (R1) implies (R2) when µ is the highest weight of a representation defined over Q. This is claimed in [Zu 2], but the proof is valid only for strongly rational representations. W. Casselman in [C ] also seems to take for granted that it suffices for the ambient representation to be defined over Q, but in fact, strongly rational over Q was meant (oral communication). 9.4. As was pointed out, the previous construction works if X¯ is a maximal Satake compactification. In that case, the rational boundary components are those with normalizers defined over Q. They are consequently parametrized by the proper parabolic Q-subgroups, as are those of X¯ B S or X¯ R B S . In fact, the discussion at the end of 7.2 ∗ implies easily the existence of a morphism of \ X¯ R B S → \ X¯ max with fibres ordinary (topological) tori, i.e., products of circles. Indeed the quotient A P /AP is in L P

112

A. Borel and L. Ji

the identity component of an algebraic torus which is anisotropic over Q, so that any arithmetic subgroup is cocompact and the quotient is a topological torus. The fibres of ν are such tori. If the Q-rank and R-rank of G are the same, then A P = AP and ν is an isomorphism. 9.5. Assume now that X is a bounded symmetric domain. It is then a direct product of irreducible bounded symmetric domains, and each of those is a quotient G/K where G is simple and K has a 1-dimensional center. X has a natural bounded realization [HC]. The closure of this bounded realization is a compactification of X , which can be proved to be isomorphic to a Satake compactification. All the boundary components are themselves bounded symmetric domains [BB]. [If X is irreducible, the Dynkin diagram of the restricted root system is either of type Cn or BCn . In either case, the highest weight defining the representation is orthogonal to all simple roots except for the endpoint which is on the double bond.] It is shown that (R1) implies (R2) and the Satake compactification V can be defined. The main point is then to prove that V admits a canonical structure as a projective variety, which induces on each boundary component its natural complex structure. This embedding is realized by means of a certain type of holomorphic automorphic forms called the Poincar´ e-Eisenstein series in [BB]. This was first carried out in the symplectic case see §2, (4), (5) for a subgroup of finite index of Siegel’s modular group Sp2n (Z) by I. Satake for the topological part and W. Baily for the projective embedding (see [BB] for references). This compactification has usually bad singularities. The book [AMRT] constructs an infinity of compactifications, all lying over V , some of which are projective, smooth, or both, called toroidal compactifications. Summary of relations between the compactifications of \X . Relations between the compactifications of \X and \G introduced in Part II can be summarized in the following proposition. See [AMRT] for detailed discustor sions of toroidal compactifications \X , and [J2] for a summary and the proof of (4) in Proposition 9.4. Proposition 9.6 1. The greatest common quotient of the geodesic compactification \X ∪ \X (∞) BS and of the Borel–Serre compactification \X is the end compactification when the Q-rank of G is equal to 1, i.e., adding one point to each end of \X , and to the one point compactification when the Q-rank of G is greater than 1. BS 2. The Borel–Serre compactification \X dominates the reductive Borel–Serre RBS compactification \X . RBS 3. The reductive Borel–Serre compactification \X dominates all Satake comS pactifications \X . BB 4. For Hermitian \X , the Baily–Borel compactification \X is a common quoBS tient of the Borel–Serre compactification \X and the toroidal compactificator tions \X but not necessarily their greatest common quotient. On the other


113

hand, the Baily–Borel compactification is the greatest common quotient of the reductive Borel–Serre and the toroidal compactifications. BS RBS 5. The compactification \G dominates \G , and their quotients by K are BS RBS isomorphic to \X and \X respectively. RBS dominates the subgroup 6. The reductive Borel–Serre compactification \G sb compactification \G , and they are isomorphic under certain conditions.

III. Smooth Compactifications in which X or Γ \X is not dense Given the symmetric space G/K , of dimension n, rank , Oshima constructed a compact real analytic G-space containing 2 open subsets isomorphic to G/K (as G spaces) such that the complement of their union consists of smooth hypersurfaces of codimension 1 with transversal intersections. His construction will be outlined in 10.4. We first describe a different procedure which exploits the structure of analytic G-manifold with corners of the maximal Satake compactification.

10 Compactifications in which all open orbits are isomorphic 10.1. We assume the notion of a manifold with corners to be known (cf. [BS][Ce]). We recall some facts and notation following [BJ1]. We first have to discuss a corner in Rn . We do this with notation adapted to the applications we have in mind. Let be a finite set (to be a system of simple roots) and R be Euclidean space with coordinates α ∈ . The open (resp. closed) positive quadrant in R is α α R >0 = {t ∈ R , t > 0, (α ∈ )}(resp. R≥0 = {t ∈ R |t ≥ 0, (α ∈ )}).

If J ⊂ , we let

Then

J R≥0 = {t ∈ R≥0 , t α = 0 for α ∈ J }

(2)

J J R>0 = {t ∈ R≥0 , t α = 0 for α ∈ J }.

(3)

R ≥0 =

J ⊂

J R>0 =

J ∈

J R≥0 .

(4)

A manifold M with corners of dimension n, rank r k(M) = , is a stratified space, the strata of which are connected manifolds of codimension j (0 ≤ j ≤ ). The closures of the strata are the boundary faces. If x lies on a stratum of codimension m . For m, it has a fundamental system of neighborhoods of the form Rn−m × R≥0

114

A. Borel and L. Ji

J then belongs to a stratum of codimension J ⊂ {1, . . . , m}, the subspace Rn−m ×R>0 m − |J |. The boundary ∂ M of M is the union of the strata of codimension ≥ 1 and the following conditions are assumed. ∂ M is the union of the boundary hypersurfaces, which are all manifolds with corners of codimension 1 and rank equal to r k(M) − 1. The boundary faces are embedded (no self-intersection) manifolds with corners. A boundary face of codimension j is an embedded manifold with corners of rank − j, and is the transversal intersection of j boundary hypersurfaces. In most cases, M will be compact. If not, it is assumed that the set H of boundary hypersurfaces is locally finite and can be written as a finite disjoint union of N subsets H j where H j consists of disjoint boundary hypersurfaces. 10.2. R. Melrose has shown (in the compact case, but the proof is the same under our assumption) that one can glue 2 N copies of M together to get a smooth manifold M˜ (cf. [BJ 1], §7). The only cases of interest here are some in which

N = r k(M).

(5)

In this case we give a somewhat more direct approach, although obviously equivalent to Melrose’s. First we need to define a partition of Rm into 3m subsets, the open quadrants in coordinate planes. We follow the conventions of 10.1, [Os1] and [OSS]. The natural set of indices for the applications to follow is a set of simple roots; so we shall label the coordinates by elements of and speak of a partition of R . Definition 1. 1. The signature of t ∈ R, denoted by sgn t, is 0 if t = 0 and t/|t| otherwise. 2. A signature ε on a finite set is a map ε : → {1, 0, −1}. Its support s(ε) is the set s(ε) = {a ∈ | ε(a) = 0}. 3. A signature is called proper if s(ε) = . We let E() be the set of all signatures on and E o () the subset of proper signatures. They have cardinalities 3|| and 2|| respectively, where || is the cardinality of . If J ⊂ , an element of E(J ) is identified with the signature of which is equal to ε on J , and is zero outside J . If I ⊂ J , and ε ∈ E(I ), ε ∈ E(J ), we write ε ⊂ ε if ε and ε coincide on I . For ε ∈ E(), let R, ε = {t ∈ R | sgn t a = ε(a)}.

(6)

It can be identified to R J, ε where J = s(ε) and ε is the restriction of ε to J . We have R = R, ε . (7) ε∈E ()

This can also be written as


R =

J ⊂,

R J, ε .

115

(8)

ε∈E o (J )

Note that J can be empty, in which case R J,ε is the origin. The closed hyperquadrants in R are exactly the subspaces R (δ) defined by R (δ) = R, ε , (9) ε⊂δ

where δ ∈ E 0 (), and R may be viewed as the space obtained by gluing them along their intersections. They are all isomorphic and so R is the manifold obtained by gluing 2|| copies of R (δ). Let M be a manifold with corners, ρ its rank, and a set of cardinality ρ. We assume that H M has a partition HM = Ha , (10) a∈

where the elements of Ha are disjoint boundary hypersurfaces of rank ρ − 1. For J ⊂ , and |J | boundary hypersurfaces Ha , with Ha ∈ Ha , a ∈ J , the intersection Z of these Ha is either empty or a manifold with corners of codimension and rank equal to ρ − |J |. Let J ⊂ and Z be similarly constructed. If J ∩ J = ∅, then Z ∩ Z is a manifold with corners of rank and codimension ρ − (|J | + |J | ) or is empty. Assume now that I = J ∩ J is not empty. If for some a ∈ I , the hypersurfaces Ha and Ha are distinct, then they are disjoint by the definition of Ha , hence Z ∩ Z is empty. If Ha = Ha for all a ∈ I , then we are back to the previous case with J replaced by J = J − I . To be consistent with the notation of the above definition, we will change slightly the one just used. For J ⊂ , let us denote by H J the set of nonempty intersections of elements Ha , where a runs through − J . Thus our previous Ha becomes H−{a} . Given a manifold with corners N , we let N o be its interior. Let HoJ be the set of interiors of the elements of H J . Then M= HoJ . (11) J ⊂

Here it is understood that if J = , then H J = M and HoJ = {M o }. If J = ∅ and Z ∈ H J , then Z = Z o is a closed manifold. The elements of the HoJ are the strata of a stratification of M in which the closed o subspace of codimension i (0 ≤ i ≤ ρ) is the union of the H J where J runs through the subsets of of cardinality ≤ ρ − i, or, simply |J |=||−i H J . We now consider objects (Z , ε), where Z ∈ HoJ and ε is a signature on with support equal to J , or, equivalently, a proper signature on J . Let = M (Z , ε). (12) J ⊂

Z ∈HoJ , ε∈E o (J )

116

A. Borel and L. Ji

with respect to which M is a Proposition There exists a suitable topology on M smooth manifold. of M by Proof. Fix δ ∈ E o (). Define a subset M(δ) M(δ) = (Z , ε).

(14)

ε⊂δ

For each Z , there is only one possibility for ε; hence we see from (11) that M(δ) is set-theoretically a copy of M. We endow it with the topology of M. We have = M M(δ). (15) δ∈E o ()

∩ M(δ ). Let Let δ ∈ E o (). We want to describe M(δ) J (δ, δ ) = {a ∈ | δ(a) = δ (a)}, and let ε(δ, δ ) be the common restriction of δ and δ to J (δ, δ ). Then ∩ M(δ ) = M(δ) (Z , ε). J ⊂J (δ, δ )

(16)

(17)

Z ∈H0J , ε⊂ε(δ, δ )

) induce the same topology on the intersection The topologies of M(δ) and M(δ with the sum topology of the topologies on the M(δ) ∩ M(δ ). We then endow M M(δ). is a smooth manifold without corners. For any point o We next indicate why M in a corner of M(δ) of codimension ||, a neighborhood of o in M(δ) is the same n−|| || as a neighborhood of the origin in R × R (δ), where n = dim M. From the fact that R|| is the manifold obtained by gluing the closed quadrants R|| (see the paragraph after (9)), we conclude that these identical neighborhoods of o in M(δ) glue into a smooth neighborhood of o in M. Similarly, for any point o in a boundary face (Z , ε) of M(δ), the neighborhoods of o in M(δ) for all the δ equal to ε on the support s(ε) glue into a smooth neighborhood. Since any two M(δ) only intersect on their common boundary faces, it can be is the manifold constructed in Proposition. seen by induction on the rank that M || corresponds to changing the proper signaFurthermore, the (Z/2Z) action on M tures δ. 10.3. In order to apply this to the maximal Satake compactification, we should first exhibit its structure of a manifold with corners. We again go back to Section 4. We let X¯ m be the maximal Satake compactification of X = G/K . Let P = M AN be the minimal standard parabolic subgroup, let P¯ be the opposite standard minimal ¯ Then parabolic subgroup and N − the unipotent radical of P. X = N − .A and N − . A¯ is an open chart in X¯ m . with the identification Here A¯ = R≥0

(1)


a → (a −α1 , . . . , α −α ),

( = {α1 , . . . , α })

117

(2)

¯ of A with R >0 . The orbits of G in X m are the subspaces O J = G/K J .A J .N J ,

(3)

where O J fibers over G/PJ , the fibres being the boundary components conjugate to X J . The subgroup K J A J N J is denoted by Q J in the following. In the terminology of 10.2, the O J are the open boundary faces and their closures are the boundary faces. O¯ J has codimension r k(X ) − |J |. The boundary hypersurfaces are the sets O¯ (α) = O¯ −{α}

(α ∈ ).

(4)

These boundary hypersurfaces are embedded and the condition 10.2 (10) is trivially fulfilled by taking Hα = O¯ (α) (α ∈ ). The space M˜ is a G-space containing 2 copies of X and 3 orbits of G. We shall see later (12.5), by means of the wonderful compactification, that X¯ m is an analytic G-space. It follows then from general ˜ considerations about gluing that this is also the case for M. ˜ Given t ∈ R , we let εt be the 10.4. We briefly give here Oshima’s definition of M. α signature εt (α) = sgn t (α ∈ ), and s(t) will stand for s(εt ): s(t) = s(εt ) = {α ∈ | t α = 0}. Let H α (α ∈ ) be the basis of a dual to . We define a map a : R → A by the rule n|t α |H α . (1) a(t) = exp − α∈s(t)

Oshima defines a quotient

X˜ = G × R / ∼

of G × R by the equivalence relation ∼, where (g, t) ∼ (g, t ) if (i) εt = εt (ii) g.a(t).Q s(t) = g .a(t ).Q s(t ) , where Q s(t) = K s(t) As(t) Ns(t) as above. (Note that s(t) = s(t ) in view of (i).) The G-action is defined by left translations on the first factor. Oshima shows that X˜ is a compact G-space, into which N − × R maps bijectively on an open chart. X˜ consists of 3 orbits, 2 being copies of X . The hard point, however, is to prove that X˜ is an analytic G-space. One has to show that the infinitesimal actions of g on the orbits match to define analytic vector fields on X˜ , and then use one of the fundamental theorems of the original Lie theory. We refer to [Os1] for details. 10.5. Now let G be defined over Q (i.e. let it be the group of real points of a semisimple connected Q-group G) and let be a neat arithmetic subgroup of G(Q). The construction of 10.3 also applies to X¯ B S and \ X¯ B S . To see this, we use the notation of Section 8. We have

118

A. Borel and L. Ji

X¯ =

e(P),

(1)

P

where P runs through the parabolic Q-subgroups of G. The boundary hypersurfaces are the e(P) where P is maximal proper. For α ∈ , let H(α) = e(P). (2) P conjugate to P−{α}

Then H = α H(α) . Moreover, the elements of H(α) are disjoint. In fact, e(P)∩e(Q) is empty if P∩Q is not a parabolic subgroup, but if P and Q are conjugate and if their intersection is parabolic, then they are identical. Therefore (10.2.(10)) is satisfied and ˜ In particular, it contains 2 subsets isomorphic to X¯ B S . we can form M. B S ¯ \ X is a compact manifold with corners, the union of faces e (P) = P \e(P) where P runs through a set of representatives of -conjugacy classes of parabolic Qsubgroups. For J ⊂ , let P J be the set of parabolic Q-subgroups conjugate to PJ . Then H= H(α) , (3) α∈

where H(α) =

e (P).

P∈\P−{α}

Again the elements of H(α) are disjoint, so that 10.2(10) is satisfied. 10.6. Oshima [Os2] has generalized the construction of 10.4 to the case of a semisimple symmetric space. We shall discuss it later (14.6).

11 Real forms and real points of complex orbits Our next goal is to consider some smooth compactifications of symmetric spaces in which the open orbits are not necessarily isomorphic G-spaces. This section contains some preliminary material. 11.1 We let σ be an involution of G and let H = G σ be its fixed point set. We want to construct a family of real forms Hε of Hc . If σ = θ, H = K , they were introduced in [OSS]. We fix a Cartan involution θ commuting with σ . We have the decompositions g=k⊕p=h⊕q

(1)

which are orthogonal with respect to the Killing form, where h, q are the ±1eigenspaces of σ . These four spaces are invariant under both σ and θ . Let ao be a maximal abelian subalgebra of p ∩ q and a a maximal abelian subalgebra of p containing it, o = (g, ao ) and = (g, a) the corresponding root systems and


W = N K (a)/Z K (a)

Wo = N K (ao )/Z K (ao )

119

(2)

their Weyl groups. It is known that Wo may be identified with the image in G L(a0 ) of the subgroup of W leaving ao stable. We choose compatible orderings on and o and denote and o the sets of corresponding simple roots. Let r : a∗ → a∗o be the restriction map. “Compatible” means that o ⊂ r () ⊂ o ∪ {0}.

(3)

11.2 A signature ε˜ on o (or an extended signature) is a map of o into {±1, 0} such that ε˜ (α) = ε˜ (−α), ε˜ (α + β) = ε˜ (α) · ε˜ (β) (α, β, α + β ∈ o ). (1) We let E(o ) be the set of extended signatures and E o (o ) the set of extended signatures which are proper, i.e., do not take the value 0. The restriction to o of an extended signature is an element of E(o ). Conversely, given ε ∈ E()o , define ε˜ by ε˜ (α) = ε(β)|n αβ | if α = n αβ β. (2) β∈o

It is easily seen that ε˜ is a signature on o and that the restriction induces bijections E(o ) E(o ) and E o (o ) = E o (o ). The support of s(ε) of a signature is the set of α on which ε is = 0. We leave it to the reader to show that s(˜ε ) = "s(ε)# ∩ o , (3) where "s(ε)# is the vector subspace spanned by s(ε). We have the decompositions g = z(ao ) ⊕ ⊕β∈o gβ , where gβ = {x ∈ g, [h, x] = β(h).x, (h ∈ ao )}

(4)

h = z(ao )σ ⊕ ⊕β>0, x∈gβ "x + σ (x)#

(5)

q = z(ao )−σ ⊕ ⊕ β>0 "x − σ (β)#

(6)

x∈gβ

Given ε ∈ E(o ), define a linear transformation σε of g by σε = σ on z(ao ), σε (x) = ε(β)σ x

x ∈ gβ

(β ∈ o ).

(7)

(Note that σε (x) ∈ g−β if x ∈ gβ .) The map σε is obviously bijective and involutive. It can be checked that it commutes with [ , ], hence that it is an involutive automorphism of g. We shall give below a somewhat more conceptual argument, and also prove that Hε := G σε (8) is a real form of Hc . 11.3. To avoid some minor technical complications, we assume that G = Ad gc (R).

120

A. Borel and L. Ji

Let Ao = exp ao and So its Zariski closure in G c = Ad gc , i.e., the smallest closed subvariety containing A0 in the group of adjoint type with Lie algebra gc . It is an R-split torus of dimension = dim ao . So (R) = (R∗ ) = Ao × So (R)2

(1)

where So (R)2 denotes the subgroup of elements of order ≤ 2 in So (R). It is a direct product of copies of Z/2Z. Let µ : So (R)2 → E o (o ) be defined by µ(s)(α) = s α := εs (α).

(2)

It is readily checked that µ is a bijection (see the remark below) and that σεs = Int s ◦ σ,

(3)

hence σεs is an automorphism. We have

So (C) = C∗ = Ao × exp ia0 ,

(4)

where the second factor is a compact torus, the largest compact subgroup of So (C). We also write Su for exp iao . It is a compact real form of So (C). We have So (R)2 = Su,2 = K ∩ So (C) = K ∩ H ∩ So (C),

(5)

since the elements of So (R)2 are fixed under both θ and σ . Any element in Su has a square root, so we can find u ε ∈ Su such that u −2 ε = sε .

(6)

Of course, Int u ε is an automorphism of G c and it is easily checked that Int u ε (Hc ) = Hε,c

(7)

hence that Hε is a real form of Hc . In fact, Int u ε is the identity on z(ao )c . Moreover if x ∈ gβ , then Int u ε (x + s(x)) = u βε x + u −β ε .s(x) = u βε (x + u −2β s(x)) = u βε (x + ε(x).s(x)) ∈ hε,c . ε

(8)

We note also that since s belongs to K ε the automorphism σεs commutes with θ; therefore θ is a Cartan involution of Hε . The subalgebras a and ao belongs to hε , and play the same role for hε as for h. Remark. We have assumed G to be of adjoint type. Therefore So is a direct product of one-dimensional subtori S α (α ∈ ) such that α is a generator of X (S α ). From this our assertion is clear. However, we shall also use this for the M J , which cannot be assumed to be of adjoint type. If G is a not of adjoint type, let q : G c → G c = Ad gc. Its restriction


121

to Su,2 has kernel Z = CG ∩ Su,2 , where CG is the center of G” after Su,2 . Let S = q(S). Then, instead of Su,2 take the group S(2) = {s ∈ Su , q(s) ∈ Su,2 and s 2 ∈ Z }.

Then given ε (proper or not), there exists sε ⊂ S(2) such that sεα = ε(α) (α ∈ ), and sε is determined up to an element of Z . The element sε does not necessarily belong to G but it can be seen to normalize it. Basically, we can argue as if sε ∈ Su,2 up to minor adjustments, which we leave to the reader. 11.4. The group Wo operates on E o (o ) as usual by w(ε)(β) = ε(w −1 .β)

(β ∈ α ).

(1)

It also acts (by inner automorphisms) on So , So (R), So (R)2 , and, clearly the map µ is Wo -equivariant; we get a bijection ∼

Su,2 /Wo →→ E o (o )/Wo .

(2)

The quotient G c /Hc is a smooth variety (in fact an affine one, since Hc is reductive) defined over R. Hence (G c /Hc )(R) is a smooth real variety and a G-space. It is a union of orbits of G and we have 11.5. Proposition Let K = H . The map s → G/K σε induces a bijection ∼

Su,2 /Wo →→ G\(G c /K c )(R). The proof, which involves Galois cohomology, will be sketched later.

12 The wonderful compactification of G c /K c and its real points 12.1. In order not to interrupt the following discussion, we start with some remarks on algebraic actions of C∗ = GL1 (C). We have P1 (C) = C∗ ∪ {0} ∪ {∞}. It is a standard fact that any morphism of C∗ into a projective (or complete) variety V over C extends to P1 (C). (In fact, when V is a projective space Pn (C), this extension is clear. In general, let V ⊂ Pn (C). Then the image of the extended map from P1 (C) is contained in V .) If this morphism is an orbit map for an algebraic action, the images of 0 and ∞ are fixed points of C∗ . We can check this directly in the only case of interest here: V = P N (C) and C∗ acts on V via a linear action on the underlying vector space W = C N +1 . Write W as a direct sum of one-dimensional subspaces Wi invariant under C∗ . Then C∗ acts on Wi by means of a character t → t m i (m i ∈ Z). We may assume that m i ≤ m j if i ≤ j. Now let w = (wi ) ∈ W, w = 0. The point t.w has coordinates (t m i .wi ). In P N (C),

122

A. Borel and L. Ji

it has the same homogeneous coordinates as (t m i −m 1 .wi ). Now all the exponents are ≥ 0, so if t → 0, the point has the limit 0.w with coordinates (0.w)i equal to 1 if m i = m 1 and to 0 otherwise. Similarly, if we divide by t m N , all exponents are ≤ 0, and if t → ∞, the point t.w tends to the point ∞.ω with homogeneous coordinates (∞.ω)i equal to 1 if m i = m N , and to 0 otherwise. 12.2. We specialize the considerations of Section 11 to the case σ = θ , hence H = K and ao = a. We already noted in 11.3 that θ is a Cartan involution of K ε . It then follows that W (K ε , a) = W (K , a) = W (g, a), (1) an assumption often used by P. Delorme in discussing semisimple symmetric spaces. We let X c = G c /K c and denote by X¯ cW , its so-called wonderful compactification, constructed by de Concini and C. Procesi [CP]. It is a smooth projective variety on which G c acts morphically. It contains X c as an open orbit, the complement of which is a union of smooth divisors with normal crossings, which is locally given by k {z = (z , · · · , z ) ∈ Cn | z = 0} for some k ≤ n. We first describe its orbit ∪i=1 n i 1 structure, starting from the notation in Sections 4 or 10.3 pertaining to the maximal Satake compactification X¯ m . Recall that O J , where O J = G/Q j and Q j = K J .A J .N J . (2) X¯ m = J ⊂

X¯ cW is a complex analogue: let S be the Zariski closure in G c of A = exp a. It is an algebraic torus, isomorphic to (C∗ ) , in which the coordinates are the simple roots, viewed globally as characters of S. The role of A¯ is now taken by C , in which S is ¯ The subgroup S J of S is defined as usual as naturally embedded, to be denoted by S. o SJ = ker α . (3) α∈J

It is the Zariski closure of A J . We have the decompositions PJ,c = M J,c .S j .N J,c , Q J,c = K J,c .S J .N J,c . Then

X¯ cW =

O J,c , where O J,c = G c /Q J,c .

(4) (5)

J ∈

If J is empty, O J = X c . The closures of the O J,c , where J runs through the sets − {α}(α ∈ ) are smooth divisors, with normal crossings. Remark. If G c does not split over R, a is not a Cartan subalgebra of gc , so that the PJ are some, but not all, of the standard parabolic subgroups of G c . They are all those which are defined over R, and any parabolic subgroup of G c defined over R is conjugate under G to one of the PJ,c . 12.3. Two definitions of X¯ cW are given in [CP]. One is the Zariski closure of the orbit of kc in the Grassmannian of m-planes (m = dim kc ) in gc , under the adjoint action,


123

but this does not make it easy to see its structure. Another is via representation theory, a complex analogue of the Satake construction. We sketch it. Let (τ, V ) be an irreducible representation of G c with regular highest weight, containing a line invariant under K c . It defines a point xo in the projective space P(V ) of one-dimensional subspaces of V . The group G c acts on the latter and, since K c is its own normalizer in G c , the orbit G c .xo of xo is isomorphic to X c = G c /K c . Then X¯ cW is its Zariski closure. To analyze it one uses, as in the case of Satake compactifications, the structure of the weights. Write Vλ = ⊕Vλ , where λ runs through the weights of τ , and Vλ = {v ∈ V, τ (t).v = t λ .v

(t ∈ S)}

The weights are of the form λ=µ−

α∈

cα (λ).α

(cα ∈ N)

where µ is the highest weight of τ . We view S as embedded in C by the map s → (s −α1 , . . . , s −α ) so that t tends to a coordinates plane if some of the t α tend to infinity, while the others remain constant. In studying the Zariski closure of G c .x, we may ignore the common factor s µ and, following 12.1, look at limits of sequences for which some s αi remains constant and the others tend to infinity. This yields an embedding of Nc− × C onto an open chart in X¯ cW , the intersection of which with X c is Nc− × S. An analysis similar to that of Section 4 shows that the complement of X c in X¯ cW is the union of orbits isomorphic to the O J,c , which intersect locally as the coordinates planes in C . 12.4. Real points of X¯ cW . All the varieties and subgroups above are defined over R, so X¯ cW (R) = (G c /Q J,c )(R). (1) J ∈

First let J = , i.e., Q J,c = K c . Then, in the notation of Section 11, we have, by 11.5 X c (R) = (G c /K c )(R) ∼ G/K ε . (2) = ε∈E o ()/W

In general, it can be shown that G\(G c /Q J,c )(R) = M J \(M J,c /K J,c )(R),

(3)

which brings us back to (2) for the symmetric space X J,c = M J,c /K J,c . The standard maximal R-split torus S J of M J,c is the identity component of S ∩ M J,c . From 11.3(3), it follows that σε leaves M J stable. Setting K J,ε = M Jσε

(4)

we see, by applying 11.5 to M J and σε , that the K J,ε are real forms of K J,c and that

124

A. Borel and L. Ji

(M J,c /K J,c )(R) =

M J /K J,ε

: (A J = S J (R)0 ),

(5)

J /W (M ,A J ) s∈Su,2 J

O J,c (R) =

G/K J,ε .A J .N J .

(6)

ε∈E o ( J )/W (M J ,A J )

12.5. We recall first some known facts about real points of complex algebraic varieties. Let V be an irreducible complex variety defined over R, and n its (complex) dimension. Then the dimension of V (R), as a real algebraic variety, is at most n. It is equal to n if and only if V (R) is Zariski-dense in V . Assume it is and let x ∈ V (R). If x is simple on V , then it is simple on V (R), and V (R) is a manifold of real dimension n around x. For all of this, see [W], especially Sections 10 and 11. X¯ cW is a smooth irreducible variety. Let n be its complex dimension. X¯ cW (R) contains a G-orbit isomorphic to G/H , which has real dimension n, hence X¯ cW (R) is Zariski dense in X¯ cW . Since it is smooth, it follows from the facts just recalled that X¯ cW (R) is a real smooth projective variety. In particular X¯ cW (R) is a real analytic G-space. The G-action is clearly algebraic. We see from 10.3, and 12.2(2), (5) that the ordinary closure of X in X¯ cW (R) is the maximal Satake compactification. Therefore the latter carries the natural structure of a real analytic (even semialgebraic) manifold with corners, with respect to which it is an analytic G-space. 12.6. An example: X n = PGLn (R)/POn . 12.6.1. The space X n,c is the space of nondegenerate quadratic forms on Cn , up to a factor in C∗ or, projectively speaking, the space of nondegenerate quadrics in W is the space of “complete quadrics” Pn−1 (C). The wonderful compactification X¯ n,c [CGMP]. To J ⊂ there is assigned a partition n = n 1 + · · · + n t (J ) of n, hence a standard parabolic subgroup PJ and we have X J,c = X n 1 ,c × · · · × X n t (J ) . Therefore what is added at infinity are flags together with nondegenerate quadrics on the successive quotients of the subspaces forming the flag. 12.7. Let In = {1, . . . , n}. The group W is the symmetric group on n letters and thus operates on E o (In ). The K ε are the groups O(ε(i).xi2 ), ε ∈ E o (In ). However, we have indexed the K ε by means of signatures of the root system . We relate the two. There is a natural map ν : E o (In ) → E o () which assigns to ε the signature ε¯ : xi − x j → ε(i).ε( j) (i = j). It commutes with W , is surjective, with kernel ±ε1 , where ε1 is the identity signature ε1 (i) = 1(i ∈ In ). A section of this map consists of the signatures with ε1 (1) = 1. Since ε and −ε define the same group, we may indeed label the K ε by E o (). Then the orthogonal groups SO( p, q) with 1 ≤ p ≤ n/2 are representative of E o ()/W . To J there is associated a partition n = n 1 + · · · + n t (J ) of n and PJ is the stability group of the standard flag defined by these integers. We see then from 12.4 that the real points of O J,c are “metrized flags”, i.e., flags in which the successive quotients are endowed with a nondegenerate (possibly indefinite) metric.


125

12.8. Let n = 2. The space of nondegenerate quadratic forms on C2 is an open subset of C3 . Modulo C∗ , it is an open subset of P2 (C) with complement P1 (C). In this case, the wonderful compactification is P2 (C). It contains two orbits of G: the nondegenerate forms and the forms of rank 1, again mod C∗ , respectively isomorphic to X 2,c and P1 (C). W (R) = P (R). The group G has three orbits: definite quadratic forms on R2 , X¯ 2,c 2 indefinite quadratic forms on R2 and the forms of rank 1. These orbits are respectively isomorphic to PGL2 (R)/PO2 = SL2 (R)/SO2 : the upper half plane X PGL2 (R)/PO(1, 1) : an open Möbius band. SL2 (R)/P = PGL2 (R)/(P/(±1)). : P1 (R). Note that we are considering PGL2 (R). The group GL2 (R) contains the group L2 (R) of matrices of determinant ±1. The group O(1, 1) has four connected components, two of determinant one and two of determinant −1. If we conjugate x 2 − y 2 to x y, then SO(1, 1) is conjugate to the group A of diagonal matrices diag(a, a −1 )(a = 0). The image of O(1, 1) in the adjoint group is the same as that of N (A). The quotient SL2 (R)/A is a cylinder and SL2 (R)/N (A) is a quotient of that cylinder by an antipodal mapping, and so is an open Möbius band. The two open orbits are glued along P1 (R).

13 The Oshima–Sekiguchi compactification of G/K 13.1. This compactification, to be denoted X¯ OsS , is constructed in [OsS]. There is no restriction there on the root system (g, a), but here we limit ourselves to the case where (R) The root system (g, a) is reduced; we wish to compare X¯ OS to X¯ cW (R). We assume again that G c is of adjoint type. Fix an ordering on (g, a), let be the set of simple roots and Po = M.A.N be the associated minimal parabolic subgroup. Recall that M ⊂ K contains representatives of all connected components of Po . By definition in Section 11, the group K ε is the full fixed point set of σε in G and the open orbits are some of the G/K ε . In X¯ OS however, the open orbits are finite coverings of the G/K ε , and a given quotient may occur several times. To describe it, we need to introduce some further notation and definitions. We use those of [OsS] to the extent possible. Given ε ∈ E(), let Jε be its support in (by 11.2(3), the support supp ε of ε on is the set of roots which are linear combinations of elements in Jε .) Let W Jε = "sα , α ∈ supp (ε)# = W (M J , A J )

(1)

ε = {α ∈ , ε(α) = 1}.

(2)

126

A. Borel and L. Ji

Wε = "sα , α ∈ ε #.

(3)

Therefore Wε is a subgroup of W Jε . Let W (ε) be the set of representatives of Wε \W Jε defined by the condition W (ε) = {w ∈ W Jε , ε ∩ w.+ = ε ∩ + }.

(4)

(This is indeed a set of representatives since W Jε is simply transitive on its own Weyl chambers.) We then put K ε∗ = K εo .M. (5) It is a subgroup of finite index of K ε . It follows from 11.3(3) that σε leaves M J stable for all J ⊂ . We let ∗ o = K J,ε .M J ⊂ K J,ε = M Jσε , K J,ε

(6)

∗ Q ∗J,ε = K J,ε .A J .N J .

(7)

As in 10.4, a : R J,ε → A J is the map a : t → a(t) = exp

−

n|t α |.H α .

(8)

α∈J

The obvious analogue of 10.4 yields a G-space with finitely many orbits of the type O ∗J,ε = G/Q ∗J,ε . However it is not compact. To get a compact space, one needs more orbits and uses W to parametrize them. 13.2. By definition

X¯ OS = (G × R × W )/ ∼

(1)

is the quotient of X˜ = G × R × W by the equivalence relation (g, t, w) ∼ (g , t , w ) if

(2)

(i) w(εt ) = w (εt ) (hence w Q ∗t =w Q ∗t ) (ii) w−1 .w ∈ W (ε) (iii) g.a(t).Q ∗t .w −1 = g .a(t ).Q ∗t .w −1 . The G-action is defined by left translations on the first factor; it is proved in [OsS] that X¯ OS is a smooth compact analytic G-space. Again, the hardest point is to show analyticity. We shall shortly indicate another way to these results. We first make some remarks about the number of orbits of a given type. As usual |Z | is the cardinality of the finite set Z . Since [W Jε : Wε ] = W (ε) by definition, we see that the number of orbits ∗ .A .N , where J = J , is O ∗J,ε = G/K J,ε J J ε [W : W J ].|Wε | = |W |. |W (ε)|−1 . If ε is proper, then W J = W , therefore

(3)


G/K ε∗ occurs |Wε | times.

127

(4)

In particular, if ε = ε1 is the signature ε(α) = 1, all α, then Wε = W , hence there are |W | copies of G/K .

(5)

If ε is the zero signature: W J = Wε = W (ε) = {1}, hence there are |W | copies of G/Po .

(6)

The space G/Po is compact, so the G/Po are (the only) closed orbits. Each copy of G/K has one such orbit in its closure. More generally, G/K ε∗ has [W : Wε ] orbits of type G/Po in its closure. Remark. Let us indicate a heuristic reason why copies parametrized by W are needed. It can be shown that Wε = K ε∗ ∩ W. (7) On the other hand (see next section), we have G = K .A.K ε∗

(8)

but, if A+ is a Weyl chamber for W , then ∪w∈W/Wε wA+ is one for Wε , hence G= K .wA+ .K ε w∈W/Wε

so that, to compactify, one has to go to infinity along all the wA+ (w ∈ W/Wε ). 13.3. Let W act on X˜ by (g, t, w) ◦ u = (g, t, u −1 · w)

(u ∈ W ),

(1)

where t ◦ u is defined by (t ◦ u)β = t u

−1 .β

(t ∈ R , β ∈ ).

(2)

This action is easily seen to be compatible with ∼, hence it induces one of W on X¯ OS : Proposition The action of W on X¯ OS just defined is free, and the quotient X¯ OS /W is G-isomorphic to X¯ cW (R). ∗ .A .N onto G/K Of course the map is a natural one. It sends G/K J,ε J J J,ε .A J .N J . OS W ¯ ¯ Thus, X is a finite unramified covering of X c (R). Hence, it is compact, smooth and analytic.

128

A. Borel and L. Ji

14 The wonderful compactification of G c /Hc and its real points 14.1. We now come back to the situation introduced in 11.1. For proofs of the results on root systems and more details, we refer to [OsS1]. Let σ be an involution of G and θ a Cartan involution of G commuting with σ . We recall the decompositions g = k ⊕ p = h ⊕ q.

(1)

(11.1(1)). There exists a Cartan subalgebra aι of g which is θ and σ stable, such that aθ = aι ∩ p aσ = aι ∩ q aθ,σ = aι ∩ p ∩ q,

(2)

are maximal abelian subspaces in p, q and p ∩ q respectively (the a and ao of Section 11 are now aθ and aθ,σ ). Any two such are conjugate under K ∩ H . We have a commutative diagram .... ..... ..... . . . . ..... ..... ..... .... . . . . θ .... ..... ..... . . . . . .... .......

a∗ι,c

r

r

rθ,σ

a∗θ,c

..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ...... ............

..... ..... ..... ..... ..... ..... ..... σ ..... ..... ..... ..... ...... ........... .

.... ... .

a∗σ,c

. ..... ..... ..... ..... . . . .... ..... ..... ..... . . . . . ..... .... ....... .......

(3)

a∗θ,σ,c

where the arrows are restriction maps. To avoid repetition, let us introduce the set = {ι, σ, θ, (σ, θ )}. (4) We have four root systems = (g, a ), ( ∈ ). In particular, ι is the absolute root system and θ , the root system of the symmetric space G/K . Note that if = ι, σ the roots may be complex-valued on a . We choose orderings on the which are compatible, denote by the set of simple roots. The -rank r k (G) is the cardinality of . Let , be the initial and endpoints of an arrow in (3) and r, the map it represents. By compatibility ⊂ r, ( ) ⊂ ∪ {0}.

(5)

14.2. Given , we have to distinguish a family of standard parabolic subgroups of G c parametrized by the subsets of . First let nι be the standard maximal nilpotent subalgebra nι = ⊕β∈ι gcβ gc,β = {x ∈ gc , [h, x] = β(h).x (h ∈ aι )} . (1)


Given I ⊂ let, as usual

a,I =

ker α.

129

(2)

α∈I

Then the standard parabolic subalgebra P,I is generated by z(a,I ) and nι . By definition the standard parabolic subgroup P,I is the normalizer in G c of p,I . It is a semidirect product P,I,c = Z(S,I ).N,I,c (3) where S,I is the torus with Lie algebra a,I,c and n,I = ⊕

gc,β ,

(4)

where the sum runs over the β ∈ + which are not linear combinations of elements of I . The Lie algebras z(a,I ) and a,I are reductive in gc and we can write z(a,I ) = m,I ⊕ a,I ,

(5)

where m,I is the orthogonal complement of a,I in z(a,I ) with respect to the Killing form (whose restrictions to z(a,I ) and a,I are nondegenerate). Let M,I be the connected algebraic group with Lie algebra m,I,c . Since the centralizer of a torus is always connected, we have Z(S,I ) = M,I .S,I P,I = M,I S,I .N,I .

(6)

By definition, the conjugates of the P,I are the -relevant parabolic subgroups of G c . The group P,I is of course a standard parabolic subgroup of G c , hence can also be written P,I = PJ (I ) with J (I ) ⊂ ι . It is clear from the definition that −1 J (I ) = r (I ∪ {0}).

(7)

If = ι, the -relevant parabolic subgroups are all parabolic subgroups. If = θ , the -relevant parabolic subgroups are the standard parabolic subgroups of G c which are defined over R, and the parabolic R-subgroups are all -relevant (recall that S is in this case a maximal R-split torus). By 2.8 in [OsS1] rσ,θ (β) = 0 ⇐⇒ rθ (β) = 0 or rσ (β) = 0 (β ∈ ι ) or, equivalently

−1 rσ,θ (0) = rθ−1 (0) ∪ rσ−1 (0).

(8) (9)

This can also be expressed by the following useful 14.3. Proposition Let Pι,J be a standard parabolic subgroup. Then the following conditions are equivalent: a) Pι,J is (σ, θ )-relevant; b) Pι,J is both θ and σ -relevant; c) Pι,J is σ -relevant and defined over R.

130

A. Borel and L. Ji W

14.3. We now go over to the wonderful compactification (G c /Hc ) and its real points. The result is a direct generalization of 12.5(5). We specialize to σ . The groups Mσ,I and Sσ,I are stable under σ . We let σ Q σ,I = Mσ,I .Sσ,I .Nσ,I .

(1)

σ .S σ Note that Mσ,I σ,I is also equal to Z(Sσ,I ) . W

Theorem 14.3 [CP]. The wonderful compactification (G c /Hc ) is a smooth projective manifold. G c /Hc is an open subset, the complement of which is a union of r kσ (G c ) smooth divisors with normal crossings. We have W (G c /Hc ) = G c /Q σ,I . (1) I ⊂σ

As in the case of G/K , this compactification can be defined in two ways. We refer to [CP] for the details. 14.4. This theorem is valid whether or not G c and Hc are defined over R. If they are, as we assume, then the wonderful compactification is also defined over R and, clearly W (G c /Hc ) (R) = (G c /Q σ,I )(R), (1) I ⊂σ

so we have first to know when a summand on the right is not empty. Let x ∈ (G c /Q σ,I )(R). Its isotropy group in G c is defined over R, hence so is its unipotent radical. But the unipotent radical of Q σ,I is the unipotent radical of Pσ,I therefore Pσ,I is defined over R, and Pσ,I also θ-relevant, or (σ, θ ) relevant (14.3). As a consequence, the orbits containing real points are effectively parametrized by the subsets of σ,θ . In the rest of this subsection, we consider only (σ, θ )-relevant standard parabolic subgroups. To simplify the notation, we replace (σ, θ ) by o. We have then W (G c /Hc ) (R) = (G c /Q o,J,c )(R). (2) J ⊂o

By definition Q o,J,c = Mo,J,c .So,J .N J,c and Mo,J is connected. In order to be closer to the conventions made earlier in the real case, we now call o . It is of finite index in Z(S σ it Mo,J o,J ) or also in the group Mo,J := ∩χ∈X (So,J ) ker χ 2 . Here X ( ) refers to rational characters, Note that So,J is R-split, so they are all defined over R. With this convention Mo,J,c (R) is the Mo,J of the Langlands decomposition of Po,J . Proposition 11.5 describes (G c /Hc )(R). For the other orbits we have, in analogy with 12.4(3)


(G c /Q o,I,c )(R) = (Mo,I /K o,I )(R).

131

(4)

W

Remark. At this point, (G c /Hc ) (R) appears as the set of real points of a subvariety of the wonderful compactification. We shall come back to it elsewhere and give more details on its structure. It appears that it is itself smooth and the complement of G c /Hc in it is a union of r k(σ,θ) G smooth divisors with normal crossings, the description being similar to that of the wonderful compactification, with Cσ being replaced by Co . If so, the closure of G/H in the set of real points would be a manifold with corners of rank equal to |o |. 14.5. Remarks on the closed orbits. Consider first the complex case, where the orbits are parametrized by the subsets of σ . For I = φ the empty set, the minimal σ -relevant standard parabolic subgroup has the decomposition Pφ,c = Mφ .So .Nφ , where Mφ1 = Mφσ is pointwise fixed under σ (since its σ -rank is zero). Therefore Q φ = Pφ and the intersection of all G-stable divisors is the projective variety G/Pφ . We now go to the real case. If σ = θ, then the closed orbit is the set of real points of G c /Pφ,c and this is again equal to G/Pφ . In the general case, there is still only one closed orbit, but its structure is more complicated. To describe it, we first note that G = K .Ao .H. (1) We sketch the proof of this well-known fact: start from the Cartan decomposition G = K .P. The space P = exp p is the direct product of X 1 = exp(p ∩ q) and X 2 = exp(p ∩ h). By the Cartan decomposition of G σ.θ with respect to K ∩ H , the space X 1 is the union of the conjugates of Ao by K ∩ H . Since K ∩ H normalizes X 1 (as well as X 2 ) we get G = K .P = K .X 1 .X 2 = K .Ao .(K ∩ H ).X 2 = K .Ao .H.

(2)

For simplicity, we again write ao for aσ,θ and Ao for Aσ,θ . We have Z(Ao ) = Mo .Ao ,

(2)

where Mo is σ - and θ-stable and has (σ, θ )-rank equal to zero; therefore by (1) Mo = Moθ .Moσ

(3)

(not direct in general)

Mo /Moσ = (K ∩ Mo )/(K ∩ Mo )σ . Moσ

(4)

Mo /Moσ

are reductive, so we can realize as an orbit of Mo in The group Mo and a linear representation. It is then also an orbit of the compact group K ∩ Mo hence is a real (affine) algebraic set, by a result of C. Chevalley [Bo3]. At first, it is an open component (in ordinary topology) of G c /Q φ (R). But, since G c /Q φ,c is irreducible, it must be the full set of real points of G c /Q φ,c . O

14.6. We end this section by sketching Oshima’s compactification (G/H ) of G/H , in which all open orbits are copies of G/H [Os2]. It is a generalization of his construction for G/K , ([Os1], cf. 10.4)). The Weyl group

132

A. Borel and L. Ji

W (g, ao ) = N K (ao )/Z K (ao )

(1)

to be denoted W , has as a subgroup. W H := W (h, ao ) = N K ∩H (ao )/Z K ∩H (ao ).

(2) O

14.6.1. Assume first that these two groups are equal. The definition (G/H ) is then the same as in the case H = K . It is the quotient of G × Ro by the equivalence relation: (g, t) ∼ (g , t ) if (i) εt = εt (ii) g.a(t).Q Jt = g .a(t ).Q J t

where a(t) is defined as before. It contains 2 ( = dim ao ) open orbits isomorphic to G/H and 3 orbits altogether. The closure of one orbit is a manifold with corners of rank . Conversely, if we start from such a compactification G/H of G/H , which W is likely to be contained in (G c /Hc ) (R), as explained in Remark 12.6, then one could get a smooth compactification by gluing 2 copies of G/H as in Section 10. 14.6.2. Assume now that W H = W . By 14.5(1), G = K .Ao .H . Let A+ be the positive Weyl chamber for W . Then w∈W H \W wA+ = B + is a Weyl chamber for W H . Its conjugates under W H fill A, hence G= K wA+ .H. (1) w∈W H \W

We want to compactify G/H . For this, one goes to infinity in each wA+ , and this leads to a compactification with [W : W H ] corners. For w ∈ W , choose representatives w¯ in N K (ao ) of w ∈ W which are contained in W H if w ∈ W H . In the notation for isotropy groups or parabolic subgroups we omit the reference o to (σ, θ ). Define ¯ Q J,w¯ = (M J ∩ w−1 H ).A J .N J

(J ∈ o ).

(2)

Then G/Q J,w¯ fibres over G/PJ with typical fiber M J /(M J ∩w¯

−1

¯ H ) = M J /M Jw.σ .

(3)

Then, by definition O

(G/H ) = G × Ro × (W H \W )/ ∼

(4)

where (g, t, w) ∼ (g , t , w ) if (i) εt = εt (ii) W H .w.W ¯ J = W H .w¯ .W J (J = s(εt )) (iii) g.a(t).Q J, w,ε ¯ t = g .a(t ).Q J, w¯ ,ε . −1

t

w¯ H ) .A .M . Here Q J,w,ε ¯ t = (M J ∩ εt J J As usual, G acts by left translations on the first factor. It is proved in [Os2] that O (G/H ) is a compact smooth analytic G-space, with orbit decomposition


G/H

O

=

J ⊂o

ε∈E o (J ) w∈W ¯ H \W/W J

Q J,w,ε ¯

(O J.w,ε ¯ = G/Q J,w,ε ¯ ).

133

(5)

([Os2], 1.10). (I have followed the notation introduced earlier in Sections 11 and 12. The one of [Os2] is different in several respects.) Some remarks on (5). a) Assume that J = o . Then W J = W and W H \W/W J is reduced to the identity. The group Q J,1 is H . We get 2 copies of G/H . These are all the open orbits. b) Let ε = εo be the zero signature. Then Jε is empty and W J = {1}. The isotropy group is Q ∅,w¯ = M∅w,σ .A∅ .N∅ and O∅,w¯ = (K ∩ M∅ )/K ∩ M∅w.σ ). We get [W : W H ] such orbits. They are compact, the only compact orbits, belong to the closure of each open orbit, and are the edges of [W : W H ] corners, which compactify G/H . o c) Assume now that J = ∅, o . The orbits O J,w,ε ¯ are parametrized by E (J ) × W H \W/W J . They have codimension | − J |. Around each edge, the closures of the open orbits are glued to form a smooth manifold. Each corner will involve Z |J | such orbits. Some however will be common faces of different corners and this leads to the parametrization by W H \W modulo W J on the right. Note that the edges are isomorphic, and so are the corners consisting of faces with a fixed w. ¯

15 Appendix: Galois cohomology and real forms This appendix is devoted to some indications regarding the proof of 11.5. The reader willing to take it on faith need not read this section, which is in any case very sketchy. For details on Galois cohomology, see [Se]. 15.1. We need only Galois cohomology for the group of order two C = {1, τ }, τ 2 = 1. Let L be a group on which C operates. We define H o (C; L) and H 1 (C; L), H 0 (C; L) = L C , (1) the fixed point of C on L. The set of 1-cocycles of C with coefficients in L is Z 1 (C; L) = {x ∈ L , x.τx = 1}.

(2)

Two cocycles x, y are cohomologous, in sign x ∼ y, if there exists z ∈ L such that y = z −1 .x.τz

(3)

H 1 (C; L) = Z 1 (C; L)/ ∼ .

(4)

(twisted conjugation). Then

134

A. Borel and L. Ji

If L is commutative, it is a group. If not, it is a set with a distinguished origin, the twisted conjugacy class of 1. If C acts trivially on L, then Z 1 (C; L) is the set L 2 of elements of order ≤ 2 in L and the equivalence relation is just conjugation H 1 (C; L) = L 2 /Int L .

(5)

Let H be a subgroup of L stable under C. There is then a long exact sequence 1 → H C → L C → (L/H )C → H 1 (C; H ) → H 1 (C; L)

(6)

(exact means the image of each map is the inverse image of the distinguished element under the next map), from which one deduces the existence of a natural bijection L C \(L/H )C ∼ (7) = ker H 1 (C; L) → H 1 (C; L) . We only recall how the map is defined. Let x ∈ (L/H )C choose g ∈ L such that g.0 = x; then τ g.0 = τ x = x = g.0, hence g −1 .τ g ∈ H . It is a cocycle z x in H which splits in L, (i.e., is a coboundary in L). The map x → z x is easily seen to define the bijection (7). 15.2. The case we are mainly interested in is where L , H are algebraic groups defined over R and τ is the complex conjugation. Then (7) takes the form L(R)\(L/H )(R) ∼ = ker(H 1 (C; H ) → H 1 (C; L).

(1)

Lemma. Let G be a real reductive linear group, K a maximal compact subgroup and τ an involutive automorphism of G leaving K stable. Then the natural map j : H 1 (C; K ) → H 1 (C; G) is bijective. If τ = θ is the Cartan involution of G with respect to K , this is well known (see e.g., [Se], III, 4.5). The proof of the lemma is an easy generalization. Note that if τ = θ, then τ acts trivially on K , hence H 1 (C; K ) = K 2 /Int K . If we take for G the group of automorphisms of a complex simple Lie algebra q, this yields E. Cartan’s classification of the real forms of q. 15.3. Let G u be the compact form of G c with Lie algebra gu = k ⊕ pu , where pu = i.p. It is stable under the complex conjugation τ of G c with respect to G, which induces in fact on G u the Cartan involution of G u with respect to K . It acts by inversion on Pu = exp pu and K ∩ Pu consists of elements of order ≤ 2. We have the commutative diagram j1

H 1 (C; K ) −−−−→ H 1 (C; G u ) ⏐ ⏐ ⏐ ⏐ j2

H 1 (C; K c ) −−−−→ H 1 (C; G c ).

(1)


135

By 15.2(1), we are interested in ker j2 . By the lemma in 15.2, the two vertical arrows in (3) are bijective. This reduces us to study ker j1 . Let S be the algebraic torus of G c with Lie algebra ac and Su = exp ia. The group Su is the maximal compact subgroup of S and Su,2 = S(R)2 = K ∩ S = S2 . The quotient G u /K is a compact symmetric space and by Cartan’s theory G u = K .Su .K .

(2)

We have W (K , A) = W (K , Su ) = W (k, ia) and this group also operates on S2 . The elements of Su,2 belong to K and define 1-cocycles for K , whence a natural map m : Su,2 → H 1 (C; K ). We claim Su,2 /W (K , Su ) = Im m = ker H 1 (C; K ) → H 1 (C; G u ) , (3) which will end the proof of 11.5. The first equality in (3) follows from 15.1(5) and the known fact that in Su conjugation under K is the same as conjugation under W (K , Su ). On Pu , τ acts by inversion and any v ∈ Pu is a cocycle. It has (at least) one square root u in Pu , whence v = u.τu −1 so v splits in G u . In particular, every element of Su,2 splits in G u and therefore Im m ⊂ ker j1 . There remains to prove the reverse inclusion. Let k ∈ Z 1 (C, K ). Then k 2 = 1. Assume it splits in G u . By (2) we can find elements x, y ∈ K and z ∈ Su such that k = y −1 .z −1 .x −1 .t(x.z.y) = y −1 .z −1 .z −1 .y = y −1 .z −2 .y,

(4)

hence the class [k] of k in H 1 (C; K ) is the same as that of z −2 . Moreover, we see that z −2 is of order two, hence belongs to Su,2 , whence ker j1 ⊂ Im m.

References [AJ]

J.P. Anker, L. Ji, Sharp bounds on the heat kernel and Green function estimates on noncompact symmetric spaces, GAFA 9 (1999), 1035–1091. [AMRT] A. Ash, D. Mumford, M. Rapoport, Y. Tai, Smooth Compactifications of Locally Symmetric Varieties, Math. Sci. Press, Brookline, MA, 1975. [BB] W.L. Baily, A. Borel, Compactification of arithmetic quotients of bounded symmetric domains, Ann. Math. 84 (1966), 442–528. [BGS] W. Ballmann, M. Gromov, V. Schroeder, Manifolds of Nonpositive Curvature, Progress in Math., Vol. 61, Birkhäuser, 1985. [Bo0] A. Borel, Ensembles fondamentaux pour les groupes arithmétiques, Colloque sur la Théorie des groupes algébriques, Bruxelles 1962, 23–40; C.P. II, 287–304. [Bo1] A. Borel, Introduction aux groupes arithmétiques, Hermann, Paris, 1969. [Bo2] A. Borel, Automorphic Forms on SL2 (R), Cambridge Tracts in Mathematics, Vol. 130, Cambridge University Press, 1997. [Bo3] A. Borel, Linear Algebraic Groups, second edition, GTM Vol. 126, Springer-Verlag, 1991. [BJ1] A. Borel, L. Ji, Compactifications of locally symmetric spaces, to appear in J. Diff. Geom.

136 [BJ2] [BS]

A. Borel and L. Ji

A. Borel, L. Ji, Compactifications of symmetric spaces I, to appear in J. Diff. Geom. A. Borel, J.P. Serre, Corners and arithmetic groups, Comment. Math. Helv. 48 (1973), 436–491. [BT] A. Borel et J. Tits, Groupes réductifs, Publ. Math. I.H.E.S. 27 (1965), 55–150. ´ ements de Mathématique, Livre VI, Intégration Hermann, Paris, [Bou] N. Bourbaki, El´ 1963. [BuS] K. Burns, R. Spatzier, On the topological Tits buildings and their classifications, Publ. Math. I.H.E.S. 65 (1987), 5–34. [C] W. Casselman, Geometric Rationality of Satake Compactifications, Australian Math. Society Lecture Series, Vol. 9, Cambridge University Press, 1997, pp. 81– 103. [Ce] J. Cerf, Topologie de certains espaces de plongements, Bull. Soc. Math. France 89 (1961), 227–380. [CGMP] C. De Concini, M. Goresky, R. MacPherson, C. Procesi, On the geometry of quadrics and their degenerations, Comm. Math. Helv. 63 (1988), 337–413. [CP] C. De Concini, C. Procesi, Complete Symmetric Varieties, Lect. Notes in Math., Vol. 996, Springer, 1983, pp. 1–44. [Fu] H. Furstenberg, A Poisson formula for semi-simple Lie groups, Ann. Math. 72 (1963), 335–386. [GJT] Y. Guivarc’h, L. Ji, J.C. Taylor, Compactifications of Symmetric Spaces, Progress in Math., Vol. 156, Birkhäuser Boston, 1998. [HC] Harish-Chandra, Representations of semisimple groups VI, Amer. J. Math. 78 (1956), 564–628; C.P. II, 90–154. [He] S. Helgason, Differential Geometry, Lie Groups, and Symmetric Spaces, Academic Press, 1978. [J1] L. Ji, Satake and Martin compactifications of symmetric spaces are topological balls, Math. Res. Letters 4 (1997), 79–89. [J2] L. Ji, The greatest common quotient of Borel–Serre and the toroidal compactifications, Geometric and Functional Analysis 8 (1998), 978–1015. [J3] L. Ji, Metric compactifications of locally symmetric spaces, International J. of Math. 9 (1998), 465–491. [J4] L. Ji, An Introduction to Symmetric Spaces and Their Compactifications, in this volume. [J5] L. Ji, Scattering flats and scattering matrices of locally symmetric spaces, in preparation. [JM] L. Ji, R. MacPherson, Geometry of compactifications of locally symmetric spaces, Ann. Inst. Fourier, 52 (2002), 457–559. [JZ] L. Ji, M. Zworski, Scattering matrices and scattering geodesics of locally symmetric ´ Norm. Sup. 34 (2001), 441–469. spaces, Ann. Sci. Ec. [Ka] F.I. Karpelevic, The geometry of geodesics and the eigenfunctions of the BeltramiLaplace operator on symmetric spaces, Trans. Moscow Math. Soc. 14 (1965), 51– 199. [Kn] A. Knapp, Lie Groups Beyond an Introduction, Progress in Math., Vol. 140, Birkhäuser, 1996. [MM] M.P. Malliavin, P. Malliavin, Factorisation et lois limites de la diffusion horizontale au-dessus d’un espace riemannien symétrique, Lect. Notes in Math, Vol. 404, Springer-Verlag, 1974. [Ma] R.S. Martin, Minimal positive harmonic functions, Trans. Amer. Math. Soc. 49 (1941), 137–172.

Compactifications of Symmetric and Locally Symmetric Spaces [Mo1] [Mo2] [Os1] [Os2]

[OsS] [OsS1] [Sa1] [Sa2] [Se] [Su] [Ta1]

[Ta2] [W] [Zu1] [Zu2]

137

C.C. Moore, Compactifications of symmetric spaces I, Amer. J. Math. 86 (1964), 201–218. C.C. Moore, Compactifications of symmetric spaces II, Amer. J. Math. 86 (1964), 358–377. T. Oshima, A realization of Riemannian symmetric spaces, J. Math. Soc. Japan 30 (1978), 117–132. T. Oshima, A realization of semisimple symmetric spaces and construction of boundary value maps, Adv. Studies in Pure Math., Representations of Lie groups, 14 (1986), pp. 603–650. T. Oshima, J. Sekiguchi, Eigenspaces of invariant differential operators on an affine symmetric spaces, Invent. Math. 57 (1980), 1–81. T. Oshima and J. Sekiguchi, The restricted root system of a semisimple symmetric pair, Advanced Studies in Pure Math. 4 (1984), 433–497. I. Satake, On representations and compactifications of symmetric spaces, Ann. Math. 71 (1960), 77–110. I. Satake, On compactifications of the quotient spaces for arithmetically defined discontinuous groups, Ann. Math. 72 (1960), 555–580. J-P. Serre, Cohomologie galoisienne, 5ème e´ dition révisée et complétée, LNM 5, Springer, 1994. D. Sullivan, Related aspects of positivity in Riemannian geometry, J. Diff. Geom. 25 (1987), 327–308. J.C. Taylor, The Martin compactification of a symmetric space of non-compact type at the bottom of the positive spectrum: An introduction, in: Potential Theory, ed. by M. Kishi, Walter de Gruyter & Co., Berlin, 1991, pp. 127–139. J.C. Taylor, Martin compactification, in: Topics in Probability and Lie groups: Boundary Theory, ed. by J.C. Taylor, in CRM-AMS, 2001, pp. 153–202. H. Whitney, Elementary structure of real algebraic varieties, Ann. Math. 66 (1957), 545–566. S. Zucker, L 2 cohomology of warped products and arithmetic groups, Invent. Math. 70 (1982), 169–218. S. Zucker, Satake compactifications, Comment. Math. Helv. 58 (1983), 312–343.

Restrictions of Unitary Representations of Real Reductive Groups Toshiyuki Kobayashi Research Institute for Mathematical Sciences, Kyoto University, Kyoto, 606-8502, Japan [email protected]

1 Reductive Lie groups Classical groups such as the general linear group G L(n, R) and the Lorentz group O( p, q) are reductive Lie groups. This section tries to give an elementary introduction to the structures of reductive Lie groups based on examples, which will be used throughout this chapter. All the materials of this section and further details may be found in standard textbooks or lecture notes such as [23, 36, 99, 104].

1.1 Smallest objects The “smallest objects” of representations are irreducible representations. All unitary representations are built up of irreducible unitary representations by means of direct integrals (see §3.1.2). The “smallest objects” for Lie groups are those without nontrivial connected normal subgroups; they consist of simple Lie groups such as S L(n, R), and onedimensional abelian Lie groups such as R and S 1 . Reductive Lie groups are locally isomorphic to these Lie groups or their direct products. Loosely, a theorem of Duflo [10] asserts that all irreducible unitary representations of a real algebraic group are built from those of reductive Lie groups. Throughout this chapter, our main concern will be with irreducible decompositions of unitary representations of reductive Lie groups. In Section 1 we summarize necessary notation and basic facts on reductive Lie groups in as elementary a way as possible. Keywords and phrases: unitary representations, branching laws, semisimple Lie group, discrete spectrum, homogeneous space 2000 MSC: primary 22E4; secondary 43A85, 11F67, 53C50, 53D20

140

T. Kobayashi

1.2 General linear group G L(N , R) The general linear group G L(N , R) is a typical example of reductive Lie groups. First, we set up notation for G := G L(N , R). We consider the map θ : G → G,

g → tg −1 .

Clearly, θ is an involutive automorphism in the sense that θ satisfies: θ ◦ θ = id, θ is an automorphism of the Lie group G. The set of the fixed points of θ K := G θ = {g ∈ G : θg = g} = {g ∈ G L(N , R) : tg −1 = g} is nothing but the orthogonal group O(N ), which is compact because O(N ) is a 2 bounded closed set in the Euclidean space in M(N , R) R N in light of the following inclusion: O(N ) ⊂ {g = (gi j ) ∈ M(N , R) :

N N

gi2j = N }.

i=1 j=1

Furthermore, there is no larger compact subgroup of G L(N , R) which contains O(N ). Thus, K = O(N ) is a maximal compact subgroup of G = G L(N , R). Conversely, any maximal compact subgroup of G is conjugate to K by an element of G. The involution θ (or its conjugation) is called a Cartan involution of G L(N , R).

1.3

Cartan decomposition

The Lie algebras of G and K will be denoted by g and k, respectively. Then, for G = G L(N , R) and K = O(N ), we have the following direct sum decomposition: g = gl(N , R) % k = o(N )

:= the set of N × N real matrices := {X ∈ gl(N , R) : X = −tX }

⊕ p = Symm(N , R) := {X ∈ gl(N , R) : X = tX }. The decomposition g = k + p is called the Cartan decomposition of the Lie algebra g corresponding to the Cartan involution θ. This decomposition lifts to the Lie group G in the sense that we have the diffeomorphism

Restrictions of Unitary Representations of Real Reductive Groups ∼

p × K → G,

(X, k) → e X k.

141

(1.3.1)

The map (1.3.1) is bijective, as (X, k) is recovered from g ∈ G L(N , R) by the formula 1 X = log(g tg), k = e−X g. 2 Here, log is the inverse of the bijection exp : Symm(N , R) → Symm+ (N , R) := {X ∈ Symm(N , R) : X 0}. It requires a small computation of the Jacobian to see the map (1.3.1) is a C ω diffeomorphism (see [23, Chapter II, Theorem 1.7]). The decomposition (1.3.1) is known as the polar decomposition of G L(N , R) in linear algebra, and is a special case of a Cartan decomposition of a reductive Lie group in Lie theory (see (1.4.2) below).

1.4

Reductive Lie groups

A connected Lie group G is called reductive if its Lie algebra g is reductive, namely, it is isomorphic to the direct sum of simple Lie algebras and an abelian Lie algebra. In the literature on representation theory, however, different authors have introduced and/or adopted several variations of the category of reductive Lie groups, in particular, with respect to discreteness, linearity and covering. In this chapter we adopt the following definition of “reductive Lie group”. Some advantages here are: • •

The structural theory of reductive Lie group can be explained in an elementary way, parallel to that of G L(n, R). It is easy to verify that (typical) classical groups are indeed real reductive Lie groups (see §1.5).

Definition 1.4. We say a Lie group G is linear reductive if G can be realized as a closed subgroup of G L(N , R) satisfying the following two conditions: i) θ G = G. ii) G has at most finitely many connected components. In this chapter we say G is a reductive Lie group if it is a finite covering of a linear reductive Lie group. Then the condition (i) of Definition 1.4 implies that its Lie algebra g is reductive. If G is linear reductive, then by using its realization in G L(N , R) we define K := G ∩ O(N ). Then K is a maximal compact subgroup of G. The Lie algebra k of K is given by g ∩ o(N ). We define p := g ∩ Symm(N , R). Similar to the case of G L(N , R), we have the following Cartan decompositions:

142

T. Kobayashi

g=k+p ∼

p×K → G

(direct sum decomposition)

(1.4.1)

(diffeomorphism)

(1.4.2)

by taking the restriction of the corresponding decompositions for G L(N , R) (see (1.3.1)) to g and G, respectively.

1.5

Examples of reductive Lie groups

Reductive Lie groups in the above definition include all compact Lie groups (especially finite groups) and classical groups such as the general linear group G L(N , R), G L(N , C), the special linear group S L(N , R), S L(N , C), the generalized Lorentz group (the indefinite orthogonal group) O( p, q), the symplectic group Sp(N , R), Sp(N , C), and some others such as U ( p, q), Sp( p, q), U ∗ (2n), O ∗ (2n), etc. In this subsection, we review some of these classical groups, and explain how to prove that they are reductive Lie groups in the sense of Definition 1.4. For this purpose, the following theorem is useful. Theorem 1.5.1. Let G be a linear algebraic subgroup of G L(N , R). If θ G = G, then G is a reductive Lie group. Here we recall that a linear algebraic group (over R) is a subgroup G ⊂ G L(N , R) which is an algebraic subset in M(N , R), that is, the set of zeros of an ideal of polynomial functions with coefficients in R. Sketch of proof. In order to prove that G has at most finitely many connected components, it is enough to verify the following two assertions: •

K = G ∩ O(N ) is compact. ∼

• The map p × K → G, sition).

(X, k) → e X k is a homeomorphism (Cartan decompo-

The first assertion is obvious because G is closed. The second assertion is deduced from the bijection (1.3.1) for G L(N , R). For this, the nontrivial part is a proof of the implication “e X k ∈ G ⇒ X ∈ p”. Let us prove this. We note that if e X k ∈ G, then e2X = (e X k)(θ (e X k))−1 ∈ G, and therefore e2n X ∈ G for all n ∈ Z. To see X ∈ p, we want to show e2t X ∈ G for all t ∈ R. This follows from Chevalley’s lemma: let X ∈ Symm(N , R); if es X satisfies a polynomial equation of entries for any s ∈ Z, then so does es X for any s ∈ R. As special cases of Theorem 1.5.1, we pin down Propositions 1.5.2, 1.5.3 and 1.5.6. Proposition 1.5.2. S L(N , R) := {g ∈ G L(N , R) : det g = 1} is a reductive Lie group. Proof. Obvious from Theorem 1.5.1.

Restrictions of Unitary Representations of Real Reductive Groups

143

Proposition 1.5.3. Let A be an N × N matrix such that A2 = cI (c = 0). Then G(A) := {g ∈ G L(N , R) : tg Ag = A} is a reductive Lie group. Proof. First, it follows from the definition that G(A) is a linear algebraic subgroup of G L(N , R). Second, let us prove θ (G(A)) = G(A). If tg Ag = A, then g −1 = A−1tg A. Since −1 A = 1c A, we have 1 I = gg −1 = g( A)tg(c A−1 ) = g Atg A−1 . c Hence g Atg = A, that is θg ∈ G(A). Then the proposition follows from Theorem 1.5.1. I O Example 1.5.4. If A = p ( p + q = n), then O −Iq G(A) = O( p, q)

(indefinite orthogonal group).

In this case K = O( p, q) ∩ O( p + q) O( p) × O(q). O B p={ t : B ∈ M( p, q; R)}. B O

O In Example 1.5.5. If A = −In O

(N = 2n), then

G(A) = Sp(n, R)

(real symplectic group).

In this case, K = Sp(n, R) ∩ O(2n) U (n). ∼ A B k={ : A = −t A, B = tB} → u(n), −B A A B → A + i B. −B A ∼ A B p={ : A = t A, B = tB} → Symm(n, C), B −A A B → A + i B. B −A

144

T. Kobayashi

Next let F = R, C or H (quaternionic number field). We regard Fn as a right Let

F× -module.

M(n, F) := the ring of endomorphisms of Fn , commuting with F× -actions ∪ G L(n, F) := the group of all invertibles in M(n, F). Proposition 1.5.6. G L(n, F) (F = R, C, H) is a reductive Lie group. Sketch of proof. Use C R2 as R-modules and H C2 as right C-modules, and then realize G L(n, C) in G L(2n, R) and G L(n, H) in G L(2n, C). For example, G L(n, H) can be realized in G L(2n, C) as an algebraic subgroup: U ∗ (2n) = {g ∈ G L(2n, C) : g¯ J = J g}, 0 −In where J = . It can also be realized in G L(4n, R) as an algebraic subgroup In 0 which is stable under the Cartan involution θ : g → tg −1 . Classical reductive Lie groups are obtained by Propositions 1.5.2, 1.5.3 and 1.5.6, and by taking their intersections and their finite coverings. For examples, in appropriate realizations, the following intersections U ( p, q) = S O(2 p, 2q) ∩ G L( p + q, C) Sp( p, q) = S O(4 p, 4q) ∩ G L( p + q, H) SU ∗ (2n) = U ∗ (2n) ∩ S L(2n, C) S O ∗ (2n) = G L(n, H) ∩ S O(2n, C) are all reductive (linear) Lie groups.

1.6

Inclusions of groups and restrictions of representations

As the constructions in the previous section indicate, these classical Lie groups enjoy natural inclusive relations such as · · · ⊂ G L(n, R) ⊂ Sp(n, R) ⊂ Sp(n, C) ⊂ Sp(2n, R) ⊂ G L(4n, R) ⊂ · · · ∪ ∪ ∪ ∪ ∪ · · · ⊂ O( p, q) ⊂ U ( p, q) ⊂ Sp( p, q) ⊂ U (2 p, 2q) ⊂ O(4 p, 4q) ⊂ · · · ∩ ∩ ∩ ∩ ∩ · · · ⊂ G L(n, R) ⊂ G L(n, C) ⊂ G L(n, H) ⊂ G L(2n, C) ⊂ G L(4n, R) ⊂ · · · ∩ ∩ ∩ ∩ ∩ .. .. .. .. .. . . . . . where p + q = n. Our objective in this chapter will be to restrict unitary representations of a group to its subgroups. This may be regarded as a representation-theoretic counterpart of inclusive relations between Lie groups as above.


145

2 Unitary representations and admissible representations To deal with infinite-dimensional representations, we need a good category to work with. This section introduces some standard notation such as continuous representations followed by a more specialized category such as discrete decomposable representations and admissible representations.

2.1

Continuous representations

Let H be a topological vector space over C. We shall write: End(H) := the ring of continuous endomorphisms of H, G L(H) := the group of all invertibles in End(H). Definition 2.1.1. Let G be a Lie group and π : G → G L(H) a group homomorphism. We say (π, H) is a continuous representation if the following map G × H → H,

(g, v) → π(g)v

(2.1.1)

is continuous. If H is a Fréchet space (a Banach space, a Hilbert space, etc.), the continuity condition in this definition is equivalent to strong continuity — that g → π(g)v is continuous from G to H for each v ∈ H. A continuous representation (π, H) is irreducible if there is no invariant closed subspace of H but for obvious ones {0} and H.

2.2

Example

Example 2.2.1. Let G = R and H = L 2 (R). For a ∈ G and f ∈ H, we define T (a) : L 2 (R) → L 2 (R),

f (x) → f (x − a).

Then it is easy to see that the map (2.1.1) is continuous. The resulting continuous representation (T, L 2 (R)) is called the regular representation of R. We note that T : G → G L(H) is not continuous if G L(H) is equipped with operator norm % % because √ lim %T (a) − T (a0 )% = 2 = 0. (2.2.1) a→a0

This observation explains that the continuity of (2.1.1) is a proper one to define the notion of continuous representations.

2.3

Unitary representations

Definition 2.3.1. A unitary representation is a continuous representation π defined on a Hilbert space H such that π(g) is a unitary operator for all g ∈ G. The set of equivalent classes of irreducible unitary representaitons of G is called the unitary dual of G, and will be denoted by G.

146

T. Kobayashi

2.4

Admissible restrictions

2.4.1

(Unitarily) discretely decomposable representations

G

Let be a Lie group and let π be a unitary representation of G on a (separable) Hilbert space. Definition 2.4.1. We say the unitary representation π is (unitarily) discretely decomposable, or simply, discretely decomposable if π is unitarily equivalent to a discrete sum of irreducible unitary representations of G : ⊕ π|G n π (σ )σ. (2.4.1) σ ∈ G

→ {0, 1, 2, . . . , ∞}, and ⊕ n π (σ )σ denotes the Hilbert completion Here, n π : G of an algebraic direct sum of irreducible representations σ ∈ G

(σ ! ⊕σ ⊕ "#· · · ⊕ σ$). n π (σ )

If we have the unitary equivalence (2.4.1), then n π (σ ) is given by n π (σ ) = dim HomG (σ, π|G ), the dimension of the space of continuous G -homomorphisms. The point of Definition 2.4.1 is that there is no continuous spectrum in the decomposition (see Theorem 3.1.2 for a general nature of irreducible decompositions). 2.4.2

Admissible representations

Let π be a unitary representation of G . Definition 2.4.2. We say π is G -admissible if it is (analytically) discretely decom . posable and if n π (σ ) < ∞ for any σ ∈ G Example. 1) Any finite-dimensional unitary representation is admissible. 2) The regular representation of a compact group K on L 2 (K ) is also admissible by the Peter–Weyl theorem. We shall give some more examples where admissible representations arise naturally in various contexts. 2.4.3

Gelfand–Piateski-Shapiro’s theorem

Let be a discrete subgroup of a unimodular Lie group (for example, any reductive Lie group is unimodular). Then we can induce a G-invariant measure on the coset space G/ from the Haar measure on G, and define a unitary representation of G on the Hilbert space L 2 (G/ ), consisting of square integrable functions on G/ . Theorem 2.4.3 (Gelfand–Piateski-Shapiro). If G/ is compact, then the representation on L 2 (G/ ) is G-admissible. Proof. See [105, Proposition 4.3.1.8] for a proof due to Langlands.


2.4.4

147

Admissible restrictions

Our main objective in this chapter is to study restrictions of a unitary representation π of G with respect to its subgroup G . Definition 2.4.4. We say the restriction π|G is G -admissible if it is G -admissible (Definition 2.4.2). Our main concern in later sections will be with the case where G is noncompact. We end up with some basic results on admissible restrictions for later purposes. 2.4.5

Chain rule of admissible restrictions

Theorem 2.4.5. Suppose G ⊃ G 1 ⊃ G 2 are (reductive) Lie groups, and π is a unitary representation of G. If the restriction π |G 2 is G 2 -admissible, then the restriction π |G 1 is G 1 -admissible. Sketch of proof. Use Zorn’s lemma. The proof parallels Langlands’ proof of Gelfand– Piateski-Shapiro. See [41, Theorem 1.2] for details. Here is an immediate consequence of Theorem 2.4.5. Corollary. Let π be a unitary representation of G, and K a compact subgroup of G. Assume that dim Hom K (σ, π| K ) < ∞ for any σ ∈ K . Then the restriction π |G is G -admissible for any G containing K . This corollary is a key to a criterion of admissible restrictions, which we shall return in later sections (see Theorem 6.3.4). 2.4.6

Harish-Chandra’s admissibility theorem

Here is a special, but very important example of admissible restrictions: Theorem 2.4.6 (Harish-Chandra [21]). Let G be a reductive Lie group with a max the restriction π | K is K-admissible. imal compact subgroup K . For any π ∈ G, Since K is compact, the point of Theorem 2.4.6 is the finiteness of K-multiplicities: . dim Hom K (σ, π| K ) < ∞ for any σ ∈ K

(2.4.6)

Remark. In a traditional terminology, a continuous (not necessarily unitary) representation π of G is called admissible if (2.4.6) holds. For advanced readers, we give a flavor of the proof of Harish-Chandra’s admissibility theorem without going into details. Sketch of proof of Theorem 2.4.6. See [104], Theorem 3.4.10; [105], Theorem 4.5.2.11.

148

T. Kobayashi

Step 1. Any principal series representation is K -admissible. This is an easy consequence of Frobenius reciprocity because the K -structure of a principal series is given as the induced representation of K from an irreducible representation of a subgroup of K . Step 2. Let π be any irreducible unitary representation of G. The center Z (g) of the enveloping algebra U (g) acts on π ∞ as scalars. Here, π ∞ denotes the representation on smooth vectors. Step 2 is due to Segal and Mautner. It may be regarded as a generalization of Schur’s lemma. Step 3. Any π can be realized in a subquotient of some principal series representation (Harish-Chandra, Lepowsky, Rader), or more strongly, in a subrepresentation of some principal series representation (Casselman’s embedding theorem). Together with Step 1, Theorem 2.4.6 follows. Casselman’s proof is to use the theory of the Jacquet module and the n-homologies of representations. There is also a more analytic proof for Casselman’s embedding theorem without using n-homologies: This proof consists of three steps: (i) to compactify G ([79]), (ii) to realize π ∞ in C ∞ (G) via matrix coefficients, (iii) to take the boundary values into principal series representations (see [80], [104] and references therein). 2.4.7

Further readings

For further details, see Chapter 4 [105] for general facts on continuous representations in Section 2.1; see [35, Chapter 8] and [104, Chapter 3] (and also an exposition [102]) for Harish-Chandra’s admissibility theorem 2.4.6 and the idea of nhomologies. For more information about G -admissible restrictions with noncompact G , see exposition [47, 50].

3 S L(2, R) and Branching Laws Branching laws of unitary representations are related to many different areas of mathematics —spectral theory of partial differential operators, harmonic analysis, combinatorics, differential geometry, complex analysis, · · · . The aim of this subsection is to give a flavor of various aspects of branching laws through a number of examples arising from S L(2, R).

3.1 3.1.1

Branching Problems Direct integral of Hilbert spaces

We recall briefly how to generalize the concept of the discrete direct sum of Hilbert spaces into the direct integral of Hilbert spaces, and explain the general theory of irreducible decomposition of unitary representations that may contain a continuous spectrum. Since the aim here is just to provide a wider perspective regarding discrete


149

decomposable restrictions which will be developed in later chapters, we shall try to minimize the exposition. See [34] for further details on §3.1. Let H be a (separable) Hilbert space, and (, µ) a measure space. We construct a Hilbert space, denoted by ⊕

Hdµ(λ),

consisting of those H-valued functions s : → H with the following two properties: i) For any v ∈ H, (s(λ), v) is measurable with respect to µ. ii) %s(λ)%2H is square integrable with respect to µ. The inner product on

⊕

Hdµ(λ) is given by (s, s ) := (s(λ), s (λ))H dµ(λ).

Example. If H = C, then

⊕

Hdµ(λ) L 2 ().

More generally, if a “measurable” family of Hilbert spaces Hλ parameterized by λ ⊕ ∈ is given, then one can also define a direct integral of Hilbert spaces Hλ dµ(λ) in a similar manner. Furthermore, if (πλ , Hλ ) is a “measurable” family of unitary representations of a Lie group G for each λ, then the map (s(λ))λ → (πλ (g)s(λ))λ ⊕ defines a unitary operator on Hλ dµ(λ). The resulting unitary representation is called the direct integral of unitary representaions (πλ , Hλ ) and will be denoted by ⊕

3.1.2

πλ dµ(λ),

⊕

Hλ dµ(λ) .

Irreducible decomposition

The following theorem holds more generally for a group G of type I in the sense of von Neumann algebras. Theorem 3.1.2. Every unitary representation π of a reductive Lie group G on a (separable) Hilbert space is unitarily equivalent to a direct integral of irreducible unitary representations of G : ⊕ π n π (σ )σ dµ(σ ). (3.1.2) G

→ N∪{∞} is a measur , n π : G Here, dµ is a Borel measure on the unitary dual G able function, and n π (σ )σ is a multiple of the irreducible unitary representation σ .

150

T. Kobayashi

3.1.3

Examples

Example 3.1.3. 1) (Decomposition of L 2 (R)). Let G = R. For a parameter ξ ∈ R, we define a one-dimensional unitary representation of the abelian Lie group R by χξ : R → C × ,

x → ei xξ .

Then we have a bijection R R, χξ ↔ ξ . The regular representation T of R on L 2 (R) is decomposed into irreducibles of R by the Fourier transform ⊕ T χξ dξ. R

L 2 (S 1 ))

2) (Decomposition of Let G = S 1 R/2π Z. By the Fourier series expansion, we have a discrete sum of Hilbert spaces: ⊕ L 2 (S 1 ) C einθ . n∈Z

This is regarded as the irreducible decomposition of the regular representation of the compact abelian Lie group S 1 on L 2 (S 1 ). Here we have identified S1 with Z by einθ ↔ n. We note that there is no discrete spectrum in (1), while there is no continuous spectrum in (2). In particular, L 2 (S 1 ) is S 1 -admissible, as already mentioned in §2.4.2 in connection with the Peter–Weyl theorem. 3.1.4

Branching problems

Let π be an irreducible unitary representation of a group G, and let G be its subgroup. By a branching law, we mean the irreducible decomposition of π when restricted to the subgroup G . Branching problems ask to find branching laws as explicitly as possible.

3.2

Unitary dual of S L(2, R)

Irreducible unitary representations of G := S L(2, R) were classified by Bargmann [2] in 1947. In this section, we recall an important family of them. 3.2.1

S L(2, R)-action on P1 C

The Riemann sphere P1 C = C ∪ {∞} splits into three orbits H+ , H− and S 1 under az + b the linear fractional transformation of S L(2, R), z → . See Figure 3.2.1 top cz + d of next page. We shall associate a family of irreducible unitary representations of S L(2, R) to these orbits in §3.2.2 and §3.2.3.


151

F IGURE 3.2.1.

3.2.2

Unitary principal series representations

First, we consider the closed orbit S 1 R ∪ {∞}. ab −1 For λ ∈ C such that Re λ = 1, and for g = ∈ G, we define cd πλ (g) : L (R) → L (R), 2

2

f → (πλ (g) f )(x) := |cx + d|

−λ

f

ax + b . cx + d

Then the following holds: Proposition 3.2.2. For any λ ∈ C such that Re λ = 1, (πλ , L 2 (R)) is an irreducible unitary representation of G = S L(2, R). Exercise. Prove Proposition 3.2.2. Hint 1) It is straightforward to see πλ (g1 g2 ) = πλ (g1 )πλ (g2 )

(g1 , g2 ∈ G),

%πλ (g) f % L 2 (R) = % f % L 2 (R)

(g ∈ G).

2) Let K = S O(2). For irreducibility, it is enough to verify the following two claims: a) Any closed G-invariant subspace W of L 2 (R) contains a nonzero K invariant vector. λ b) Any K -invariant vector in L 2 (R) is a scalar multiple of |1 + x 2 |− 2 . 3.2.3

Holomorphic discrete series representations

Next we construct a family of representations attached to the open orbit H+ (see Figure 3.2.1). Let O(H+ ) be the space of holomorphic functions on the upper half plane H+ . For an integer n ≥ 2, we define Vn+ := O(H+ ) ∩ L 2 (H+ , y n−2 d x d y).

152

T. Kobayashi

Then it turns out that Vn+ is a nonzero closed subspace of the Hilbert space L 2 (H+ , y n−2 dx d y), from which the Hilbert structure is induced. a b For g −1 = ∈ G, we define cd az + b πn+ (g) : Vn+ → Vn+ , f (z) → (cz + d)−n f . cz + d Then the following proposition holds: Proposition 3.2.3 (Holomorphic discrete series). Each (πn+ , Vn+ ) (n ≥ 2) is an irreducible unitary representation of G. Let us discuss this example in the following exercises: Exercise. Verify that πn+ (g) (g ∈ G) is a unitary operator on Vn+ by a direct computation. We shall return to the irreducibility of (πn+ , Vn+ ) in §3.3.4. cos θ − sin θ Exercise. Let kθ := , and f n (z) := (z + i)−n . Prove the following sin θ cos θ formula: πn+ (kθ ) f n = einθ f n . As we shall see in Proposition 3.3.3 (1), the K -types occurring in πn+ are χn , χn+2 , χn+4 , . . . . Hence, the vector f n ∈ Vn+ is called a minimal K -type vector of the representation (πn+ , Vn+ ). In §7.1, we shall construct a family of unitary representations attached to elliptic coadjoint orbits. The above construction of (πn+ , Vn+ ) is a simplest example in which the Dolbeault cohomology group turns up in the degree 0 because the elliptic coadjoint orbit is biholomorphic to a Stein manifold H+ in this case (see Theorem 7.1.4 for a general feature). Similar to (πn+ , Vn+ ), we can construct another family of irreducible unitary representations πn− (n = 2, 3, 4, . . . ) of G on the Hilbert space Vn− := O(H− ) ∩ L 2 (H− , |y|n−2 d x d y). The representation (πn− , Vn− ) is called the anti-holomorphic discrete series representation. 3.2.4

Restriction and proof for irreducibility

Given a representation π of G, how can one find finer properties of π such as irreducibility, Jordan–Hölder series, etc.? A naive (and sometimes powerful) approach is to take the restriction of π to a suitable subgroup H . For instance, the method of (g, K )-modules (see Section 4) is based on “discretely decomposable” restrictions to


153

a maximal compact subgroup K . Together with “transition coefficients” that describe the actions of p on g-modules, the method of (g, K )-modules provides an elementary and alternative proof of irreducibility of both πλ and πn± , simultaneously (e.g. [95, Chapter 2]; see also Howe and Tan [26] for some generalization to the Lorentz group O( p, q)). On the other hand, the restriction to noncompact subgroups is sometimes effective in studying representations of G. We shall return to this point in Section 8. For a better understanding, let us observe a number of branching laws of unitary representations with respect to both compact and noncompact subgroups in the case S L(2, R).

Branching laws of S L(2, R)

3.3 3.3.1

Subgroups of S L(2, R)

Let us consider three subgroups of G = S L(2, R): ⎧ ⎪ K := {kθ : θ ∈ R/2π Z} S 1 , ⎪ ⎪ ⎪ ) ⎪ ⎪ ⎪ 1b ⎪ ⎨ N := : b ∈ R R, 01 G ⊃ H := ⎪ ⎪

) ⎪ ⎪ ⎪ es 0 ⎪ ⎪ : s ∈ R R. ⎪ ⎩ A := 0 e−s For ξ ∈ R, we define a one-dimensional unitary representation of R by χξ : R → C× ,

x → ei xξ .

For n ∈ Z, χn is well defined as a unitary representation of S 1 R/2π Z. 3.3.2

Branching laws G ↓ K , A, N

Proposition 3.3.2. Fix λ ∈ C such that Re λ = 1. Then the branching laws of a principal series representation πλ of G = S L(2, R) to the subgroups K , N and A are given by the following: ⊕ * 1) πλ * K χ2n . n∈Z

* 2) πλ * N * 3) πλ * A

⊕

R ⊕ R

χξ dξ . 2χξ dξ .

Z, and N A R. Here we have used the identifications K

154

T. Kobayashi

Sketch of proof. In all three cases, the proof reduces to (Euclidean) harmonic analysis. Here are some more details. 1) We define a unitary map (up to scalar) Tλ by Tλ : L 2 (R) → L 2 (S 1 ),

* ψ **−λ + ψ, * . f → *cos * f tan 2 2

(3.3.2)

Then Tλ respects the K -actions as follows: Tλ (πλ (kϕ ) f )(θ ) = (Tλ f )(θ + 2ϕ). Thus, the branching law (1) follows from the (discrete) decomposition of L 2 (S 1 ) by the Fourier series expansion: ⊕ L 2 (S 1 ) C einθ , n∈Z

as given in Example 3.1.3 (2). * 1b 2) Since (πλ f )(x) = f (x − b), the restriction πλ * N is nothing but the 01 regular representation of R on L 2 (R). Hence the branching law (2) is given by the Fourier transform as in Example 3.1.3. 3) We claim that the multiplicity in the continuous spectrum is uniformly two. To see this, we consider the decomposition L 2 (R) L 2 (R+ ) ⊕ L 2 (R− ) − −−−→ L 2 (R) ⊕ L 2 (R) T+ +T−

where T+ (likewise, T− ) is defined by λ

T+ : L 2 (R+ ) → L 2 (R), f (x) → e 2 t f (et ). * Then the unitary representation (πλ * A , L 2 (R+ )) of A is unitarily equivalent to the regular representation of A R on L 2 (R) because s e 0 T+ (πλ f )(t) = (T+ f )(t − 2s). 0 e−s * Hence, the branching law πλ * A follows from the Plancherel formula for L 2 (R) as was given in Example 3.1.3 (1). 3.3.3

Restriction of holomorphic discrete series

Without a proof, we present branching laws (abstract Plancherel formulas) of holomorphic discrete series representations πn+ (n = 2, 3, 4, . . . ) of G = S L(2, R) with respect to its subgroups K , N and A:


155

Proposition 3.3.3. Fix n = 2, 3, 4, . . . . 1)

*

πn+ * K

∞ ⊕

χn+2k .

k=0

* 2) πn+ * N * 3) πn+ * A

⊕

R+ ⊕

R

χξ dξ . χξ dξ .

It is remarkable that the multiplicity is free in all of the above three cases in Proposition 3.3.3 (compare with the multiplicity 2 results in Proposition 3.3.2 (3)). This multiplicity-free result holds in more general branching laws of unitary highest weight representations (see [42]). 3.3.4

Irreducibility of πn+ (n = 2, 3, 4, . . . )

One of the traditional approaches to show the irreducibility of πn+ is to use (g, K )modules together with explicit transition coefficients of Lie algebra actions; (e.g., [2], [26], Chapter 2 [95]). Aside from this, we shall explain another approach based on the restriction to noncompact subgroups N and A. This is a small example that restrictions to noncompact subgroups are also useful in studying representations of the whole group. (See Subsection 8.1 for this line of research.) Suppose that W is a G-invariant closed subspace in Vn+ . In view of the branching law of the restriction to the subgroup N (Proposition 3.3.3 (2)), the representation (πn+ , W ) of N is unitarily equivalent to the direct integral ⊕ χξ dξ E

for some measurable set E in R+ , as a unitary representation of N R. Furthermore, since W is invariant also by the subgroup A, E must be stable under the dilation, namely, E is either the ∅ or R+ (up to a measure zero set). Hence, the invariant subspace W is either {0} or Vn+ . This shows that (πn+ , Vn+ ) is already irreducible as a representation of the subgroup AN .

3.4 ⊗-product representations of S L(2, R) 3.4.1

Tensor product representations

Tensor product representations are a special case of restrictions and their decompositions are a special case of branching laws. In fact, suppose π and π are unitary representations of G. Then the outer tensor product π π is a unitary representation of the direct product group G × G. Its restriction to the diagonally embedded subgroup G:

156

T. Kobayashi

G → G × G,

g → (g, g)

gives rise to the tensor product representation π ⊗ π of the group G. Let us consider the irreducible decomposition of the tensor product representations, as a special case of branching laws. 3.4.2

πλ ⊗ πλ (principal series)

Proposition 3.4.2. Let Re λ = Re λ = 1. Then, the tensor product representation of two (unitary) principal series representations πλ and πλ decomposes into irreducibles of G = S L(2, R) as follows: πλ ⊗ πλ

⊕ Re ν=1 Im ν≥0

2πν dν +

∞ ⊕

+ − (π2n + π2n ).

n=1

A distinguishing feature here is that both continuous and discrete spectrum occur. Discrete spectrum occurs with multiplicity free, while continuous spectrum with multiplicity two. Sketch of proof. Unlike the branching laws in §3.3, we shall use noncommutative harmonic analysis. That is, the decomposition of the tensor product representation reduces to the Plancherel formula for the hyperboloid. To be more precise, we divide the proof into four steps: Step 1. The outer tensor product πλ πλ is realized on L 2 (R) L 2 (R2 ), L 2 (R)⊗ or equivalently, on L 2 (T2 ) via the intertwining operator Tλ ⊗ Tλ (see (3.3.2)). Step 2. Consider the hyperboloid of one sheet: X := {(x, y, z) ∈ R3 : x 2 + y 2 − z 2 = 1}. We identify X with the set of matrices: z x+y {B = : Trace B = 0, det B = 1}. x − y −z Then, G = S L(2, R) acts on X by X → X, B → g Bg −1 . Step 3. Embed the G-space X into an open dense subset of the (G × G)-space T2 so that it is equivariant with respect to the diagonal homomorphism G → G × G. (This is a conformal embedding with respect to natural pseudo-Riemannian metrics.)


157

Step 4. The pullback ι∗ followed by a certain twisting (depending on λ and λ ) sends L 2 (T2 ) onto L 2 (X ). The Laplace–Beltrami operator on X (a wave operator) commutes with the action of G (not by G × G), and its spectral decomposition gives rise to the Plancherel formula for L 2 (X ), or equivalently the branching law πλ ⊗ πλ . For advanced readers who are already familiar with the basic theory of representations of semisimple Lie groups, it would be easier to understand some of the above steps by the (abstract) Mackey theory. For example, the above embedding ι : X → T2 may be written as X G/M A G/(P ∩ P) → (G × G)/(P × P), parabolic subgroups of G such that P ∩ P = M A := where P and P are opposite a 0 : a ∈ R× . 0 a −1 A remaining part is to prove that the irreducible decomposition is essentially independent of λ and λ (see [45]). See, for example, [14, 81] for the Plancherel formula for the hyperboloid O( p, q)/O( p − 1, q) (the above case is essentially the same when ( p, q) = (2, 1)), and chapters of van den Ban, Delorme and Schlichtkrull in [3] for the more general case (reductive symmetric spaces). 3.4.3

πm+ ⊗ πn+ (holomorphic discrete series)

Proposition 3.4.3. Let m, n ≥ 2. Then πm+

⊗ πn+

∞ ⊕

+ πm+n+2 j

j=0

A distinguishing feature here is that there is no continuous spectrum (i.e. “discretely decomposable restriction”), even though it is a branching law with respect to a noncompact subgroup (i.e., diagonally embedded G). Sketch of proof. Realize πm+ ⊗ πn+ as holomorphic functions of two variables on H+ × H+ . Take their restrictions to the diagonally embedded submanifold

158

T. Kobayashi

ι : H+ → H+ × H+ . + Then the “bottom” representation πm+n (see irreducible summands in Proposition + 3.4.3) arises. Other representations πm+n+2 j ( j = 1, 2, . . . ) are obtained by taking normal derivatives with respect to the embedding ι.

4 (g, K )-modules and infinitesimal discrete decomposition 4.1

Category of (g, K )-modules

Section 4 starts with a brief summary of basic results on (g, K )-modules. Advanced readers can skip Subsection 4.1, and go directly to Subsection 4.2 where the concept of infinitesimal discrete decomposition is introduced. This concept is an algebraic analog of the property “having no continuous spectrum”, and is powerful in the algebraic study of admissible restrictions of unitary representations. 4.1.1

K -finite vectors

Let G be a reductive Lie group with a maximal compact subgroup K . Suppose (π, H) is a (K -)admissible representation of G on a Fréchet space. (See §2.4.6 and the remark there for the definition.) We define a subset of H by H K := {v ∈ H : dimC "K · v# < ∞}. Here "K ·v# denotes the complex vector space spanned by {π(k)v : k ∈ K }. Elements in H K are called K -finite vectors. Then it turns out that H K is a dense subspace of H, and decomposes into an algebraic direct sum of irreducible K -modules: HK (σ,Vσ )∈ K

4.1.2

Hom K (σ, π| K ) ⊗ Vσ

(algebraic direct sum).

Underlying (g, K )-modules

Retain the setting as before. Suppose (π, H) is a continuous representation of G on a Fréchet space H. If π | K is K -admissible, then the limit π(et X )v − v t→0 t

dπ(X )v := lim

exists for v ∈ H K and X ∈ g, and dπ(X )v is again an element of H K . Thus, g and K act on H K . Definition 4.1.2. With the action of g and K , H K is called the underlying (g, K )module of (π, H).


159

To axiomize “abstract (g, K )-modules”, we pin down the following three properties of H K (here, we omit writing π or dπ ): k · X · k −1 · v = Ad(k)X · v (X ∈ g, k ∈ K ), dimC "K · v# < ∞, d ** et X · v − v Xv = * (X ∈ k). dt t=0 t

(4.1.2)(a) (4.1.2)(b) (4.1.2)(c)

(Of course, (4.1.2)(c) holds for X ∈ g in the above setting.) 4.1.3

(g, K )-modules

Now we forget (continuous) representations of a group G and consider only the action of g and K . Definition (Lepowsky). We say W is a (g, K )-module if W is a g-module and at the same time a K -module satisfying the axioms (4.1.2)(a), (b) and (c). The point here is that no topology is specified. Nevertheless, many of the fundamental properties of a continuous representation are preserved when passing to the underlying (g, K )-module. For example, irreducibility (or more generally, Jordan– Hölder series) of a continuous representation is reduced to that of (g, K )-modules, as the following theorem indicates: Theorem 4.1.3 (Harish-Chandra [21]). Let (π, H) be a (K -)admissible continuous representation of G on a Fréchet space H. Then there is a lattice isomorphism between { closed G-invariant subspaces of H} and {g-invariant subspaces of H K }. The correspondence is given by V −→ V K := V ∩ H K , V K −→ V K (closure in H). In particular, (π, H) is irreducible if and only if its underlying (g, K )-module is irreducible. 4.1.4

Infinitesimally unitary representations

Unitary representations form an important class of continuous representations. Unitarity can be also studied algebraically by using (g, K )-modules. Let us recall some known basic results in this direction. A unitary representation of G gives rise to a representation of g by essentially skew-adjoint operators (Segal and Mautner; see Vergne [92] for references). Conversely, let us start with a representation of g having this property:

160

T. Kobayashi

Definition. A (g, K )-module W is infinitesimally unitary if it admits a positive definite invariant Hermitian form. Here by “invariant” we mean the following two conditions: for any u, v ∈ W , (X u, v) + (u, X v) = 0 (k · u, k · v) = (u, v)

(X ∈ g), (k ∈ K ).

The point of the following theorem is that analytic objects (unitary representations of G) can be studied by algebraic objects (their underlying (g, K )-modules). Theorem 4.1.4. 1) Any irreducible infinitesimally unitary (g, K )-module is the underlying (g, K )-modules of some irreducible unitary representation of G on a Hilbert space. 2) Two irreducible unitary representations of G on Hilbert spaces are unitarily equivalent if and only if their underlying (g, K )-modules are isomorphic as (g, K )modules. 4.1.5

Scope

Our interest throughout this chapter focuses on the restriction of unitary representations. The above mentioned theorems suggest that we deal with restrictions of unitary representations using the algebraic methods of (g, K )-modules. We shall see that this idea works quite successfully for admissible restrictions. Along this line, we shall introduce the concept of infinitesimally discretely decomposable restrictions in §4.2. 4.1.6

For further reading

See [95, Chapter 1], [104, Chapter 3], and [105, Chapter 4] for §4.1.

4.2

Infinitesimal discrete decomposition

The aim of Section 4 is to establish an algebraic formulation of the condition “having no continuous spectrum”. We are ready to explain this notion by means of (g, K )modules with some background of motivations. 4.2.1

Wiener subspace

We begin with an observation in the opposite extremal case — “having no discrete spectrum”. Observation 4.2.1. There is no discrete spectrum in the Plancherel formula for L 2 (R). Equivalently, there is no nonzero closed R-irreducible subspace in L 2 (R).


161

An easy proof for this is given simply by observing ei xξ ∈ / L 2 (R)

for any ξ ∈ R.

A less easy proof is to deduce from the following claim: Claim. For any nonzero closed R-invariant subspace W in L 2 (R), there exists an infinite decreasing sequence {W j } of closed R-invariant subspaces: W W1 W2 · · · . Exercise. Prove that there is no discrete spectrum in the Plancherel formula for L 2 (R) by using the above claim. Sketch of the proof of Claim. Let W be a closed R-invariant subspace in L 2 (R) (a Wiener subspace). Then one can find a measurable subset E of R such that W = F(L 2 (E)), that is, the image of the Fourier transform F of square integrable functions supported on E. Then take a decreasing sequence of measurable sets E E1 E2 · · · and put W j := F(L 2 (E j )). This is what we wanted. 4.2.2

Discretely decomposable modules

Let us consider an opposite extremal case, namely, the property “having no continuous spectrum”. We shall introduce a notion of this nature for representations of Lie algebras as follows. Definition 4.2.2. Let g be a Lie algebra and let X be a g -module. We say X is discretely decomposable as a g -module if there is an increasing sequence {X j } of g -modules satisfying both (1) and (2): (1) X =

∞

X j.

j=0

(2) X j is of finite length as a g -module for any j. Here we note that X j is not necessarily finite dimensional. 4.2.3

Infinitesimally discretely decomposable representations

Let us turn to representations of Lie groups. Suppose G ⊃ G is a pair of reductive Lie groups with maximal compact subgroups K ⊃ K , respectively. Let (π, H) be a (K -)admissible representation of G. Definition 4.2.3. The restriction π|G is infinitesimally discretely decomposable if the underlying (g, K )-module (π K , H K ) is discretely decomposable as a g -module (Definition 4.2.2).

162

T. Kobayashi

4.2.4

Examples

1) It is always the case √ if G is compact, especially if G = {e}. 2) Let πλ (λ ∈ 1 + −1R) be a principal * series representation of G = S L(2, R) * is infinitesimally discretely decompos(see §3.3.2). Then, the restriction π λ K * * able, while the restriction πλ A is not infinitesimally discretely decomposable. See also explicit branching laws given in Proposition 3.3.2.

4.2.5

Unitary case

So far, we have not assumed the unitarity of π . The terminology “discretely decomposable” fits well if π is unitary, as seen in the following theorem. Theorem 4.2.5. Let G ⊃ G be a pair of reductive Lie groups with maximal compact Then the following three subgroups K ⊃ K , respectively. Suppose (π, H) ∈ G. conditions on the triple (G, G , π) are equivalent: i) The restriction π|G is infinitesimally discretely decomposable. ii) The underlying (g, K )-module (π K , H K ) decomposes into an algebraic direct sum of irreducible (g , K )-modules: πK

n π (Y )Y

(algebraic direct sum),

Y

where Y runs over all irreducible (g , K )-modules and we have defined n π (Y ) := dim Homg ,K (Y, H K ) ∈ N ∪ {∞}.

(4.2.5)

iii) There exists an irreducible (g , K )-module Y such that n π (Y ) = 0. Sketch of proof. (i) ⇒ (iii) : Obvious. (i) ⇒ (ii) : Use the assumption that π is unitary. (iii) ⇒ (i) : Use the assumption that π is irreducible. (ii) ⇒ (i): If X is decomposed into the algebraic direct sum of irreducible j ∞ (g , K )-modules, say Yi , then we put X j := Yi ( j = 0, 1, 2, . . . ). i=0

i=0

We end this section with two important theorems without proof (their proof is not very difficult).


4.2.6

163

Infinitesimal discrete decomposability ⇒ discrete decomposability

, we define and σ ∈ G For π ∈ G m π (σ ) := dim HomG (σ, π|G ), the dimension of continuous G -intertwining operators. Then, in general, the following inequality holds: n π (σ K ) ≤ m π (σ ). This inequality becomes an equality if the restriction π |G is infinitesimally discretely decomposable. More precisely, we have: Theorem 4.2.6. In the setting of Theorem 4.2.5, if one of (therefore, any of) the three equivalent conditions is satisfied, then the restriction π |G is discretely decomposable (Definition 2.4.1), that is, we have an equivalence of unitary representations of G : ⊕ π|G m π (σ )σ (discrete Hilbert sum). σ ∈ G

Furthermore, the above multiplicity m π (σ ) coincides with the algebraic multiplicity n π (σ K ) given in (4.2.5). It remains an open problem (see [48, Conjecture D]) whether discrete decomposability implies infinitesimal discrete decomposability. 4.2.7

K -admissibility ⇒ infinitesimally discrete decomposability

Theorem 4.2.7. Retain the setting of Theorem 4.2.5. If π is K -admissible, then the restriction π |G is infinitesimally discretely decomposable. 4.2.8

For further reading

Materials of §4.2 are taken from the author’s papers [44] and [48]. See also [47] and [49] for related topics.

5 Algebraic theory of discretely decomposable restrictions The goal of this section is to introduce associated varieties to the study of infinitesimally discretely decomposable restrictions.

164

5.1

T. Kobayashi

Associated varieties

Associated varieties give a coarse approximation of modules of Lie algebras. This subsection summarizes quickly some known results on associated varieties (see Vogan’s treatise [98] for more details). 5.1.1

Graded modules

Let V be a finite-dimensional vector space over C. We use the following notation: V ∗ : the dual vector space of V. ∞ S k (V ) : the symmetric algebra of V. S(V ) = k=0 k -

Sk (V ) :=

S j (V ).

j=0

Let M =

∞ -

Mk be a finitely generated S(V )-module. We say M is a graded S(V )-

k=0

module if S i (V )M j ⊂ Mi+ j

for any i, j.

The annihilator Ann S(V ) (M) is an ideal of S(V ) defined by Ann S(V ) (M) := { f ∈ S(V ) : f · u = 0

for any u ∈ M}.

Then, Ann S(V ) (M) is a homogeneous ideal, namely, Ann S(V ) (M) =

∞

(Ann S(V ) (M) ∩ S k (V )),

k=0

and thus, Supp S(V ) (M) := {λ ∈ V ∗ : f (λ) = 0

for any f ∈ Ann S(V ) (M)}

is a closed cone in V ∗ . Hence, we have defined a functor {graded S(V )-modules} ; {closed cones in V ∗ } M → Supp S(V ) (M). 5.1.2

Associated varieties of g-modules

Let gC be a Lie algebra over C. For each integer n ≥ 0, we define a subspace Un (gC ) of the enveloping algebra U (gC ) of the Lie algebra gC by Un (gC ) := C-span{Y1 · · · Yk ∈ U (gC ) : Y1 , · · · , Yk ∈ gC , k ≤ n}.


165

It follows from the definition that U (gC ) =

∞ n=0

Un (gC ),

C = U0 (gC ) ⊂ U1 (gC ) ⊂ U2 (gC ) ⊂ · · · . Then the graded ring ∞

gr U (gC ) :=

Uk (gC )/Uk−1 (gC )

k=0

is isomorphic to the symmetric algebra ∞

S(gC ) =

S k (gC )

k=0

by the Poincaré–Birkhoff–Witt theorem. The point here is that the Lie algebra structure on gC is forgotten in the graded ring gr U (gC ) S(gC ). Suppose X is a finitely generated gC -module. We fix its generators v1 , . . . , vm ∈ X , and define a filtration {X j } of X by X j :=

m

U j (gC )vi .

i=1

Then the graded module gr X :=

∞ -

X j / X j−1 becomes naturally a finitely gen-

j=0

erated graded module of gr U (gC ) S(gC ). Thus, as in §5.1.1, we can define its support by VgC (X ) := Supp S(gC ) (gr X ). It is known that the variety VgC (X ) is independent of the choice of generators v1 , . . . , vm of X . Definition 5.1.2. We say VgC (X ) is the associated variety of a gC -module X . Its complex dimension is called the Gelfand–Kirillov dimension, denote by Dim(X ). By definition, the associated variety VgC (X ) is a closed cone in g∗C . For a reductive Lie algebra, we shall identify g∗C with gC , and thus regard VgC (X ) as a subset of gC . 5.1.3

Associated varieties of G-representations

So far we have considered a representation of a Lie algebra. Now we consider the case where X comes from a continuous representation of a reductive Lie group G as the underlying (g, K )-module. The scheme is as follows:

166

T. Kobayashi

(π, H) : an admissible representation of G of finite length, ⇓ (π K , H K ) : its underlying (g, K )-module, ⇓ HK : regarded as a U (gC )-module, ⇓ gr H K : its graded module (an S(gC )-module), ⇓ VgC (H K ) : the associated variety (a subset of g∗C ). 5.1.4

Nilpotent cone

Let g be the Lie algebra of a reductive Lie group G. We define the nilpotent cone by NgC := {H ∈ gC : ad(H ) is a nilpotent endomorphism}. Then NgC is an Ad(G C )-invariant closed cone in gC . Let gC = kC + pC be the complexification of a Cartan decomposition g = k + p. We take a connected complex Lie group K C with Lie algebra kC . then its associated variety Vg (H K ) is a K C Theorem 5.1.4 ([98]). If (π, H) ∈ G, C invariant closed subset of NpC := NgC ∩ pC . Theorem 5.1.4 holds in a more general setting where π is a (K -)admissible (nonunitary) representation of G of finite length. Sketch of proof. Here are key ingredients: 1) The center Z (gC ) of U (gC ) acts on H K as scalars ⇒ VgC (X ) ⊂ NgC . 2) K acts on H K ⇒ VgC (X ) is K C -invariant. 3) H K is locally K -finite ⇒ VgC (X ) ⊂ pC .

In [60], Kostant and Rallis studied the K C -action on the nilpotent cone NpC and proved that the number of K C -orbits on NpC is finite ([60, Theorem 2]; see also [8, 89]). Therefore, there are only finitely many possibilities of associated varieties VgC (H K ) for admissible representations (π, H) of G. As an illustrative example, we shall give an explicit combinatorial description of all K C -orbits on NpC in §5.2 for the G = U (2, 2) case.

5.2

Restrictions and associated varieties

This subsection presents the behavior of associated varieties with respect to infinitesimally discretely decomposable restrictions.


5.2.1

167

Associated varieties of irreducible summands

Let g be a reductive Lie algebra and g its subalgebra which is reductive in g. This is the case if g ⊂ g ⊂ gl(n, R) are both stable under the Cartan involution X → −tX . We write prg→g : g∗C → (gC )∗ for the natural projection dual to gC → gC . Suppose X is an irreducible g-module, and Y is an irreducible g -module. Then, we can define their associated varieties in g∗C and (gC )∗ , respectively. Let us compare them in the following diagram: VgC (X ) ⊂ ⏐g∗C ⏐ prg→g Vg (Y ) ⊂ (gC )∗ C

Theorem 5.2.1. If Homg (Y, X ) = {0}, then prg→g (VgC (X )) ⊂ Vg (Y ). C

(5.2.1)

Sketch of proof. In order to compare two associated varieties VgC (X ) and Vg (Y ), we C shall take a double filtration as follows: First take an ad(g )-invariant complementary subspace q of g in g: g = g ⊕ q. Choose a finite-dimensional vector space F ⊂ Y , and we define a subspace of X by ⎧ ⎫ A1 , · · · , A p ∈ g ( p ≤ i), ⎬ ⎨ X i j := C-span A1 · · · A p B1 · · · Bq v : B1 , · · · , Bq ∈ g (q ≤ j), . ⎩ ⎭ v∈F Then

-

1 i+ j≤N 2 X 0, j j

) is a filtration of X ,

X i, j N

is a filtration of Y ,

through which we can define and compare the associated varieties VgC (X ) and Vg (Y ). This gives rise to (5.2.1). C

Remark 5.1. The above approach based on double filtration is taken from [44, Theorem 3.1]; Jantzen [30, page 119] gave an alternative proof to Theorem 5.2.1.

168

T. Kobayashi

5.2.2

G : compact case

Let us apply Theorem 5.2.1 to a very special case, namely, G = K = K (a maximal compact subgroup). Obviously, there exists a (finite-dimensional) irreducible representation Y of K such that Homk(Y, X ) = {0}. Since Y is finite dimensional, therefore VkC (Y ) = {0}. Then Theorem 5.2.1 means that prg→k(VgC (X )) = {0}, or equivalently, VgC (X ) ⊂ pC

(5.2.2)

if we identify g∗C with gC . This inclusion (5.2.2) is nothing but the one given in Theorem 5.1.4. Therefore, Theorem 5.2.1 may be regarded as a generalization of Theorem 5.1.4. 5.2.3

Criterion for infinitesimally discretely decomposable restrictions

For a noncompact G , Theorem 5.2.1 leads us to a useful criterion for infinitesimal discrete decomposability by means of associated varieties. and G ⊃ G be a pair of reCorollary 5.2.3. ([44, Corollary 3.4]). Let (π, H) ∈ G, ductive Lie groups. If the restriction π|G is infinitesimally discretely decomposable, then prg→g (VgC (H K )) ⊂ Ng . C

Here Ng is the nilpotent cone of gC . C

Proof. The corollary readily follows from Theorem 5.2.1 and Theorem 5.1.4.

The converse statement also holds if (G, G ) is a reductive symmetric pair. That is, if prg→g (VgC (H K )) is contained in the nilpotent cone Ng , then the restricC tion π |G is infinitesimally discretely decomposable. This result was formulated and proved in [44] in the case where π K is of the form Aq(λ) (e.g. discrete series representations). A general case was conjectured in [48, Conjecture B], and Huang and Vogan have announced an affirmative solution to it. See Remark 8.1 after Theorem 8.2.2 for further illuminating examples of associated varieties regarding the theta correspondence for reductive dual pairs.

5.3

Examples

In this subsection, we examine Corollary 5.2.3 for the pair of reductive Lie groups (G, G i ) (i = 1, 2, 3):


169

The goal of this subsection is to classify all K C -orbits in the nilpotent variety NpC corresponding to the conditions in Corollary 5.2.3 for infinitesimally discretely decomposable restrictions with respect to the subgroups G i (i = 1, 2, 3). (The case i = 3 is trivial.) 5.3.1

Strategy

We divide into two steps: Step 1. Classify K C -orbits on NpC (§5.3.2). Step 2. List all K C -orbits O on NpC such that prg→gi (O) ⊂ NpiC (§5.3.4, §5.3.5, §5.3.6). K C -orbits on NpC

5.3.2

For G = U (2, 2), let us write down explicitly all K C -orbits on NgC by using elementary linear algebra. Relation to representation theory will be discussed in §5.3.3. We realize in matrices the group G = U (2, 2) as {g ∈ G L(4, C) : t g I2,2 g = I2,2 }, where I2,2 := diag(1, 1, −1, −1). We shall identify g1 O K C G L(2, C) × G L(2, C), ↔ (g1 , g2 ), Og2 O A pC M(2, C) ⊕ M(2, C), ↔ (A, B). B O Then the adjoint action of K C on pC is given by (A, B) → (g1 Ag2−1 , g2 Bg1−1 )

(5.3.2)

for (g1 , g2 ) ∈ G L(2, C) × G L(2, C). Furthermore, the nilpotent cone NpC = NgC ∩ pC is given by {(A, B) ∈ M(2, C) ⊕ M(2, C) : Both AB and B A are nilpotent matrices}. We define a subset (possibly an empty set) of NpC for 0 ≤ i, j, k, l ≤ 2 by Oi j := {(A, B) ∈ NpC : rank A = i, rank B = j, kl

rank AB = k, rank B A = l}.

Since rank A, rank B, rank AB, and rank B A are invariants of the K C -action (5.3.2), all Oi j are K C -invariant sets. Furthermore, the following proposition holds: kl

170

T. Kobayashi

Proposition 5.3.2. 1) Each Oi j is a single K C -orbit if it is not empty. kl 2) The nilpotent cone NpC splits into 10 K C -orbits. These orbits together with their dimensions are listed in the following diagram:

F IGURE 5.3.2.

O Here | stands for the closure relation O ⊃ O . O There is also known a combinatorial description of nilpotent orbits of classical reductive Lie groups. For example, nilpotent orbits in u( p, q) are parameterized by signed Young diagrams ([8, Theorem 9.3.3]). For the convenience of readers, let us illustrate briefly it by the above example. A signed Young diagram of signature ( p, q) is a Young diagram in which every box is labeled with a + or − sign with the following two properties: 1) The number of boxes labeled + is p, while that of boxes labeled − is q. 2) Signs alternate across rows (they do not need to alternate down columns). Two such signed diagrams are regarded as equivalent if and only if one can be obtained by interchanging rows of equal length. For a real reductive Lie group G, the number of K C -orbits on NpC is finite ([60]). Furthermore, there is a bijection (the Kostant–Sekiguchi correspondence [89]) {K C -orbits on NpC } ↔ {G-orbits on Ng}. Therefore, K C -orbits on NpC are also parameterized by signed Young diagrams. For example, Figure 5.3.2 may be written as


171

F IGURE 5.3.3.

5.3.3

Interpretations from representation theory

We have given a combinatorial description of all K C -orbits on NpC . The goal of this subsection is to provide (without proof) a list of the orbits which are realized as the associated varieties of specific representations of G = U (2, 2). 1) Oi j is the associated variety of some highest (or lowest) weight module of G if kl and only if i = 0 or j = 0, namely, Oi j corresponds to one of the circled points kl (Fig. 5.3.4 below). 2) Oi j is the associated variety of some (Harish-Chandra) discrete series representakl

tions of G (i.e. irreducible unitary representations that can be realized in L 2 (G)) if and only if Oi j corresponds to one of the circled points (Fig. 5.3.5 below). kl

To see this, we recall the work of Beilinson and Bernstein which realizes irreducible (g, K )-modules with regular infinitesimal characters by using D-modules on the flag variety of G C G L(4, C). The D-module which corresponds to a discrete series representation is supported on a closed K C -orbit W, and its associated variety is given by the image of the moment map of the conormal bundle of W ([4]). A combinatorial description of this map for some classical groups may be found in [109]. For the case of G = U (2, 2), compare [41, Example 3.7] for the list of all K C -orbits on the flag variety of G C with [44, §7] (a more detailed explanation of Figure 5.3.2) for those of the image of the moment map of the conormal bundles.

172

T. Kobayashi

F IGURE 5.3.4.

F IGURE 5.3.5.

5.3.4

Case (G, G 1 ) = (U (2, 2), Sp(1, 1)) (essentially, (S O(4, 2), S O(4, 1)))

The goal of this subsection is to classify the K C -orbits Oi j on NpC such that kl prg→g1 (Oi j ) is contained in the nilpotent cone of g1C , in the case G 1 = Sp(1, 1) kl Spin(4, 1). Let g1 = k1 + p1 be a Cartan decomposition of g1 , and we shall identify p1C with M(2, C). The projection prg→g1 : gC → g1C when restricted to pC M(2, C) ⊕ M(2, C) is given by prg→g1 : M(2, C) ⊕ M(2, C) → M(2, C), (A, B) → A + τ B, ab −d b where τ := . cd c −a Then we have


173

Proposition 5.3.4. prg→g1 (Oi j ) ⊂ Np1C if and only if 0 ≤ i, j, k, l ≤ 1, namely, kl Oi j corresponds to one of the circled points (Fig. 5.3.6 below): kl

F IGURE 5.3.6.

5.3.5

Case (G, G 2 ) = (U (2, 2), U (2, 1) × U (1))

We write down the result without proof: Proposition 5.3.5. Let Oi j be a K C -orbit on NpC . Then, prg→g2 (Oi j ) ⊂ Np2C if kl kl and only if l = 0, namely, Oi j corresponds to one of the circled points (Fig. 5.3.7): kl

F IGURE 5.3.7.

174

T. Kobayashi

5.3.6

Case (G, G 3 ) = (U (2, 2), U (2) × U (2))

Since G 3 is compact, we have obviously Proposition 5.3.6. prg→g3 (Oi j ) ⊂ Np3C for any K C -orbit Oi j in NpC , namely Oi j kl kl kl is one of the circled points below (Fig. 5.3.8):

F IGURE 5.3.8.

It would be an interesting exercise to compare the above circled points with those in §5.3.3 (meaning from representation theory).

6 Admissible restriction and microlocal analysis Section 6 tries to explain how an idea coming from microlocal analysis leads to a criterion for admissibility of restrictions of unitary representations. The main idea is to look at the singularity spectrum (or the wave front set) of the hyperfunction (or distribution) character of an irreducible representation. Its flavor is explained in §6.1. Basic concepts such as asymptotic K -support are introduced in §6.2, and they will play a crucial role in the main results of §6.3. In §6.4, we give a sketch of the idea of an alternative (new) proof of Theorem 6.3.3 using symplectic geometry.

6.1

Hyperfunction characters

This subsection explains very briefly reasons why we need distributions or hyperfunctions in defining characters of infinite-dimensional representations and how


175

they work. The exposition of Schwartz’s distribution characters (§6.1.1- 6.1.3) is influenced by Atiyah’s article [1], and the exposition for Sato’s hyperfunction characters (§6.1.4–6.1.6) is intended to explain the idea of our main applications to restricting unitary representations in the spirit of the papers [33, 43] of Kashiwara–Vergne and the author. 6.1.1

Finite group

Let G be a finite group. Consider the (left) regular representation π of G on L 2 (G)

Cg. g∈G

In light of a matrix expression of π(g) with respect to the basis on the right-hand side, the character χπ of π is given by the following form: #G (g = e), Trace π(g) = 0 (g = e). 6.1.2

Dirac’s delta function

How does the character formula look like as the order #G of G goes to infinity? The character will not be a usual function if #G = ∞. We shall take G = S 1 as an example of an infinite group below, and consider the character of the regular representation π. Then we shall interpret Trace π as the Dirac delta function both from Schwartz’s distributions and Sato’s hyperfunctions. 6.1.3

Schwartz’s distributions

Let G = S 1 R/2π Z. Let π be the regular representation π of S 1 on L 2 (S 1 ). From the viewpoint of Schwartz’s distributions, the character χπ = Trace π is essentially Dirac’s delta function: Lemma 6.1.3. Trace π(θ )dθ = 2π δ(θ ) Sketch of proof. Take a test function f ∈ C ∞ (S 1 ), and we develop f into the Fourier series: f (θ ) =

1 an ( f )einθ , 2π n∈Z

where an ( f ) := S 1 f (θ )e−inθ dθ. as the proof of the fact that We do not go into technical details here2 (such 1 )), but present only a formal comf (θ )π(θ )dθ is a trace class operator on L (S S1 putation

176

T. Kobayashi

" f, Trace π(θ )dθ# =

f (θ ) Trace π(θ )dθ = f (θ )e−imθ dθ S1

1 m∈Z S

=

am ( f )

m∈Z

= 2π f (0) = " f, 2π δ(θ )#.

This is what we wanted to verify. 6.1.4

Sato’s hyperfunctions

Let us give another interpretation of the character of the regular representation of G = S 1 . We regard S 1 as the unit circle in C. From the viewpoint of Sato’s hyperfunctions [32, 83], the character χπ is given as boundary values of holomorphic functions: Lemma 6.1.4. Trace π(θ ) = lim

z→eiθ |z|↑1

1 −1 + lim . 1 − z z→eiθ 1 − z |z|↓1

Sketch of proof (heuristic argument). Formally, we may write Trace π(θ ) = einθ = zn , n∈Z

n∈Z

where we put z = eiθ . Let us “compute” this infinite sum as follows: zn = zn + zn n∈Z

n≥0

=

(6.1.4)

n 1, there is no intersection of domains where the above formula makes sense in a usual way. However, it can be justified as boundary values of holomorphic functions, in the theory of hyperfunctions. This is Lemma 6.1.4. 6.1.5

Distributions or hyperfunctions

The distribution character in Lemma 6.1.3 is essentially the same as the hyperfunction character in Lemma 6.1.4. Cauchy’s integral formula bridges them.


177

To see this, we take an arbitrary real analytic test function F(z) ∈ A(S 1 ). Then F(z) extends holomorphically in the complex neighborhood of S 1 , say, 1 − 2ε < |z| < 1 + 2ε for some ε > 0. Then, 3 3 1 1 −1 1 F(z)dz + F(z)dz 2πi |z|=1−ε 1 − z 2πi |z|=1+ε 1 − z 3 1 F(z) = dz = F(1) (6.1.5) 2πi γ z − 1 where γ is a contour surrounding z = 1. We define a function f on R/2π Z by f (θ ) = eiθ F(eiθ ). We note F(1) = f (0). If z = eiθ , then F(z)dz = i f (θ )dθ. Hence, the formula (6.15) amounts to 1 2π

2π 0

( lim

z→eiθ |z|↑1

1 −1 + lim ) f (θ )dθ = f (0). iθ 1 − z z→e 1 − z

(6.1.6)

|z|↓1

Since any smooth test function on S 1 can be approximated by real analytic functions, the formula (6.16) holds for any f ∈ C ∞ (S 1 ). Hence we have shown the relation between Lemma 6.1.3 and Lemma 6.1.4. 6.1.6

Strategy

We recall that our main goal in this chapter is the restriction π |G in the setting where • •

π is an irreducible unitary representation of G, G is a reductive subgroup of G.

We are particularly interested in the (non-)existence of continuous spectrum in the irreducible decomposition of the restriction π |G . For this, our strategy is summarized as follows: 1) Restriction of a representation π to a subgroup G . ⇑ 2) Restriction of its character Trace π to a subgroup. ⇑ 3) Restriction of a holomorphic function to a complex submanifold. For (2), we shall find that the interpretation of a character as a hyperfunction fits well. Then (3) makes sense if the domain of holomorphy is large enough. In turn, the domain of holomorphy will be estimated by the invariants of the representation π, namely the asymptotic K -support AS K (π ), which we are going to explain in the next section.

178

6.2

T. Kobayashi

Asymptotic K -support

Associated to a representation π of a compact Lie group K , the asymptotic K support AS K (π ) is defined as a closed cone in a positive Weyl chamber. Here, we are mostly interested in the case where π is infinite dimensional (therefore, not irreducible as a K -module). The goal of this subsection is to introduce the definition of the asymptotic K -support and to explain how it works for the criterion of admissibility of the restriction of a unitary representation of a (noncompact) reductive Lie group G based on the ideas explained in §6.1. 6.2.1

Asymptotic cone

Let S be a subset of a real vector space R N . The asymptotic cone S∞ is an important notion in microlocal analysis (e.g. [32], Definition 2.4.3), which is a closed cone defined by S∞ := {y ∈ R N : there exists a sequence (yn , εn ) ∈ S × R>0 such that lim εn yn = y and lim εn = 0}. n→∞

Example 6.2.1. We illustrate the correspondence S ⇒ S∞ with two-dimensional examples (see Fig. 6.2.1).

Figure 6.2.1

n→∞


6.2.2

179

Cartan–Weyl highest weight theory

We review quickly a well-known fact about the finite-dimensional representations of compact Lie groups and fix notation as follows. Let K be a connected compact Lie group, and take a maximal torus T . We write + k and t for their √ Lie∗ algebras, respectively. Fix a positive root system (k, t), and as a write C+ √ (⊂ −1t ) for the corresponding closed Weyl chamber. We regard T ∗ lattice of −1t and put ∩ C+ . + := T We note that the asymptotic cone + ∞ is equal to C+ . For λ ∈ + , we shall denote by τλ an irreducible (finite-dimensional) representation of K whose highest weight is λ. Then Cartan–Weyl highest weight theory (for and + : a connected compact Lie group) establishes a bijection between K

τλ ↔ 6.2.3

√ −1t∗

∈

Λ + ⊂ C+ ⊂

∈

K

λ

Asymptotic K -support

For a representation π of K , we define the K -support of π by Supp K (π ) := {λ ∈ + : Hom K (τλ , π) = 0}. Later, π will be a representation of a noncompact reductive Lie group G when restricted to its maximal compact subgroup K . Thus, we have in mind the case where π is infinite dimensional (and therefore, not irreducible as a K -module). Its asymptotic cone AS K (π ) := Supp K (π )∞ is called the asymptotic K -support of π. We note that it is a closed cone contained in the closed Weyl chamber C+ , because Supp K (π ) ⊂ + and C+ = + ∞. The asymptotic K -support AS K (π ) was introduced by Kashiwara and Vergne [33] and has the following property (see [43, Proposition 2.7] for a rigorous statement): The smaller the asymptotic K -support AS K (π ), the larger the domain of K C in which the character Trace(π ) extends holomorphically. For further properties of the asymptotic K -support AS K (π ), see [33, 43, 53]. In particular, we mention: or more generally, π is a (K -)admissible repTheorem 6.2.3. [53] Suppose π ∈ G, resentation of G of finite length. (π is not necessarily unitary.) Then the asymptotic K -support AS K (π ) is a finite union of polytopes, namely, there exists a finite set {αi j } ⊂ C+ such that AS K (π ) =

l k=1

R≥0 - span{αk1 , αk2 , . . . , αkm k }.

180

T. Kobayashi

6.2.4

Examples from S L(2, R)

Let G = S L(2, R). We recall the notation in Section 3; in particular, we identify K with Z. Here is a list of Supp K (π ) and AS K (π ) for some typical representations π of G. π

Supp K (π )

AS K (π )

(1) 1

{0}

{0}

(2) πλ

2Z

R

{n, n + 2, n + 4, . . . }

R≥0

{−n, −n − 2, −n − 4, . . . }

R≤0

(3) (4)

πn+ πn−

Here 1 denotes the trivial one-dimensional representation, πλ a principal series representation, and πn± (n ≥ 2) a holomorphic or anti-holomorphic representation of G. Now let us recall §6.1.4 for a hyperfunction character. In the case (2), the character Trace(π ) = k∈2Z eikθ has a similar nature to the character of L 2 (S 1 ) computed in §6.1. In the cases (3) and (4), the asymptotic K -support is smaller, and the characters Trace(π ) = k∈n+2N e±ikθ converges in a larger complex domain, as we have seen in a similar property for each of the summands in (6.1.4).

6.3 6.3.1

Criterion for admissible restriction The closed cone C K (K )

Let K be a connected compact Lie group, and K its closed subgroup. Dual to the inclusion k ⊂ k of Lie algebras, we write prk→k : k∗ → (k )∗ for the projection, and (k )⊥ for the kernel of prk→k . We fix a positive definite and Ad(K )-invariant bilinear form on k, and regard t∗ as a subspace of k∗ . to the pair (K , K ) of compact Lie groups, we define a closed cone √Associated ∗ in −1t by √ C(K ) ≡ C K (K ) := C+ ∩ −1 Ad∗ (K )(k )⊥ . (6.3.1) 6.3.2

Symmetric pair

For a compact symmetric pair (K , K ), the closed cone C K (K ) takes a very simple form. Let us describe it explicitly. Suppose that (K , K ) is a symmetric pair defined by an involutive automorphism σ of K . As usual, the differential of σ will be denoted by the same letter. Taking a conjugation by K if necessary, we may and do assume that t and + (k, t) are chosen so that


181

−σ := t ∩ k−σ is a maximal abelian subspace of k−σ , 1) t + (k, t−σ ) := {λ|t−σ : λ ∈ + (k, t)} \ {0} is a positive system of the restricted 2) root system (k, t−σ ). √ We write (t−σ )∗+ (⊂ −1(t−σ )∗ ) for the corresponding dominant Weyl chamber. Then we have (see [23]).

Proposition 6.3.2. C K (K ) = (t−σ )∗+ . One can also give an alternative proof of Proposition 6.3.2 by using Theorem 6.4.3 below and a Cartan–Helgason theorem [105], §3.3.1. 6.3.3

Criterion for K -admissibility

We are ready to explain the main results of Section 6, namely, Theorem 6.3.3 and Theorem 6.3.4 regarding a criterion for admissible restrictions. Theorem 6.3.3 ([43, 53]). Let K ⊃ K be a pair of compact Lie groups, and X a K -module. Then the following two conditions are equivalent: 1) X is K -admissible. 2) C K (K ) ∩ AS K (π ) = {0}. Sketch of proof. Let us explain an idea of the proof (2) ⇒ (1). The K -character Trace(π | K ) of X is a distribution (or a hyperfunction) on K . The condition (2) implies that its wave front set (or its singularity spectrum) is transversal to the submanifold K . Then the restriction of this distribution (or hyperfunction) to K is well defined, and coincides with the K -character of X (see [43], Theorem 2.8 for details). 6.3.4

Sufficient condition for G -admissibility

Now, let us return our original problem, namely, the restriction to a noncompact reductive subgroup. and G ⊃ G be a pair of reductive Lie groups. Theorem 6.3.4. ([43]) Let π ∈ G, Take maximal compact subgroups K ⊃ K , respectively. If C K (K ) ∩ AS K (π ) = {0}, then the restriction π |G is G -admissible, that is, π |G splits discretely: ⊕ π|G n π (τ )τ τ ∈G

. into irreducible representations of G with n π (τ ) < ∞ for any τ ∈ G

182

T. Kobayashi

6.3.5

Remark

The assumption of Theorem 6.3.4 is obviously fulfilled if C K (K ) = {0} or if AS K (π ) = {0}. What do these extremal cases mean? Here is the answer: 1) C K (K ) = {0} ⇔ K = K . In this case, Theorem 6.3.4 is nothing but Harish-Chandra’s admissibility theorem, as we explained in Section 2 (see Theorem 2.4.6). 2) AS K (π ) = {0} ⇔ dim π < ∞. In this case, the conclusion of Theorem 6.3.4 is nothing but the complete reducibility of a finite-dimensional unitary representation. Thus, the second case is more or less trivial. In this connection, one might be tempted to ask when AS K (π ) becomes the second smallest, namely, of the form R+ v generated by a single element v? It follows from the following result of Vogan [94] that this is always the case if π is a minimal representation of G (in the sense that its annihilator is the Joseph ideal). Theorem 6.3.5 (Vogan). Let π be a minimal representation of G. Then there exists a weight ν such that πK τmv+ν , m∈N

as K -modules, where v is the highest weight of p, and τmv+ν is the irreducible representation of K with highest weight mv + ν. In particular, AS K (π ) = R+ v. This reflects the fact that there are fairly rich examples of a discretely decomposable restriction of the minimal representation π of a reductive group G with respect to a noncompact reductive subgroup G (e.g. Howe [24] for G = M p(n, R), Kobayashi–Ørsted [57] for G = O( p, q), and Gross–Wallach [19] for some exceptional Lie groups G).

6.4 6.4.1

Application of symplectic geometry Hamiltonian action

Let (M, ω) be a symplectic manifold on which a compact Lie group K acts as symplectic automorphisms by τ : K × M → M. We write k → X(M), X → X M for the vector field induced from the action. The action τ is called Hamiltonian if there exists a map : M → k∗ such that d X = ι(X M )ω for all X ∈ k, where we put X (m) = "X, (m)#. We say is the momentum map. For a subset Y of M, The momentum set (Y ) is defined by √ (Y ) = −1(Y ) ∩ C+ .


6.4.2

183

Affine varieties

Let V be a complex Hermitian vector space. Assume that a compact Lie group K acts on V as a unitary representation. Then the action of K is also symplectic if we equip V with the symplectic form ωV (u, v) = − Im(u, v), as we saw the inclusive relation U (n) ⊂ Sp(n, R) in Section 1 (see Example 1.5.5). Then, V is Hamiltonian with the momentum map √ −1 ∗ : V → k , (v)(X ) = (X v, v), (X ∈ k). 2 We extend the K -action to a complex linear representation of K C . Suppose V is a K C -stable closed irreducible affine variety of V . We have naturally a representation of K on the space of regular functions C[V]. Then Supp K (C[V]) is a monoid, namely, there exists a finite number of elements in + , say λ1 , . . . , λk , such that Supp K (C[V]) = Z≥0 -span{λ1 , . . . , λk }.

(6.4.2)

The momentum set (V) is a “classical” analog of the set of highest weights Supp K (C). That is, the following theorem holds: Proposition 6.4.2. [88] (V) = R≥0 Supp K (C[V]). Together with (6.4.2), we have (V) = R≥0 -span{λ1 , . . . , λk } = AS K (Supp K (C[V])). 6.4.3

Cotangent bundle

Let M = T ∗ (K /K ), the cotangent bundle of the homogeneous space K /K . We define an equivalence relation on the direct product K × k⊥ by (k, λ) ∼ (kh, Ad∗ (h)−1 λ) for some h ∈ H , and write [k, λ] for its equivalence class. Then the set of equivalence classes, denoted by K × K k⊥ , becomes a K -equivariant homogeneous bundle over K /K , and is identified with T ∗ (K /K ). Then the symplectic manifold T ∗ (K /K ) is naturally a Hamiltonian with the momentum map : T ∗ (K /K ) → k∗ , [k, λ] → Ad∗ (k)λ. The momentum set (T ∗ (K /K )) equals to C K (K ), as follows from the definition (6.3.1). The closed cone C K (K ) can be realized as the asymptotic K -support of a certain induced representation of K . That is, we can prove: Theorem 6.4.3. [53] Let τ be an arbitrary finite-dimensional representation of K . Then, C K (K ) = AS K (Ind K K (τ )). In particular, C K (K ) = AS K (L 2 (K /K )).

(6.4.3)

184

T. Kobayashi

Here we note that the asymptotic K -support AS K (Ind K K (τ )) does not change whatever we take as the class of functions (algebraic, square integrable, hyperfunctions, . . . ) in the definition of the induced representation Ind K K (τ ).

7 Discretely decomposable restriction of Aq (λ) In the philosophy of the Kostant–Kirillov orbit method, there is an important family of irreducible unitary representations of a reductive Lie group G, “attached to” elliptic coadjoint orbits. Geometrically, they are realized in dense subspaces of Dolbeault cohomology groups of certain equivariant holomorphic line bundles. This is a vast generalization due to Langlands, Schmid, and others of the Borel–Weil–Bott theorem: from compact to noncompact; from finite dimensional to infinite dimensional. Algebraically, they are also expressed as Zuckerman’s derived functor modules Aq(λ) by using so-called cohomological parabolic induction. Among all of them in importantance is the unitarizability theorem of these modules under certain positivity conditions on the parameter, as proved by Vogan and Wallach. Analytically, some of them can be realized in L 2 -spaces on homogeneous spaces. For example, Harish-Chandra’s discrete series representations for group manifolds and Flensted-Jensen’s for symmetric spaces are such cases. Subsection 7.1 collects some of basic results about these representations from these three — geometric, algebraic, and analytic — aspects. Subsection 7.2 speculates about their restrictions to noncompact subgroups, and examines the results of Sections 5 and 6 for these modules.

7.1 7.1.1

Elliptic orbits and geometric quantization Elliptic orbits

Consider the adjoint action of a Lie group G on its Lie algebra g. For X ∈ g, we define the adjoint orbit O X by O X := Ad(G)X G/G X . Here, G X is the isotropy subgroup at X , given by {g ∈ G : Ad(g)X = X }. Definition 7.1.1. An element X ∈ g is elliptic if ad(X ) ∈ EndC (gC ) is diagonalizable and if all eigenvalues are purely imaginary. Then, O X is called an elliptic orbit. Example. If G is a compact Lie group, then any adjoint orbit is elliptic. Let g = k + p be a Cartan decomposition of the Lie algebra g of G. We fix a maximal abelian subspace t of k.


185

F IGURE 7.1. (Co)adjoint orbits of G = S L(2, R)

Any elliptic element is conjugate to an element in k under the adjoint action of G. Furthermore, any element of k is conjugate to an element in t under the adjoint action of K . Hence, for X ∈ g, we have the following equivalence: O X is an elliptic orbit ⇔ O X ∩ k = φ ⇔ O X ∩ t = φ. From now on, without loss of generality, we can and do take X ∈ t when we deal with an elliptic orbit. 7.1.2

Complex structure on an elliptic orbit

Every elliptic orbit O X carries a G-invariant complex structure. There are several choices of G-invariant complex structures on O X . We shall choose one in the following manner. (See, for √ example, [56] for further details.) First, we note that −1 ad(X ) ∈ End(gC ) is a semisimple √ transformation with all eigenvalues real. Then the eigenspace decomposition of −1 ad(X ) leads to the Gelfand–Naimark decomposition gC = u− + (g X )C + u+ .

(7.1.2)

√ Here u+ (respectively, u− ) is the direct sum of eigenspaces of −1 ad(X ) with positive (respectively, negative) eigenvalues. We define a parabolic subalgebra of gC by

186

T. Kobayashi

q := (g X )C + u+ . For simplicity, suppose G is a connected reductive Lie group contained in a (connected) complex Lie group G C with Lie algebra gC . Let Q be the parabolic subgroup of G C with Lie algebra q. We note that Q is a connected complex subgroup of G C . Then the key ingredients here are G ∩ Q = GX, g + q = gC . Hence the natural inclusion G ⊂ G C induces an open embedding of O X into the generalized flag variety G C /Q: O X G/G X → G C /Q. open Hence we can define a complex structure on the adjoint orbit O X from the one on G C /Q. Obviously, the action of G on O X is biholomorphic. For a further structure on O X , we note that O X contains another (smaller) generalized flag variety O XK := Ad(K )X K /K X , of which the complex dimension will be denoted by S. In summary, we have O XK

⊂ O X → G C /Q, open

closed

and in particular, Proposition 7.1.2. Any elliptic orbit O X carries a G-invariant complex structure through an open embedding into the generalized flag variety G C /Q. Furthermore, O X contains a compact complex submanifold O XK . 7.1.3

Elliptic coadjoint orbit

For a reductive Lie group G, adjoint orbits on g and coadjoint orbits on g∗ can be identified via a nondegenerate G-invariant bilinear form. For example, such a bilinear form is given by g × g → R, (X, Y ) → Trace(X Y ) if g is realized , R) such that t g√= g. √in gl(N ∗ Let λ ∈ −1g . We write X λ ∈ −1g for the corresponding element via the isomorphism √ √ −1g∗ −1g. √ We write X := − −1X λ ∈ g. Then isotropy subgroups of the adjoint action and the coadjoint action coincide — G X = G λ .


187

Assume that X is an elliptic element. With analogous notation as in §7.1.2, Proposition 7.1.2 states that the coadjoint orbit Oλ := Ad∗ (G) · λ G/G λ = G/G X Ad(G) · X =: O X carries a G-invariant complex structure. The Lie algebra √ of G λ is given by gλ := {X ∈ g : λ([X, Y ]) = 0 for all Y ∈ g}. We define ρλ ∈ −1g∗λ by ρλ (Y ) := Trace(ad(Y ) : u+ → u+ ), for Y ∈ gλ . We say λ is integral if the Lie algebra homomorphism λ + ρλ : gλ → C lifts to a character of G λ . For simplicity, we shall write Cλ+ρλ for the lifted character. Then Lλ := G ×G λ Cλ+ρλ → Oλ is a G-equivariant holomorphic line bundle over the coadjoint orbit Oλ . 7.1.4

Geometric quantization a` la Schmid–Wong

This subsection completes the following scheme of “geometric quantization” of an elliptic coadjoint orbit Oλ . √ λ ∈ −1g∗ an elliptic and integral element > ↓ Lλ → Oλ a G-equivariant holomorphic line bundle > ↓ H∂¯∗ (Oλ , Lλ ) a representation (λ) of G We have already explained the first step. Here is a summary of the second step: √ Theorem 7.1.4. Let λ ∈ −1g∗ be elliptic and integral. j

1) (topology) The Dolbeault cohomology group H∂¯ (Oλ , Lλ ) carries a Fréchet topology, on which G acts continuously. j 2) (vanishing theorem) H∂¯ (Oλ , Lλ ) = 0 if j = S. 3) (unitarizability) There is a dense subspace H in H∂¯S (Oλ , Lλ ) with which a Ginvariant Hilbert structure can be equipped. 4) (irreducibility) If λ is “sufficiently regular”, then the unitary representation of G on H is irreducible and nonzero. We shall denote by (λ) the unitary representation constructed in Theorem 7.1.4 (3). Here are some comments on and further introductions to Theorem 7.1.4.

188

T. Kobayashi

¯ 1) The nontrivial part of the statement (1) is that the range of the ∂-operator is closed with respect to the Fréchet topology on the space of (0, q)-forms. The difficulty arises from the fact that Oλ G/G λ is noncompact. This closed range problem was solved affirmatively by Schmid in the case G λ compact, and by H. Wong for general G λ early in 1990s [107]. 2) This vanishing result is an analog of Cartan’s Theorem for Stein manifolds. We note that Oλ is Stein if and only if S = 0. In this case, Oλ is biholomorphic to a Hermitian symmetric space of noncompact type, and the statement (2) asserts the vanishing of all cohomologies in higher degrees. The resulting representations in the 0th degree in this special case are highest weight representations. An opposite extremal case is when G is compact (see the next subsection §7.1.5). In this case, our choice of the complex structure on O X implies that the dual of Lλ is ample, and the statement (2) asserts that all the cohomologies vanish except for the top degree. 3) The unitarizability was conjectured by Zuckerman, and proved under a certain positivity condition on the parameter λ by Vogan [96] and Wallach independently [103] in the 1980s. See also a proof in the works of Knapp–Vogan [37] or Wallach [104]. In our formulation which is suitable for the orbit method, this positivity condition is automatically satisfied. 4) Vogan introduced the condition “good range” and a slightly weaker one “fair range”. The statement (4) of Theorem 7.1.4 holds if λ is in the good range ([96]). Special cases of Theorem 7.1.4 contain many interesting representations as we shall see in §7.1.5 ∼§7.1.7. 7.1.5

Borel–Weil–Bott theorem

If G is a compact Lie group, then any coadjoint orbit Oλ is elliptic and becomes a compact complex manifold (a generalized flag variety). Then by a theorem of j Kodaira–Serre, the Dolbeault cohomology groups H∂¯ (Oλ , Lλ ) are finite dimensional. The representations constructed in Theorem 7.1.4 are always irreducible, and exhaust all irreducible (finite dimensional, unitary) representations of G. This is known as the Borel–Weil–Bott construction. 7.1.6

Discrete series representations

Suppose (X, µ) is a G-space with G-invariant measure µ. Then on the Hilbert space L 2 (X, dµ) of square integrable functions, there is a natural unitary representation of G by translations. Definition 7.1.6. An irreducible unitary representation π of G is called a discrete series representation for L 2 (X, dµ) (or simply, for X ) if π can be realized in a G-invariant closed subspace of L 2 (X, dµ), or equivalently, if dim HomG (π, L 2 (X )) = 0,


189

where HomG denotes the space of continuous G-intertwining operators. consisting of all discrete series repreWe shall write Disc(X ) for the subset of G 2 sentations for L (X, dµ). It may happen that Disc(X ) = φ. 7.1.7

Harish-Chandra’s discrete series representations

Let G be a real reductive linear Lie group. If (X, µ) = (G, Haar measure) with left G-action, then Disc(G) was classified by Harish-Chandra. In the context of Theorem 7.1.4, Disc(G) is described as follows: Theorem 7.1.7. Let G be a real reductive linear Lie group. Disc(G) = {(λ) : λ is integral and elliptic, G λ is a compact torus.} This theorem presents a geometric construction of discrete series representations. Such a construction was conjectured by Langlands in the 1960s via L 2 -cohomology and proved by Schmid [85]. We can see easily that there exists λ such that G λ is a compact torus if and only if rank G = rank K . In this case, there are countably many integral and elliptic λ such that G λ is a compact torus. In particular, the above formulation of Theorem 7.1.7 includes a Harish-Chandra celebrated criterion: Disc(G) = φ ⇔ rank G = rank K . 7.1.8

(7.1.7)

Discrete series representations for symmetric spaces

Suppose G/H is a reductive symmetric space. Here are some typical examples. S L( p + q, R)/S O( p, q), G L( p + q, R)/(G L( p, R) × G L(q, R)), G L(n, C)/G L(n, R). Without loss of generality, we may assume that H is stable under a fixed Cartan involution θ of G. Then we may formulate the results of Flensted-Jensen, Matsuki– Oshima and Vogan on discrete series representations for reductive symmetric spaces as follows: Theorem 7.1.8. Let G/H be a reductive symmetric space. Then ⎧ ⎫ λ is elliptic, and satisfies a certain ⎬ ⎨ Disc(G/H ) = (λ) : integral condition, λ|h ≡ 0, . ⎩ ⎭ G λ /(G λ ∩ H ) is a compact torus We note that the original construction of discrete series representations for G/H did not use Dolbeault cohomology groups but used the Poisson transform of hyperfunctions (or distributions) on real flag varieties. It follows from the duality theorem

190

T. Kobayashi

due to Hecht–Miliˇcić–Schmid–Wolf [22] that these discrete series representations are isomorphic to some (λ). The above formulation of the description of discrete series representations is taken from the author’s exposition ([45, Example 2.9]). As in the case of Harish-Chandra’s discrete series representations, we can see easily that such λ exists if and only if rank G/H = rank K /H ∩ K . Hence Theorem 7.1.8 contains a criterion for the existence of discrete series representations for reductive symmetric spaces: Disc(G/H ) = φ ⇔ rank G/H = rank K /H ∩ K . This generalizes (7.1.7) (since the group case can be regarded as a symmetric space (G × G)/ diag(G)), and was proved by Flensted-Jensen, Matsuki and Oshima (see [16, 45, 84] for further reading).

7.2 7.2.1

Restriction of (λ) attached to elliptic orbits Asymptotic K -support, associated variety

Throughout this section, (λ) will be a unitary representation of G attached to an integral elliptic coadjoint orbit Oλ . We assume that (λ) is nonzero. This is the case if λ is in the good range ([96, 97]). √ We may and do assume X λ ∈ −1t (see (7.1.1)). Let gC = kC + pC be the complexification of a Cartan decomposition, and + gC = u− λ + (gλ )C + uλ

the Gelfand–Naimark decomposition (7.1.2) corresponding to the action of √ √ −1 ad(− −1X λ ) = ad(X λ ). We define the set of t-weights (positive noncompact roots) by √ + ∗ + (7.2.1) λ (p) := (uλ ∩ pC , t) ⊂ −1t . Here is an explicit estimate on the asymptotic K -support, and the associated variety of (λ). Theorem 7.2.1. Suppose that (λ) = 0 in the above setting. 1) AS K ((λ)) ⊂ R≥0 -span + λ (p). 2) VgC ((λ)) = Ad∗ (K C )(u− λ ∩ pC ). Proof. See [43, §3] for the Statement (1). See [4] or [97] for the Statement (2) for a regular λ; and [44] for a general case. 7.2.2

Restriction to a symmetric subgroup

We recall the notation and convention in §6.3.2, where (G, G ) is a reductive symmetric pair defined by an involutive automorphism σ of G. In particular, we have


191

C K (K ) = (t−σ )∗+ . Then the following theorem tells us explicitly when the restriction (λ)|G is infinitesimally discretely decomposable. We note that the approach by associated varieties and wave front sets leads us to the same criterion for admissible restrictions. Theorem 7.2.2 ([44, Theorem 4.2]). Let (λ) be a nonzero unitary representation of G attached to an integral elliptic coadjoint orbit, and (G, G ) a reductive symmetric pair defined by an involutive automorphism σ of G. Then the following four conditions on (G, σ, λ) are equivalent: i) ii) iii) iv)

(λ)| K is K -admissible. (λ)|G is infinitesimally discretely decomposable. −σ )∗ = {0}. R≥0 -span + + λ (p) ∩ (t prg→g (Ad(K C )(u− ∩ pC )) ⊂ Np . C

Sketch of proof. (iii)⇒(i) This follows from the criterion of K -admissibility given in Theorem 6.3.3 together with Theorem 7.2.1. (i)⇒(ii) See Theorem 4.2.7. (ii)⇒(iv) This follows from the criterion for infinitesimally discretely decomposable restrictions by means of associated varieties. Use Corollary 5.2.3 and Theorem 7.2.1. (iv)⇒(iii) This part can be proved only by techniques of Lie algebras (without representation theory).

7.3 U (2, 2) ↓ Sp(1, 1) This subsection examines Theorem 7.2.2 by an example (G, G ) = (U (2, 2), Sp(1, 1)). More precisely, we shall take a discrete series representation of U (2, 2) (with Gelfand–Kirillov dimension 5), and explain how to verify the criteria (iii) (root data) and (iv) (associated varieties) in Theorem 7.2.2. 7.3.1

Non-holomorphic discrete series representations for U (2, 2)

Let t be a maximal √ abelian subspace of k u(2) + u(2), and take a standard basis {e1 , e2 , e3 , e4 } of −1t∗ such that (pC , t) = {±(ei − e j ) : 1 ≤ i ≤ 2, 3 ≤ j ≤ 4}. We shall fix once and for all λ = λ1 e1 + λ2 e2 + λ3 e3 + λ4 e4 ∈

√

−1t∗

192

T. Kobayashi

such that λ1 > λ3 > λ4 > λ2 , λ j ∈ Z (1 ≤ j ≤ 4).

(7.3.1)

Such λ is integral and elliptic, and the resulting unitary representation (λ) is nonzero irreducible. We note that G λ is isomorphic to a compact torus T4 . Then (λ) is a Harish-Chandra’s discrete series representation by Theorem 7.1.7. Its Gelfand– Kirillov dimension is 5, as we shall see in (7.3.3) that its associated variety is five dimensional. Furthermore, (λ) is not a holomorphic discrete series representation which has Gelfand–Kirillov dimension 4 for G = U (2, 2). 7.3.2

Criterion for K -admissibility for U (2, 2) ↓ Sp(1, 1)

Retain the setting as in §7.3.1. In light of (7.3.1), the set of noncompact positive roots + λ (p) (see (7.2.1) for definition) is given by + λ (p) = {e1 − e3 , e1 − e4 , e3 − e2 , e4 − e2 }. √ Hence, via the identification −1t∗ R4 , we have ⎧⎛ ⎫ ⎞ a+b ⎪ ⎪ ⎪ ⎪ ⎨⎜ ⎬ ⎟ −c − d + ⎟ ; a, b, c, d ≥ 0 . R≥0 -span λ (p) = ⎜ ⎝ −a + c ⎠ ⎪ ⎪ ⎪ ⎪ ⎩ ⎭ −b + d

(7.3.2)

On the other hand, for (K , K ) ≡ (K , K σ ) = (U (2) × U (2), Sp(1) × Sp(1)), we have ⎛ ⎞ ⎛ ⎞ 1 0 ⎜ ⎟ ⎜ ⎟ 1 ⎟ ⎜ 0⎟ (t−σ )∗+ = R ⎜ ⎝0⎠ + R ⎝1⎠ . 0 1 −σ )∗ = {0} because the condition Thus, R≥0 -span + + λ (p) ∩ (t ⎛ ⎞ ⎛ ⎞ ⎛ ⎞ a+b 1 0 ⎜−c − d ⎟ ⎜1⎟ ⎜0⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎝ −a + c ⎠ ∈ R ⎝0⎠ + R ⎝1⎠ −b + d 0 1

leads to a + b = −c − d, which occurs only if a = b = c = d under the assumption a, b, c, d ≥ 0 (see (7.3.2)). Thus, the condition (iii) of Theorem 7.2.2 holds. Therefore the restriction (λ)| K is K -admissible. 7.3.3

Associated variety of (λ)

Retain the setting of the example as above. Although we have already known that all of the equivalent conditions in Theorem 7.2.2 hold, it is illuminating to verify them


193

directly. So let us compute the associated variety VgC ((λ)) and verify the condition (iv) of Theorem 7.2.2 using the computation given in Proposition 5.3.4. In light of (u− λ ∩ pC , t) = {−e1 + e3 , −e1 + e4 , e2 − e3 , e2 − e4 }, (= −+ λ (p))

we have u− λ ∩ pC

00 z 0 , : x, y, z, w ∈ C x y w0

via the identification pC M(2, C) ⊕ M(2, C) (see (5.3.2) in §5). Hence we have VgC ((λ)) = Ad(K C )(u− ∩ pC ) = O1 1

(7.3.3)

10

by Theorem 7.2.1, where we recall that O1 1 is a five-dimensional manifold defined 10 by O1 1 = {(A, B) : rank A = rank B = rank AB = 1, rank B A = 0}. 10

F IGURE 7.3.

In Figure 7.3 above, the closure of O1 1 consists of 5 K C -orbits, which are de10 scribed by circled points on the left. In view of the classification of K C -orbits O on NpC such that prg→g (O) ⊂ Np by Proposition 5.3.4, one can observe that all the C circled points on the left are also those on the right. Hence we conclude that prg→g (O1 1 ) ⊂ Np . 10

C

Therefore prg→g (Ad(K C )(u− ∩ pC )) ⊂ Np , that is, the condition (iv) of TheoC rem 7.2.2 has been verified directly.

194

T. Kobayashi

8 Applications of branching problems So far we have explained a basic theory of restrictions of unitary representations of reductive Lie groups, with emphasis on discrete spectrum for noncompact subgroups. In Section 8, let us discuss what restrictions can do for representation theory. Furthermore, we shall discuss briefly some new interactions of unitary representation theory with other branches of mathematics through restrictions of representations to subgroups. For further details on applications, we refer to [47, Sections 3 and 4], and [50, Sections 4, 5 and 6] and references therein.

8.1 8.1.1

Understanding representations via restrictions Analysis and Synthesis

An idea of “analysis through decomposition” is to try to decompose the object into the smallest units, and to try to understand how it can be built up from the smallest units. This method may be pursued until one reaches the smallest units. Then, what can we do with the analysis on the smallest units? Only a complete change of viewpoint allows us to go further. For example, molecules may be regarded as the “smallest units”; but they could be decomposed into atoms which may be regarded as the “smallest” in another sense, and then they consist of electrons, protons and neutrons, and then... Each step of these decompositions requires a different viewpoint. An irreducible representation π of a group G is the “smallest unit” as representations of G. Now a subgroup G could provide a “different viewpoint”. In fact, π would no longer then be the “smallest unit” as representations of G . This leads us to a method to study the irreducible representations of G by taking the restriction to G , still in the spirit of “analysis through decomposition”. 8.1.2

Cartan–Weyl highest weight theory, revisited

Let G be a connected compact Lie group, and take G := T to be a maximal toral subgroup. Let π be an irreducible representation of G. Obviously π is T -admissible. We write the branching law of the restriction π|T as π|T λ

n π (λ)Cλ ,

(8.1.2)

where Cλ is a one-dimensional representation of T , and n π (λ) is its multiplicity. Of course, the whole branching law (that is, weight decomposition) (8.1.2) determines the representation π because finite-dimensional representations are determined by their characters and because characters are determined on their restrictions to the maximal torus T . Much more strongly, Cartan–Weyl highest weight theory asserts that a single element λ (the “largest” piece in the decomposition (8.1.2)) is sufficient to determine π . This may be regarded as an example of the spirit: to understand representations through their restrictions to subgroups.


8.1.3

195

Vogan’s minimal K -type theory

Let G be a reductive linear Lie group, and take G := K to be a maximal compact subgroup of G. Let π be an irreducible unitary representation of G. Then π is K -admissible (Harish-Chandra’s admissibility theorem, see Theorem 2.4.6). As we have already mentioned in Section 4, Harish-Chandra’s admissibility laid the foundation of the theory of (g, K )-modules, an algebraic approach to infinite-dimensional representations of reductive Lie group G. Let us write the branching law as ⊕ π |K n π (τ )τ. (8.1.3) τ ∈K

Then Vogan’s minimal K -type theory shows that a single element (or a small number of elements) τ (the “smallest ones” in the branching law (8.1.3)) gives critical information for the classification of irreducible (g, K )-modules ([93, 95]) and also ([82, 100]). for understanding the unitary dual G This may be regarded as another example of the same spirit of understanding representations through their restrictions to subgroups. 8.1.4

Restrictions to noncompact groups

In contrast to the restriction to compact subgroups as we discussed in §8.1.2 and §8.1.3, not much is known about the restriction of irreducible unitary representations to noncompact subgroups. I try now to indicate a few examples where the restriction to noncompact subgroups G provides a successful clue to understanding representations π of G. This idea works better, particularly, in the case where π belongs to “singular” (or “small”) in the currepresentations, which are the most mysterious part of the unitary dual G rent status of this field. 1) (Parabolic restriction) Some of “small” representations of G remain irreducible when restricted to parabolic subgroups. (See, for example, [59, 106].) Conversely, Torasso [90] constructed minimal representations by making use of their restriction to maximal parabolic subgroups. 2) (Nonvanishing condition for Aq(λ)) In the philosophy of the orbit method, it is perhaps a natural problem to classify all elliptic and integral orbits Ad(G)λ such that the corresponding unitary representation (λ) = 0 (or equivalently, the underlying (g, K )-module Aq(λ) = 0, see §7.1.4). This is always the case if λ is in the good range in the sense of Vogan [96]. For a general O-stable parabolic subalgebra q, the set λ (outside the good range) such that Aq(λ) = 0 may be very complicated. Such a set was found explicitly in [39, Chapter 4] in the special setting where there exists a noncompact reductive subgroup G such that the restriction (λ)|G is G -admissible (see Theorem 7.2.2). Some other (combinatorial) approaches to this problem may be seen in [39, Chapter 5] and [91], but the general case remains open.

196

T. Kobayashi

3) (Jordan–Hölder series) Even in the case where π is neither unitary nor irreducible, the restriction to subgroups sometimes gives us a good tool to study the representation. Very recently, Lee and Loke [62] determined explicitly the Jordan– Hölder series of certain degenerate principal series representations and classified which irreducible subquotients are unitarizable. Previously known cases in this direction were limited to the special setting in which degenerate principal series representations have a multiplicity free K -type decomposition as was in the Howe and Tan paper [26]. Lee and Loke’s idea was to generalize this method by replacing a maximal compact subgroup K by a certain noncompact subgroup G such that the restriction to G splits discretely with multiplicity free. These three examples may be also regarded as concrete cases where one can have a better understanding of infinite-dimensional representations through their restrictions to noncompact subgroups.

8.2

Construction of representations of subgroups

Suppose we are given a representation π of G. Then the knowledge of (a part of) the branching law of the restriction π|G may be regarded as a construction of irreducible representations of G . One of the advantages of admissible restrictions is that there is no continuous spectrum in the branching law so that each irreducible summand could be explicitly captured. In this way admissible restrictions may also serve as a method to study representations of subgroups. 8.2.1

Finite-dimensional representations

Consider the natural representation of G = G L(n, C) on V = Cn . Then the m-th tensor power T m (V ) = V ⊗ · · · ⊗ V becomes an irreducible representation of the direct product group G = G × · · · × G . The restriction from G to the diagonally embedded subgroup of G is no longer irreducible, and its branching law gives rise to irreducible representations of G as irreducible summands. When restricted to the diagonally embedded subgroup G G L(n, C), it decomposes into irreducible representations of G . (A precise description is given by the Schur–Weyl duality which treats T m (V ) as a representation of the direct product group G L(n, C)×Sm .) Conversely, any polynomial representation of G can be obtained in this way for some m ∈ N. This may be regarded as a construction of irreducible representations of the subgroup G L(n, C) via branching laws. 8.2.2

Highest weight modules

The above idea can be extended to construct some irreducible infinite-dimensional representations, called highest weight representations. For example, let us consider the Weil representation π of the metaplectic group G 1 = M p(n, R). Then, the m-th tensor power π ⊗ · · · ⊗ π becomes a representation of G 1 × O(m), which decomposes discretely into multiplicity free irreducible representations of G 1 × O(m), and


197

in particular, gives rise to irreducible highest weight representations of G 1 . More generally, we have the following theorem which is part of Howe’s theory regarding reductive dual pairs: Theorem 8.2.2 (Theta correspondence [24]). Let π be the Weil representation of the metaplectic group G = M p(n, R), and G = G 1 G 2 forms a dual pair with G 2 compact. Then the restriction π|G decomposes discretely into irreducible multiplicity free representations of G = G 1 G 2 . In particular, the restriction π |G is G -admissible. Furthermore, each irreducible summand is a highest weight representation of G . A typical example of the setting of Theorem 8.2.2 is given locally as (g, g1 , g2 ) = (sp(nm, R), sp(n, R), o(m)), (sp(( p + q)m, R), u( p, q), u(m)) (sp(2 pm, R), o∗ (2 p), sp(m)). The classification of irreducible highest weight representations was accomplished in the early 1980s by Enright–Howe–Wallach and Jakobsen, independently (see [12, 28]). Quite a large part of irreducible highest weight representations were constructed as irreducible summands of the restriction of the Weil representation in the setting of Theorem 8.2.2 by Howe, Kashiwara, Vergne and others in the 1970s. Remark 8.1 Theorem 8.2.2 fits into our framework of Sections 4 –7. Very recently, Nishiyama–Ochiai–Taniguchi [75] and Enright–Willenbring [13] have made a detailed study of irreducible summands occurring in the restriction π |G in the setting of Theorem 8.2.2 under the assumption that (G, G ) is in the stable range with G 2 smaller, that is, m ≤ R-rank G 1 . Then, as was pointed out in Enright–Willenbring [13, Theorem 6], the Gelfand–Kirillov dimension of each irreducible summand Y is dependent only on the dual pair and is independent of Y . They proved this result based on case-by-case argument. We note that this result follows directly from a general theory ([44, Theorem 3.7]) of discretely decomposable restrictions. Furthermore, one can show by using the results in [13, 44] that the associated variety Vg (Y ) coincides with the projection prg→g (VgC (π )). (See Theorem 5.2.1 for the C inclusive relation in the general case). t 8.2.3

Small representations

The idea of constructing representations of subgroups as irreducible summands works also for non-highest weight representations. As one can observe from the criterion for the admissibility of restrictions (see Theorem 6.3.4), the restriction π |G tends to be discretely decomposable if its asymptotic K -support AS K (π ) is small. In particular, if π is a minimal representation, then AS K (π ) is one dimensional (Remark 6.3.5). Thus, there is a good possibility that AS K (π ) is discretely decomposable if π is “small”.

198

T. Kobayashi

For example, if G = O( p, q) ( p, q ≥ 2; p + q ≥ 8 even) and π is the minimal representation of G, then the restriction to its natural subgroup G = O( p, q ) × O(q ) is G -admissible for any q , q such that q + q = q (see [57, Theorem 4.2]). In this case, branching laws give rise to unitary representations “attached to” minimal elliptic orbits (recall the terminology from §7.1). It is also G -admissible if G = U ( p , q ), p = 2 p and q = 2q . The idea of constructing representations via branching laws was also used in a paper by Gross and Wallach [19], where they constructed interesting “small” unitary representations of exceptional Lie groups G by taking the restrictions of another small representation of G. Discretely decomposable branching laws for noncompact G are used also in the theory of automorphic forms for exceptional groups by J.-S. Li [64]. Note also that Neretin [74] recently constructed some of irreducible unitary representations of O( p, q) with K -fixed vector (namely, so-called spherical unitary representations) as discrete spectrum of the restriction of irreducible representations of U ( p, q). In his case, the restriction contains both continuous and discrete spectrum.

8.3

Branching problems

In general, it is a hard problem to find branching laws of unitary representations. Except for highest weight modules (e.g. the Weil representation, holomorphic discrete series representations) or principal series representations, not much has been studied systematically on the branching laws of irreducible unitary representations of reductive Lie groups with respect to noncompact subgroups until recently. From the viewpoint of finding explicit branching laws, an advantage of admissible restrictions is that we may employ algebraic techniques because there is no continuous spectrum. Recently, a number of explicit branching laws have been found (e.g. [19, 20, 38, 40, 41, 57, 64, 65, 66, 108]) in the context of admissible restrictions to noncompact reductive subgroups.

8.4 Global analysis Let G/H be the homogeneous space of a Lie group G by a closed subgroup H . The idea of noncommutative harmonic analysis is to try to understand functions on G/H by means of representations of the group G. For example, we refer in this Lie Theory series to the volume [3] and chapters by van den Ban, Delorme, and Schlichtkrull on this subject where G/H is a reductive symmetric space. Our main concern here is with non-symmetric spaces for which very little is known. 8.4.1

Global analysis and restriction of representations

Our approach to this problem is different from traditional approaches: The main machinery here is the restriction of representations.


199

H . 1) Embed G/H into a larger homogeneous space G/ H ). 2) Realize a representation π of G on a subspace V → C ∞ (G/ 3) Restrict the space V of functions to the submanifold G/H , and understand it by the restriction π |G of representations. One may also consider variants of this idea by replacing C ∞ by L 2 , holomorphic functions, sections of vector bundles, or cohomologies. Also, one may consider the restriction to submanifolds after taking normal derivatives (e.g. [72, 29]). Our optimistic idea here (which is used in [38, 41], for example) is that the knowledge of any two of the three would be useful in understanding the remaining one. to G Restriction from G

Analysis on G/H

H Analysis on G/

In particular, we shall consider the setting where G/H is the space on which H is the space on which we we wish to develop harmonic analysis, and where G/ have already a good understanding of harmonic analysis (e.g. a group manifold or a symmetric space). 8.4.2

Discrete series and admissible representations

As we explained in §7.1.7 and §7.1.8, a necessary and sufficient condition for the existence of discrete series representations is known for a reductive group and also for a reductive symmetric space. However, in the generality that H ⊂ G is a pair of reductive subgroups, it is still an open problem to determine which homogeneous space G/H admits discrete series representations. Let us apply the strategy in §8.4.1 to a non-symmetric homogeneous space G/H . H , and consider the branching law of the We start with a discrete series π for G/ restriction π |G . H into three cases, from the We divide the status of an embedding G/H → G/ best to more general settings. H . 1) The case G/H G/ ⊂G satisfy G H =G then we have a natural diffeomorphism If subgroups G, H H . For example, G/H G/ U ( p, q)/U ( p − 1, q) S O0 (2 p, 2q)/S O0 (2 p − 1, 2q), G 2 (R)/S L(3, R) S O0 (4, 3)/S O0 (3, 3), G 2 (R)/SU (2, 1) S O0 (4, 3)/S O0 (4, 2). (see [41, Example 5.2] for a list of such examples). In this case, any discrete spectrum of the branching law contribute to discrete series representations for G/H , and conversely, all discrete series representations for G/H are obtained in this way.

200

T. Kobayashi

H in the sense of 2) (Generic orbit) Suppose G/H is a principal orbit in G/ Richardson, namely, there is a G-open subset U in G/ H such that any G-orbit in U is isomorphic to G/H . Then, any discrete spectrum of the branching law π |G contributes to discrete series representations for G/H [46, §8]. See [38, 41, 63] for concrete examples. H the 3) (General case; e.g. sigular orbits) For a general embedding G/H → G/ above strategy may not work; the restriction of L 2 -functions does not always yield L 2 -functions on submanifolds. A remedy for this is to impose the G-admissibility of the restriction of π , which justifies again the above strategy ([46]). For instance, let us consider the action of a group G on G itself. If we consider the left action, then the action is transitive and we cannot get new results from the above strategy. However, the action is nontrivial if we consider it from both the left and the right such as G → G, x → gxσ (g)−1 for some group automorphism σ of G (e.g. σ is the identity, a Cartan involution, etc.). For example, if G = Sp(2n, R) and take an involutive automorphism σ such that G σ Sp(n, C), then the following homogeneous manifolds G/H = Sp(2n, R)/(Sp(n 0 , C) × G L(n 1 , C) × · · · × G L(n k , C)) occur as G-orbits on G for any partition (n 0 , . . . , n k ) of n. (These orbits arise as “general case”, namely, in the case (3).) Then, we can prove that there always exist discrete series for the above homogeneous spaces for any partition (n 0 , . . . , n k ) of n by using the criterion of admissible restrictions (Theorem 6.3.4). This is new except for the case n 1 = · · · = n k = 0, where G/H = Sp(2n, R)/Sp(n, C) is a symmetric space. We refer to [46] for further details. There are also further results, for example, by Neretin, Olshanski–Neretin, and Ørsted–Vargas that interact the restriction of representations and harmonic analysis on homogeneous manifolds [73, 74, 78].

8.5 Discrete groups and restriction of unitary representations 8.5.1

Matsushima–Murakami’s formula

Let G be a reductive linear Lie group, and a cocompact discrete subgroup without torsion. By Gelfand–Piateski-Shapiro’s theorem (see Theorem 2.4.3), the right regular representation on L 2 (\G) is G-admissible, so that it decomposes discretely: L 2 (\G) n (π )π π ∈G

with n (π ) < ∞. Since acts on the Riemannian symmetric space G/K properly discontinuously and freely, the quotient space X = \G/K becomes a compact smooth manifold. Its cohomology group can be described by means of the multiplicities n (π ) by a theorem of Matsushima–Murakami (see a treatise of Borel–Wallach [6]):


201

Theorem 8.5.1. Let X = \G/K be as above. Then H ∗ (X ; C)

Cn (π) ⊗ H ∗ (g, K ; π K ). π∈G

Here H ∗ (g, K ; π K ) denotes the (g, K )-cohomology of the (g, K )-module π K ([6, 95]). We say Cn (π) ⊗ H ∗ (g, K ; π K ) is the π -component of H ∗ (X ; C), and denote by H ∗ (X )π . Furthermore, H ∗ (g, K ; π K ) is nonzero except for a finite number of π , for which the (g, K )-cohomologies are explicitly computed by Vogan and Zuckerman ([101]). (More precisely, such π is exactly the representations that we explained in §7.1.3, namely, π is isomorphic to certain (λ) with λ = ρλ .) 8.5.2

Vanishing theorem for modular varieties

Matsushima–Murakami’s formula interacts the topology of a compact manifold X = \G/K with unitary representations of G. Its object is a single manifold X . Let us consider the topology of morphisms, that is our next object is a pair of manifolds Y, X . For this, we consider the following setting:

⊂

G

∩

∩ ⊂ G

⊃

K ,

⊃

∩ K,

such that G ⊂ G are a pair of reductive linear Lie groups, K := K ∩G is a maximal compact subgroup, and := ∩ G is a cocompact in G . Then Y := \G /K is also a compact manifold. We have a natural map ι : Y → X. Then, the modular variety ι(Y ) is totally geodesic in X . We write [Y ] ∈ Hm (Y ; Z) for the fundamental class defined by Y , where we put m = dim Y . Then, the cycle ι∗ [Y ] in the homology group Hm (X ; Z) is called the modular symbol. Theorem 8.5.2 (a vanishing theorem for modular symbols). If AS K(π ) ∩C K(K ) = {0} (see Theorem 6.3.4) and if π = 1 (the trivial one-dimensional representation), then the modular symbol ι∗ [Y ] is annihilated by the π -component H m (X )π in the perfect pairing Hm (X ; C) × H m (X ; C) → C. The discreteness of irreducible decomposition plays a crucial role both in Matsushima–Murakami’s formula and in the above-mentioned vanishing theorem for modular varieties. In the former, L 2 (\G) is G-admissible (Gelfand–Piateski-Shapiro), while the restriction π |G is G -admissible (see Theorem 6.3.4) in the latter.

202

T. Kobayashi

8.5.3

Clifford–Klein problem

A Clifford–Klein form of a homogeneous space G/H is the quotient manifold \G/H where is a discrete subgroup of G acting properly discontinuously and freely on G/H . Any Riemannian symmetric space G/K admits a compact Clifford– Klein form (Borel [5]). On the other hand, there is no compact Clifford–Klein form of O(n, 1)/O(n−1), namely, any complete Lorentz manifold with constant sectional curvature is noncompact (Calabi–Markus phenomenon [7]). It remains an unsolved problem to classify homogeneous spaces G/H which admit compact Clifford–Klein forms even for the special case where G/H is a symmetric space such as S L(n, R)/S O( p, n − p). Recently, Margulis revealed a new connection of this problem with restrictions of unitary representations. He found an obstruction for the existence of compact Clifford–Klein forms for G/H . His approach is to consider the unitary representation of G on the Hilbert space L 2 (\G) from the right ( is a discrete subgroup of G), and to take the restriction to the subgroup H . The key technique is to study the asymptotic behavior of matrix coefficients of these unitary representations (see a paper of Margulis[70] and also of Oh [76]). We refer to [49, 71] and references therein for an overall exposition and open questions related to this problem. Acknowledgement. This exposition is based on courses given by the author at the European School on Group Theory in Odense, in August 2000. I am very grateful to the organizers, Bent Ørsted and Henrik Schlichtkrull for their warm hospitality during my visit. I have had also the good fortune to give courses relavant to this material, at the Summer School at Yonsei University, organized by W. Schmid and J.-H. Yang in 1999, at the graduate course at Harvard University in 2001, and also at the University of Tokyo in 2002. I am very grateful to many colleagues at these institutions for the supportive atmosphere. The comments, questions, and valuable advice of all readers have been a great help in writing up this work. It is my pleasure to acknowledge my deep indebtness to the secretariat of RIMS, for indispensable help in preparing the LATEX manuscript.

References [1] M. F. Atiyah, The Harish-Chandra character, London Math. Soc. Lecture Note Series, Vol. 34, 1979, 176–181. [2] V. Bargmann, Irreducible unitary representations of the Lorentz group. Ann. of Math. (2) 48 (1947), 568–640. [3] E. van den Ban, P. Delorme, and H. Schlichtkrull. Lie Theory : Harmonic Analysis on Symmetric Spaces, Progress in Mathematics, Vol. 230, (eds. J.-P. Anker and B. Ørsted), Birkhäuser, Boston, 2004. [4] W. Borho and J. L. Brylinski, Differential operators on homogeneous spaces I, Invent. Math., 69 (1982), 437–476; Part III, ibid. 80 (1985), 1–68.


203

[5] A. Borel, Compact Clifford–Klein forms of symmetric spaces, Topology 2 (1963), 111–122. [6] A. Borel and N. Wallach, Continuous Cohomology, Discrete Subgroups, and Representations of Reductive Groups, Second edition, Mathematical Surveys and Monographs, Vol. 67, American Mathematical Society, Providence, RI, 2000. [7] E. Calabi and L. Markus, Relativistic space forms, Ann. of Math., 75 (1962), 63–76. [8] D. H. Collingwood and W. M. McGovern, Nilpotent orbits in semisimple Lie algebras, Van Nostrand Reinhold Mathematics Series, Van Nostrand Reinhold Co., New York, 1993. [9] P. Delorme, Formule de Plancherel pour les espaces symétriques réductifs, Annals of Math., 147 (1998), 412–452. [10] M. Duflo, Théorie de Mackey pour les groupes de Lie algébriques, Acta Math., 149 (1982), 153–213. [11] M. Duflo and J. Vargas, Proper map and multiplicities; in preparation. [12] T. Enright, R. Howe and N. Wallach, A classification of unitary highest weight modules, Representation theory of reductive groups (Park City, Utah, 1982), 97–143, Prog. Math., Vol. 40, Birkhäuser, Boston, 1983. [13] T. Enright and J. Willenbring, Hilbert series, Howe duality and branching for classical groups, Ann. of Math. (2), 159 (2004), 337–375. [14] J. Faraut, Distributions sphériques sur les espaces hyperboliques, J. Math. Pures Appl., (9) 58 (1979), 369–444. [15] M. Flensted-Jensen, Discrete series for semisimple symmetric spaces, Annals of Math., 111 (1980), 253–311. [16] M. Flensted-Jensen, Analysis on Non-Riemannian Symmetric Spaces, CBMS Regional Conference Series in Mathematics, Vol. 61, Amer. Math. Soc., 1986. [17] I. M. Gelfand and N. Ya. Vilenkin, Generalized Functions. Vol. 4: Applications of harmonic analysis, (English translation), Academic Press, New York, London, 1964. [18] I. M. Gelfand, M. I. Graev and I. Piateski-Shapiro, Representation Theory and Automorphic Functions, Saunders Math. Books, 1969. [19] B. H. Gross and N. R. Wallach, A distinguished family of unitary representations for the exceptional groups of real rank = 4, Lie Theory and Geometry, In honor of Bertram Kostant, Progress in Math., Vol. 123, 1994, Birkhäuser, 289–304. [20] B. Gross and N. Wallach, Restriction of small discrete series representations to symmetric subgroups, Proc. Sympos. Pure Math., 68 (2000), A.M.S., 255–272. [21] Harish-Chandra, Representations of semi-simple Lie groups, I, III, Trans. A.M.S., 75 (1953), 185–243; 76 (1954), 234–253; IV, Amer. J. Math., 77 (1955), 743–777. [22] H. Hecht, D. Miliˇcić, W. Schmid, and J. A. Wolf, Localization and standard modules for real semisimple Lie groups. I. The duality theorem, Invent. Math. 90 (1987), 297–332. [23] S. Helgason, Differential Geometry, Lie Groups and Symmetric Spaces, Pure Appl. Math. Vol. 80, (1978). [24] R. Howe, θ-series and invariant theory, Proc. Symp. Pure Math., 33 (1979), Amer. Math. Soc., 275–285. [25] R. Howe, Transcending classical invariant theory, J. Amer. Math. Soc., 2 (1989), 535– 552. [26] R. Howe and E.-C. Tan, Homogeneous functions on light cones: the infinitesimal structure of some degenerate principal series representations, Bull. Amer. Math. Soc., 28 (1993), 1–74. [27] J-S. Huang, P. Pandˇzić and G. Savin, New dual pair correspondences, Duke Math. J., 82 (1996), 447–471.

204

T. Kobayashi

[28] H. P. Jakobsen, Hermitian symmetric spaces and their unitary highest weight modules, J. Funct. Anal. 52 (1983), 385–412. [29] H. P. Jakobsen and M. Vergne, Restrictions and expansions of holomorphic representations, J. Funct. Anal., 34, (1979), 29–53. [30] J. C. Jantzen, Nilpotent orbits in representation theory. Lie Theory, Prog. Math., Vol. 228, 2003, Birkhäuser, Boston, 1–211. [31] A. Joseph, Annihilators and associated varieties of unitary highest weight modules, ´ Ann. Scient. Ecole Normale Supérieure, (1992), 1–45. [32] M. Kashiwara, T. Kawai and T. Kimura, Foundations of Algebraic Analysis, Princeton Math. Series, Vol. 37, Princeton Univ. Press, 1986. [33] M. Kashiwara and M. Vergne, K -types and singular spectrum, Lect. Notes in Math., Vol. 728, 1979, Springer-Verlag, 177–200. [34] A. A. Kirillov, Elements of the Theory of Representations, Grundlehren der Mathematischen Wissenschaften, Band 220, Springer-Verlag, Berlin-New York, 1976. xi + 315 pp. [35] A. W. Knapp, Representation Theory of Semisimple Groups, An overview based on examples, Princeton Mathematical Series, Vol. 36, Princeton University Press, Princeton, NJ, 1986. [36] A. W. Knapp, Lie Groups Beyond an Introduction, Second edition, Progress in Mathematics, Vol. 140, Birkhäuser, Boston, 2002. [37] A. W. Knapp and D. Vogan, Jr., Cohomological Induction and Unitary Representations, Princeton U.P., 1995. [38] T. Kobayashi, Unitary representations realized in L 2 -sections of vector bundles over semisimple symmetric spaces, Proceedings at the 27-28th Symp. of Functional Analysis and Real Analysis (1989), Math. Soc. Japan, 39–54. [39] T. Kobayashi, Singular Unitary Representations and Discrete Series for Indefinite Stiefel Manifolds U ( p, q; F)/U ( p−m, q; F), Mem. Amer. Math. Soc., Vol. 462, 1992. [40] T. Kobayashi, The restriction of Aq(λ) to reductive subgroups, Proc. Japan Acad., 69 (1993), 262–267. [41] T. Kobayashi, Discrete decomposability of the restriction of Aq(λ) with respect to reductive subgroups and its application, Invent. Math., 117 (1994), 181–205. [42] T. Kobayashi, Multiplicity free branching laws for unitary highest weight modules, Proceedings of the Symposium on Representation Theory held at Saga, Kyushu 1997, ed. K. Mimachi, 1997, 9–17. [43] T. Kobayashi, Discrete decomposability of the restriction of Aq(λ), II. —micro-local analysis and asymptotic K -support, Annals of Math., 147 (1998), 709–729. [44] T. Kobayashi, Discrete decomposability of the restriction of Aq(λ), III. —restriction of Harish-Chandra modules and associated varieties, Invent. Math., 131 (1998), 229–256. [45] T. Kobayashi, Harmonic analysis on homogeneous manifolds of reductive type and unitary representation theory, Sugaku, 46 (1994), Math. Soc. Japan (in Japanese), 124–143; Translations, Series II, Selected Papers on Harmonic Analysis, Groups, and Invariants (K. Nomizu, ed.), Vol. 183 (1998), A.M.S., 1–31. [46] T. Kobayashi, Discrete series representations for the orbit spaces arising from two involutions of real reductive Lie groups, J. Funct. Anal., 152 (1998), 100–135. [47] T. Kobayashi, Theory of discrete decomposable branching laws of unitary representations of semisimple Lie groups and some applications, Sugaku, 51 (1999), Math. Soc. Japan (in Japanese), 337–356; English translation, to appear in Sugaku Exposition, Transl. Series, A.M.S.


205

[48] T. Kobayashi, Discretely decomposable restrictions of unitary representations of reductive Lie groups — examples and conjectures, Advanced Study in Pure Math., 26 (2000), 98–126. [49] T. Kobayashi, Discontinuous groups for non-Riemannian homogeneous spaces, Mathematics unlimited—2001 and beyond, Springer, Berlin, 2001, 723–747. [50] T. Kobayashi, Unitary representations and branching laws, Proceedings of the I.C.M. 2002 at Beijing, 2 (2002), 615–627. [51] T. Kobayashi, Conformal geometry and analysis on minimal representations, Lecture Notes of the Winter School 2002 on geometry and physics, Czech Republic, Rendiconti del Circolo Matematico di Palermo, Serie II, Suppl. 71 (2003), 15–40. [52] T. Kobayashi, Introduction to actions of discrete groups on pseudo-Riemannian homogeneous manifolds, The 2000 Twente Conference on Lie Groups (Enschede). Acta Appl. Math., 73 (2002), 115–131. [53] T. Kobayashi, Admissible restrictions of unitary representations of reductive Lie groups – discrete decomposability and symplectic geometry, in preparation. [54] T. Kobayashi, Geometry of multiplicity-free representations of G L(n), visible actions on flag varieties, and tri-unity, Acta Appl. Math., 81 (2004), 129–146. [55] T. Kobayashi and T. Oda, Vanishing theorem of modular symbols on locally symmetric spaces, Comment. Math. Helvetici, 73 (1998), 45–70. [56] T. Kobayashi and K. Ono, Note on Hirzebruch’s proportionality principle, J. Fac. Sci. Univ. Tokyo 37 (1990), 71–87. [57] T. Kobayashi and B. Ørsted, Analysis on minimal representations of O( p, q), Part II. Branching Laws, Adv. Math., 180 (2003), 513–550. [58] T. Kobayashi and B. Ørsted, Analysis on minimal representations of O( p, q), Part III. Ultrahyperbolic equations on R p−1,q−1 , Adv. Math., 180 (2003), 551–595. [59] T. Kobayashi and T. Oshima, Lie Groups and Lie Algebras, Iwanami Shoten, (in Japanese), (2001), pp. 610; English translation (to appear). [60] B. Kostant and S. Rallis, Orbits and representations associated with symmetric spaces. Amer. J. Math. 93 (1971), 753–809. [61] R. Langlands, On the classification of irreducible representations of real algebraic groups, mimeographed notes, 1973. [62] S-T. Lee and H-Y. Loke, Degenerate principal series of U ( p, q) and Spin( p, q), Compositio Math., 132 (2002), 311–348. [63] J-S. Li, On the discrete series of generalized Stiefel manifolds, Trans. Amer. Math. Soc., Vol. 340, 1993, 753–766. [64] J-S. Li, Two reductive dual pairs in groups of type E, Manuscripta Math., 91 (1996), 163–177. [65] H-Y. Loke, Restrictions of quaternionic representations, J. Funct. Anal., 172 (2000), 377–403. [66] H-Y. Loke, Howe quotients of unitary characters and unitary lowest weight modules, preprint. [67] G. W. Mackey, Induced representations of locally compact groups I, Annals of Math., 55 (1952), 101–139. [68] D. Manchon, Distributions a` support compact et représentations unitaires. J. Lie Theory, 9 (1999), 403–424. [69] D. Manchon, Front d’onde et propagation des singularités pour un vecteur-distribution, Colloq. Math., 81 (1999), 161–191. [70] G. Margulis, Existence of compact quotients of homogeneous spaces, measurably proper actions, and decay of matrix coefficients, Bull. Soc. Math. France, 125 (1997), 1–10.

206

T. Kobayashi

[71] G. Margulis, Problems and conjectures in rigidity theory, Mathematics: Frontiers and Perspectives, 161–174, Amer. Math. Soc., Providence, RI, 2000. [72] S. Martens, The characters of the holomorphic discrete series, Proc. Nat. Acad. Sci. USA, 72, (1975), 3275–3276. [73] Y. A. Neretin and G. I. Ol’shanskii, Boundary values of holomorphic functions, special unitary representations of O( p, q), and their limits, Zapiski Nauchnykh Seminarov POMI, 223 (1995), 9–91. [74] Y. Neretin, Plancherel formula for Berezin deformation of L 2 on Riemannian symmetric space, J. Funct. Anal. 189 (2002), 336–408. [75] K. Nishiyama, H. Ochiai, and K. Taniguchi, Bernstein degree and associated cycles of Harish-Chandra modules — Hermitian symmetric case —, Astérisque 273 (2001), 13–80. [76] H. Oh, Tempered subgroups and representations with minimal decay of matrix coefficients, Bull. Soc. Math. France, 126 (1998), 355–380. ´ [77] G. Olafsson and B. Ørsted, The holomorphic discrete series of an affine symmetric space, I, J. Funct. Anal., 81 (1988), 126–159. [78] B. Ørsted and J. Vargas, Restriction of square integrable representations: discrete spectrum, preprint. [79] T. Oshima, A realization of Riemannian symmetric spaces, J. Math. Soc. Japan, 30 (1978), 117–132. [80] T. Oshima, A realization of semisimple symmetric spaces and construction of boundary value maps, Adv. Stud. Pure. Math., 14 (1988), 603–650. [81] W. Rossmann, Analysis on real hyperbolic spaces, J. Funct. Anal. 30 (1978), 448–477. [82] S. A. Salamanca-Riba and D. A. Vogan Jr., On the classification of unitary representations of reductive Lie groups. Ann. of Math. 148 (1998), 1067–1133. [83] M. Sato, Theory of hyperfunctions. I, J. Fac. Sci. Univ. Tokyo, Sect. I, 8 (1959), 139– 193. [84] H. Schlichtkrull, Hyperfunctions and Harmonic Analysis on Symmetric Spaces, Progress in Mathematics, Vol. 49, Birkhäuser, Boston, 1984. [85] W. Schmid, On a conjecture of Langlands, Ann. Math. 93 (1971), 1–42. [86] W. Schmid and K. Vilonen, Characteristic cycles and wave front cycles of representations of reductive Lie groups, Ann. of Math., 151 (2000), 1071–1118. [87] Y. Shalom, Rigidity, unitary representations of semisimple groups, and fundamental groups of manifolds with rank one transformation group, Ann. of Math., 152 (2000), 113–182. [88] R. Sjamaar, Convexity properties of the moment mapping, re-examined, Adv. Math., 138 (1998), 46–91. [89] J. Sekiguchi, Remarks on real nilpotent orbits of a symmetric pair, J. Math. Soc. Japan, 39 (1987), 127–138. [90] P. Torasso, Méthode des orbites de Kirillov–Duflo et représentations minimales des groupes simples sur un corps local de caractéristique nulle, Duke Math. J., 90 (1997), 261–377. [91] P. E. Trapa, Annihilators and associated varieties of Aq(λ) modules for U ( p, q), Compositio Math., 129 (2001), 1–45. [92] M. Vergne, Some comments on the work of I. E. Segal on group representations. Special issue dedicated to the memory of I. E. Segal. J. Funct. Anal., 190 (2002), 29–37. [93] D. A. Vogan, Jr., The algebraic structure of the representation of semisimple Lie groups. I, Ann. of Math., 109 (1979), 1–60. [94] D. A. Vogan, Jr., Singular unitary representations, Lecture Notes in Mathematics, Vol. 880, 1980, Springer, 506–535.


207

[95] D. A. Vogan, Jr., Representations of Real Reductive Lie Groups, Progress in Math., Vol. 15, Birkhäuser, 1981. [96] D. A. Vogan, Jr., Unitarizability of certain series of representations, Annals of Math., 120 (1984), 141–187. [97] D. A. Vogan, Jr., Irreducibility of discrete series representations for semisimple symmetric spaces, Advanced Studies in Pure Math. 14 (1988), 191–221. [98] D. A. Vogan, Jr., Associated varieties and unipotent representations, Harmonic Analysis on Reductive Lie Groups, Progress in Math., Vol. 101, Birkhäuser, 1991, 315–388. [99] D. A. Vogan, Jr., The orbit method and unitary representations for reductive Lie groups, Algebraic and analytic methods in representation theory (Sonderborg, 1994), (eds. B. Ørsted and H. Schlichtkrull) 243–339, Perspect. Math., Vol. 17, Academic Press, San Diego, 1996. [100] D. A. Vogan, Jr., A Langlands classification for unitary representations, Analysis on Homogeneous Spaces and Representation Theory of Lie Groups, Okayama–Kyoto 1997, (eds. T. Kobayashi et al.) Adv. Stud. Pure Math., Vol. 26, 2000, 299–324. [101] D. A. Vogan, Jr. and G. J. Zuckerman, Unitary representations with nonzero cohomology, Compositio Math., 53 (1984), 51–90. [102] N. R. Wallach, Representations of semi-simple Lie groups and Lie algebras, Lie theories and their applications, Proc. Ann. Sem. Canad. Math. Congr., Queen’s Univ., Kingston, Ont., 1977, 154–246, Academic Press, New York, 1978. [103] N. R. Wallach, On the unitarizability of derived functor modules, Invent. Math., 78 (1984), 131–141. [104] N. R. Wallach, Real Reductive Groups I, II, Pure and Appl. Math., Vol. 132, Academic Press, 1988, 1992. [105] G. Warner, Harmonic Analysis on Semisimple Lie Groups I, II, Springer Verlag, Berlin, 1972. [106] J. A. Wolf, Representations that remain irreducible on parabolic subgroups, Differential geometrical methods in mathematical physics Proc. Conf., Aix-enProvence/Salamanca, 1979, 129–144, Lecture Notes in Math., Vol. 836, Springer, Berlin, 1980. [107] H. Wong, Dolbeault cohomological realization of Zuckerman modules associated with finite rank representations, J. Funct. Anal., 129 (1995), 428–454. [108] J. Xie, Restriction of discrete series of SU (2, 1) to S(U (1) × U (1, 1)), J. Funct. Anal., 122 (1994), 478–518, Ph.D dissertation, Rutgers University. [109] A. Yamamoto, Orbits in the flag variety and images of the moment map for classical groups. I, Represent. Theory , 1 (1997), 329–404.

Lie Theory: Unitary Representations and Compactifications of Symmetric Spaces (Progress in Mathematics)

Lie Theory. Unitary Representations and Compactifications of Symmetric Spaces

Compactifications of Symmetric Spaces

Compactifications of Symmetric and Locally Symmetric Spaces (Mathematics, Theory & Applications)

Compactifications of Symmetric and Locally Symmetric Spaces (Mathematics: Theory & Applications)

Lie Theory: Harmonic Analysis on Symmetric Spaces - General Plancherel Theorems (Progress in Mathematics)

Differential geometry, Lie groups, and symmetric spaces

Differential Geometry, Lie Groups and Symmetric Spaces

Unitary Representations and Harmonic Analysis

Unitary representations of topological groups

Unitary representations and harmonic analysis

Lie Theory: Harmonic Analysis on Symmetric Spaces General Plancherel Theorems

Smooth compactifications of locally symmetric varieties

Unitary Group Representations in Physics, Probability and Number Theory

Integrable systems on Lie algebras and symmetric spaces

Linear and projective representations of symmetric groups

Integrable Systems on Lie Algebras and Symmetric Spaces (Advanced Studies in Contemporary Mathematics)

Integrable Systems on Lie Algebras and Symmetric Spaces (Advanced Studies in Contemporary Mathematics, Vol 2)

Differential geometry and symmetric spaces

Differential Geometry and Symmetric Spaces

Lie groups, Lie algebras and their representations

Theory of Stein Spaces (Classics in Mathematics)

Twistor Theory for Riemannian Symmetric Spaces

Unitary Group Representations in Physics, Probability and Number Theory (Mathematics Lecture Notes Series; 55)

Generalized Symmetric Spaces

Lie Groups, Lie Algebras, and Their Representations

Unitary Representations and Harmonic Analysis: An Introduction

Symmetries, Lie algebras and representations

The Symmetric Group: Representations, Combinatorial Algorithms, and Symmetric Functions

Representations of compact Lie groups

Naive Lie Theory: Undergraduate Texts in Mathematics

Lie Theory: Unitary Representations and Compactifications of Symmetric Spaces (Progress in Mathematics)