Advanced Linear Algebra

Graduate Texts in Mathematics S. Axler 135 Editorial Board F.W. Gehring K.A. Ribet Graduate Texts in Mathematics 1...

Author: Steven Roman

1227 downloads 1456 Views 4MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Graduate Texts in Mathematics

S. Axler

135

Editorial Board F.W. Gehring K.A. Ribet

Graduate Texts in Mathematics 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33

TAKEUTI/ZARING.Introduction to Axiomatic Set Theory. 2nd ed. OXTOBY.Measure and Category. 2nd ed. SCHAEFER.Topological Vector Spaces. 2nd ed. HILTON/STAMMBACH.A Course in Homological Algebra. 2nd ed. MAC LANE. Categories for the Working Mathematician. 2nd ed. HUGHES/PIPER.Projective Planes. J.-P. SERRE.A Course in Arithmetic. TAKEUTI/ZARING.Axiomatic Set Theory. HUMPHREYS.Introduction to Lie Algebras and Representation Theory. COHEN.A Course in Simple Homotopy Theory. CONWAY.Functions of One Complex Variable I. 2nd ed. BEALS.Advanced Mathematical Analysis. ANDERSON/FULLER.Rings and Categories of Modules. 2nd ed. GOLUBITSKY/GUILLEMIN.Stable Mappings and Their Singularities. BERBERIAN.Lectures in Functional Analysis and Operator Theory. WrNTER.The Structure of Fields. ROSENBLATT.Random Processes. 2nd ed. HALMOS.Measure Theory. HALMOS.A Hilbert Space Problem Book. 2nd ed. HUSEMOLLER.Fibre Bundles. 3rd ed. HUMPHREYS.Linear Algebraic Groups. BARNES/]VIACK.An Algebraic Introduction to Mathematical Logic. GREUB.Linear Algebra. 4th ed. HOLMES.Geometric Functional Analysis and Its Applications. HEWITT/STROMBERG.Real and Abstract Analysis. MANES.Algebraic Theories. KELLEY.General Topology. ZARISKI/SAMUEL.Commutative Algebra. Vol.I. ZAPdSrO/SAMUEL.Commutative Algebra. Vol.II. JACOBSON.Lectures in Abstract Algebra I. Basic Concepts. JACOBSON.Lectures in Abstract Algebra II. Linear Algebra. JACOBSON.Lectures in Abstract Algebra III. Theory of Fields and Galois Theory. HIRSCH.Differential Topology.

34 35 36 37 38 39 40 41

42 43 44 45 46 47 48 49 50 51 52 53 54 55

56 57 58 59 60 61 62 63

SPITZER.Principles of Random Walk. 2nd ed. ALEXANDE~ERMER.Several Complex Variables and Banach Algebras. 3rd ed. KELLEY/NAMIOKAet al. Linear Topological Spaces. MONK.Mathematical Logic. GRAUERT/FRITZSCHE.Several Complex Variables. ARVESON.An Invitation to C*-Algebras. KEMENY/SNELL/KNAPP.Denumerable Markov Chains. 2nd ed. APOSTOL.Modular Functions and Dirichlet Series in Number Theory. 2nd ed. J.-P. SERRE.Linear Representations of Finite Groups. GILLMAN/JERISON.Rings of Continuous Functions. KENDIG.Elementary Algebraic Geometry. LOEVE.Probability Theory I. 4th ed. LoI~vE.Probability Theory II. 4th ed. MOISE.Geometric Topology in Dimensions 2 and 3. SACHS/~qVu.General Relativity for Mathematicians. GRUENBERG/~vVEIR.Linear Geometry. 2nd ed. EDWARDS.Fermat's Last Theorem. KLINGENBERG.A Course in Differential Geometry. HARTSHORNE.Algebraic Geometry. MANN.A Course in Mathematical Logic. GRAVEK/V~ATKINS.Combinatorics with Emphasis on the Theory of Graphs. BROWN/PEARCY.Introduction to Operator Theory I: Elements of Functional Analysis. MASSEY.Algebraic Topology: An Introduction. CROWELL/Fox.Introduction to Knot Theory. KOBLITZ.p-adic Numbers, p-adic Analysis, and Zeta-Functions. 2nd ed. LANG.Cyclotomic Fields. ARNOLD.Mathematical Methods in Classical Mechanics. 2nd ed. WHITEHEAD.Elements of Homotopy Theory. KARGAPOLOV/]~ERLZJAKOV.Fundamentals of the Theory of Groups. BOLLOBAS.Graph Theory.

(continued after index)

Steven Roman

Advanced Linear Algebra Second Edition

___ S p r i n g e r

Steven Roman University of California, Irvine Irvine, California 92697-3875 USA [email protected]

Editorial Board." S. Axler Mathematics Department San Francisco State University San Francisco, CA 94132 USA [email protected]

EW. Gehring Mathematics Department East Hall University of Michigan Ann Arbor, MI 48109 USA [email protected]

K.A. Ribet Mathematics D e p a r t m e n t University of California, Berkeley Berkeley, CA 94720-3840 USA [email protected]

Mathematics Subject Classification (2000): 15-xx Library of Congress Cataloging-in-Publication Data Roman, Steven. Advanced linear algebra / Steven Roman.--2nd ed. p. cm. Includes bibliographical references and index. ISBN 0-387-24766-1 (acid-free paper) 1. Algebras, Linear. I. Title. QA184.2.R66 2005 512'.5-- dc22 2005040244 ISBN 0-387-24766-1

Printed on acid-free paper.

© 2005 Steven Roman All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+ Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed in the United States of America. 98765432 springer.com

To Donna and to my poker buddies Rachelle, Carol and Dan

Preface to the Second Edition

Let me begin by thanking the readers of the first edition for their many helpful comments and suggestions. The second edition represents a major change from the first edition. Indeed, one might say that it is a totally new book, with the exception of the general range of topics covered. The text has been completely rewritten. I hope that an additional 12 years and roughly 20 books worth of experience has enabled me to improve the quality of my exposition. Also, the exercise sets have been completely rewritten. The second edition contains two new chapters: a chapter on convexity, separation and positive solutions to linear systems (Chapter 15) and a chapter on the QR decomposition, singular values and pseudoinverses (Chapter 17). The treatments of tensor products and the umbral calculus have been greatly expanded and I have included discussions of determinants (in the chapter on tensor products), the complexification of a real vector space, Schur's lemma and Geršgorin disks. Steven Roman

Irvine, California February 2005

Preface to the First Edition

This book is a thorough introduction to linear algebra, for the graduate or advanced undergraduate student. Prerequisites are limited to a knowledge of the basic properties of matrices and determinants. However, since we cover the basics of vector spaces and linear transformations rather rapidly, a prior course in linear algebra (even at the sophomore level), along with a certain measure of “mathematical maturity,” is highly desirable. Chapter 0 contains a summary of certain topics in modern algebra that are required for the sequel. This chapter should be skimmed quickly and then used primarily as a reference. Chapters 1–3 contain a discussion of the basic properties of vector spaces and linear transformations. Chapter 4 is devoted to a discussion of modules, emphasizing a comparison between the properties of modules and those of vector spaces. Chapter 5 provides more on modules. The main goals of this chapter are to prove that any two bases of a free module have the same cardinality and to introduce noetherian modules. However, the instructor may simply skim over this chapter, omitting all proofs. Chapter 6 is devoted to the theory of modules over a principal ideal domain, establishing the cyclic decomposition theorem for finitely generated modules. This theorem is the key to the structure theorems for finite-dimensional linear operators, discussed in Chapters 7 and 8. Chapter 9 is devoted to real and complex inner product spaces. The emphasis here is on the finite-dimensional case, in order to arrive as quickly as possible at the finite-dimensional spectral theorem for normal operators, in Chapter 10. However, we have endeavored to state as many results as is convenient for vector spaces of arbitrary dimension. The second part of the book consists of a collection of independent topics, with the one exception that Chapter 13 requires Chapter 12. Chapter 11 is on metric vector spaces, where we describe the structure of symplectic and orthogonal geometries over various base fields. Chapter 12 contains enough material on metric spaces to allow a unified treatment of topological issues for the basic

x

Preface

Hilbert space theory of Chapter 13. The rather lengthy proof that every metric space can be embedded in its completion may be omitted. Chapter 14 contains a brief introduction to tensor products. In order to motivate the universal property of tensor products, without getting too involved in categorical terminology, we first treat both free vector spaces and the familiar direct sum, in a universal way. Chapter 15 [Chapter 16 in the second edition] is on affine geometry, emphasizing algebraic, rather than geometric, concepts. The final chapter provides an introduction to a relatively new subject, called the umbral calculus. This is an algebraic theory used to study certain types of polynomial functions that play an important role in applied mathematics. We give only a brief introduction to the subject c emphasizing the algebraic aspects, rather than the applications. This is the first time that this subject has appeared in a true textbook. One final comment. Unless otherwise mentioned, omission of a proof in the text is a tacit suggestion that the reader attempt to supply one. Steven Roman

Irvine, California

Contents

Preface to the Second Edition, vii Preface to the First Edition, ix

Preliminaries, 1 Part 1 Preliminaries, 1 Part 2 Algebraic Structures, 16

Part I—Basic Linear Algebra, 31 1

Vector Spaces, 33 Vector Spaces, 33 Subspaces, 35 Direct Sums, 38 Spanning Sets and Linear Independence, 41 The Dimension of a Vector Space, 44 Ordered Bases and Coordinate Matrices, 47 The Row and Column Spaces of a Matrix, 48 The Complexification of a Real Vector Space, 49 Exercises, 51

2

Linear Transformations, 55 Linear Transformations, 55 Isomorphisms, 57 The Kernel and Image of a Linear Transformation, 57 Linear Transformations from - to - , 59 The Rank Plus Nullity Theorem, 59 Change of Basis Matrices, 60 The Matrix of a Linear Transformation, 61 Change of Bases for Linear Transformations, 63 Equivalence of Matrices, 64 Similarity of Matrices, 65 Similarity of Operators, 66 Invariant Subspaces and Reducing Pairs, 68

xii

Contents

Topological Vector Spaces, 68 Linear Operators on = d , 71 Exercises, 72 3

The Isomorphism Theorems, 75 Quotient Spaces, 75 The Universal Property of Quotients and the First Isomorphism Theorem, 77 Quotient Spaces, Complements and Codimension, 79 Additional Isomorphism Theorems, 80 Linear Functionals, 82 Dual Bases, 83 Reflexivity, 84 Annihilators, 86 Operator Adjoints, 88 Exercises, 90

4

Modules I: Basic Properties, 93 Modules, 93 Motivation, 93 Submodules, 95 Spanning Sets, 96 Linear Independence, 98 Torsion Elements, 99 Annihilators, 99 Free Modules, 99 Homomorphisms, 100 Quotient Modules, 101 The Correspondence and Isomorphism Theorems, 102 Direct Sums and Direct Summands, 102 Modules Are Not As Nice As Vector Spaces, 106 Exercises, 106

5

Modules II: Free and Noetherian Modules, 109 The Rank of a Free Module, 109 Free Modules and Epimorphisms, 114 Noetherian Modules, 115 The Hilbert Basis Theorem, 118 Exercises, 119

6

Modules over a Principal Ideal Domain, 121 Annihilators and Orders, 121 Cyclic Modules, 122 Free Modules over a Principal Ideal Domain, 123 Torsion-Free and Free Modules, 125

Contents

Prelude to Decomposition: Cyclic Modules, 126 The First Decomposition, 127 A Look Ahead, 127 The Primary Decomposition, 128 The Cyclic Decomposition of a Primary Module, 130 The Primary Cyclic Decomposition Theorem, 134 The Invariant Factor Decomposition, 135 Exercises, 138 7

The Structure of a Linear Operator, 141 A Brief Review, 141 The Module Associated with a Linear Operator, 142 Orders and the Minimal Polynomial, 144 Cyclic Submodules and Cyclic Subspaces, 145 Summary, 147 The Decomposition of = , 147 The Rational Canonical Form, 148 Exercises, 151

8

Eigenvalues and Eigenvectors, 153 The Characteristic Polynomial of an Operator, 153 Eigenvalues and Eigenvectors, 155 Geometric and Algebraic Multiplicities, 157 The Jordan Canonical Form, 158 Triangularizability and Schur's Lemma, 160 Diagonalizable Operators, 165 Projections, 166 The Algebra of Projections, 167 Resolutions of the Identity, 170 Spectral Resolutions, 172 Projections and Invariance, 173 Exercises, 174

9

Real and Complex Inner Product Spaces , 181 Norm and Distance, 183 Isometries, 186 Orthogonality, 187 Orthogonal and Orthonormal Sets, 188 The Projection Theorem and Best Approximations, 192 Orthogonal Direct Sums, 194 The Riesz Representation Theorem, 195 Exercises, 196

10

Structure Theory for Normal Operators, 201 The Adjoint of a Linear Operator, 201

xiii

xiv

Contents

Unitary Diagonalizability, 204 Normal Operators, 205 Special Types of Normal Operators, 207 Self-Adjoint Operators, 208 Unitary Operators and Isometries, 210 The Structure of Normal Operators, 215 Matrix Versions, 222 Orthogonal Projections, 223 Orthogonal Resolutions of the Identity, 226 The Spectral Theorem, 227 Spectral Resolutions and Functional Calculus, 228 Positive Operators, 230 The Polar Decomposition of an Operator, 232 Exercises, 234

Part II—Topics, 235 11

Metric Vector Spaces: The Theory of Bilinear Forms, 239 Symmetric, Skew-Symmetric and Alternate Forms, 239 The Matrix of a Bilinear Form, 242 Quadratic Forms, 244 Orthogonality, 245 Linear Functionals, 248 Orthogonal Complements and Orthogonal Direct Sums, 249 Isometries, 252 Hyperbolic Spaces, 253 Nonsingular Completions of a Subspace, 254 The Witt Theorems: A Preview, 256 The Classification Problem for Metric Vector Spaces, 257 Symplectic Geometry, 258 The Structure of Orthogonal Geometries: Orthogonal Bases, 264 The Classification of Orthogonal Geometries: Canonical Forms, 266 The Orthogonal Group, 272 The Witt's Theorems for Orthogonal Geometries, 275 Maximal Hyperbolic Subspaces of an Orthogonal Geometry, 277 Exercises, 279

12

Metric Spaces, 283 The Definition, 283 Open and Closed Sets, 286 Convergence in a Metric Space, 287 The Closure of a Set, 288

Contents

Dense Subsets, 290 Continuity, 292 Completeness, 293 Isometries, 297 The Completion of a Metric Space, 298 Exercises, 303 13

Hilbert Spaces, 307 A Brief Review, 307 Hilbert Spaces, 308 Infinite Series, 312 An Approximation Problem, 313 Hilbert Bases, 317 Fourier Expansions, 318 A Characterization of Hilbert Bases, 328 Hilbert Dimension, 328 A Characterization of Hilbert Spaces, 329 The Riesz Representation Theorem, 331 Exercises, 334

14

Tensor Products, 337 Universality, 337 Bilinear Maps, 341 Tensor Products, 343 When Is a Tensor Product Zero? 348 Coordinate Matrices and Rank, 350 Characterizing Vectors in a Tensor Product, 354 Defining Linear Transformations on a Tensor Product, 355 The Tensor Product of Linear Transformations, 357 Change of Base Field, 359 Multilinear Maps and Iterated Tensor Products, 363 Tensor Spaces, 366 Special Multilinear Maps, 371 Graded Algebras, 372 The Symmetric Tensor Algebra, 374 The Antisymmetric Tensor Algebra: The Exterior Product Space, 380 The Determinant, 387 Exercises, 391

15

Positive Solutions to Linear Systems: Convexity and Separation 395 Convex, Closed and Compact Sets, 398 Convex Hulls, 399

xv

xvi

Contents

Linear and Affine Hyperplanes, 400 Separation, 402 Exercises, 407 16

Affine Geometry, 409 Affine Geometry, 409 Affine Combinations, 41 Affine Hulls, 412 The Lattice of Flats, 413 Affine Independence, 416 Affine Transformations, 417 Projective Geometry, 419 Exercises, 423

17

Operator Factorizations: QR and Singular Value, 425 The QR Decomposition, 425 Singular Values, 428 The Moore–Penrose Generalized Inverse, 430 Least Squares Approximation, 433 Exercises, 434

18

The Umbral Calculus, 437 Formal Power Series, 437 The Umbral Algebra, 439 Formal Power Series as Linear Operators, 443 Sheffer Sequences, 446 Examples of Sheffer Sequences, 454 Umbral Operators and Umbral Shifts, 456 Continuous Operators on the Umbral Algebra, 458 Operator Adjoints, 459 Umbral Operators and Automorphisms of the Umbral Algebra, 460 Umbral Shifts and Derivations of the Umbral Algebra, 465 The Transfer Formulas, 470 A Final Remark, 471 Exercises, 472

References, 473 Index, 475

Chapter 0

Preliminaries

In this chapter, we briefly discuss some topics that are needed for the sequel. This chapter should be skimmed quickly and used primarily as a reference.

Part 1 Preliminaries Multisets The following simple concept is much more useful than its infrequent appearance would indicate. Definition Let : be a nonempty set. A multiset 4 with underlying set : is a set of ordered pairs 4 ~ ¸² Á ³

:Á {b Á

£

for £ ¹

where {b ~ ¸Á Á Ã ¹. The number is referred to as the multiplicity of the elements in 4 . If the underlying set of a multiset is finite, we say that the multiset is finite. The size of a finite multiset 4 is the sum of the multiplicities of all of its elements. For example, 4 ~ ¸²Á ³Á ²Á ³Á ²Á ³¹ is a multiset with underlying set : ~ ¸Á Á ¹. The elements has multiplicity . One often writes out the elements of a multiset according to multiplicities, as in 4 ~ ¸Á Á Á Á Á ¹ . Of course, two mutlisets are equal if their underlying sets are equal and if the multiplicity of each element in the comon underlying set is the same in both multisets.

Matrices The set of d matrices with entries in a field - is denoted by CÁ ²- ³ or by CÁ when the field does not require mention. The set CÁ ²< ³ is denoted by C ²- ³ or C À If ( C, the ²Á ³-th entry of ( will be denoted by (Á . The identity matrix of size d is denoted by 0 . The elements of the base

2

Advanced Linear Algebra

field - are called scalars. We expect that the reader is familiar with the basic properties of matrices, including matrix addition and multiplication. The main diagonal of an d matrix ( is the sequence of entries (Á Á (Á Á Ã Á (Á where ~ min¸Á ¹. Definition The transpose of ( CÁ is the matrix (! defined by ²(! ³Á ~ (Á A matrix ( is symmetric if ( ~ (! and skew-symmetric if (! ~ c(. Theorem 0.1 (Properties of the transpose) Let (, ) CÁ . Then 1) ²(! ³! ~ ( 2) ²( b )³! ~ (! b ) ! 3) ²(³! ~ (! for all 4) ²()³! ~ ) ! (! provided that the product () is defined 5) det²(! ³ ~ det²(³.

Partitioning and Matrix Multiplication Let 4 be a matrix of size d . If ) ¸Á Ã Á ¹ and * ¸Á Ã Á ¹ then the submatrix 4 ´)Á *µ is the matrix obtained from 4 by keeping only the rows with index in ) and the columns with index in * . Thus, all other rows and columns are discarded and 4 ´)Á *µ has size () ( d (* (. Suppose that 4 CÁ and 5 CÁ . Let 1) F ~ ¸) Á Ã Á ) ¹ be a partition of ¸Á Ã Á ¹ 2) G ~ ¸* Á Ã Á * ¹ be a partition of ¸Á Ã Á ¹ 3) H ~ ¸+ Á Ã Á + ¹ be a partition of ¸Á Ã Á ¹ (Partitions are defined formally later in this chapter.) Then it is a very useful fact that matrix multiplication can be performed at the block level as well as at the entry level. In particular, we have ´4 5 µ´) Á + µ ~ 4 ´) Á * µ5 ´* Á + µ * G

When the partitions in question contain only single-element blocks, this is precisely the usual formula for matrix multiplication

´4 5 µÁ ~ 4Á 5Á ~

Preliminaries

3

Block Matrices It will be convenient to introduce the notational device of a block matrix. If )Á are matrices of the appropriate sizes then by the block matrix 4~

v )Á Å w )Á

)Á Å )Á

Ä

)Á y Å Ä )Á zblock

we mean the matrix whose upper left submatrix is )Á , and so on. Thus, the )Á 's are submatrices of 4 and not entries. A square matrix of the form v ) x 4 ~x Å w

Ä y Æ Æ Å { { Æ Æ Ä ) zblock

where each ) is square and is a zero submatrix, is said to be a block diagonal matrix.

Elementary Row Operations Recall that there are three types of elementary row operations. Type 1 operations consist of multiplying a row of ( by a nonzero scalar. Type 2 operations consist of interchanging two rows of (. Type 3 operations consist of adding a scalar multiple of one row of ( to another row of (. If we perform an elementary operation of type to an identity matrix 0 , the result is called an elementary matrix of type . It is easy to see that all elementary matrices are invertible. In order to perform an elementary row operation on ( CÁ we can perform that operation on the identity 0 , to obtain an elementary matrix , and then take the product ,(. Note that multiplying on the right by , has the effect of performing column operations. Definition A matrix 9 is said to be in reduced row echelon form if 1) All rows consisting only of 's appear at the bottom of the matrix. 2) In any nonzero row, the first nonzero entry is a . This entry is called a leading entry. 3) For any two consecutive rows, the leading entry of the lower row is to the right of the leading entry of the upper row. 4) Any column that contains a leading entry has 's in all other positions. Here are the basic facts concerning reduced row echelon form.

4


Theorem 0.2 Matrices (Á ) CÁ are row equivalent, denoted by ( ) , if either one can be obtained from the other by a series of elementary row operations. 1) Row equivalence is an equivalence relation. That is, a) ( ( b) ( ) ¬ ) ( c) ( ) , ) * ¬ ( * . 2) A matrix ( is row equivalent to one and only one matrix 9 that is in reduced row echelon form. The matrix 9 is called the reduced row echelon form of (. Furthermore, ( ~ , Ä, 9 where , are the elementary matrices required to reduce ( to reduced row echelon form. 3) ( is invertible if and only if its reduced row echelon form is an identity matrix. Hence, a matrix is invertible if and only if it is the product of elementary matrices. The following definition is probably well known to the reader. Definition A square matrix is upper triangular if all of its entries below the main diagonal are . Similarly, a square matrix is lower triangular if all of its entries above the main diagonal are . A square matrix is diagonal if all of its entries off the main diagonal are .

Determinants We assume that the reader is familiar with the following basic properties of determinants. Theorem 0.3 Let ( CÁ ²- ³. Then det²(³ is an element of - . Furthermore, 1) For any ) C ²- ³, det²()³ ~ det²(³det²)³ 2) ( is nonsingular (invertible) if and only if det²(³ £ . 3) The determinant of an upper triangular or lower triangular matrix is the product of the entries on its main diagonal. 4) If a square matrix 4 has the block diagonal form v ) x 4 ~x Å w then det²4 ³ ~ det²) ³.

Ä y Æ Æ Å { { Æ Æ Ä ) zblock

Preliminaries

5

Polynomials The set of all polynomials in the variable % with coefficients from a field - is denoted by - ´%µ. If ²%³ - ´%µ, we say that ²%³ is a polynomial over - . If ²%³ ~ b % b Ä b % is a polynomial with £ then is called the leading coefficient of ²%³ and the degree of ²%³ is , written deg ²%³ ~ . For convenience, the degree of the zero polynomial is cB. A polynomial is monic if its leading coefficient is . Theorem 0.4 (Division algorithm) Let ²%³Á ²%³ - ´%µ where deg ²%³ . Then there exist unique polynomials ²%³Á ²%³ - ´%µ for which ²%³ ~ ²%³²%³ b ²%³ where ²%³ ~ or deg ²%³ deg ²%³. If ²%³ divides ²%³, that is, if there exists a polynomial ²%³ for which ²%³ ~ ²%³²%³ then we write ²%³ ²%³. Theorem 0.5 Let ²%³Á ²%³ - ´%µ. The greatest common divisor of ²%³ and ²%³, denoted by gcd² ²%³Á ²%³³, is the unique monic polynomial ²%³ over for which 1) ²%³ ²%³ and ²%³ ²%³ 2) if ²%³ ²%³ and ²%³ ²%³ then ²%³ ²%³. Furthermore, there exist polynomials ²%³ and ²%³ over - for which gcd² ²%³Á ²%³³ ~ ²%³ ²%³ b ²%³²%³

Definition The polynomials ²%³Á ²%³ - ´%µ are relatively prime if gcd² ²%³Á ²%³³ ~ . In particular, ²%³ and ²%³ are relatively prime if and only if there exist polynomials ²%³ and ²%³ over - for which ²%³ ²%³ b ²%³²%³ ~

Definition A nonconstant polynomial ²%³ - ´%µ is irreducible if whenever ²%³ ~ ²%³²%³ then one of ²%³ and ²%³ must be constant. The following two theorems support the view that irreducible polynomials behave like prime numbers. Theorem 0.6 A nonconstant polynomial ²%³ is irreducible if and only if it has the property that whenever ²%³ ²%³²%³ then either ²%³ ²%³ or ²%³ ²%³.

6


Theorem 0.7 Every nonconstant polynomial in - ´%µ can be written as a product of irreducible polynomials. Moreover, this expression is unique up to order of the factors and multiplication by a scalar.

Functions To set our notation, we should make a few comments about functions. Definition Let ¢ : ¦ ; be a function from a set : to a set ; . 1) The domain of is the set : . 2) The image or range of is the set im² ³ ~ ¸ ² ³ :¹. 3) is injective (one-to-one), or an injection, if % £ & ¬ ²%³ £ ²&³. 4) is surjective (onto ; ), or a surjection, if im² ³ ~ ; . 5) is bijective, or a bijection, if it is both injective and surjective. 6) Assuming that ; , the support of is supp² ³ ~ ¸ : ² ³ £ ¹

If ¢ : ¦ ; is injective then its inverse c ¢ im² ³ ¦ : exists and is welldefined as a function on im² ³. It will be convenient to apply to subsets of : and ; . In particular, if ? : and if @ ; , we set ²?³ ~ ¸ ²%³ % ?¹ and c ²@ ³ ~ ¸ : ² ³ @ ¹ Note that the latter is defined even if is not injective. Let ¢ : ¦ ; . If ( : , the restriction of to ( is the function O( ¢ ( ¦ ; defined by O( ²³ ~ ²³ for all (. Clearly, the restriction of an injective map is injective.

Equivalence Relations The concept of an equivalence relation plays a major role in the study of matrices and linear transformations. Definition Let : be a nonempty set. A binary relation on : is called an equivalence relation on : if it satisfies the following conditions:

Preliminaries

7

1) (Reflexivity) for all : . 2) (Symmetry) ¬ for all Á : . 3) (Transitivity) Á ¬ for all Á Á : . Definition Let be an equivalence relation on : . For : , the set of all elements equivalent to is denoted by ´µ ~ ¸ : ¹ and called the equivalence class of . Theorem 0.8 Let be an equivalence relation on : . Then 1) ´µ ¯ ´µ ¯ ´µ ~ ´µ 2) For any Á : , we have either ´µ ~ ´µ or ´µ q ´µ ~ J. Definition A partition of a nonempty set : is a collection ¸( Á Ã Á ( ¹ of nonempty subsets of : , called the blocks of the partition, for which 1) ( q ( ~ J for all £ 2) : ~ ( r Ä r ( . The following theorem sheds considerable light on the concept of an equivalence relation. Theorem 0.9 1) Let be an equivalence relation on : . Then the set of distinct equivalence classes with respect to are the blocks of a partition of : . 2) Conversely, if F is a partition of : , the binary relation defined by if and lie in the same block of F is an equivalence relation on : , whose equivalence classes are the blocks of F . This establishes a one-to-one correspondence between equivalence relations on : and partitions of : . The most important problem related to equivalence relations is that of finding an efficient way to determine when two elements are equivalent. Unfortunately, in

8


most cases, the definition does not provide an efficient test for equivalence and so we are led to the following concepts. Definition Let be an equivalence relation on : . A function ¢ : ¦ ; , where ; is any set, is called an invariant of if it is constant on the equivalence classes of , that is, ¬ ²³ ~ ²³ and a complete invariant if it is constant and distinct on the equivalence classes of , that is, ¯ ²³ ~ ²³ A collection ¸ Á Ã Á ¹ of invariants is called a complete system of invariants if ¯ ²³ ~ ²³ for all ~ Á Ã Á

Definition Let be an equivalence relation on : . A subset * : is said to be a set of canonical forms (or just a canonical form) for if for every : , there is exactly one * such that . Put another way, each equivalence class under contains exactly one member of * . Example 0.1 Define a binary relation on - ´%µ by letting ²%³ ²%³ if and only if ²%³ ~ ²%³ for some nonzero constant - . This is easily seen to be an equivalence relation. The function that assigns to each polynomial its degree is an invariant, since ²%³ ²%³ ¬ deg²²%³³ ~ deg²²%³³ However, it is not a complete invariant, since there are inequivalent polynomials with the same degree. The set of all monic polynomials is a set of canonical forms for this equivalence relation. Example 0.2 We have remarked that row equivalence is an equivalence relation on CÁ ²- ³. Moreover, the subset of reduced row echelon form matrices is a set of canonical forms for row equivalence, since every matrix is row equivalent to a unique matrix in reduced row echelon form. Example 0.3 Two matrices (, ) C ²- ³ are row equivalent if and only if there is an invertible matrix 7 such that ( ~ 7 ) . Similarly, ( and ) are column equivalent, that is, ( can be reduced to ) using elementary column operations if and only if there exists an invertible matrix 8 such that ( ~ )8. Two matrices ( and ) are said to be equivalent if there exist invertible matrices 7 and 8 for which

Preliminaries

9

( ~ 7 )8 Put another way, ( and ) are equivalent if ( can be reduced to ) by performing a series of elementary row and/or column operations. (The use of the term equivalent is unfortunate, since it applies to all equivalence relations, not just this one. However, the terminology is standard, so we use it here.) It is not hard to see that an d matrix 9 that is in both reduced row echelon form and reduced column echelon form must have the block form 1 ~ >

0 cÁ

Ác cÁc ?block

We leave it to the reader to show that every matrix ( in C is equivalent to exactly one matrix of the form 1 and so the set of these matrices is a set of canonical forms for equivalence. Moreover, the function defined by ²(³ ~ , where ( 1 , is a complete invariant for equivalence. Since the rank of 1 is and since neither row nor column operations affect the rank, we deduce that the rank of ( is . Hence, rank is a complete invariant for equivalence. In other words, two matrices are equivalent if and only if they have the same rank. Example 0.4 Two matrices (, ) C ²- ³ are said to be similar if there exists an invertible matrix 7 such that ( ~ 7 )7 c Similarity is easily seen to be an equivalence relation on C . As we will learn, two matrices are similar if and only if they represent the same linear operators on a given -dimensional vector space = . Hence, similarity is extremely important for studying the structure of linear operators. One of the main goals of this book is to develop canonical forms for similarity. We leave it to the reader to show that the determinant function and the trace function are invariants for similarity. However, these two invariants do not, in general, form a complete system of invariants. Example 0.5 Two matrices (, ) C ²- ³ are said to be congruent if there exists an invertible matrix 7 for which ( ~ 7 )7 ! where 7 ! is the transpose of 7 . This relation is easily seen to be an equivalence relation and we will devote some effort to finding canonical forms for congruence. For some base fields - (such as s, d or a finite field), this is relatively easy to do, but for other base fields (such as r), it is extremely difficult.

10


Zorn's Lemma In order to show that any vector space has a basis, we require a result known as Zorn's lemma. To state this lemma, we need some preliminary definitions. Definition A partially ordered set is a pair ²7 Á ³ where 7 is a nonempty set and is a binary relation called a partial order, read “less than or equal to,” with the following properties: 1) (Reflexivity) For all 7 , 2) (Antisymmetry) For all Á 7 , and implies ~ 3) (Transitivity) For all Á Á 7 , and implies Partially ordered sets are also called posets. It is customary to use a phrase such as “Let 7 be a partially ordered set” when the partial order is understood. Here are some key terms related to partially ordered sets. Definition Let 7 be a partially ordered set. 1) A maximal element is an element 7 with the property that there is no larger element in 7 , that is 7Á ¬ ~ 2) A minimal element is an element 7 with the property that there is no smaller element in 7 , that is 7Á ¬ ~ 3) Let Á 7 . Then " 7 is an upper bound for and if " and " The unique smallest upper bound for and , if it exists, is called the least upper bound of and and is denoted by lub¸Á ¹. 4) Let Á 7 . Then M 7 is a lower bound for and if M and M The unique largest lower bound for and , if it exists, is called the greatest lower bound of and and is denoted by glb¸Á ¹.

Preliminaries

11

Let : be a subset of a partially ordered set 7 . We say that an element " 7 is an upper bound for : if " for all : . Lower bounds are defined similarly. Note that in a partially ordered set, it is possible that not all elements are comparable. In other words, it is possible to have %Á & 7 with the property that % & and & %. Definition A partially ordered set in which every pair of elements is comparable is called a totally ordered set, or a linearly ordered set. Any totally ordered subset of a partially ordered set 7 is called a chain in 7 . Example 0.6 1) The set s of real numbers, with the usual binary relation , is a partially ordered set. It is also a totally ordered set. It has no maximal elements. 2) The set o ~ ¸Á Á Ã ¹ of natural numbers, together with the binary relation of divides, is a partially ordered set. It is customary to write to indicate that divides . The subset : of o consisting of all powers of is a totally ordered subset of o, that is, it is a chain in o. The set 7 ~ ¸Á Á Á Á Á ¹ is a partially ordered set under . It has two maximal elements, namely and . The subset 8 ~ ¸Á Á Á Á ¹ is a partially ordered set in which every element is both maximal and minimal! 3) Let : be any set and let F ²:³ be the power set of : , that is, the set of all subsets of : . Then F ²:³, together with the subset relation , is a partially ordered set. Now we can state Zorn's lemma, which gives a condition under which a partially ordered set has a maximal element. Theorem 0.10 (Zorn's lemma) If 7 is a partially ordered set in which every chain has an upper bound then 7 has a maximal element. We will not prove Zorn's lemma. Indeed, Zorn's lemma is a result that is so fundamental that it cannot be proved or disproved in the context of ordinary set theory. (It is equivalent to the famous Axiom of Choice.) Therefore, Zorn's lemma (along with the Axiom of Choice) must either be accepted or rejected as an axiom of set theory. Since almost all mathematicians accept it, we will do so as well. Indeed, we will use Zorn's lemma to prove that every vector space has a basis.

Cardinality Two sets : and ; have the same cardinality, written (: ( ~ (; ( if there is a bijective function (a one-to-one correspondence) between the sets.

12


The reader is probably aware of the fact that ({( ~ (o( and (r( ~ (o( where o denotes the natural numbers, { the integers and r the rational numbers. If : is in one-to-one correspondence with a subset of ; , we write (: ( (; (. If : is in one-to-one correspondence with a proper subset of ; but not all of ; then we write (: ( (; (. The second condition is necessary, since, for instance, o is in one-to-one correspondence with a proper subset of { and yet o is also in one-to-one correspondence with { itself. Hence, (o( ~ ({(. This is not the place to enter into a detailed discussion of cardinal numbers. The intention here is that the cardinality of a set, whatever that is, represents the “size” of the set. It is actually easier to talk about two sets having the same, or different, size (cardinality) than it is to explicitly define the size (cardinality) of a given set. Be that as it may, we associate to each set : a cardinal number, denoted by (: ( or card²:³, that is intended to measure the size of the set. Actually, cardinal numbers are just very special types of sets. However, we can simply think of them as vague amorphous objects that measure the size of sets. Definition 1) A set is finite if it can be put in one-to-one correspondence with a set of the form { ~ ¸Á Á Ã Á c ¹, for some nonnegative integer . A set that is not finite is infinite. The cardinal number (or cardinality) of a finite set is just the number of elements in the set. 2) The cardinal number of the set o of natural numbers is L (read “aleph nought”), where L is the first letter of the Hebrew alphabet. Hence, (o( ~ ({( ~ (r( ~ L 3) Any set with cardinality L is called a countably infinite set and any finite or countably infinite set is called a countable set. An infinite set that is not countable is said to be uncountable. Since it can be shown that (s( (o(, the real numbers are uncountable. If : and ; are finite sets then it is well known that (: ( (; ( and (; ( (: ( ¬ (: ( ~ (; ( The first part of the next theorem tells us that this is also true for infinite sets. The reader will no doubt recall that the power set F²:³ of a set : is the set of all subsets of : . For finite sets, the power set of : is always bigger than the set

Preliminaries

13

itself. In fact, (: ( ~ ¬ (F ²:³( ~ The second part of the next theorem says that the power set of any set : is bigger (has larger cardinality) than : itself. On the other hand, the third part of this theorem says that, for infinite sets : , the set of all finite subsets of : is the same size as : . Theorem 0.11 ¨ –Bernstein theorem) For any sets : and ; , 1) (Schroder (: ( (; ( and (; ( (: ( ¬ (: ( ~ (; ( 2) (Cantor's theorem) If F²:³ denotes the power set of : then (: ( (F ²:³( 3) If F ²:³ denotes the set of all finite subsets of : and if : is an infinite set then (: ( ~ (F ²:³( Proof. We prove only parts 1) and 2). Let ¢ : ¦ ; be an injective function from : into ; and let ¢ ; ¦ : be an injective function from ; into : . We want to use these functions to create a bijective function from : to ; . For this purpose, we make the following definitions. The descendants of an element : are the elements obtained by repeated alternate applications of the functions and , namely ² ³Á ² ² ³³Á ²² ² ³³³Á Ã If ! is a descendant of then is an ancestor of !. Descendants and ancestors of elements of ; are defined similarly. Now, by tracing an element's ancestry to its beginning, we find that there are three possibilities: the element may originate in : , or in ; , or it may have no point of origin. Accordingly, we can write : as the union of three disjoint sets I: ~ ¸ : originates in :¹ I; ~ ¸ : originates in ; ¹ IB ~ ¸ : has no originator¹ Similarly, ; is the disjoint union of J: , J; and JB . Now, the restriction OI: ¢ I: ¦ J: is a bijection. To see this, note that if ! J: then ! originated in : and therefore must have the form ² ³ for some : . But ! and its ancestor ² ³ have the

14


same point of origin and so ! J: implies I: . Thus, OI: is surjective and hence bijective. We leave it to the reader to show that the functions ²OJ; ³c ¢ I; ¦ J; and OIB ¢ IB ¦ JB are also bijections. Putting these three bijections together gives a bijection between : and ; . Hence, (: ( ~ (; (, as desired. We now prove Cantor's Theorem. The map ¢ : ¦ F ²:³ defined by ² ³ ~ ¸ ¹ is an injection from : to F ²:³ and so (: ( (F ²:³(. To complete the proof we must show that if no injective map ¢ : ¦ F ²:³ can be surjective. To this end, let ?~¸ :

¤ ² ³¹ F ²:³

We claim that ? is not in im² ³. For suppose that ? ~ ²%³ for some % : . Then if % ? , we have by the definition of ? that % ¤ ? . On the other hand, if % ¤ ? , we have again by the definition of ? that % ? . This contradiction implies that ? ¤ im² ³ and so is not surjective. Cardinal Arithmetic Now let us define addition, multiplication and exponentiation of cardinal numbers. If : and ; are sets, the cartesian product : d ; is the set of all ordered pairs : d ; ~ ¸² Á !³

:Á ! ; ¹

The set of all functions from ; to : is denoted by : ; . Definition Let and denote cardinal numbers. Let : and ; be any sets for which (: ( ~ and (; ( ~ . 1) The sum b is the cardinal number of : r ; . 2) The product is the cardinal number of : d ; . 3) The power is the cardinal number of : ; . We will not go into the details of why these definitions make sense. (For instance, they seem to depend on the sets : and ; , but in fact they do not.) It can be shown, using these definitions, that cardinal addition and multiplication are associative and commutative and that multiplication distributes over addition. Theorem 0.12 Let , and be cardinal numbers. Then the following properties hold: 1) (Associativity) b ² b ³ ~ ² b ³ b and ²³ ~ ²³

Preliminaries

15

2) (Commutativity) b ~ b and ~ 3) (Distributivity) ² b ³ ~ b 4) (Properties of Exponents) a) b ~ b) ² ³ ~ c) ²³ ~ On the other hand, the arithmetic of cardinal numbers can seem a bit strange, as the next theorem shows. Theorem 0.13 Let and be cardinal numbers, at least one of which is infinite. Then b ~ ~ max¸Á ¹

It is not hard to see that there is a one-to-one correspondence between the power set F²:³ of a set : and the set of all functions from : to ¸Á ¹. This leads to the following theorem. Theorem 0.14 For any cardinal 1) If (: ( ~ then (F ²:³( ~ 2) We have already observed that (o( ~ L . It can be shown that L is the smallest infinite cardinal, that is, L0 ¬ is a natural number It can also be shown that the set s of real numbers is in one-to-one correspondence with the power set F ²o³ of the natural numbers. Therefore, (s( ~ L The set of all points on the real line is sometimes called the continuum and so L is sometimes called the power of the continuum and denoted by . Theorem 0.13 shows that cardinal addition and multiplication have a kind of “absorption” quality, which makes it hard to produce larger cardinals from smaller ones. The next theorem demonstrates this more dramatically. Theorem 0.15 1) Addition applied a countable number of times or multiplication applied a finite number of times to the cardinal number L , does not yield anything

16


more than L . Specifically, for any nonzero o, we have L h L ~ L and L ~ L 2) Addition and multiplication, applied a countable number of times to the cardinal number L does not yield more than L . Specifically, we have L h L ~ L and ²L ³L ~ L

Using this theorem, we can establish other relationships, such as L ²L ³L ²L ³L ~ L which, by the Schro¨der–Bernstein theorem, implies that ²L ³L ~ L We mention that the problem of evaluating in general is a very difficult one and would take us far beyond the scope of this book. We will have use for the following reasonable–sounding result, whose proof is omitted. Theorem 0.16 Let ¸( 2¹ be a collection of sets, indexed by the set 2 , with (2 ( ~ . If (( ( for all 2 then e ( e

2

Let us conclude by describing the cardinality of some famous sets. Theorem 0.17 1) The following sets have cardinality L . a) The rational numbers r. b) The set of all finite subsets of o. c) The union of a countable number of countable sets. d) The set { of all ordered -tuples of integers. 2) The following sets have cardinality L . a) The set of all points in s . b) The set of all infinite sequences of natural numbers. c) The set of all infinite sequences of real numbers. d) The set of all finite subsets of s. e) The set of all irrational numbers.

Part 2 Algebraic Structures We now turn to a discussion of some of the many algebraic structures that play a role in the study of linear algebra.

Preliminaries

17

Groups Definition A group is a nonempty set ., together with a binary operation denoted by *, that satisfies the following properties: 1) (Associativity) For all Á Á . ²i³i ~ i²i³ 2) (Identity) There exists an element . for which i ~ i ~ for all .. 3) (Inverses) For each ., there is an element c . for which ic ~ c i ~

Definition A group . is abelian, or commutative, if i ~ i for all Á . . When a group is abelian, it is customary to denote the operation i by +, thus writing i as b . It is also customary to refer to the identity as the zero element and to denote the inverse c by c, referred to as the negative of . Example 0.7 The set < of all bijective functions from a set : to : is a group under composition of functions. However, in general, it is not abelian. Example 0.8 The set CÁ ²- ³ is an abelian group under addition of matrices. The identity is the zero matrix 0Á of size d . The set C ²- ³ is not a group under multiplication of matrices, since not all matrices have multiplicative inverses. However, the set of invertible matrices of size d is a (nonabelian) group under multiplication. A group . is finite if it contains only a finite number of elements. The cardinality of a finite group . is called its order and is denoted by ².³ or simply (.(. Thus, for example, { ~ ¸Á Á Ã Á c ¹ is a finite group under addition modulo , but CÁ ²s³ is not finite. Definition A subgroup of a group . is a nonempty subset : of . that is a group in its own right, using the same operations as defined on . .

Rings Definition A ring is a nonempty set 9 , together with two binary operations, called addition (denoted by b ) and multiplication (denoted by juxtaposition), for which the following hold: 1) 9 is an abelian group under addition

18


2) (Associativity) For all Á Á 9 , ²³ ~ ²³ 3) (Distributivity) For all Á Á 9 , ² b ³ ~ b and ² b ³ ~ b A ring 9 is said to be commutative if ~ for all Á 9 . If a ring 9 contains an element with the property that ~ ~ for all 9 , we say that 9 is a ring with identity. The identity is usually denoted by . Example 0.9 The set { ~ ¸Á Á Ã Á c¹ is a commutative ring under addition and multiplication modulo l ~ ² b ³ mod Á

p ~ mod

The element { is the identity. Example 0.10 The set , of even integers is a commutative ring under the usual operations on {, but it has no identity. Example 0.11 The set C ²- ³ is a noncommutative ring under matrix addition and multiplication. The identity matrix 0 is the identity for C ²- ³. Example 0.12 Let - be a field. The set - ´%µ of all polynomials in a single variable %, with coefficients in - , is a commutative ring, under the usual operations of polynomial addition and multiplication. What is the identity for - ´%µ? Similarly, the set - ´% Á Ã Á % µ of polynomials in variables is a commutative ring under the usual addition and multiplication of polynomials. Definition A subring of a ring 9 is a subset : of 9 that is a ring in its own right, using the same operations as defined on 9 and having the same multiplicative identity as 9 . The condition that a subring : have the same multiplicative identity as 9 is required. For example, the set : of all d matrices of the form ( ~ >

?

for - is a ring under addition and multiplication of matrices (isomorphic to - ). The multiplicative identity in : is the matrix ( , which is not the identity 0 of CÁ ²- ³. Hence, : is a ring under the same operations as CÁ ²- ³ but it is not a subring of CÁ ²- ³.

Preliminaries

19

Applying the definition is not generally the easiest way to show that a subset of a ring is a subring. The following characterization is usually easier to apply. Theorem 0.18 A nonempty subset : of a ring 9 is a subring if and only if 1) The multiplicative identity 9 of 9 is in : 2) : is closed under subtraction, that is Á : ¬ c : 3) : is closed under multiplication, that is, Á : ¬ :

Ideals Rings have another important substructure besides subrings. Definition Let 9 be a ring. A nonempty subset ? of 9 is called an ideal if 1) ? is a subgroup of the abelian group 9, that is, ? is closed under subtraction Á ? ¬ c ? 2) ? is closed under multiplication by any ring element, that is, ? Á 9 ¬ ? and ?

Note that if an ideal ? contains the unit element then ? ~ 9 . Example 0.13 Let ²%³ be a polynomial in - ´%µ. The set of all multiples of ²%³ º²%³» ~ ¸²%³²%³ ²%³ - ´%µ¹ is an ideal in - ´%µ, called the ideal generated by ²%³. Definition Let : be a subset of a ring 9 with identity. The set º:» ~ ¸

b Ä b

9Á

:Á ¹

of all finite linear combinations of elements of : , with coefficients in 9 , is an ideal in 9 , called the ideal generated by : . It is the smallest (in the sense of set inclusion) ideal of 9 containing : . If : ~ ¸ Á Ã Á ¹ is a finite set, we write º Á Ã Á

»

~ ¸

b Ä b

9Á

:¹

Note that in the previous definition, we require that 9 have an identity. This is to ensure that : º:». Theorem 0.19 Let 9 be a ring.

20


1) The intersection of any collection ¸? 2¹ of ideals is an ideal. 2) If ? ? Ä is an ascending sequence of ideals, each one contained in the next, then the union ? is also an ideal. 3) More generally, if 9 ~ ¸? 0¹ is a chain of ideals in 9 then the union @ ~ 0 ? is also an ideal in 9 . Proof. To prove 1), let @ ~ ? . Then if Á @ , we have Á ? for all 2 . Hence, c ? for all 2 and so c @ . Hence, @ is closed under subtraction. Also, if 9 then ? for all 2 and so @ . Of course, part 2) is a special case of part 3). To prove 3), if Á @ then ? and ? for some Á 0 . Since one of ? and ? is contained in the other, we may assume that ? ? . It follows that Á ? and so c ? @ and if 9 then ? @ . Thus @ is an ideal. Note that in general, the union of ideals is not an ideal. However, as we have just proved, the union of any chain of ideals is an ideal. Quotient Rings and Maximal Ideals Let : be a subset of a commutative ring 9 with identity. Let be the binary relation on 9 defined by ¯ c : It is easy to see that is an equivalence relation. When , we say that and are congruent modulo : . The term “mod” is used as a colloquialism for modulo and is often written mod : As shorthand, we write . To see what the equivalence classes look like, observe that ´µ ~ ¸ 9 ¹ ~ ¸ 9 c :¹ ~ ¸ 9 ~ b for some :¹ ~ ¸ b :¹ ~b: The set b : ~ ¸ b

:¹

is called a coset of : in 9 . The element is called a coset representative for b :.

Preliminaries

21

Thus, the equivalence classes for congruence mod : are the cosets b : of : in 9 . The set of all cosets is denoted by 9 ~ ¸ b : 9¹ : This is read “9 mod : .” We would like to place a ring structure on 9°: . Indeed, if : is a subgroup of the abelian group 9 then 9°: is easily seen to be an abelian group as well under coset addition defined by ² b :³ b ² b :³ ~ ² b ³ b : In order for the product ² b :³² b :³ ~ b : to be well defined we must have b : ~ Z b : ¬ b : ~ Z b : or, equivalently, c Z : ¬ ² c Z ³ : But c Z may be any element of : and may be any element of 9 and so this condition implies that : must be an ideal. Conversely, if : is an ideal then coset multiplication is well defined. Theorem 0.20 Let 9 be a commutative ring with identity. Then the quotient 9°? is a ring under coset addition and multiplication if and only if ? is an ideal of 9. In this case, 9°? is called the quotient ring of 9 modulo ? , where addition and multiplication are defined by ² b :³ b ² b :³ ~ ² b ³ b : ² b :³² b :³ ~ b :

Definition An ideal ? in a ring 9 is a maximal ideal if ? £ 9 and if whenever @ is an ideal satisfying ? @ 9 then either @ ~ ? or @ ~ 9 . Here is one reason why maximal ideals are important. Theorem 0.21 Let 9 be a commutative ring with identity. Then the quotient ring 9°? is a field if and only if ? is a maximal ideal. Proof. First, note that for any ideal ? of 9, the ideals of 9°? are precisely the quotients @ °? where @ is an ideal for which ? @ 9. It is clear that @ °? is an ideal of 9°? . Conversely, if AZ is an ideal of 9°? then let A ~ ¸ 9 b ? AZ ¹ It is easy to see that A is an ideal of 9 for which ? A 9 .

22


Next, observe that a commutative ring : with identity is a field if and only if : has no nonzero proper ideals. For if : is a field and ? is an ideal of : containing a nonzero element then ~ c ? and so ? ~ : . Conversely, if : has no nonzero proper ideals and £ : then the ideal º » must be : and so there is an : for which ~ . Hence, : is a field. Putting these two facts together proves the theorem. The following result says that maximal ideals always exist. Theorem 0.22 Any commutative ring 9 with identity contains a maximal ideal. Proof. Since 9 is not the zero ring, the ideal ¸¹ is a proper ideal of 9 . Hence, the set I of all proper ideals of 9 is nonempty. If 9 ~ ¸? 0¹ is a chain of proper ideals in 9 then the union @ ~ 0 ? is also an ideal. Furthermore, if @ ~ 9 is not proper, then @ and so ? , for some 0 , which implies that ? ~ 9 is not proper. Hence, @ I . Thus, any chain in I has an upper bound in I and so Zorn's lemma implies that I has a maximal element. This shows that 9 has a maximal ideal.

Integral Domains Definition Let 9 be a ring. A nonzero element r 9 is called a zero divisor if there exists a nonzero 9 for which ~ . A commutative ring 9 with identity is called an integral domain if it contains no zero divisors. Example 0.14 If is not a prime number then the ring { has zero divisors and so is not an integral domain. To see this, observe that if is not prime then ~ in {, where Á . But in { , we have p ~ mod ~ and so and are both zero divisors. As we will see later, if is a prime then { is a field (which is an integral domain, of course). Example 0.15 The ring - ´%µ is an integral domain, since ²%³²%³ ~ implies that ²%³ ~ or ²%³ ~ . If 9 is a ring and % ~ & where Á %Á & 9 then we cannot in general cancel the 's and conclude that % ~ &. For instance, in { , we have h ~ h , but canceling the 's gives ~ . However, it is precisely the integral domains in which we can cancel. The simple proof is left to the reader. Theorem 0.23 Let 9 be a commutative ring with identity. Then 9 is an integral domain if and only if the cancellation law

Preliminaries

23

% ~ &Á £ ¬ % ~ & holds.

The Field of Quotients of an Integral Domain Any integral domain 9 can be embedded in a field. The quotient field (or field of quotients) of 9 is a field that is constructed from 9 just as the field of rational numbers is constructed from the ring of integers. In particular, we set 9 b ~ ¸²Á ³ Á 9Á £ ¹ Thinking of ²Á ³ as the “fraction” ° we define addition and multiplication of fractions in the same way as for rational numbers ²Á ³ b ²Á ³ ~ ² b Á ³ and ²Á ³ h ²Á ³ ~ ²Á ³ It is customary to write ²Á ³ in the form ° . Note that if 9 has zero divisors, then these definitions do not make sense, because may be even if and are not. This is why we require that 9 be an integral domain.

Principal Ideal Domains Definition Let 9 be a ring with identity and let 9 . The principal ideal generated by is the ideal º» ~ ¸ 9¹ An integral domain 9 in which every ideal is a principal ideal is called a principal ideal domain. Theorem 0.24 The integers form a principal ideal domain. In fact, any ideal ? in { is generated by the smallest positive integer a that is contained in ? . Theorem 0.25 The ring - ´%µ is a principal ideal domain. In fact, any ideal ? is generated by the unique monic polynomial of smallest degree contained in ? . Moreover, for polynomials ²%³Á Ã Á ²%³, º ²%³Á Ã Á ²%³» ~ ºgcd¸ ²%³Á Ã Á ²%³¹» Proof. Let ? be an ideal in - ´%µ and let ²%³ be a monic polynomial of smallest degree in ? . First, we observe that there is only one such polynomial in ? . For if ²%³ ? is monic and deg²²%³³ ~ deg²²%³³ then ²%³ ~ ²%³ c ²%³ ? and since deg²²%³³ deg²²%³³, we must have ²%³ ~ and so ²%³ ~ ²%³.

24


We show that ? ~ º²%³». Since ²%³ ? , we have º²%³» ? . To establish the reverse inclusion, if ²%³ ? then dividing ²%³ by ²%³ gives ²%³ ~ ²%³²%³ b ²%³ where ²%³ ~ or deg ²%³ deg ²%³. But since ? is an ideal, ²%³ ~ ²%³ c ²%³²%³ ? and so deg ²%³ deg ²%³ is impossible. Hence, ²%³ ~ and ²%³ ~ ²%³²%³ º²%³» This shows that ? º²%³» and so ? ~ º²%³». To prove the second statement, let ? ~ º ²%³Á Ã Á ²%³». Then, by what we have just shown, ? ~ º ²%³Á Ã Á ²%³» ~ º²%³» where ²%³ is the unique monic polynomial ²%³ in ? of smallest degree. In particular, since ²%³ º²%³», we have ²%³ ²%³ for each ~ Á Ã Á . In other words, ²%³ is a common divisor of the ²%³'s. Moreover, if ²%³ ²%³ for all , then ²%³ º²%³» for all , which implies that ²%³ º²%³» ~ º ²%³Á Ã Á ²%³» º²%³» and so ²%³ ²%³. This shows that ²%³ is the greatest common divisor of the ²%³'s and completes the proof. Example 0.16 The ring 9 ~ - ´%Á &µ of polynomials in two variables % and & is not a principal ideal domain. To see this, observe that the set ? of all polynomials with zero constant term is an ideal in 9 . Now, suppose that ? is the principal ideal ? ~ º²%Á &³». Since %Á & ? , there exist polynomials ²%Á &³ and ²%Á &³ for which % ~ ²%Á &³²%Á &³ and & ~ ²%Á &³²%Á &³

(0.1)

But ²%Á &³ cannot be a constant for then we would have ? ~ 9 . Hence, deg²²%Á &³³ and so ²%Á &³ and ²%Á &³ must both be constants, which implies that (0.1) cannot hold. Theorem 0.26 Any principal ideal domain 9 satisfies the ascending chain condition, that is, 9 cannot have a strictly increasing sequence of ideals ? ? Ä where each ideal is properly contained in the next one.

Preliminaries

25

Proof. Suppose to the contrary that there is such an increasing sequence of ideals. Consider the ideal < ~ ? which must have the form < ~ º» for some < . Since ? for some , we have ? ~ ? for all , contradicting the fact that the inclusions are proper.

Prime and Irreducible Elements We can define the notion of a prime element in any integral domain. For Á 9 , we say that divides (written ) if there exists an % 9 for which ~ %. Definition Let 9 be an integral domain. 1) An invertible element of 9 is called a unit. Thus, " 9 is a unit if "# ~ for some # 9 . 2) Two elements Á 9 are said to be associates if there exists a unit " for which ~ " . 3) A nonzero nonunit 9 is said to be prime if ¬ or 4) A nonzero nonunit 9 is said to be irreducible if ~ ¬ or is a unit

Note that if is prime or irreducible then so is " for any unit ". Theorem 0.27 Let 9 be a ring. 1) An element " 9 is a unit if and only if º"» ~ 9 . 2) and are associates if and only if º» ~ º ». 3) divides if and only if º » º». 4) properly divides , that is, ~ % where % is not a unit, if and only if º » º». In the case of the integers, an integer is prime if and only if it is irreducible. In any integral domain, prime elements are irreducible, but the converse need not hold. (In the ring {´jc µ ~ ¸ b jc Á {¹ the prime element divides the product ² b jc ³² c jc ³ ~ but does not divide either factor.) However, in principal ideal domains, the two concepts are equivalent. Theorem 0.28 Let 9 be a principal ideal domain. 1) An 9 is irreducible if and only if the ideal º» is maximal. 2) An element in 9 is prime if and only if it is irreducible.

26


3) The elements Á 9 are relatively prime, that is, have no common nonunit factors if and only if there exist Á 9 for which b ~ Proof. To prove 1), suppose that is irreducible and that º» º» 9 . Then º» and so ~ % for some % 9. The irreducibility of implies that or % is a unit. If is a unit then º» ~ 9 and if % is a unit then º» ~ º%» ~ º». This shows that º» is maximal. (We have º» £ 9 , since is not a unit.) Conversely, suppose that is not irreducible, that is, ~ where neither nor is a unit. Then º» º» 9 . But if º» ~ º» then and are associates, which implies that is a unit. Hence º» £ º». Also, if º» ~ 9 then must be a unit. So we conclude that º» is not maximal, as desired. To prove 2), assume first that is prime and ~ . Then or . We may assume that . Therefore, ~ % ~ % . Canceling 's gives ~ % and so is a unit. Hence, is irreducible. (Note that this argument applies in any integral domain.) Conversely, suppose that is irreducible and let . We wish to prove that or . The ideal º» is maximal and so ºÁ » ~ º» or ºÁ » ~ 9 . In the former case, and we are done. In the latter case, we have ~ % b & for some %Á & 9 . Thus, ~ % b & and since divides both terms on the right, we have . To prove 3), it is clear that if b ~ then and are relatively prime. For the converse, consider the ideal ºÁ » which must be principal, say ºÁ » ~ º%». Then % and % and so % must be a unit, which implies that ºÁ » ~ 9 . Hence, there exists Á 9 for which b ~ .

Unique Factorization Domains Definition An integral domain 9 is said to be a unique factorization domain if it has the following factorization properties: 1) Every nonzero nonunit element 9 can be written as a product of a finite number of irreducible elements ~ Ä . 2) The factorization into irreducible elements is unique in the sense that if ~ Ä and ~ Ä are two such factorizations then ~ and after a suitable reindexing of the factors, and are associates. Unique factorization is clearly a desirable property. Fortunately, principal ideal domains have this property.

Preliminaries

27

Theorem 0.29 Every principal ideal domain 9 is a unique factorization domain. Proof. Let 9 be a nonzero nonunit. If is irreducible then we are done. If not then ~ , where neither factor is a unit. If and are irreducible, we are done. If not, suppose that is not irreducible. Then ~ , where neither nor is a unit. Continuing in this way, we obtain a factorization of the form (after renumbering if necessary) ~ ~ ² ³ ~ ² ³² ³ ~ ² ³² ³ ~ Ä Each step is a factorization of into a product of nonunits. However, this process must stop after a finite number of steps, for otherwise it will produce an infinite sequence Á Á Ã of nonunits of 9 for which b properly divides . But this gives the ascending chain of ideals º » º » º » º » Ä where the inclusions are proper. But this contradicts the fact that a principal ideal domain satisfies the ascending chain condition. Thus, we conclude that every nonzero nonunit has a factorization into irreducible elements. As to uniqueness, if ~ Ä and ~ Ä are two such factorizations then because 9 is an integral domain, we may equate them and cancel like factors, so let us assume this has been done. Thus, £ for all Á . If there are no factors on either side, we are done. If exactly one side has no factors left then we have expressed as a product of irreducible elements, which is not possible since irreducible elements are nonunits. Suppose that both sides have factors left, that is, Ä ~ Ä where £ . Then Ä , which implies that for some . We can assume by reindexing if necessary that ~ . Since is irreducible must be a unit. Replacing by and canceling gives Äc ~ Äc This process can be repeated until we run out of 's or 's. If we run out of 's first then we have an equation of the form " Ä ~ where " is a unit, which is not possible since the 's are not units. By the same reasoning, we cannot run out of 's first and so ~ and the 's and 's can be paired off as associates.

Fields For the record, let us give the definition of a field (a concept that we have been using).

28


Definition A field is a set - , containing at least two elements, together with two binary operations, called addition (denoted by b ) and multiplication (denoted by juxtaposition), for which the following hold: 1) - is an abelian group under addition. 2) The set - i of all nonzero elements in - is an abelian group under multiplication. 3) (Distributivity) For all Á Á - , ² b ³ ~ b and ² b ³ ~ b

We require that - have at least two elements to avoid the pathological case, in which ~ . Example 0.17 The sets r, s and d, of all rational, real and complex numbers, respectively, are fields, under the usual operations of addition and multiplication of numbers. Example 0.18 The ring { is a field if and only if is a prime number. We have already seen that { is not a field if is not prime, since a field is also an integral domain. Now suppose that ~ is a prime. We have seen that { is an integral domain and so it remains to show that every nonzero element in { has a multiplicative inverse. Let £ { . Since , we know that and are relatively prime. It follows that there exist integers " and # for which " b # ~ Hence, " ² c #³ mod and so " p ~ in { , that is, " is the multiplicative inverse of . The previous example shows that not all fields are infinite sets. In fact, finite fields play an extremely important role in many areas of abstract and applied mathematics. A field - is said to be algebraically closed if every nonconstant polynomial over - has a root in - . This is equivalent to saying that every nonconstant polynomial splits into linear factors over - . For example, the complex field d is algebraically closed but the real field s is not. We mention without proof that every field - is contained in an algebraically closed field - , called the algebraic closure of - .

The Characteristic of a Ring Let 9 be a ring with identity. If is a positive integer then by h , we simply mean

Preliminaries

29

h ~ bÄb terms Now, it may happen that there is a positive integer for which h~ For instance, in { , we have h ~ ~ . On the other hand, in {, the equation h ~ implies ~ and so no such positive integer exists. Notice that, in any finite ring, there must exist such a positive integer , since the infinite sequence of numbers h Á h Á h Á Ã cannot be distinct and so h ~ h for some , whence ² c ³ h ~ . Definition Let 9 be a ring with identity. The smallest positive integer for which h ~ is called the characteristic of 9 . If no such number exists, we say that 9 has characteristic . The characteristic of 9 is denoted by char²9³. If char²9³ ~ then for any 9 , we have h ~ b Ä b ~ ² b Ä b ³ ~ h ~ terms terms Theorem 0.30 Any finite ring has nonzero characteristic. Any finite field has prime characteristic. Proof. We have already seen that a finite ring has nonzero characteristic. Let be a finite field and suppose that char²- ³ ~ . If ~ , where Á then h ~ . Hence, ² h ³² h ³ ~ , implying that h ~ or h ~ . In either case, we have a contradiction to the fact that is the smallest positive integer such that h ~ . Hence, must be prime. Notice that in any field - of characteristic , we have ~ for all - . Thus, in ~ c for all This property takes a bit of getting used to and makes fields of characteristic quite exceptional. (As it happens, there are many important uses for fields of characteristic .)

Algebras The final algebraic structure of which we will have use is a combination of a vector space and a ring. (We have not yet officially defined vector spaces, but we will do so before needing the following definition, which is placed here for easy reference.)

30


Definition An algebra 7 over a field - is a nonempty set 7, together with three operations, called addition (denoted by b ), multiplication (denoted by juxtaposition) and scalar multiplication (also denoted by juxtaposition), for which the following properties hold: 1) 7 is a vector space over - under addition and scalar multiplication. 2) 7 is a ring under addition and multiplication. 3) If - and Á 7 then ²³ ~ ²³ ~ ²³

Thus, an algebra is a vector space in which we can take the product of vectors, or a ring in which we can multiply each element by a scalar (subject, of course, to additional requirements as given in the definition).

Part I—Basic Linear Algebra

Chapter 1

Vector Spaces

Vector Spaces Let us begin with the definition of one of our principal objects of study. Definition Let - be a field, whose elements are referred to as scalars. A vector space over - is a nonempty set = , whose elements are referred to as vectors, together with two operations. The first operation, called addition and denoted by b , assigns to each pair ²"Á #³ of vectors in = a vector " b # in = . The second operation, called scalar multiplication and denoted by juxtaposition, assigns to each pair ²rÁ "³ - d = a vector " in = . Furthermore, the following properties must be satisfied: 1) (Associativity of addition) For all vectors "Á #Á $ = " b ²# b $³ ~ ²" b #³ b $ 2) (Commutativity of addition) For all vectors "Á # = "b#~#b" 3) (Existence of a zero) There is a vector = with the property that b"~"b~" for all vectors " = . 4) (Existence of additive inverses) For each vector " = , there is a vector in = , denoted by c", with the property that " b ²c"³ ~ ²c"³ b " ~

34


5) (Properties of scalar multiplication) For all scalars Á F and for all vectors "Á # = ²" b #³ ~ " b # ² b ³" ~ " b " ²³" ~ ²"³ " ~ "

Note that the first four properties in the definition of vector space can be summarized by saying that = is an abelian group under addition. Any expression of the form # b Ä b # where - and # = for all , is called a linear combination of the vectors # Á Ã Á # . If at least one of the scalars is nonzero, then the linear combination is nontrivial. Example 1.1 1) Let - be a field. The set - - of all functions from - to - is a vector space over - , under the operations of ordinary addition and scalar multiplication of functions ² b ³²%³ ~ ²%³ b ²%³ and ² ³²%³ ~ ² ²%³³ 2) The set CÁ ²- ³ of all d matrices with entries in a field - is a vector space over - , under the operations of matrix addition and scalar multiplication. 3) The set - of all ordered -tuples, whose components lie in a field - , is a vector space over - , with addition and scalar multiplication defined componentwise ² Á Ã Á ³ b ² Á Ã Á ³ ~ ² b Á Ã Á b ³ and ² Á Ã Á ³ ~ ² Á Ã Á ³ When convenient, we will also write the elements of - in column form. When - is a finite field - with elements, we write = ²Á ³ for - . 4) Many sequence spaces are vector spaces. The set Seq²- ³ of all infinite sequences with members from a field - is a vector space under componentwise operations ² ³ b ²! ³ ~ ²

b ! ³

Vector Spaces

35

and ² ³ ~ ² ³ In a similar way, the set of all sequences of complex numbers that converge to is a vector space, as is the set MB of all bounded complex sequences. Also, if is a positive integer then the set M of all complex sequences ² ³ for which B

( ( B ~

is a vector space under componentwise operations. To see that addition is a binary operation on M , one verifies Minkowski's inequality °

B

8 ( ~

b ! ( 9

B

8 ( ( 9 ~

°

B

°

b 8 (! ( 9 ~

which we will not do here.

Subspaces Most algebraic structures contain substructures, and vector spaces are no exception. Definition A subspace of a vector space = is a subset : of = that is a vector space in its own right under the operations obtained by restricting the operations of = to : . Since many of the properties of addition and scalar multiplication hold a fortiori in a nonempty subset : , we can establish that : is a subspace merely by checking that : is closed under the operations of = . Theorem 1.1 A nonempty subset : of a vector space = is a subspace of = if and only if : is closed under addition and scalar multiplication or, equivalently, : is closed under linear combinations, that is Á - Á "Á # : ¬ " b # :

Example 1.2 Consider the vector space = ²Á ³ of all binary -tuples, that is, -tuples of 's and 's. The weight M ²#³ of a vector # = ²Á ³ is the number of nonzero coordinates in #. For instance, M ²³ ~ . Let , be the set of all vectors in = of even weight. Then , is a subspace of = ²Á ³. To see this, note that M ²" b #³ ~ M ²"³ b M ²#³ c M ²" q #³ where " q # is the vector in = ²Á ³ whose th component is the product of the

36


th components of " and #, that is, ²" q #³ ~ " h # Hence, if M ²"³ and M ²#³ are both even, so is M ²" b #³. Finally, scalar multiplication over - is trivial and so , is a subspace of = ²Á ³, known as the even weight subspace of = ²Á ³. Example 1.3 Any subspace of the vector space = ²Á ³ is called a linear code. Linear codes are among the most important and most studied types of codes, because their structure allows for efficient encoding and decoding of information.

The Lattice of Subspaces The set I²= ³ of all subspaces of a vector space = is partially ordered by set inclusion. The zero subspace ¸¹ is the smallest element in I ²= ³ and the entire space = is the largest element. If :Á ; I ²= ³ then : q ; is the largest subspace of = that is contained in both : and ; . In terms of set inclusion, : q ; is the greatest lower bound of : and ; : q ; ~ glb¸:Á ; ¹ Similarly, if ¸: 2¹ is any collection of subspaces of = then their intersection is the greatest lower bound of the subspaces

: ~ glb¸: 2¹ 2

On the other hand, if :Á ; I ²= ³ (and - is infinite) then : r ; I ²= ³ if and only if : ; or ; : . Thus, the union of two subspaces is never a subspace in any “interesting” case. We also have the following. Theorem 1.2 A nontrivial vector space = over an infinite field - is not the union of a finite number of proper subspaces. Proof. Suppose that = ~ : r Ä r : , where we may assume that : \ : r Ä r : Let $ : ± ²: r Ä r : ³ and let # ¤ : . Consider the infinite set ( ~ ¸$ b # - ¹ which is the “line” through #, parallel to $. We want to show that each : contains at most one vector from the infinite set (, which is contrary to the fact that = ~ : r Ä r : . This will prove the theorem.

Vector Spaces

37

If $ b # : for £ then $ : implies # : , contrary to assumption. Next, suppose that $ b # : and $ b # : , for , where £ . Then : ² $ b #³ c ² $ b #³ ~ ² c ³$ and so $ : , which is also contrary to assumption. To determine the smallest subspace of = containing the subspaces : and ; , we make the following definition. Definition Let : and ; be subspaces of = . The sum : b ; is defined by : b ; ~ ¸" b # " :Á # ; ¹ More generally, the sum of any collection ¸: 2¹ of subspaces is the set of all finite sums of vectors from the union : : ~ H 2

bÄb

: I

2

It is not hard to show that the sum of any collection of subspaces of = is a subspace of = and that in terms of set inclusion, the sum is the least upper bound : b ; ~ lub¸:Á ; ¹ More generally, : ~ lub¸: 2¹ 2

If a partially ordered set 7 has the property that every pair of elements has a least upper bound and greatest lower bound, then 7 is called a lattice. If 7 has a smallest element and a largest element and has the property that every collection of elements has a least upper bound and greatest lower bound, then 7 is called a complete lattice. Theorem 1.3 The set I²= ³ of all subspaces of a vector space = is a complete lattice under set inclusion, with smallest element ¸¹, largest element = , glb¸: 2¹ ~ : 2

and lub¸: 2¹ ~ : 2

38


Direct Sums As we will see, there are many ways to construct new vector spaces from old ones.

External Direct Sums Definition Let = Á Ã Á = be vector spaces over a field - . The external direct sum of = Á Ã Á = , denoted by = ~ = ^ Ä ^ = is the vector space = whose elements are ordered -tuples = ~ ¸²# Á Ã Á # ³ # = Á ~ Á Ã Á ¹ with componentwise operations ²" Á Ã Á " ³ b ²# Á Ã Á # ³ ~ ²" b # Á Ã Á " b #³ and ²# Á Ã Á # ³ ~ ²# Á Ã Á # ³

Example 1.4 The vector space - is the external direct sum of copies of - , that is, - ~ - ^ Ä ^ where there are summands on the right-hand side. This construction can be generalized to any collection of vector spaces by generalizing the idea that an ordered -tuple ²# Á Ã Á # ³ is just a function ¢ ¸Á Ã Á ¹ ¦ = from the index set ¸Á Ã Á ¹ to the union of the spaces with the property that ²³ = . Definition Let < ~ ¸= 2¹ be any family of vector spaces over - . The direct product of < is the vector space = ~ H ¢ 2 ¦ = d ²³ = I

2

2

thought of as a subspace of the vector space of all functions from 2 to = . It will prove more useful to restrict the set of functions to those with finite support. Definition Let < ~ ¸= 2¹ be a family of vector spaces over - . The support of a function ¢ 2 ¦ = is the set supp² ³ ~ ¸ 2 ²³ £ ¹

Vector Spaces

39

Thus, a function has finite support if ²³ ~ for all but a finite number of 2. The external direct sum of the family < is the vector space ext

= ~ H ¢ 2 ¦ = d ²³ = , has finite supportI 2

2

thought of as a subspace of the vector space of all functions from 2 to = . An important special case occurs when = ~ = for all 2 . If we let = 2 denote the set of all functions from 2 to = and ²= 2 ³ denote the set of all functions in = 2 that have finite support then ext

= ~ = 2 and = ~ ²= 2 ³ 2

2

Note that the direct product and the external direct sum are the same for a finite family of vector spaces.

Internal Direct Sums An internal version of the direct sum construction is often more relevant. Definition Let = be a vector space. We say that = is the (internal) direct sum of a family < ~ ¸: 2¹ of subspaces of = if every vector # = can be written, in a unique way (except for order), as a finite sum of vectors from the subspaces in < , that is, if for all # = , # ~ " b Ä b " for " : and furthermore, if # ~ $ b Ä b $ where $ : then ~ and (after reindexing if necessary) $ ~ " for all ~ Á Ã Á . If = is the direct sum of < , we write = ~ : 2

and refer to each : as a direct summand of = . If < ~ ¸: Á Ã Á : ¹ is a finite family, we write = ~ : l Ä l : If = ~ : l ; then ; is called a complement of : in = .

40


Note that a sum is direct if and only if whenever " b Ä b " ~ where " : and £ then " ~ for all , that is, if and only if has a unique representation as a sum of vectors from distinct subspaces. The reader will be asked in a later chapter to show that the concepts of internal and external direct sum are essentially equivalent (isomorphic). For this reason, we often use the term “direct sum” without qualification. Once we have discussed the concept of a basis, the following theorem can be easily proved. Theorem 1.4 Any subspace of a vector space has a complement, that is, if : is a subspace of = then there exists a subspace ; for which = ~ : l ; . It should be emphasized that a subspace generally has many complements (although they are isomorphic). The reader can easily find examples of this in s . We will have more to say about the existence and uniqueness of complements later in the book. The following characterization of direct sums is quite useful. Theorem 1.5 A vector space = is the direct sum of a family < ~ ¸: 2¹ of subspaces if and only if 1) = is the sum of the : = ~ : 2

2) For each 2 , : q 4: 5 ~ ¸¹ £

Proof. Suppose first that = is the direct sum of < . Then 1) certainly holds and if # : q 4: 5 £

then # ~

for some

: and #~

bÄb

where : and £ for all ~ Á Ã Á . Hence, by the uniqueness of direct sum representations, ~ and so # ~ . Thus, 2) holds. For the converse, suppose that 1) and 2) hold. We need only verify the uniqueness condition. If #~ and

bÄb

Vector Spaces

41

# ~ ! b Ä b ! where : and ! : then by including additional terms equal to we may assume that the index sets ¸ Á Ã Á ¹ and ¸ Á Ã Á ¹ are the same set ¸ Á Ã Á ¹, that is #~

bÄb

and # ~ ! b Ä b ! Thus, ²

c ! ³ b Ä b ²

c ! ³ ~

Hence, each term " c !" :" is a sum of vectors from subspaces other than :" , which can happen only if " c !" ~ . Thus, " ~ !" for all " and = is the direct sum of < . Example 1.5 Any matrix ( C can be written in the form (~

²( b (! ³ b ²( c (! ³ ~ ) b *

(1.1)

where (! is the transpose of (. It is easy to verify that ) is symmetric and * is skew-symmetric and so (1.1) is a decomposition of ( as the sum of a symmetric matrix and a skew-symmetric matrix. Since the sets Sym and SkewSym of all symmetric and skew-symmetric matrices in C are subspaces of C , we have C ~ Sym b SkewSym Furthermore, if : b ; ~ : Z b ; Z , where : and : Z are symmetric and ; and ; Z are skew-symmetric, then the matrix < ~ : c :Z ~ ; Z c ; is both symmetric and skew-symmetric. Hence, provided that char²- ³ £ , we must have < ~ and so : ~ : Z and ; ~ ; Z . Thus, C ~ Sym l SkewSym

Spanning Sets and Linear Independence A set of vectors spans a vector space if every vector can be written as a linear combination of some of the vectors in that set. Here is the formal definition.

42


Definition The subspace spanned (or subspace generated) by a set : of vectors in = is the set of all linear combinations of vectors from : º:» ~ span²:³ ~ ¸ # b Ä b # - Á # = ¹ When : ~ ¸# Á Ã Á # ¹ is a finite set, we use the notation º# Á Ã Á # », or span²# Á Ã Á # ³. A set : of vectors in = is said to span = , or generate = , if = ~ span²:³, that is, if every vector # = can be written in the form # ~ # b Ä b # for some scalars Á Ã Á and vectors # Á Ã Á # . It is clear that any superset of a spanning set is also a spanning set. Note also that all vector spaces have spanning sets, since the entire space is a spanning set. Definition A nonempty set : of vectors in = is linearly independent if for any # Á Ã Á # in : , we have # b Ä b # ~ ¬ ~ Ä ~ ~ If a set of vectors is not linearly independent, it is said to be linearly dependent. It follows from the definition that any nonempty subset of a linearly independent set is linearly independent. Theorem 1.6 Let : be a set of vectors in = . The following are equivalent: 1) : is linearly independent. 2) Every vector in span²:³ has a unique expression as a linear combination of the vectors in : . 3) No vector in : is a linear combination of the other vectors in : . The following key theorem relates the notions of spanning set and linear independence. Theorem 1.7 Let : be a set of vectors in = . The following are equivalent: 1) : is linearly independent and spans = . 2) For every vector # = , there is a unique set of vectors # Á Ã Á # in : , along with a unique set of scalars Á Ã Á in - , for which # ~ # b Ä b # 3) : is a minimal spanning set, that is, : spans = but any proper subset of : does not span = . 4) : is a maximal linearly independent set, that is, : is linearly independent, but any proper superset of : is not linearly independent. Proof. We leave it to the reader to show that 1) and 2) are equivalent. Now suppose 1) holds. Then : is a spanning set. If some proper subset : Z of : also

Vector Spaces

43

spanned = then any vector in : c : Z would be a linear combination of the vectors in : Z , contradicting the fact that the vectors in : are linearly independent. Hence 1) implies 3). Conversely, if : is a minimal spanning set then it must be linearly independent. For if not, some vector : would be a linear combination of the other vectors in : and so : c ¸ ¹ would be a proper spanning subset of : , which is not possible. Hence 3) implies 1). Suppose again that 1) holds. If : were not maximal, there would be a vector # = c : for which the set : r ¸#¹ is linearly independent. But then # is not in the span of : , contradicting the fact that : is a spanning set. Hence, : is a maximal linearly independent set and so 1) implies 4). Conversely, if : is a maximal linearly independent set then : must span = , for if not, we could find a vector # = c : that is not a linear combination of the vectors in : . Hence, : r ¸#¹ would be a linearly independent proper superset of :, which is a contradiction. Thus, 4) implies 1). Definition A set of vectors in = that satisfies any (and hence all) of the equivalent conditions in Theorem 1.7 is called a basis for = . Corollary 1.8 A finite set : ~ ¸# Á Ã Á # ¹ of vectors in = is a basis for = if and only if = ~ º# » l Ä l º# »

Example 1.6 The th standard vector in - is the vector that has s in all coordinate positions except the th, where it has a . Thus, ~ ²Á Á Ã Á ³Á

~ ²Á Á Ã Á ³ Á Ã Á

~ ²Á Ã Á Á ³

The set ¸ Á Ã Á ¹ is called the standard basis for - . The proof that every nontrivial vector space has a basis is a classic example of the use of Zorn's lemma. Theorem 1.9 Let = be a nonzero vector space. Let 0 be a linearly independent set in = and let : be a spanning set in = containing 0 . Then there is a basis 8 for = for which 0 8 : . In particular, 1) Any vector space, except the zero space ¸¹, has a basis. 2) Any linearly independent set in = is contained in a basis. 3) Any spanning set in = contains a basis. Proof. Consider the collection 7 of all linearly independent subsets of = containing 0 and contained in : . This collection is not empty, since 0 7 . Now, if

44


9 ~ ¸0 2¹ is a chain in 7 then the union < ~ 0 2

is linearly independent and satisfies 0 < : , that is, < 7 . Hence, every chain in 7 has an upper bound in 7 and according to Zorn's lemma, 7 must contain a maximal element 8 , which is linearly independent. Now, 8 is a basis for the vector space º:» ~ = , for if any : is not a linear combination of the elements of 8 then 8 r ¸ ¹ : is linearly independent, contradicting the maximality of 8 . Hence : º8 » and so = ~ º:» º8 ». The reader can now show, using Theorem 1.9, that any subspace of a vector space has a complement.

The Dimension of a Vector Space The next result, with its classical elegant proof, says that if a vector space = has a finite spanning set : then the size of any linearly independent set cannot exceed the size of : . Theorem 1.10 Let = be a vector space and assume that the vectors # Á Ã Á # are linearly independent and the vectors Á Ã Á span = . Then . Proof. First, we list the two sets of vectors: the spanning set followed by the linearly independent set Á Ã Á Â # Á Ã Á # Then we move the first vector # to the front of the first list # Á Á Ã Á Â # Á Ã Á # Since Á Ã Á span = , # is a linear combination of the 's. This implies that we may remove one of the 's, which by reindexing if necessary can be , from the first list and still have a spanning set # Á Á Ã Á Â # Á Ã Á # Note that the first set of vectors still spans = and the second set is still linearly independent. Now we repeat the process, moving # from the second list to the first list # Á # Á Á Ã Á Â # Á Ã Á # As before, the vectors in the first list are linearly dependent, since they spanned = before the inclusion of # . However, since the # 's are linearly independent, any nontrivial linear combination of the vectors in the first list that equals

Vector Spaces

45

must involve at least one of the 's. Hence, we may remove that vector, which again by reindexing if necessary may be taken to be and still have a spanning set # Á # Á Á Ã Á Â # Á Ã Á # Once again, the first set of vectors spans = and the second set is still linearly independent. Now, if , then this process will eventually exhaust the list # Á # Á Ã Á # Â #b Á Ã Á #

's

and lead to the

where # Á # Á Ã Á # span = , which is clearly not possible since # is not in the span of # Á # Á Ã Á # . Hence, . Corollary 1.11 If = has a finite spanning set then any two bases of = have the same size. Now let us prove Corollary 1.11 for arbitrary vector spaces. Theorem 1.12 If = is a vector space then any two bases for = have the same cardinality. Proof. We may assume that all bases for = are infinite sets, for if any basis is finite then = has a finite spanning set and so Corollary 1.11 applies. Let 8 ~ ¸ 0¹ be a basis for = and let 9 be another basis for = . Then any vector 9 can be written as a finite linear combination of the vectors in 8 , where all of the coefficients are nonzero, say ~ is denoted by B²= Á > ³ and the set of all linear operators on = is denoted by B²= ³. We should mention that some authors use the term linear operator for any linear transformation from = to > . Definition The following terms are also employed: 1) homomorphism for linear transformation 2) endomorphism for linear operator 3) monomorphism (or embedding) for injective linear transformation 4) epimorphism for surjective linear transformation 5) isomorphism for bijective linear transformation. 6) automorphism for bijective linear operator. Example 2.1 1) The derivative +¢ = ¦ = is a linear operator on the vector space = of all infinitely differentiable functions on s.

56


2) The integral operator ¢ - ´%µ ¦ - ´%µ defined by %

² ³ ~ ²!³!

is a linear operator on - ´%µ. 3) Let ( be an d matrix over - . The function ( ¢ - ¦ - defined by ( ²#³ ~ (#, where all vectors are written as column vectors, is a linear transformation from - to - . This function is just multiplication by ( . 4) The coordinate map ¢ = ¦ - of an -dimensional vector space is a linear transformation from = to - . The set B²= Á > ³ is a vector space in its own right and B²= ³ has the structure of an algebra, as defined in Chapter 0. Theorem 2.1 1) The set B²= Á > ³ is a vector space under ordinary addition of functions and scalar multiplication of functions by elements of - . 2) If B²< Á = ³ and B²= Á > ³ then the composition is in B²< Á > ³. 3) If B²= Á > ³ is bijective then c B²> Á = ³. 4) The vector space B²= ³ is an algebra, where multiplication is composition of functions. The identity map B²= ³ is the multiplicative identity and the zero map B²= ³ is the additive identity. Proof. We prove only part 3). Let ¢ = ¦ > be a bijective linear transformation. Then c ¢ > ¦ = is a well-defined function and since any two vectors $ and $ in > have the form $ ~ ²# ³ and $ ~ ²# ³, we have c ²$ b $ ³ ~ c ² ²# ³ b ²# ³³ ~ c ² ²# b # ³³ ~ # b # ~ c ²$ ³ b c ²$ ³ which shows that c is linear. One of the easiest ways to define a linear transformation is to give its values on a basis. The following theorem says that we may assign these values arbitrarily and obtain a unique linear transformation by linear extension to the entire domain. Theorem 2.2 Let = and > be vector spaces and let 8 ~ ¸# 0¹ be a basis for = . Then we can define a linear transformation B²= Á > ³ by specifying the values of ²# ³ > arbitrarily for all # 8 and extending the domain of to = using linearity, that is, ² # b Ä b # ³ ~ ²# ³ b Ä b ²# ³

Linear Transformations

57

This process uniquely defines a linear transformation, that is, if Á B²= Á > ³ satisfy ²# ³ ~ ²# ³ for all # 8 then ~ . Proof. The crucial point is that the extension by linearity is well-defined, since each vector in = has a unique representation as a linear combination of a finite number of vectors in 8. We leave the details to the reader. Note that if B²= Á > ³ and if : is a subspace of = , then the restriction O: of to : is a linear transformation from : to > .

The Kernel and Image of a Linear Transformation There are two very important vector spaces associated with a linear transformation from = to > . Definition Let B²= Á > ³. The subspace ker² ³ ~ ¸# = ²#³ ~ ¹ is called the kernel of and the subspace im² ³ ~ ¸ ²#³ # = ¹ is called the image of . The dimension of ker² ³ is called the nullity of and is denoted by null² ³. The dimension of im² ³ is called the rank of and is denoted by rk² ³. It is routine to show that ker² ³ is a subspace of = and im² ³ is a subspace of > . Moreover, we have the following. Theorem 2.3 Let B²= Á > ³. Then 1) is surjective if and only if im² ³ ~ > 2) is injective if and only if ker² ³ ~ ¸¹ Proof. The first statement is merely a restatement of the definition of surjectivity. To see the validity of the second statement, observe that ²"³ ~ ²#³ ¯ ²" c #³ ~ ¯ " c # ker² ³ Hence, if ker² ³ ~ ¸¹ then ²"³ ~ ²#³ ¯ " ~ #, which shows that is injective. Conversely, if is injective and " ker² ³ then ²"³ ~ ²³ and so " ~ . This shows that ker² ³ ~ ¸¹.

Isomorphisms Definition A bijective linear transformation ¢ = ¦ > is called an isomorphism from = to > . When an isomorphism from = to > exists, we say that = and > are isomorphic and write = > . Example 2.2 Let dim²= ³ ~ . For any ordered basis 8 of = , the coordinate map 8 ¢ = ¦ - that sends each vector # = to its coordinate matrix

58


´#µ8 - is an isomorphism. Hence, any -dimensional vector space over - is isomorphic to - . Isomorphic vector spaces share many properties, as the next theorem shows. If B²= Á > ³ and : = we write ²:³ ~ ¸ ² ³

:¹

Theorem 2.4 Let B²= Á > ³ be an isomorphism. Let : = . Then 1) : spans = if and only if ²:³ spans > . 2) : is linearly independent in = if and only if ²:³ is linearly independent in >. 3) : is a basis for = if and only if ²:³ is a basis for > . An isomorphism can be characterized as a linear transformation ¢ = ¦ > that maps a basis for = to a basis for > . Theorem 2.5 A linear transformation B²= Á > ³ is an isomorphism if and only if there is a basis 8 of = for which ²8 ³ is a basis of > . In this case, maps any basis of = to a basis of > . The following theorem says that, up to isomorphism, there is only one vector space of any given dimension. Theorem 2.6 Let = and > be vector spaces over - . Then = > if and only if dim²= ³ ~ dim²> ³. In Example 2.2, we saw that any -dimensional vector space is isomorphic to - . Now suppose that ) is a set of cardinality and let ²- ) ³ be the vector space of all functions from ) to - with finite support. We leave it to the reader to show that the functions ²- ) ³ defined for all ) , by ²%³ ~ F

if % ~ if % £

form a basis for ²- ) ³ , called the standard basis. Hence, dim²²- ) ³ ³ ~ () (. It follows that for any cardinal number , there is a vector space of dimension . Also, any vector space of dimension is isomorphic to ²- ) ³ . Theorem 2.7 If is a natural number then any -dimensional vector space over - is isomorphic to - . If is any cardinal number and if ) is a set of cardinality then any -dimensional vector space over - is isomorphic to the vector space ²- ) ³ of all functions from ) to - with finite support.


59

The Rank Plus Nullity Theorem Let B²= Á > ³. Since any subspace of = has a complement, we can write = ~ ker² ³ l ker² ³ where ker² ³ is a complement of ker² ³ in = . It follows that dim²= ³ ~ dim²ker² ³³ b dim²ker² ³ ³ Now, the restriction of to ker² ³ ¢ ker² ³ ¦ > is injective, since ker² ³ ~ ker² ³ q ker² ³ ~ ¸¹ Also, im² ³ im² ³. For the reverse inclusion, if ²#³ im² ³ then since # ~ " b $ for " ker² ³ and $ ker² ³ , we have ²#³ ~ ²"³ b ²$³ ~ ²$³ ~ ²$³ im² ³ Thus im² ³ ~ im² ³. It follows that ker² ³ im² ³ From this, we deduce the following theorem. Theorem 2.8 Let B²= Á > ³. 1) Any complement of ker² ³ is isomorphic to im² ³ 2) (The rank plus nullity theorem) dim²ker² ³³ b dim²im² ³³ ~ dim²= ³ or, in other notation, rk² ³ b null² ³ ~ dim²= ³

Theorem 2.8 has an important corollary. Corollary 2.9 Let B²= Á > ³, where dim²= ³ ~ dim²> ³ B. Then is injective if and only if it is surjective. Note that this result fails if the vector spaces are not finite-dimensional.

Linear Transformations from - to - Recall that for any d matrix ( over - the multiplication map ( ²#³ ~ (#

60


is a linear transformation. In fact, any linear transformation B²- Á - ³ has this form, that is, is just multiplication by a matrix, for we have 2 ² ³ Ä ² ³3 ~ 2 ² ³ Ä ² ³3²³ ~ ² ³ and so ~ ( where ( ~ 2 ² ³ Ä ² ³3 Theorem 2.10 1) If ( is an d matrix over - then ( B²- Á - ³. 2) If B²- Á - ³ then ~ ( where ( ~ ² ² ³ Ä ² ³³ The matrix ( is called the matrix of . Example 2.3 Consider the linear transformation ¢ - ¦ - defined by ²%Á &Á '³ ~ ²% c &Á 'Á % b & b '³ Then we have, in column form

v % y v %c& y v & ~ ' ~ w ' z w %b&b' z w

c

y v%y & z w' z

and so the standard matrix of is (~

v w

c

y z

If ( CÁ then since the image of ( is the column space of (, we have dim²ker²( ³³ b rk²(³ ~ dim²- ³ This gives the following useful result. Theorem 2.11 Let ( be an d matrix over - . 1) ( ¢ - ¦ - is injective if and only if rk²(³ ~ n. 2) ( ¢ - ¦ - is surjective if and only if rk²(³ ~ m.

Change of Basis Matrices Suppose that 8 ~ ² Á Ã Á ³ and 9 ~ ² Á Ã Á ³ are ordered bases for a vector space = . It is natural to ask how the coordinate matrices ´#µ8 and ´#µ9 are related. The map that takes ´#µ8 to ´#µ9 is 8Á9 ~ 9 8c and is called the change of basis operator (or change of coordinates operator). Since 8Á9 is an operator on - , it has the form ( where


61

( ~ ²8Á9 ² ³Á Ã Á 8Á9 ² ³³ ~ ²9 8c ²´ µ8 ³Á Ã Á 9 8c ²´ µ8 ³³ ~ ²´ µ9 Á Ã Á ´ µ9 ³³ We denote ( by 48,9 and call it the change of basis matrix from 8 to 9 . Theorem 2.12 Let 8 ~ ² Á Ã Á ³ and 9 be ordered bases for a vector space = . Then the change of basis operator 8Á9 ~ 9 8c is an automorphism of - , whose standard matrix is 48,9 ~ ²´ µ9 Á Ã Á ´ µ9 ³³ Hence ´#µ9 ~ 48Á9 ´#µ8 and 49Á8 ~ 48c ,9 . Consider the equation ( ~ 48 Á 9 or equivalently, ( ~ ²´ µ9 Á Ã Á ´ µ9 ³³ Then given any two of ( (an invertible d matrix)Á 8 (an ordered basis for - ) and 9 (an order basis for - ), the third component is uniquely determined by this equation. This is clear if 8 and 9 are given or if ( and 9 are given. If ( and 8 are given then there is a unique 9 for which (c ~ 49Á8 and so there is a unique 9 for which ( ~ 48Á9 . Theorem 2.13 If we are given any two of the following: 1) An invertible d matrix (. 2) An ordered basis 8 for - . 3) An ordered basis 9 for - . then the third is uniquely determined by the equation ( ~ 48 Á 9

The Matrix of a Linear Transformation Let ¢ = ¦ > be a linear transformation, where dim²= ³ ~ and dim²> ³ ~ and let 8 ~ ² Á Ã Á ³ be an ordered basis for = and 9 an ordered basis for > . Then the map ¢ ´#µ8 ¦ ´ ²#³µ9 is a representation of as a linear transformation from - to - , in the sense


62

that knowing (along with 8 and 9, of course) is equivalent to knowing . Of course, this representation depends on the choice of ordered bases 8 and 9. Since is a linear transformation from - to - , it is just multiplication by an d matrix (, that is ´ ²#³µ9 ~ (´#µ8 Indeed, since ´ µ8 ~ , we get the columns of ( as follows: (²³ ~ ( ~ (´#µ8 ~ ´ ² ³µ9 Theorem 2.14 Let B²= Á > ³ and let 8 ~ ² Á Ã Á ³ and 9 be ordered bases for = and > , respectively. Then can be represented with respect to 8 and 9 as matrix multiplication, that is ´ ²#³µ9 ~ ´ µ8,9 ´#µ8 where ´ µ8,9 ~ ²´ ² ³µ9 Ä ´ ² ³µ9 ³ is called the matrix of with respect to the bases 8 and 9. When = ~ > and 8 ~ 9, we denote ´ µ8,8 by ´ µ8 and so ´ ²#³µ8 ~ ´ µ8 ´#µ8

Example 2.4 Let +¢ F ¦ F be the derivative operator, defined on the vector space of all polynomials of degree at most . Let 8 ~ 9 ~ ²Á %Á % ³. Then ´+²³µ9 ~ ´µ9 ~

vy vy vy , ´+²%³µ9 ~ ´µ9 ~ Á ´+²% ³µ9 ~ ´%µ9 ~ wz wz wz

and so ´+µ8 ~

v w

y z

Hence, for example, if ²%³ ~ b % b % then ´+²%³µ9 ~ ´+µ8 ´²%³µ8 ~

v w

yv y v y ~ zw z w z

and so +²%³ ~ b %. The following result shows that we may work equally well with linear transformations or with the matrices that represent them (with respect to fixed


63

ordered bases 8 and 9). This applies not only to addition and scalar multiplication, but also to matrix multiplication. Theorem 2.15 Let = and > be vector spaces over - , with ordered bases 8 ~ ² Á Ã Á ³ and 9 ~ ² Á Ã Á ³, respectively. 1) The map ¢ B²= Á > ³ ¦ CÁ ²- ³ defined by ² ³ ~ ´ µ8,9 is an isomorphism and so B²= Á > ³ CÁ ²- ³. 2) If B²< Á = ³ and B²= Á > ³ and if 8 , 9 and : are ordered bases for < , = and > , respectively then ´µ8,: ~ ´ µ9,: ´µ8,9 Thus, the matrix of the product (composition) is the product of the matrices of and . In fact, this is the primary motivation for the definition of matrix multiplication. Proof. To see that is linear, observe that for all ´ b ! µ8Á9 ´ µ8 ~ ´² b ! ³² ³µ9 ~ ´ ² ³ b ! ² ³µ9 ~ ´² ³µ9 b !´ ² ³µ9 ~ ´µ8Á9 ´ µ8 b !´ µ8Á9 ´ µ8 ~ ² ´µ8Á9 b !´ µ8Á9 ³´ µ8 and since ´ µ8 ~ is a standard basis vector, we conclude that ´ b ! µ8Á9 ~ ´µ8Á9 b !´ µ8Á9 and so is linear. If ( CÁ , we define by the condition ´ ² ³µ9 ~ (²³ , whence ² ³ ~ ( and is surjective. Since dim²B²= Á > ³³ ~ dim²CÁ ²- ³³, the map is an isomorphism. To prove part 2), we have ´µ8Á: ´#µ8 ~ ´ ²²#³³µ: ~ ´ µ9,: ´²#³µ9 ~ ´ µ9Á: ´µ8Á9 ´#µ8

Change of Bases for Linear Transformations Since the matrix ´ µ8,9 that represents depends on the ordered bases 8 and 9, it is natural to wonder how to choose these bases in order to make this matrix as simple as possible. For instance, can we always choose the bases so that is represented by a diagonal matrix? As we will see in Chapter 7, the answer to this question is no. In that chapter, we will take up the general question of how best to represent a linear operator by a matrix. For now, let us take the first step and describe the relationship between the matrices ´ µ8Á9 and ´ µ8Z Á9Z of with respect to two different pairs ²8 Á 9³ and ²8 Z Á 9Z ³ of ordered bases. Multiplication by ´ µ8Z Á9Z sends ´#µ8Z to

64


´ ²#³µ9Z . This can be reproduced by first switching from 8 Z to 8 , then applying ´ µ8Á9 and finally switching from 9 to 9Z , that is, ´ µ8Z ,9Z ~ 49Á9Z ´ µ8,9 48Z Á8 ~ 49Á9Z ´ µ8,9 48c Á8 Z Theorem 2.16 Let B²= ,> ³ and let ²8 Á 9³ and ²8 Z Á 9Z ³ be pairs of ordered bases of = and > , respectively. Then ´ µ8Z Á9Z ~ 49Á9Z ´ µ8Á9 48Z Á8

(2.1)

When B²= ³ is a linear operator on = , it is generally more convenient to represent by matrices of the form ´ µ8 , where the ordered bases used to represent vectors in the domain and image are the same. When 8 ~ 9, Theorem 2.16 takes the following important form. Corollary 2.17 Let B²= ³ and let 8 and 9 be ordered bases for = . Then the matrix of with respect to 9 can be expressed in terms of the matrix of with respect to 8 as follows ´ µ9 ~ 48Á9 ´ µ8 48c Á9

(2.2)

Equivalence of Matrices Since the change of basis matrices are precisely the invertible matrices, (2.1) has the form ´ µ8Z Á9Z ~ 7 ´ µ8Á9 8c where 7 and 8 are invertible matrices. This motivates the following definition. Definition Two matrices ( and ) are equivalent if there exist invertible matrices 7 and 8 for which ) ~ 7 (8c

We remarked in Chapter 0 that ) is equivalent to ( if and only if ) can be obtained from ( by a series of elementary row and column operations. Performing the row operations is equivalent to multiplying the matrix ( on the left by 7 and performing the column operations is equivalent to multiplying ( on the right by 8c . In terms of (2.1), we see that performing row operations (premultiplying by 7 ) is equivalent to changing the basis used to represent vectors in the image and performing column operations (postmultiplying by 8c ) is equivalent to changing the basis used to represent vectors in the domain.


65

According to Theorem 2.16, if ( and ) are matrices that represent with respect to possibly different ordered bases then ( and ) are equivalent. The converse of this also holds. Theorem 2.18 Let = and > be vector spaces with dim²= ³ ~ and dim²> ³ ~ . Then two d matrices ( and ) are equivalent if and only if they represent the same linear transformation B²= Á > ³, but possibly with respect to different ordered bases. In this case, ( and ) represent exactly the same set of linear transformations in B²= Á > ³. Proof. If ( and ) represent , that is, if ( ~ ´ µ8,9 and ) ~ ´ µ8Z ,9Z for ordered bases 8 Á 9Á 8 Z and 9Z then Theorem 2.16 shows that ( and ) are equivalent. Now suppose that ( and ) are equivalent, say ) ~ 7 (8c where 7 and 8 are invertible. Suppose also that ( represents a linear transformation B²= Á > ³ for some ordered bases 8 and 9, that is, ( ~ ´ µ8Á9 Theorem 2.13 implies that there is a unique ordered basis 8 Z for = for which 8 ~ 48Á8Z and a unique ordered basis 9Z for > for which 7 ~ 49Á9Z . Hence ) ~ 49Á9Z ´ µ8Á9 48Z Á8 ~ ´ µ8Z Á9Z Hence, ) also represents . By symmetry, we see that ( and ) represent the same set of linear transformations. This completes the proof. We remarked in Example 0.3 that every matrix is equivalent to exactly one matrix of the block form 1 ~ >

0 cÁ

Ác cÁc ?block

Hence, the set of these matrices is a set of canonical forms for equivalence. Moreover, the rank is a complete invariant for equivalence. In other words, two matrices are equivalent if and only if they have the same rank.

Similarity of Matrices When a linear operator B²= ³ is represented by a matrix of the form ´ µ8 , equation (2.2) has the form ´ µ8Z ~ 7 ´ µ8 7 c where 7 is an invertible matrix. This motivates the following definition.

66


Definition Two matrices ( and ) are similar if there exists an invertible matrix 7 for which ) ~ 7 (7 c The equivalence classes associated with similarity are called similarity classes. The analog of Theorem 2.18 for square matrices is the following. Theorem 2.19 Let = be a vector space of dimension . Then two d matrices ( and ) are similar if and only if they represent the same linear operator B²= ³, but possibly with respect to different ordered bases. In this case, ( and ) represent exactly the same set of linear operators in B²= ³. Proof. If ( and ) represent B²= ³, that is, if ( ~ ´ µ8 and ) ~ ´ µ9 for ordered bases 8 and 9 then Corollary 2.17 shows that ( and ) are similar. Now suppose that ( and ) are similar, say ) ~ 7 (7 c Suppose also that ( represents a linear operator B²= ³ for some ordered basis 8 , that is, ( ~ ´ µ8 Theorem 2.13 implies that there is a unique ordered basis 9 for = for which 7 ~ 48Á9 . Hence ) ~ 48Á9 ´ µ8 48c Á 9 ~ ´ µ9 Hence, ) also represents . By symmetry, we see that ( and ) represent the same set of linear operators. This completes the proof. We will devote much effort in Chapter 7 to finding a canonical form for similarity.

Similarity of Operators We can also define similarity of operators. Definition Two linear operators Á B²= ³ are similar if there exists an automorphism B²= ³ for which ~ c The equivalence classes associated with similarity are called similarity classes.


67

The analog of Theorem 2.19 in this case is the following. Theorem 2.20 Let = be a vector space of dimension . Then two linear operators and on = are similar if and only if there is a matrix ( C that represents both operators (but with respect to possibly different ordered bases). In this case, and are represented by exactly the same set of matrices in C . Proof. If and are represented by ( C , that is, if ´ µ8 ~ ( ~ ´µ9 for ordered bases 8 and 9 then ´µ9 ~ ´ µ8 ~ 49Á8 ´ µ9 48Á9 Let B²= ³ be the automorphism of = defined by ² ³ ~ , where 8 ~ ¸ Á Ã Á ¹ and 9 ~ ¸ Á Ã Á ¹. Then ´µ9 ~ ²´² ³µ9 Ä ´² ³µ9 ³ ~ ²´ µ9 Ä ´µ9 ³ ~ 48Á9 and so c ´µ9 ~ ´µc 9 ´ µ9 ´µ9 ~ ´ µ9

from which it follows that and are similar. Conversely, suppose that and are similar, say ~ c Suppose also that is represented by the matrix ( C , that is, ( ~ ´ µ8 for some ordered basis 8 . Then ´µ8 ~ ´c µ8 ~ ´µ8 ´ µ8 ´µc 8 If we set ~ ² ³ then 9 ~ ² Á Ã Á ³ is an ordered basis for = and ´µ8 ~ ²´² ³µ8 Ä ´² ³µ8 ³ ~ ²´ µ8 Ä ´µ8 ³ ~ 49Á8 Hence ´µ8 ~ 49Á8 ´ µ8 49c Á8 It follows that ( ~ ´ µ8 ~ 48Á9 ´µ8 48c Á9 ~ ´ µ9 and so ( also represents . By symmetry, we see that and are represented by the same set of matrices. This completes the proof.

68


Invariant Subspaces and Reducing Pairs The restriction of a linear operator B²= ³ to a subspace : of = is not necessarily a linear operator on : . This prompts the following definition. Definition Let B²= ³. A subspace : of = is said to be invariant under or -invariant if ²:³ : , that is, if ² ³ : for all : . Put another way, : is invariant under if the restriction O: is a linear operator on : . If = ~: l; then the fact that : is -invariant does not imply that the complement ; is also -invariant. (The reader may wish to supply a simple example with = ~ s .) Definition Let B²= ³. If = ~ : l ; and if both : and ; are -invariant, we say that the pair ²:Á ; ³ reduces . A reducing pair can be used to decompose a linear operator into a direct sum as follows. Definition Let B²= ³. If ²:Á ; ³ reduces we write ~ O: l O; and call the direct sum of O: and O; . Thus, the expression ~l means that there exist subspaces : and ; of = for which ²:Á ; ³ reduces and ~ O: and ~ O;

The concept of the direct sum of linear operators will play a key role in the study of the structure of a linear operator.

Topological Vector Spaces This section is for readers with some familiarity with point-set topology. The standard topology on s is the topology for which the set of open rectangles ) ~ ¸0 d Ä d 0 0 's are open intervals in s¹ is a basis (in the sense of topology), that is, a subset of s is open if and only if it is a union of sets in ) . The standard topology is the topology induced by the Euclidean metric on s .


69

The standard topology on s has the property that the addition function 7¢ s d s ¦ s ¢ ²#Á $³ ¦ # b $ and the scalar multiplication function C¢ s d s ¦ s ¢ ²Á #³ ¦ # are continuous. As such, s is a topological vector space. Also, any linear functional ¢ s ¦ s is a continuous map. More generally, any real vector space = endowed with a topology J is called a topological vector space if the operations of addition 7¢ = d = ¦ = and scalar multiplication C¢ s d = ¦ = are continuous under J . Let = be a real vector space of dimension and fix an ordered basis 8 ~ ²# Á Ã Á # ³ for = . Consider the coordinate map ~ 8 ¢ = ¦ s ¢ # ¦ ´#µ8 and its inverse 8 ~ 8c ¢ s ¦ = ¢ ² Á Ã Á ³ ¦ # We claim that there is precisely one topology J ~ J= on = for which = becomes a topological vector space and for which all linear functionals are continuous. This is called the natural topology on = . In fact, the natural topology is the topology for which 8 (and therefore also 8 ) is a homeomorphism, for any basis 8. (Recall that a homeomorphism is a bijective map that is continuous and has a continuous inverse.) Once this has been established, it will follow that the open sets in J are precisely the images of the open sets in s under the map 8 . A basis for the natural topology is given by ¸8 ²0 d Ä d 0 ³ 0 's are open intervals in s¹ ~ D # 0 's are open intervals in sE 0

First, we show that if = is a topological vector space under a topology J then is continuous. Since ~ where ¢ s ¦ = is defined by ² Á Ã Á ³ ~ # it is sufficient to show that these maps are continuous. (The sum of continuous maps is continuous.) Let 6 be an open set in J . Then Cc ²6³ ~ ¸²Á %³ s d = % 6¹ is open in s d = . We need to show that the set

70


c ²6³ ~ ¸² Á Ã Á ³ s # 6¹

is open in s , so let ² Á Ã Á ³ c ²6³. Thus, # 6 . It follows that ² Á # ³ Cc ²6³, which is open, and so there is an open interval 0 s and an open set ) J of = for which ² Á # ³ 0 d ) Cc ²6³ Then the open set < ~ s d Ä d s d 0 d s d Ä d s, where the factor 0 is in the th position, has the property that ²< ³ 6. Thus ² Á Ã Á ³ < c ²6³ and so c ²6³ is open. Hence, , and therefore also , is continuous. Next we show that if every linear functional on = is continuous under a topology J on = then the coordinate map is continuous. If # = denote by ´#µ8Á the th coordinate of ´#µ8 . The map ¢ = ¦ s defined by ²#³ ~ ´#µ8Á is a linear functional and so is continuous by assumption. Hence, for any open interval 0 s the set ( ~ ¸# = ´#µ8Á 0 ¹ is open. Now, if 0 are open intervals in s then c ²0 d Ä d 0 ³ ~ ¸# = ´#µ8 0 d Ä d 0 ¹ ~ ( is open. Thus, is continuous. Thus, if a topology J has the property that = is a topological vector space and every linear functional is continuous, then and ~ c are homeomorphisms. This means that J , if it exists, must be unique. It remains to prove that the topology J on = that makes a homeomorphism has the property that = is a topological vector space under J and that any linear functional on = continuous. As to addition, the maps ¢ = ¦ s and ² d ³¢ = d = ¦ s d s are homeomorphisms and the map 7Z ¢ s d s ¦ s is continuous and so the map 7¢ = d = ¦ = , being equal to c k 7Z k ² d ³, is also continuous. As to scalar multiplication, the maps ¢ = ¦ s and ² d ³¢ s d = ¦ s d s are homeomorphisms and the map CZ ¢ s d s ¦ s is continuous and so the map C¢ = d = ¦ = , being equal to c k CZ k ² d ³, is also continuous. Now let be a linear functional. Since is continuous if and only if k c is continuous, we can confine attention to = ~ s . In this case, if Á Ã Á is the


71

standard basis for s and ( ² ³( 4 , then for any % ~ ² Á Ã Á ³ s we have ( ²%³( ~ c ² ³c ( (( ² ³( 4 ( ( Now, if (%( °4 then ( ( °4 and so ( ²%³( , which implies that is continuous. According to the Riesz representation theorem and the Cauchy–Schwarz inequality, we have ) ²%³) )H ))%) Hence, % ¦ implies ²% ³ ¦ and so by linearity, % ¦ % implies ²% ³ ¦ % and so is continuous. Theorem 2.21 Let = be a real vector space of dimension . There is a unique topology on = , called the natural topology for which = is a topological vector space and for which all linear functionals on = are continuous. This topology is determined by the fact that the coordinate map ¢ = ¦ s is a homeomorphism.

Linear Operators on = d A linear operator on a real vector space = can be extended to a linear operator d on the complexification = d by defining d ²" b #³ ~ ²"³ b ²#³ Here are the basic properties of this complexification of . Theorem 2.22 If Á B²= ³ then 1) ² ³d ~ d , s 2) ² b ³d ~ d b d 3) ²³d ~ d d 4) ´ ²#³µd ~ d ²#d ³. Let us recall that for any ordered basis 8 for = and any vector # = we have ´# b µcpx²8 ³ ~ ´#µ8 Now, if 8 is a basis for = , then the th column of ´ µ8 is ´ ² ³µ8 ~ ´ ² ³ b µcpx²8³ ~ ´ d ² b ³µcpx²8³ which is the th column of the coordinate matrix of d with respect to the basis cpx²8 ³. Thus we have the following theorem.

72


Theorem 2.23 Let B²= ³ where = is a real vector space. The matrix of d with respect to the basis cpx²8 ³ is equal to the matrix of with respect to the basis 8 ´ d µcpx²8³ ~ ´ µ8 Hence, if a real matrix ( represents a linear operator on = then ( also represents the complexification d of on = d .

Exercises 1.

2. 3. 4.

Let ( CÁ have rank . Prove that there are matrices ? CÁ and @ CÁ , both of rank , for which ( ~ ?@ . Prove that ( has rank if and only if it has the form ( ~ %! & where % and & are row matrices. Prove Corollary 2.9 and find an example to show that the corollary does not hold without the finiteness condition. Let B²= Á > ³. Prove that is an isomorphism if and only if it carries a basis for = to a basis for > . If B²= Á > ³ and B²= Á > ³ we define the external direct sum ^ B²= ^ = Á > ^ > ³ by ² ^ ³²²# Á # ³³ ~ ² ²# ³Á ²# ³³

5. 6.

7.

8. 9.

Show that ^ is a linear transformation. Let = ~ : l ; . Prove that : l ; : ^ ; . Thus, internal and external direct sums are equivalent up to isomorphism. Let = ~ ( b ) and consider the external direct sum , ~ ( ^ ) . Define a map ¢ ( ^ ) ¦ = by ²#Á $³ ~ # b $. Show that is linear. What is the kernel of ? When is an isomorphism? Let J be a subset of B²= ³. A subspace : of = is J -invariant if : is invariant for every J . Also, = is J -irreducible if the only J -invariant subspaces of = are ¸¹ and = . Prove the following form of Schur's lemma. Suppose that J= B²= ³ and J> B²> ³ and = is J= -irreducible and > is J> -irreducible. Let B²= Á > ³ satisfy J= ~ J> , that is, for any J= there is a J> such that ~ . Prove that ~ or is an isomorphism. Let B²= ³ where dim²= ³ B. If rk² ³ ~ rk² ³ show that im² ³ q ker² ³ ~ ¸¹. Let B²< ,= ³ and B²= Á > ³. Show that rk²³ min¸rk² ³Á rk²³¹

10. Let B²< Á = ³ and B²= Á > ³. Show that null²³ null² ³ b null²³ 11. Let Á B²= ³ where is invertible. Show that rk²³ ~ rk² ³ ~ rk²³


73

12. Let Á B²= Á > ³. Show that rk² b ³ rk² ³ b rk²³ 13. Let : be a subspace of = . Show that there is a B²= ³ for which ker² ³ ~ : . Show also that there exists a B²= ³ for which im²³ ~ : . 14. Suppose that Á B²= ³. a) Show that ~ for some B²= ³ if and only if im²³ im² ³. b) Show that ~ for some B²= ³ if and only if ker² ³ ker²³. 15. Let = ~ : l : . Define linear operators on = by ² b ³ ~ for ~ Á . These are referred to as projection operators. Show that 1) ~ 2) b ~ 0 , where 0 is the identity map on = . 3) ~ for £ where is the zero map. 4) = ~ im² ³ l im² ³ 16. Let dim²= ³ B and suppose that B²= ³ satisfies ~ . Show that rk² ³ dim²= ³. 17. Let ( be an d matrix over - . What is the relationship between the linear transformation ( ¢ - ¦ - and the system of equations (? ~ ) ? Use your knowledge of linear transformations to state and prove various results concerning the system (? ~ ) , especially when ) ~ . 18. Let = have basis 8 ~ ¸# Á Ã Á # ¹. Suppose that for each Á we define Á B²= ³ by Á ²# ³ ~ F

# # b #

if £ if ~

Prove that the Á are invertible and form a basis for B²= ³. 19. Let B²= ³. If : is a -invariant subspace of = must there be a subspace ; of = for which ²:Á ; ³ reduces ? 20. Find an example of a vector space = and a proper subspace : of = for which = : . 21. Let dim²= ³ B. If , B²= ³ prove that ~ implies that and are invertible and that ~ ² ³ for some polynomial ²%³ - ´%µ. 22. Let B²= ³ where dim²= ³ B. If ~ for all B²= ³ show that ~ , for some - , where is the identity map. 23. Let (Á ) C ²- ³. Let 2 be a field containing - . Show that if ( and ) are similar over 2 , that is, if ) ~ 7 (7 c where 7 C ²2³ then ( and ) are also similar over - , that is, there exists 8 C ²- ³ for which ) ~ 8(8c . Hint: consider the equation ?( c )? ~ as a homogeneous system of linear equations with coefficients in - . Does it have a solution? Where? 24. Let ¢ s ¦ s be a continuous function with the property that ²% b &³ ~ ²%³ b ²&³ Prove that is a linear functional on s .

74


25. Prove that any linear functional ¢ s ¦ s is a continuous map. 26. Prove that any subspace : of s is a closed set or, equivalently, that : ~ s ± : is open, that is, for any % : there is an open ball )² Á ³ centered at % with radius for which )²%Á ³ : . 27. Prove that any linear transformation ¢ = ¦ > is continuous under the natural topologies of = and > . 28. Prove that any surjective linear transformation from = to > (both finitedimensional topological vector spaces under the natural topology) is an open map, that is, maps open sets to open sets. 29. Prove that any subspace : of a finite-dimensional vector space = is a closed set or, equivalently, that : is open, that is, for any % : there is an open ball )² Á ³ centered at % with radius for which )²%Á ³ : . 30. Let : be a subspace of = with dim²= ³ B. a) Show that the subspace topology on : inherited from = is the natural topology. b) Show that the natural topology on = °: is the topology for which the natural projection map ¢ = ¦ = °: continuous and open. 31. If = is a real vector space then = d is a complex vector space. Thinking of = d as a vector space ²= d ³s over s, show that ²= d ³s is isomorphic to the external direct product = ^ = . 34. (When is a complex linear map a complexification?) Let = be a real vector space with complexification = d and let B²= d ³. Prove that is a complexification, that is, has the form d for some B²= ³ if and only if commutes with the conjugate map ¢ = d ¦ = d defined by ²" b #³ ~ " c #. 35. Let > be a complex vector space. a) Consider replacing the scalar multiplication on > by the operation ²'Á $³ ¦ '$ where ' d and $ > . Show that the resulting set with the addition defined for the vector space > and with this scalar multiplication is a complex vector space, which we denote by > . b) Show, without using dimension arguments, that ²>s ³d > ^ > . 36. a) Let be a linear operator on the real vector space < with the property that ~ c. Define a scalar multiplication on < by complex numbers as follows ² b ³ h # ~ # b ²#³ for Á s and # < . Prove that under the addition of < and this scalar multiplication < is a complex vector space, which we denote by

Advanced Linear Algebra

Advanced Linear Algebra

Advanced linear algebra