George Bachman Lawrence Narici Edward Beckenstein
FOURIER AND WAVELET ANALYSIS
Springer
George Bachman Lawrence Narici Edward Beckenstein
Fourier and Wavelet Analysis
,
Springer
George Bachman Professor Emeritus
Lawrence Narici
of Mathematics Polytechnic University
5 Metrotech Center Brooklyn, NY 11201 USA
Department of Mathematics and Computer Science St. John's University Jamaica, NY 11439 USA
Edward Beckenstein Science Division St. John's University Staten Island, NY 10301 USA
Editorial Board (North America): S. Axler
F.W. Gehring
Mathematics Department
Mathematics Department
San Francisco State UniversIty
East Hall
San Francisco, CA 94132
University of Michigan
Ann Arbor, MI 48109-1109
USA
USA
K.A. Ribet Department of Mathematics University of California at Berkeley
Berkeley, CA 94720-3840 USA
Mathematics Subject Classification (1991): 42Axx, 42Cxx, 41-xx
Library of Congress Cataloging-in-Publication Data Bachman, George, 1929Fourier and wavelet analysis Edward Beckenstein. p.
cm. - (Universitext)
/ George Bachman, Lawrence Narici,
Includes bibliographical references and index. ISBN 0-387-98899-8 (alk. paper) 1. Fourier, analysis. Edward, 1940QA403.5.B28
2. Wavelets (Mathematics) II. Narici, Lawrence.
I. Beckenstein,
III. Title.
IV. Series.
2000
515'.2433--dc21
99-36217
Printed on acid-free paper.
© 2000 Springer-Verlag New York, Inc.
All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer-Verlag New York, Inc., 175 Fifth Avenue, New York, NY 10010, USA), except for brief excerpts jn connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use of general descriptive names, trade names, trademarks, etc., in this publication, even if the former arc not especially identified, is not to be taken as a sign that such names, as understood by the Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone. Production managed by Michael Koy; manufacturing supervised by Jeffrey Taub. Camera-ready copy prepared by the author. Printed and bound by
R.R. Donnelley and Sons, Harrisonburg, VA.
Printed in the United States of America. 9 8 7 6 5 4 3 2 1
ISBN 0-387-98899-8 Springer-Verlag New York Berlin Heidelberg
SPIN 10491067
Not long ago many thought that the mathematical world was created out of analytic functions. It was the Fourier series which disclosed a terra incognita in a second hemisphere. -E. B. van Vleck, 1 9 14 The Fast Fourier transform-the most valuable numerical algo rithm of our lifetime. -G. Strang, 1993 . . . wavelets are without any doubt an exciting and intuitive concept . This con cept brings with it a new way of thinking . . . . -)T. �eyer, 1993 Foreword
Fourier, the nineteenth (and not the last!) child in his family, wanted to join an artillery regiment . His commoner status prevented it and he went on to other things. Goethe's dictum that boldness has a magic all its own found life in Fourier. He was so rash at times that his work was rejected by his peers (see the introduction to Chapter 4) . He never worried about convergence and said that any periodic function could be expressed in a Fourier series. Nevertheless he was so original that others-Cauchy and Lagrange, among them-were inspired to attempt to place his creations on a firm foundation. They both failed. The first proof that Fourier series converged pointwise was Dirichlet 's in 1829 for piecewise smooth functions (Sec. 4.6) . As a result of Dirichlet's work, the idea of function was trans mogrified. No more would it apply only to the aristocratic society of poly nomials, exponentials and sines and cosines; disorderly conduct now had to be tolerated. By the mid-nineteenth century, it inspired (as a trigonomet ric series) Weierstrass's continuous-but-not-differentiable map (Sec. 4.3). It was such a shock at the time that Weierstrass was apparently in no hurry to disseminate it widely. In order to generalize Dirichlet's pointwise convergence theorem for piece wise smooth functions to a wilder sort , Jordan invented the concept of 'function of bounded variation ; ' he proved his pointwise convergence the orem of Fourier series for such functions in 1881 (Sec. 4.6) . As it became necessary to deal with this wider class of functions, the conception of in tegral was also transmuted. At Dirichlet's urging, it went from integral-as antiderivative to being defined as area under a curve. Cauchy developed the integral from this perspective for continuous functions. Riemann extended it to discontinuous functions, although not too discontinuous. Fejer ( 1 904) went beyond functions of bounded variation. He discovered that for many functions I, f can be recovered by summing the arithmetic means of its Fourier series; even if the Fourier series diverges at a poin t , the series o f arithmetic means converges t o (f (t- ) + I (t+)) / 2 (Sec. 4 . 1 5 ) . What happens a t t's where the one-sided limits d o not exist? By removing the requirement concerning the existence of I ( r ) and I (t+) , Lebesgue
vi
Foreword
globalized Fejer's theorem; he showed that the Fourier series for any f E Ll [ -11", 11"] converges (C, 1) to f (t) a.e. The desire to do this was part of the reason that Lebesgue invented his integral; the theorem mentioned above was one of the first uses he made of it (Sec. 4.18). Denjoy, with the same motivation, extended the integral even further. Concurrently, the emerging point of view that things could be decom p osed into waves and then reconstituted infused not j ust mathematics but all of science. It is imp ossible to quantify the role that this perspective played in the development of the physics of the nineteenth and twentieth centuries, but it was certainly great . Imagine physics without it. We develop the standard features of Fourier analysis-Fourier series, Fourier transform, Fourier sine and cosine transforms. We do NOT do it in the most elegant way. Instead, we develop it for the reader who has never seen them before. We cover more recent developments such as the discrete and fast Fourier transforms and wavelets in Chapters 6 and 7 . Our treatment of these topics is strictly introductory, for the novice . ( Wavelets for idiots?) To do them properly, especially the applications, would take at least a whole book. What do you need to read the book? Not a lot of facts per se, but a little sophistication. We have helped ourselves to what we needed about the Lebesgue integral and given references. It's not much and if you don't know them exactly-if you know the analogous results (when they exist) for the Riemann integral-you can still read the book . We use some things about Hilb ert space, too, and we have included a short development of what is needed in the first three short chapters. You can use them as a short course in functional analysis or start in Chapter 4 on Fourier series, referring to them on an as-needed basis. The chapters are sufficiently independent so that you could start in Chapter 5 (Fourier transform) or 6 (discrete, fast) or 7 (wavelets) and refer back as needed. One caveat : To appreciate Chapter 7, you should read the theory of the L2 Fourier transform in Chapter 5. The L2 theory is really quite pretty, anyway. Notation: Our notation is all standard. On some rare occasions we use C for complement . If we say "by Exercise 3," we mean Exercise 3 at the end of the current section; otherwise we say "Exercise 2.4-3," meaning Exercise 3 at the end of Sec. 2.4. Hints are provided for lots (not all) of the exercises. Rather than a separate index of symbols, the symbols are blended into the index. We prepared the book using Scientific Word and Scientific Workplace. The experience has been. . .interesting. We hope that the result is fun.
Contents Foreword 1
2
3
Metric and
1.1 1.2 1.3 1.4 1.5 1.6
v
Normed Spaces
Metric Spaces. Normed Spaces
1
. . . . . .
Inner Product Spaces Orthogonality.
. . .
Linear Isometry..
.
Holder and Minkowski Inequalities;
Lp
and
Qp
Spaces.
35
Analysis
2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 2.10
Balls Convergence and Continuity. Bounded Sets .
.
.
.
.
.
Closure and Closed Sets. Open Sets...
.
..
Completeness .... Uniform Continuity. Compactness
.
. .
Equivalent Norms. Direct Sums.
Bases
3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10
1 9 12 18 24 28
Best Approximation. .
. . . . . . . . . . . . . . . .
Orthogonal Complements and the Projection Theorem. Orthonormal Sequences . Orthonormal Bases .
. .
The Haar Basis .....
.
.
Unconditional Convergence Orthogonal Direct Sums. Continuous Linear Maps Dual Spaces. Adjoints........
.
35 38 49 52 58 60 66 69 75 83 89
90 94 103 107 114 119 123 126 131 135
viii 4
5
Contents
139
Fourier Series
4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9 4.10 4.11 4.12 4.13 4.14 4.15 4.16 4.17 4.18 4.19 4.20
Warmup Fourier Sine Series and Cosine Series Smoothness . The Riemann-Lebesgue Lemma . The Dirichlet and Fourier Kernels. Pointwise Convergence of Fourier Series. Uniform Convergence . The Gibbs Phenomenon. A Divergent Fourier Series Termwise Integration . Trigonometric vs. Fourier Series. Termwise Differentiation Dido's Dilemma . Other Kinds of Summability. Fejer Theory The Smoothing Effect of
(C, 1 )
Summation
Weierstrass's Approximation Theorem. Lebesgue's Pointwise Convergence Theorem . Higher Dimensions Convergence of Multiple Series
The Fourier Transform
5.1 The Finite Fourier Transform . 5.2 Convolution on T . 5.3 The Exponential Form of Lebesgue's Theorem. 5.4 Motivation and Definition. 5.5 Basics/ Examples 5.6 The Fourier Transform and Residues 5.7 The Fourier Map 5.8 Convolution on R . 5.9 Inversion, Exponential Form 5.10 Inversion, Trigonometric Form 5.11 (C, 1 ) Summability for Integrals. 5.12 The Fejer-Lebesgue Inversion Theorem 5.13 Convergence Assistance . 5.14 Approximate Identity . 5.15 Transforms of Derivatives and Integrals . 5.16 Fourier Sine and Cosine Transforms. 5.17 Parseval's Identities.
143 154 159 169 174 188 202 207 210 213 218 221 224 226 235 242 244 245 249 257 263
264 267 273 275 278 284 289 291 294 295 303 306 317 330 334 340 351
5.18 5.19 5.20 5.21 5.22 5.23 6
7
Contents
The L2 Theory . . . . . . . . . . .
356 361 366 372 375 378
The Plancherel Theorem . . . . . .
Pointwise Inversion and Summability A Sampling Theorem . The Mellin Transform . Variations. . .
. .
.
.
383
The Discrete and Fast Fourier Transforms
6.1 6.2 6.3 6.4 6.5
The Discrete Fourier Transform . .
.
The Inversion Theorem for the DFT.
Fast Fourier Transform for N Cyclic Convolution .
. . . . . . . . 2k. .
The Fast Fourier Transform for N =
RC.
=
Wavelets
7.1 7.2 7.3 7.4 7.5
Orthonormal Basis from One Function Multiresolution Analysis. . . . . . .
Construction of
a
.
.
.
Shannon Wavelets.
. .
. .
Riesz Bases and MRAs Franklin Wavelets. Frames . . . . . . . Splines . . . . . . .
. .
The Continuous Wavelet Transform.
Index
.
383 390 396 399 406 411
413 414 419 422
Scaling Function with
Compact Support .
7.6 7.7 7.8 7.9 7.10 7.11
.
Mother Wavelets Yield Wavelet Bases. From MRA to Mother Wavelet
ix
435 448 449 459 464 476 480 497
1
Met ric and N ormed Spaces I t is natural t o think about distance between physical objects-people, say, or buildings or stars. In what follows, we explore a notion of "closeness" for such things as functions and sequences. (How far is f (x) = x 3 from g (x} = sin x? How far is the sequence ( lin) from (2/n 2)?) The way we answer such a question is. through the idea of a metric space. In principle , it enables us to talk about the distance between colors or ideas or songs. When we can measure "distance," we can take limits or "perform analysis." Special distance-measuring devices called norms are introduced for vector spaces. The analysis we care most about in this book involves norms . This type of analysis is known as functional analysis because the vector spaces of greatest interest are spaces of functions.
1.1
Metric Spaces
We define a metric space here, then give some examples. The idea is to abstract the properties of the usual notion of distance.
Definition 1.1.1 METRIC SPACE
Two things are required to define a M ETRIC SPACE: a set X of elements called P O INTS or ELEMENTS and a M ETRIC (or DISTANCE FUNCTION ) d, defined on pairs (x, y) of points that satisfy the following conditions . For all x and y in X , •
• •
(Positive) d( x , y) � 0, and d(x , y ) = 0 if and only if x = y. Thus, not only is the distance from any point x to itself 0, but x is the only point 0 units. away from x. (Symmetric) d(x , y) = d(y, x).
(Triangle Inequality) For any z E X, d(x , y) :s: d(x , z) + d(z, y) . 0
As a rule, the only property that is difficult to verify about a proposed metric is the triangle inequality. We use the following notation throughout the book . • •
R C
denotes the real numbers. denotes the complex numbers.
2
1 . Metric and Normed Spaces •
K=
R or e without specifying which.
Example 1.1.2 THE EUCLIDEAN SPACES (K n ,
d2 ) = £2 (n ) , n E N With distance defined by dl (x, y) = I x - YI , (K, dJ ) is a metric space. Let K 2 denote the Cartesian product K K = {(x, y) x, Y E K } . A 2 :
x
metric d 2 is defined on K by
x (X I , X 2 ) Y ( Y2 , Y2 ). (Xl , X 2 , X 3 ) Y ( YI , Y2 , Y3 ) d2 (x, y) = [ lYI - xl 1 2 + I Y2 - x 2 1 2 + IY3 - x 3 1 2P / 2 .
and = where = Let K 3 = K x K x K. A distance i s defined between the points of K 3 by and =
x
The triangle inequality is verified by using the Minkowski inequality 1 .6 . 3 ( a) with p = 2. In either case d2 is called the EUCLIDEAN or USUAL metric on K 2 and K 3 . The spaces ( K 2 , d2 ) and are called EU CLIDEAN 2-SPACE ( the EUCLIDEAN PLANE if K = and EUCLIDEAN 3-SPACE, re spectively. If we say only or we mean respec and tively. We extend this idea of distance to n-dimensional space K n by taking, for EK n ,
R2 R3 ,
(K3 , d ) R) 2 (R2 , d2 )
(R3 , d2 ),
(x; ), ( Yi )
The triangle inequality follows from the Minkowski inequality with p = 2. The metric d2 is also called the EUCLIDEAN METRIC , the pair d2) EUCLIDEAN n-SPACE. Many authors reserve Eu clidean for d2) and refer to as UNITARY n-SPACE . We say real or complex Euclidean n-space, respectively; we also denote ( K n , by ( n) . 0
(Kn , n (R ,
(e n , d2 )
d2 ) £ 2
In many imp ortant respects , the spaces we care most about in this book can be thought of as though they were Euclidean 2- or 3-space , a point that will become clearer as our story unfolds. For now , you might glance at Exercise 1 .3-8 . I t is common t o view as a "subset" of the plane even though i t is actually x {O} , not that is the subset of R2 . As metric spaces, there is no difference between and R x {O} C (2): For two real numbers a and b compare the distance (a, b) = l a between them with the d 2 -distance between (a, 0) and (b, O) :
R
R R, (R,dl )
R2 ,
dl
£2 - bl
d2 [(a, 0) , (b, 0)] = j(a - b) 2 + (0 - 0) 2 = l a - b l .
1.
Metric and Normed Spaces
3
Likewise , as metric spaces, there is no difference between ( R 2 , d 2 ) and the complex numbers C-distance is measured in exactly the same way in each case. The point is that there can be concretely different things that are the same as metric spaces. Definition 1.1.3 ISOMETRY
Let (X, d) and (Y, d' ) be metric spaces. A map f : (X, d) ---+ (Y, d') is an ISOMETRY if for all x, y E X,
d' (f(x) , f(y)) = d(x , y) . If the isometry f i s onto (surjective) i n the sense that for all y i n Y there is some x in X such that f(x) = y, then the spaces (X, d) and (Y, d' ) are called ISOMETRIC . 0 Notice that isometries f must be 1- 1 (injective) , since
x =F y ¢::::? d (x , y) =F 0 ¢::::? d' (f(x) , f(y)) =F 0 ¢::::? f(x) =F f (y) · Some obvious isometries of R2 onto R2 are translation, rotation through
a fixed angle , and reflection about a line . Indeed , these are the only ones , something we say a little more about in Exercise 1 .5-2 . We commonly identify-i.e ., treat as equal-isometric spaces, as in "R is a subspace of R2 " rather than "R is isometric to a subspace of R2 . " Let Pn denote the set of polynomials p(t) of degree less than or equal to n with real coefficients and distance defined by
n
I)ai - b; ) 2 . ;=0
The space Pn is isometric to Rn + l = £2 ( n + 1) under the mapping
f : Rn + l
(ao,al , . . . , an )
--+
1----+
Pn L�o a; t i '
At times it is preferable to view Pn as a set of polynomials, at other times as a set of ( n + 1 )-tuples. As metric spaces, they are the same. Example 1.1.4 PRODUCT METRICS
The square-root-of-the-sum-of-the-squares method used to create a met ric for the Cartesian products R x R and R x R x R in Example 1 . 1 .2 can be generalized to Cartesian products of arbitrary metric spaces as follows. Suppose that (X, d) and (Y, d' ) are metric spaces. A metric d2 on X x Y = {(x, y) : x E X, Y E Y} is defined by ( 1 . 1)
4
1 . Metric and Normed Spaces
The triangle inequality follows from the Minkowski inequality 1 .6 .3( a) with p = 2. More generally, the Minkowski inequality implies that for 1 � p < 00 , defines a metric o n X x Y. Aside from p = 2 , another imp ortant special case is p = 1: a special case of which is discussed below in Example 1 . 1 .5. For p = 00, The triangle inequality follows from the Minkowski inequality of Exercise 1 .6-2. X Xn Each of the dp can be extended to any finite product Xl x X2 X of metric spaces (Xi , di) as the pth root of the sum of pth powers •
•
.
Are any of the dp to be preferred? In a sense to be made clear in Section 2 . 9 , no, because the convergent sequences are the same with respect to any of them. Suppose that (X, d) and (Y, d') are metric spaces. An imp ortant property that each of these metrics dp on X x Y possesses is that they "extend" the original metrics-for any fixed y in Y, the space X x { y} is metrically a copy of X. Since d' (y, y) = it follows that for any Xl , X 2 E X , by equation ( 1 .2) ,
0,
p 1 dp «X 1 , y) , (X 2 , Y)) = (d(X 1 , x 2t + d, (y, y) ) /P = d (X 1 , X 2 ) , 1 � p �oo. In other words, X {y} is isometric to X under the "projection" map (x , y)�x . 0 In the next example we emphasize the point that it is the pair (X, d) and not just the set X that defines a metric space. x
Example 1.1.5 TAXICAB METRIC
As follows from Example 1 . 1 .4, for X = (Xl , yd and y = (X 2 , Y2 ) in the set R2 ,
d 1 (x, y) = I X 2 - x d + IY2 - yd defines a metric on R 2 . It is called the TAXICAB metric because distance
is measured by adding horizontal and vertical distances, as it would be computed by a taxi ' s odometer on a grid of city streets. It is much easier
1 . Metric and Narmed Spaces
5
to work with than the Euclidean metric, withthe itsorigin cumbersome square(1, roots d1 -didstance from to the point l) in ofthissums of squares. The metric is 2, not V2, the 2 -dis tance; thus, even though the points are the same (namely the points of R2 ), the distances have changed. 0 Example 1.1.6 THE TRIVIAL METRIC
The trivial metric is great for counterexamples (and not much else). On d any points(or yDISCRETE E X , take (x, y) = 1 if =P y and 0 otherwis e; d is setcalledX, fortheanyTRIVIAL . The space (X, d) is called 0 a (not "the") TRIVIAL METRIC SPACE.) METRIC If inn-tuples the spaces '-2 ( ) of Example 1 . 1 .2 we let 00,usually i.e., wecalledgo from to square-summable sequences, we get what is HILBERT SPACE £2 , known also as '-2 (N) or '-2 (00). Be careful about the usage Hilbert "Hilbertspace space,(Definition " because 2.there.1).is a more general type of space also called 6 SQUARE-SUMMABLE SEQUENCES '-2 (N) = £2 (00) = '-2 = £Example 2 (Z) The square-root-of-sums-of-squares of measuring distance is taken about far it(00)can ofgoallin SQUARE-SUMMABLE the space way ( the Hilb ert space, "little ell-two") = £2 £complex 2 = £2 (N)numbers, sequences (an ) such that sequences (an) of real or LneN 1 an 1 2< 00. toWebedefine the distance between two such sequences1 / 2 = (an) and y = ( bn) d2 ( x, y) = ( L 1 bn an 1 2) n eN The triangle inequality follows from the Minkowski inequality 1.6 . 3 (b) for infinite Let Zseries. denote the setto beof integers. ofC a, two-sided series (biseries) m,n_ooTheL�sum lim =E N; for convergent m m k Lbiseries n eZ CnLnis defined terms, limn_oo L�(an) Cn of nonnegative =- n Ck · LneZ Cn =bisequences eZcollection Consider the '(Z) of square-summable neZ 2 such that x,
x
n
n -+
1.1.7
as
as
x
_
n,
and take
6
1. Metric and Normed Spaces
asisa any metric on.e2 (Z). It is easy to show that .e2 (N) is isometric to.e2 (Z); if ( i. e ., 1- 1 and onto map ) ofN onto Z, then (xn ) (xf( n ») fmaps .e bijection Exercise2 (9N. ) 0isometrically onto .e2 (Z). We ask the reader to verify this in By adistance SUBSPACE of a metric space (X, d) we mean a subset S of X in which is still measured using d; we denote such a subspace by s) (S, d i . forof.e any(N ) . > 2; the spaces Rn =.e2 ( ) E N, are (isometric to) subspaces 2 took the "square root of sums of squares" idea to a limit in Example We 1 . 1 .7. In Example 1 . 1 .8 we go further; we go from sums to integrals. Example SQUARE-SUMMABLE FUN CTIO NS L 2 [ a, b) We say thaton two complex-valued functions f and are "equal almost everywhere" the closed interval [a, b) if f (t) = 9 (t) at each t E [a, b) but for ina setSection of Lebesgue measure 0 ( cf. Exercise 15 and the brief dis cussion 1 . 6 ) . We abbreviate "almost everywhere" to "a. e." We treat functions that are equal a.e. as the same ( we "identify" functions that are equal a. e . ) . This means, in particular, that we treat as equal the CHARACTERISTI C (or INDICATOR ) function lQ of the rational numbers Q (lQ = 1 on the rationals and 0 everywhere else ) and the function that is identically O. The convention of identifying almost-everywhere-equal tions can therefore make for some drastic differences and simplify thingsfuncin certain cases. Withspace, a.e . equal treated(foras thethe Lebesgue same function, collection (vector actuallfunctions y) of functions integralthe) L 2 [a, b) = { x [a, b]-+ K lb I x(t) l 2 dt < 00 } istionsknown space of S QUARE-SUMMABLE or SQUARE-INTEGRABLE func on [a,as the b] . The distance between x , y E L 2 [a, b] is defined to be ( ) 1/2 b 2 d(x , y) = l l x(t) - y(t) 1 dt IfL the( R)functions x are square-integrable on R, J�oo I x(t) 1 2 dt < 00 , we write 2 . 1The6 .3 (triangle namely c) . 0 inequality follows from another Minkowski inequality, �
n , n
n
1.1.8
9
:
:
.
Exercises 1 . 1
(X, d)
is a metric space in the problems below.
1.
Metric and Normed Spaces
7
themust only thepossible routes(usualbetween "cities" arethem specified by aalong road map, distances notion) between measured those routes satisfy the triangle inequality? 2. Show that a metric space is still obtain ed if the real numbers are replaced by the complex numbers C in any of the parts of Example 1 . 1 .2. 3 . PSEUDOMETRICS If a function on X fails to be a metric only in that p(x , y) = for certain x # y in X, then is called a PSEUDO METRIC . Let X be any collection of functions on a set T and let t E T. Show that p(x , y) = Ix(t) y(t) l , X , y E X, defines a pseudometric on X . 4. PRO DUCT METRICS NOT ISOMETRIC Show that for p # q , the spaces (X x Y, dp ) and (X Y, dq) of Example 1 .1 .4 are not isometric spaces under the identity map x ..-. x . 5 . POSITIVE MULTIPLES O F METRICS If a > 0, show that da , defined by da (x, y) = ad(x, y) for all x, y E X , is metric. This shows that it is possible to define infinitely many metrics on any metric space. 6. NEW METRICS FRO M OLD Let 1 : [0,00) -+ R be increasing with I(t) > for t > 0, and I( + t) I(s) + I(t) . For any 1metric (0) =d0,show that d' (x, y)r = J[d (x , y)] also defines a metric. Some suitable s are f' inf ( 1 , t) . I(t) = t (0::; 1 ) , In( l + t; ) , tl (l + t) , and 7. EXTENDED REALS Use the result of the preceding exercise to metrize the EX TENDED REAL N UMBERS = Ru {±oo} by means of the map 1 [-1 , 1] , x l.Hxl for x E R, 1 ( -00) = - 1 , I ( 00) = 1 . (For x E ( - 1 , 1 ) , 1 - 1 (x) = xl (1 - I x!) .) For x > 0, what is d' (x, oo)? d' (1, oo)? 8. BIJECTIONS DEFINE METRICS If I X Y is bij ective, then show that 1 . If
p
°
p
-
x
a
°
:
R
r::;
::;
s
R
-+
..-.
:
-+
defines a metric on Y .
9. SOME ISOMETRIES
(a) Show that £2 (N) is isometric to £2 (Z) . (b) isometric For any twoto Lclosed intervals [a , bj and [e, d) show that L2 [a , bj is 2 [e, d) . 10. Identi fy themetric set of space; all pointsthe that t away from a point x in a trivial set ofarealldistance points distance 2 away?
8
1.
Metric and Normed Spaces
MAX AND SUP METRICS ( cf. Example 2.2.8) We explore the metric doo of Example 1.1.4 a little further here. ( a ) Let E N. In real £00 ( ) = (R n , doo), the distance between any two points x = (ai ) and y = (bi ) in Rn is doo(x, y) = m!lX I bi - ai l· VerifyOnthat 1dooforisXa metric. Identi f£y the(3)"unit ball" U = {x E X: £00 d (x, ::; (2) and 00 = ( b ) If x = (a i) and y = (b i ) are points in Roo = I1 i EN R, the space of all sequences ofberealnonumbers, theconsider same idea will not work because there may maximum: x = (0) and y = ( ) If we restrict consideration to the set Roo of all bounded sequences of real or complex numbers and use sup instead of then doo(x, y) = SUPi Ibi - ail defines a metric. 12. THE HAMMING METRIC Let X(n) denote the set of all n-tuples of O's and l 's . For example (without the customary commas and paren theses) , X(3) = {000, 001, 010, 011, 100, 101, 110, I ll } . The HAMMING METRIC d measures distance by similarity: For x, y E X ( n ) , d(x, y) is the number of places where x and y have different entries. For example, d( O O O , 001) = 1 and d(010, 101) = 3. Verify that d is a metric. This metric space has applications in coding and automata theory, combinatorics, and computer science.
11.
n
n
t
.
max,
13.
n .
ULTRAMETRICS
a The metric space indefined here, knowntheory. as the Let BAIRE N ULL SPACE, has applications communication B be the set of all sequences of positive integers. For x = ( a 2 , . .) and y = (b l , b2 , .. .) in B define { 0, ifif iaiis=thebi forfirstallindexi, for which ai =P bi. d(x, y) = As infurther the previous exerci sdifference e, this metricoccurs, is based onclosersimilarity the out the first the x and y are. Veri f y that thi s i s an ULTRAMETRIC in that it satisfies the STRO NG TRIANGLE, or U LTRAMETRIC , INEQUALITY , namely, d(x, y) ::; max {d(x, z), d(z, y)} for all x, y, and
( )
aI,
_zl.'
z.
.
1.
9
Metric and Normed Spaces
b Show that the metric doc of Exercise ll(a) is not an ultrametric. 14. Let N 2 = N N denote the grid of all ordered pairs of positive integers. Does d[(a, b), (c, d)] = I da - b l / ( bd) define a metric? 15. In Example 1 . 1 . 8 we mentioned that we treated functions in L 2 [a, b) that werex -equalifalmost everywhere as the same. Show that the re lation y and only if x = y a. e . is an equivalence relation onclasses. L 2 [a, b]; as such, it decomposes L 2 [a, b] into disjoint equivalence It is these classes that actually constitute the "points" of L ( )
x
c
2 [a, b] .
Hints
and d' ( 1 , 00) = �. 1 3 ( a) . If the distance from x = (aI , a 2 , "') to y = (b l , b 2 , . . . ) in B is Iii, then for j <element i. Now compare x and y to = ( C I' C 2 , ) , whereaj is= abjsuitable of B. 14. What is the distance from ( 1 , 2) to ( 2, 4) ? 7. d' (x, 00 ) = 1/ (1 + x)
z
1.2
z
.
.
•
Normed Spaces
When we say "vector space" we always mean real or complex vector space. we will specify which by saying something like "Let X be a real Sometimes vector space. " If we do not specify, we mean the result to apply to both. The space R or C is denoted generically by K . Real or complex numbers S CALARS . By a SUBSPACE of a vector space we always mean a liare nAnalysis, eacalled rsubspace, a subset thatlimits, is also acannot vectorbespace; we say subse the ability to take doneotherwise in an arbitrary vectort. space.involve In particular, nosums, consideration ofjust infinitesums.seriesIn order is possible because li m i t s of rather than to be able toa they donormanalysis in vector spaces, we introduce a special kind of metric called a vectorintoin theR2 space. or R3 . It generalizes the notion of length, or magnitude, of Definition 1.2.1 N ORM
�
The combination (X, 1 1 · II II : X R isare:called a N O RMED (LINEAR) SPACE . The defining properties of a norm (Positive) I I x i l � 0 for all x E X , and II x i l = 0 if and only if x = O . (Multiplicative) I I axi l = l a l I I x il for all a E K and x E X. •
•
10
1 . Metric and Normed Spaces
+ yl l � I I x l l + I l yl l for all x , y E X. 0 Any normed space is a metric space with d(x , y) = Il x - yl l , since d(x, y) = d(y, x) = I I x - y l l � 0, and II x - y ll = 0 if and only if x = y and (add and subtract y) d(x , z) = I I x - y + y - z l l � I I x - y l l + lI y - z l l = d(x , y) + d(y, ) Whenever we speak of a normed space a metric space, it is always to this norm-in du ced metric that we refer. The prototype normed space is Rn = i2 ( n ) with the usual term-by-term ( also called "pointwise" ) operations and scalar multiplication and the inequality EUCLIDEAN NORM, I I x l b = of addition satisfies the tri a ngle by the Minkowski = (ad l I l l 2 v'L:�=1 ali 11 1 1 2 inequality 1. 6 .3( a ) and induces the nEuclidean metric of Example 1.1. 2 . The a normforonx K= (a( cf.) EExample following 1.1.4) known the n K or sup normalso11 defines namely, , i 11 00 ' (1. 5 ) I Ix l ioo = sup l ai I · i The taxicab norm ( Example 1.1.5) n •
(Triangle Inequality) Il x
z .
as
as
I l x l ll =
� )ail ;=1
max
(1.6)
and the Euclidean norm are each a special kind of p-NORM ( cf. Example ' /' (1.7) II z l l , = (t. l a; I' ) , 1';: p < 00, TheWereason for treating 11·11to00sequences as a p-normandisfunctions, discussed toin the Exercivector se 4.spaces pass from n-tuples i2 (N) and L 2 [a, b] of Examples 1.1.7 and 1.1.8, by taking ( ') 1 2 1/2 ' and I Izl l, = ([ I z(') I d') , I I z l l , = E.. I a n I respectively. These are special cases of the following. Example 1 .2.2 p-NORMS ip ( n ) , ip (N) , and Lp [a, b], 1 � � 00 (a ) ip ( n ) = (K n , 11 ' l Ip ) and ip (N), 1 � p < 00. The space ip ( n ) is the normed linear space Kn of n-tuples of scalars equipped with the p-norm: 1.1.4) ,
/
p
1.
Metric and Normed Spaces p
11
TheNowtriangle followstofrom the Minkowski we passinequality from n-tuples sequences. For 1 ::; inequality let 1.6.3 ( a). < lp (N) = lp = { (an ) E Koo : L l a n l P < oo} , n N the of pTH P O WER SUMMABLE sequences. With pointwise opera tions,collection l is a vector space. We always view l as equip ped with the p-N O RM 00 ,
E
p
p
Toinfinite verifseries. y that 1I · l Ip is a norm, we use the Minkowski inequality 1.6.3(b) for ( b ) Let E be a Lebesgue-measurable set. For ::; 00, the class of K-valued Lebesgue-measurable functions x : E -+1K such< that h l x(tW dt < isE denoted by Lp (E). If E is a finite interval [a, b] we write Lp [a , b] ; if = R, we write Lp (R). The elements of Lp (E) are called pTH POWER SUMMABLE functions on E; L ; (E) and L; (R) denote the subspaces ofreal (E) and L ; (R), respectively. As in Example 1.1.8, valued functions of L;that weformidenti f y functions a vector space, normedarebyequal a.e. As discussed in Section 1.6, they p
00
which is also called the p-N ORM. (c ) The collection loo (N) , or just loo , of all bounded sequences x = (an ) ofnormed elements space with pointwise operations; it is by an E K forms a vector I l x l i oo = sup I an I · n For more details see Section 1.6 and Exercise 1.6-2 ( a) . 0 The following is a very important normed space.
Example 1.2.3 CONTINU OUS FUNCTIO NS C [a, b]
Let Cinterval [a , b] denote the space of continuous K-valued functions on the closed [a , b], ::; a < b::; with sUP or UNIFO RM norm: II x l i oo = sup {I x (t) 1 : t E [0, In . - 00
00 ,
12
1 . Metric and N armed Spaces
Unless we say otherwise, C [a , b] always carries the sup norm. 0 As theI I x l lorigin. = d(x , O), the norm is seen to measure the distance of a vector from vector of norm 1 is said to be a UNIT VECTOR. For any nonzero vectornormsx , xlinduce II x l l is a unit vector. Although metrics, notts every metric defines a(X,norm. For one d) need thing, the underlying set X of poin of the metric space not be a vector space, but even if it is, notice that nonzero vectors x must get bigger2 as they are multiplied by larger scalars: l a l ---+ 00 ==> lI ax l l ---+ 00 . If R carries the trivial metric of 1 . 1 . 6, then d(ax , 0) = 1 for all a =J 0, so metric 2.cannot come from a norm. ofthisExercise 9-7 is have not norm-induced either.Similarly, the bounded metric A
Exercises 1 . 2 q q
q
map is proposed as a norm on a vector space X: 1. The following . O Is a norm? q (x) = 1 for x =J 0, (0) 2. SECOND TRIANGLE INEQU ALITY Show that any norm satisfies the SECOND TRIANGLE INEQUALITY : I l I x l l - I I Y I I I ::; II x - y l l for all vectors x and y. 3 . THE MAX NORM Let {X l , X 2 , , xn } , E N, be a basis for a vector " , a } be scalars. Show that 1 1 2:7:: ai xi l l oo = Let {ala, norm mspacei l aiX.l defines on X; 11 · 11 00 is called the MAX N O RM for X WITH RESPECT TO THE BASIS {Xl , X 2 , . . . , Xn } . denote 4. 1 I ' l Ip on K n , E N. Let 1 I · lI p , 1 p < the p-norm of equation (1. 7 ) on Kn . To see why the notation 11· 11 00 is used to denote the norm (equation ( 1 .5) ) on K n , for any E K n , show that lp I I x l l oo ::; I I x ll p ::; n / I l x l i oo . Deduce from this that I Ixlloo = limp-+ oo I I x lip . .
ax
max
=
n
.
.
•
n
n
1
::;
00 ,
x
Hints 4.
1 .3
Inner Product Spaces
Another way toof generate by means produ ct. It is a generalization the familiarnormsdot isproduct of Rof2 andan inner R3 .
1.
13
Metric and Normed Spaces
Definition 1 . 3 . 1 INNER PRODUCT
vectorKspace K ( = R or C) and an INNER PRO DUCT ( , ) is calledX over an INNER PRODUCT SPACE if the following conditions are met. (Conjugate Symmetry) (x , y) = (y, x) , where the bar denotes com plex conjugation. (Positive Definite) (x , x) 0, and ( x , x) = 0 if and only if x = O. A XxX •
•
•
�
-+
�
(Linear in the First Argument) (ax + by, z ) = o
a (x ,
n
(x , y) =L a i bi
z
) + b (y , ) . z
(1. 8) is =an2inner product; thisin the is therealfamiliar dotconjugate product when K = R and or 3. Note that case, the symmetry reduces tothe(x,mosty) =important (y , x) and that (x , ay) = a (x , y) for any a E R. For many of results aboutspace.innerTheproduct spaces, itofisequation vital that(1.8)we have a complex inner product inner products are essentially the only inner products on Kn (1.5.4). The spaces £2 and spaces with respect toL2 [a, b] of Examples 1.1.7 and 1.1.8 are inner product b (1. 9 ) (x, y) = L a n bn and (x, y) = l x(t)y(t) dt , neN respecti vely.already seen that norms are more special than metrics. Inner We have productsa arenormevenby more induces way ofspecial than norms. Note that an inner product ( 1.10 ) I l x l l =� , weinnerproveproducts after 1.just 3.2. mentioned. The norms Whenever of £2 ( ) £the byin 2 are induced 2 , and11·11 Lsymbol is used the an inner (1.10). productTospace, itthatrefers� to thesatisfies inner theproduct-induced norm weof equation prove triangle inequality, first generalize the following result about the dot product of two vectors x, y in R 2 : I (x , y) 1 = I l I x l l I /y l l cos 01 � I I x l i l l yl i for any i= l
n
a
as
n ,
O.
14
1 . Metric and Normed Spaces
1 . 3 . 2 CAU CHy-S CHWARZ INEQU ALITY For any two vectors x and y of an inner produ ct spa ce l (x , y) 1 � II x l i ll y l i.
X, Proof. If x or y is 0, both sides of the inequali ty are 0, so suppose that neither x nor y is O. For any scalar a , o �
(x - ay, x - ay) .
Fora = (x , y) /(y, y) ,
���� 1 2 0 (1.11) Now weandcanmultiplicative complete theproperties proof thatof inner productsdirectly definefrom norms.the The positive norm follow def inition of inner product: I I x l l = � 0 and I I x l l = 0 if and only if x = 0, and for any scalar a , o � (x - ay, x
_
ay) = II x l 1 2
_
1(
�
l I ax l l = v(ax, ax) = l a l �. �
As for the triangle inequality, since Re (x , y) I (x , y) l , o < I I x + Yl l 2 = (x + y, x + y) I I x l1 2 + 2 Re (x , y) + I I y l 1 2 < I I x l 1 2 + 2 1 (x , y) 1 + l I y l l 2 . By the Cauchy-Schwarz inequality of 1.3.2, it follows that which yields the triangle inequality. Cauchy-Schwarz magni magnitude Inof the dot tude,Whenwhendoesis theequality inner hold productin thea product? In R2 theinequality? of twobetween vectors isthem the same as the product of the lengths if and only thethe angle is 0 or i. e., one vector is a scalar multiple ifofproduct other. As is frequently the case, what happens in R2 can be used to predict what happens generally. In any inner product space, for any scalar a and vector I (x , ax) 1 = l a l ll x ll 2 = II x l i ll ax l l ; thus, equality holds if one vector is a multiple of the other. Conversely, if equality holds for nonzero vectors x and y, then the right side of (1.11) is 0, and x - ay = x - y = 0 , or x = i=.Jll( and only if the vectors are scalar multiples of each other. 0
x,
Y,Y
11",
(X,y)) ( Y,Y
1.
Metric and Normed Spaces
15
Exercises 1 . 3
Inofantheinner productx +space, show that, iftheandlength ofif only + a =sumax isforthesomesumscalar lengths, x y ll = I I ll l I y l l II a � O. Not all norms have this property. Those y that do are called STRICT N ORMS . Show that neither the taxicab norm ( equation (1.6) of Section 1 . 2) nor the max norm ( equation (1.5) of Section 1.2) is a strict norm.
1 . STRICT NORMS
2.
ANG LE
( a)
Define aspace. notionShow of "angl e"your between anyof the twoangle vectors() between in an inner product that notion the vectors x and y in a real inner product space satisfies the law of cosines, i.e ., that
(b) In the real space '-2 (2), show that the �ap A defined by ( � ) (� 0 1 ) ( � ) amounts to rotating a vector counterclock wisesobyin90complex degrees,'- that {Ax , x } = 0 for all x; show that this is not 2 (2). 3. D IFFERENTIABLE FUN CTIO NS The collection C1 [a, b] of continuously differentiabl complex-val uedto functions onaddition the closed intervalmultipli [a, b] is acation. vectorShow spacee that withanrespect pointwise and scalar inner product is defined by (x , y) = lb ( x(t)y(t) + Xl(t)yl (t) ) dt . 4. TRIG O N O METRI C POLYNO MIALS A function x(t) = a + � )a k cos kt + b k sin kt) or L ck e ik t k=l k=-n with complex Tcoefficients is knownpolyasnomials a trigonom etric polyn o mial. The collection of trigonometric obviously tor1, 2,space. Show that ( anticipating Fourier coefficients) forformsanya vec k = 1'lr f�'Ir x (t) dt, a 2 .; D'Ir x(t) cos kt dt , ak bk .; f�'Ir x(t) sin kt dt .
f--+
n
. . . , n,
n
1.
16
5.
Metric and Normed Spaces
Let RH (S l ) denote the space ofrational functions that are holomor phic (analytic) on the circumference S l of the unit circle in C , i. e ., have no pole of magnitude 1. The space RH (31 ) is a vector space with respect to pointwise addition and scalar multiplication. (a) Show that an inner product is defined on RH (Sl ) by 1 1 - dz (f, g ) = 2 . f(z) g (z) -Z . (b) bers Use the ZCauchy integral formula tocircle, show that for complex num strictly inside the unit 2 ( z � Z l ' Z � 2 ) = 1 - �1 2 �z
SI
Zl,
6.
Z
CAUCHy-SCHWARZ INEQU ALITY REVISITED
Z
•
�
(a) Ifo fora, b,allandrealaret , show real numbers with a > 0 and f (t) = at 2 + bt + c 2 that b ::; 4ac. (b) aApply the preceding result to f(A) = I I AX + y l1 2 , where A is complexspace, number and xtheandCauchy-Schwarz y are fixed vectors in an 1.inner product to deduce inequality 3.2. 7 . RIGHT TRIANG LES (cf. Exercise 1.2-2.) Let x, y, and z be three vec tors in an inner product space X. Define "triangle" to mean the triple T (x, y, z) . We say that T is right-angle d at y if (x - y, z - y) = O. (a) Show that T is right-angled at y if and only if I I x - zl l 2 2 2 I I x - y I I + l i z Y1 l . (b) Show that a triangle cannot be right-angled at x and at y . 8. LINES AND PLANES In R3 with its usual inner product, the plane through b, c) normal to the vector (A, B, C) is defined to be thetheset point of all (a,points (x, y, z) such that ( x - a, y - b, z - ) (A, B, C)) = O. Bydefineanalogy withTHROUGH this, in anXoarbitrary real inner product space X we a PLANE NORMAL TO w to be {x E X : (x - Xo , w ) = O}. Similarly, R3 , the line through the point Xo = (a, b , c) parallel to (orthat"in theindirection of") the vector (A, B, C) is the set (x, y, z) such (x, y, z ) t (A, B, C) + (a, b, c) c
=
_
c
=
,
1.
17
Metric and Normed Spaces
the similarity to y = mx + b.) In a general inner forproduct all realspacet . (Note we define the LINE L THROUGH x PARALLEL TO to be the set of all vectors of the form z = ty + x where y # 0 and x and y are fixed vectors and t E These ideas are the basis for the theory of linear programming. In a real inner product space X, (a) Show that if Il x - yll + II y - z ll = II x - z ll , then x, y, and z lie on a line. ( b ) For real k, show that a plane ( x, n) = k divides the space into two parts: (x, ) 2 k and (x, ) < k. ( c ) Show that if a line meets a plane or another line, then the in tersection is a singleton. ( d ) Show that a line z = ty+x meets a SPHERE {v E X: !I v ull = r} oftheradius r > 0 and center u in at most two points. If it meets sphere point w (in which case the line is called a "tangent"in) , only then one ( w - u, y) = O. Draw a picture and interpret this geometrically. A similar result holds for tangent planes to spheres. Compare this to the situation in part (f) . ( e ) The lines z = ty + x and w = tu + are called PARALLEL if y = au for some nonzero scalar a. Show that if the only two possibilities are that that they any meettwoor lines are parallel, dimX = 2. forThelinesproposition meet orthen are parallel is fundamental inbeplane Euclidean geometry. The real ization that there might geometries in which the proposition some isthing false-of there beingin human "non-Euclidean" of a milestone intellectualgeometries-was development. (f) LINES IN NORMED SPACES In any vector space X, define lin e as in the previous exercise. The LINE SEGMENT connecting x and y is [x , y] = {tx + (l - t) y : O :S t :S I } . If X is (apartnormed space, unlike what happens in inner product spaces ( d)) , show that a line can intersect a sphere in infinitely many points. (g) CONVEXITY subset of a vector space is called CON VEX if it contains the ball line insegment two ofIs itsanypoints. Show that the unit a normedthrough space isanyconvex. ball convex? Y
R.
n
n
-
v
A
18
1 . Metric and Normed Spaces
Hints
First, useinequality. the remarkThenaftershow1.3.2thatconcerning equalitynumber in the aCauchy Schwarz for any complex , 11 + a l = 1 + l a l if and only if a � o . 3. Use the fact that if a nonnegative continuous function x(t) is such that J: x(t) dt = 0, then x(t) = of magnitude 1 such that (x , y) 6 (b). Let ,beletat complex be real, andnumber consider A = tao I {x, y) I 8(a) . Use Exercise 1.2-1. 8 (d). This has to do with solutions of quadratic equations. By considering thelinearly linesindependent z = ty and w = tu + show that there 8 (e) . cannot be three vectors {y, u, v} in the space. If dimX = I, then all lines meet. 8(f) . Consider the unit sphere in the real space Roo (2), i. e ., R 2 with norm . 1.
o.
a
ii
v,
max
11· 11 00
1.4
Orthogonality
A wonderful thing about inner product spaces is their "geometry." Among other things, therevectors. is a Pythagorean theorem, a paral lelogram law, and "angles" between We say that two vectors x and y are O RTH O G ONAL if (x , y) = 0; we write x y. Since (ax , y) = a (x , y) for any scalar implies that x is orthogonal to any scalar multiple ay of y, a, x i.e., is yorthogonal to the line [y] = {ay: a EO,K}x) =determined by As 0, so the 0 vector (O, x) = (o + 0, x) = 2 (O, x) , it follows that ( is orthogonal to every vector. Conversely, if x is orthogonal to every vec tor, then (forx , x)showing = 0; since ( , ) is positive definite, x = O. This provides a technique equalityIn Rof2 two vectors x and Show that x - y is orthogonal to every vector. the projection of x asontoa Fourier a unit vector is (x , y) y. As will be seen later, expressing a function series isy very much like writing it a sum of projections of this type. Definition 1 .4 . 1 ORTHOGONAL SETS A subset S of an inner product space X is called an O RTH O G O N A L SET iorthogonal f y for all distinct x, y E S. If is denumerable, we say that S is an SEQUENCE. If S is an orthogonal set of unit vectors, it is called anandORTHONORMAL SET . We say x S if x y for every y E S. For sets S T we say S 1. T if x for each x E S and E T. 0 1.
1.
x
y.
y:
as
x
S
1.
1.
y
1.
1.
y
1 . Metric and Normed Spaces •
•
•
•
•
R2 ,
19
R,"
In (0, 1 ) i s orthogonal t o any vector "in i.e . , any vector ( a, 0) in x {OJ . Similarly, (0 , 0 , ) is orthogonal to any vector in the "plane" { ( a, b, 0) : a, b = R2 X {O J .
R
R
1
E R}
5 0 , any vector of the form (0, 0, a3 , . . . , a5 ) i s orthogonal t o any 0
In vector whose last 48 entries are O.
Let T be any set . The COZERO SET of a scalar-valued function x : T is coz x = {t T : x(t) # OJ . If two functions x and y of L 2 [0, 211"] have disjoint cozero sets, then x y = Hence x and y are orthogonal: ( x , y) = xy = O.
K
E
O.
f�7r
The sets
{ 1 / y'2;, cos
N } and { cos
E N} are orthonormal sequences in L 2 [ -1I", 11"] ; so is { e i n t /y'2; : E Z} . n
t / J?r : n E
n
t / J?r : n
n
A function x is EVEN if x (t) = x (- t ) for all t in its domain , O D D if x (t) = - x ( - t) for all t. In L 2 [ - 1I", 11"] , any even function is orthogonal to any odd function.
The linear subspace [S] SPANNED by a set S, or just the LINEAR SPAN K and of S, is the set of all linear combinations L �= l ai X i where ai S. For finite sets {Xl , X 2 , . . . or sequences we omit the braces and write [Xl , x 2 , . . . , x ] instead of [{Xl , X 2 , . . . , x } ] . Evidently, if x .1 S, then
Xi E
x .l [S] .
, Xn}
n
E
n
Example 1 .4.2 ORTHONORMAL SETS
(a) The so-called STANDARD BASIS for £ 2 ( n ) , the vectors
e i = (0, 0, . . . , 0, 1 , 0, . . . , 0) ,
1�i�
n,
with 1 i n the ith position and O's elsewhere, form an orthonormal set. If the l 's are replaced by arbitrary scalars ai , the vectors so obtained still form an orthogonal set . If we take sequences e i = where is the Kronecker delta, with all entries 0 except the ith, which is 1 , the vectors e i , i are an orthonormal set in £2 ; they are called the STANDARD BASIS for £ 2 . (b) An orthonormal subset of L 2 [ - 11", 11"] is given by the unit vectors
(8in),
E N,
1
sin t cos t sin t cos 2t .,f2-i ' ..ji ' ..ji ' ..ji2 ' ..ji , . . . . (c) In the complex space L 2 [ - 1I", 11"] the vectors
in Xn = e 2 11"t E Z , rn= ' n V
8i n
20
1. Metric and N ormed Spaces
comprise an orthonormal set . 0
=( +
+ ) = II ll +
As to the "Pythagorean theorem" in an inner product space, note that 2 2 2 ( 1 . 12 ) x y, x y x 2 Re x , y y , x y
/ I + ll
so
( ) + lI ll
( 1 . 13)
The Pythagorean relation holds for n orthogonal vectors Xl , . . . , X n 2 n
;= 1
;= 1
(
:
Exercise 5) . In real vector spaces, it follows from equation ( 1 . 12) that two vectors satisfy the Pythagorean equality ( 1 . 13) if and only if they are orthogonal. For any vector x and a unit vector y , observe that x- x , y y is orthogonal to y :
( )
( 1 . 14) ( x - ( x , y) y , y) ( x , y) - ( x , y) /l y ll 2 = 0 . Therefore , x - ( x , y) y is also orthogonal to ( x , y) y . By equation ( 1 . 13) , it =
follows that 2 x = x - x, y y
II
/l l1
( ) + ( x , y) Y ll 2 = II x - ( x , y) y ll 2 + I ( x , y) 1 2 ,
or, equivalently,
( 1 . 15) This is a special case of BESSEL'S EQ U ALITY, a topic we return to in Exercise 6, 3 .3 . 1 , and elsewhere. A LINEARLY INDEPENDENT subset S of any vector space is one such that for any X l , . . . , Xn E S, 1 a; Xi = ° implies that each ai equals 0. If two vectors x and y are not multiples of each other, then they are linearly independent . If S is an orthogonal set of nonzero vectors , then S is linearly independent : For any Xl , . . . , x n E S and any scalars aI , . . . , an and any j, if = 1 a i x i 0 , then
2:�
2::
=
Therefore, each aj is equal to 0 . Since they are mutually orthogonal, it follows that the standard basis vectors
(
ei
=
))
(0, 0, . . . , 0 , 1 , 0 , . ) , i E .
.
N,
of £2 Example 1 .4.2( a are linearly independent. The Gram-Schmidt the orem exhibits a process that converts an arbitrary linearly independent sequence Yl , . . . , Yn , . . . into an orthonormal one.
1 . Metric and Normed Spaces
21
1 .4.3 GRA M-SCHMIDT ff YI , " Yn , . . . is a sequence of lin early indepen dent vectors, then there exists an orth onormal sequence Xl , X 2 , . . " Xn , . . such that the lin ear spans [Xl , X 2 , . . . , xn] and [Y l , . . . , Yn] are equal for every "
.
n.
Proof. L e t a , b , c, d E Two simple observations: ( 1 ) If X and y are linearly independent , then ax + b y = ex + dy if and only if a = c and b = d. (2) For nonzero a and b, the linear spans [x , y] and [ax , b y] are equal. Let Y I and Y2 be linearly independent . The argument for Y I , Y2 demon strates the crucial inductive step . Let Xl = yI / I I Yl ll . Now subtract the projection of Y2 onto Xl from Y2 : Let w = Y2 - (Y2 , X l ) X l . As observed in equation ( 1 . 14) , w 1. X l . Could w be equal to O? If so, then Y I and Y2 would be linearly dependent. Since w =P 0 , we may consider the unit vector X 2 = wi I I w l l j {Xl , X 2 } is an orthonormal set such that [Yl , Y2 ] = [Xl , X 2 ] ' Assume that the result of the theorem holds for sets of n - 1 linearly independent vectors, that {YI , " " Yn } is linearly independent, and that {Xl , X 2 , . . . , Xn - l } is orthonormal and such that the linear spans [Xl , X 2 , . . . . . . , Xn - l] and [Y I , . . . , Yn - l] are equal . Let
K.
Wn = Yn Then for 1 � j �
n
n-l -
(Yn, x ; ) X ; . L ;= 1
- 1,
( wn , Xj ) = (Yn , Xj )
n- l -
L (Yn , X; ) (x ; , Xj ) = (Yn , Xj ) - (Yn , Xj ) = 0 . ;=1
As in the case for two vectors , Wn =P 0 . With
n Xn = W I I wn ll i t i s straightforward t o verify that {Xl , . . . , x n } has the desired properties. o
A BASIS , or HAMEL BASE, in a vector space X is a linearly independent collection B of vectors such that any X in X may be expressed as a (finite) linear combination X = ai X; of vectors from B. The standard basis
2:7= 1
e i = (0 , 0 , . .
.
, O, � O, . . .) , i E N , i th
is a Hamel base for the space cp of sequences all but a finite number of whose entries are OJ the members of cp are also called "finitely nonzero sequences ." Despite the name "standard basis," it is not a Hamel base for £ 2 j however, it is a "Schauder basis" for £ 2 (see Definition 7 . 7 . 1 ) . Note that the possibility of using one finite set of basis vectors to express X and
22
1. Metric and Normed Spaces
another set to express a different vector y is perfectly possible . If there is a finite basis { X l , X 2 , , xn } for a vector space X, then all bases for X have n elements . This fact enables us to define DIMENSION as dim X = n for vector spaces with finite bases; we also say that X is FINITE-DIMEN SIONAL or n-DIMENSIONAL. If B is an infinite basis for X , then any basis for X has the same cardinality as B , and this cardinal is defined to be the DIMENSION of X. Some other facts about bases are : •
• •
•
•
Every vector space has
a
basis .
Any linearly independent set can be extended to a basis.
The proofs of these statements involve what is known as "the standard Zorn's lemma argument" (see, for example, Bachman and Narici 1966 , pp. 1 57-160) . The argument that a vector space X has a basis runs as follows. Choose a nonzero vector Xl E X . If the linear span [xd is not X, there must be a vector X2 E X not in [X l] . If [Xl , X2] :/; X, choose X s fi. [X l , X2]. Then consider the question Is [X l , X 2 , xs] = X. The problem is, Does the process ever end? This is where Zorn 's lemma is used. Zorn's lemma guarantees the existence of a maximal linearly independent set of vectors , which is indeed a basis . As will be seen later, the term "basis" has other distinct meanings (orthonormal, Riesz, and Schauder bases) . As an immediate corollary to the Gram-Schmidt theorem 1 .4.3, it follows that 1.4.4 Every finite- dimensional inner produ ct space has a basis of orthonor mal vectors.
The parallelogram law of plane geometry states that the sum of the squares of the diagonals of a parallelogram is equal to the sum of the squares of the sides . For vectors X, y E R2 , this is equation ( 1 . 16) below. Equation ( 1 . 16) survives in inner product spaces. The formulation below follows directly from equation ( 1 . 12) applied to J l x y J l 2 and J l x - yJ l 2 . The terms ± Re (x , y) drop out.
+
1 .4.5 PARALLELO GRAM LAW For all X and y in an inner produ ct space,
( 1 . 16)
An imp ortant use of the parallelogram law is a test to see whether a norm comes from an inner product . Although an inner product-derived norm satisfies the parallelogram law , it is not satisfied by every norm. For example , consider £ 1 (2) = R2 with the taxicab norm (equation ( 1 . 6) of Section 1 .2 ) ; let X = ( 1 , 0) and y = (0, 1 ) . Then Il x yJ l = J l x - yJ l = 2, so
+
I l x + Yl 1 2 + J l x - y J l 2 = 8 to 2 1 1 x J l 2 + 2 J 1 y J l 2 = 4 .
1 . Metric an d N armed Spaces
23
The taxicab norm is therefore not inner-product-induced. Another feature that distinguishes inner-pro duct-derived norms from general norms is the polarization identity of Exercise 3.
Exercises 1 . 4 1 . Find two vectors x and Y from the complex inner product space £2 (2) for which the Pythagorean relation holds but that are not orthogonal. 2. Orthonormal vectors cannot be too close to each other. For any two orthogonal unit vectors x and y show that I I x - yl l = .../2 .
3 . POLARIZATION IDENTITY (a) Show that any inner product satisfies the P O LARIZATION IDEN TITY
{x, y} =
� { l i x + yl l 2 - l i x - yl 1 2 + i I I x + iyl l 2 - i I I x - iy11 2 }
( 1 . 17)
for all x, y E X . All it takes is fortitude to verify it. (b) The polarization identity provides a way to test a norm for inner product status. Suppose that X is a complex normed space whose norm satisfies the parallelogram law 1 .4.5. Show that equation ( 1 . 17) defines an inner product on X such that I I x l l = �. An inner product space is therefore a normed space whose norm satisfies the parallelogram law . (c) Show that neither .ep (n) nor .ep i s an inner product space for P :F 2 . 4. Show that i f x .1 S, then x i s orthogonal t o any linear combination L:?= l a i X i of vectors Xi from S, i.e . , x .1 [S] .
5.
PYTHAGOREAN THEOREM If {Xl , X 2 , show that 1 I L:?= l x i l l 2 = L:?= l l l x i Il 2 .
.
.
•
, xn } is an orthogonal set ,
6 . BESSEL'S EQUALITY AND INEQU ALITY (cf. 3 . 3 . 1 )
(a) Let {Xl , X 2 , , xn } b e an orthonormal set . Use the Pythagorean theorem of Exercise and argue as in equation ( 1 . 15) to show that for any vector x, BESSEL'S EQUALITY holds : .
•
.
5
24
1 . Metric and Normed Spaces
(b) Deduce BESSEL'S INEQ UALITY from this:
n I (x , x i} 1 2 ::; I I x l l 2 . L ; =1
(c) If (xn ) is an orthonormal sequence and x any vector, show that I (x , x i} 1 2 ::; I I x l l 2 for any n. Deduce from this that I (x , x i} 1 � ° for any x.
E� 1
Hints
3 (c) . See how the parallelogram law works on ( 1 , 0) and (0, 1 ) in ip (2) . 6 . Peek ahead to 3 .3 . 1 .
1.5
Linear Isometry
We have already discussed our criterion for saying that two metric spaces are "the same as metric spaces" in Section 1 . 1 , namely, isometry. We pro mulgate analogous criteria for normed and inner product spaces in this section . By this means, we can prove that for any n E N, there is essen tially only one n-dimensional inner product space: (Exercise 5 ) . We show in 3.4.9 that any infinite-dimensional separable Hilbert space is "in ner product isomorphic" to i2 • This leads to an interesting choice: One can view L 2 [0, 1] as a function space or as the sequence space i2 • For any two closed intervals and the spaces of continuous functions C and C are linearly isometric (Exercise 7) , so there is no loss in focusing attention on C [0 , 1] . Thus, the notion of linear isometry also enables us to exclude many apparently different objects as only trivially different . A map A : X � Y between vector spaces X and Y is LINEAR if for any scalars and and any vectors x and
Kn
[a, b)
[c, d] a
[c, d] ,
b
[a, b)
y,
A (ax + by) = aAx + bAy. An injective linear map is called a LINEAR ISOMORPHISM or j ust an ISO MORPHISM; if a linear bijection exists between two vector spaces, then they are called LINEARLY (or ALGEBRAICALLY ) ISOMORPHIC . Example 1 . 5 . 1 LINEAR MAPS
(a) Let m , n E N. Let T be an m x n matrix and let Tx denote mul tiplication of a column vector x with n elements by T. The map � X 1-+ Tx , is linear . Matrix multiplication is essentially the only linear map between finite-dimensional vector spaces.
Rm ,
Rn
1.
Metric and Normed Spaces
25
(b) Differentiation and integration are linear when defined on vector spaces of differentiable and integrable functions, respectively. ( c) Any n-dimensional vector space over K is linearly isomorphic to K n (see the proof of 1 .5 .4) . Hence , any two n-dimensional spaces over K are linearly isomorphic to each other. If {Xl , x 2 , . . . , xn is a Hamel base for X and ei = (0, 0, . . . , 0, 1 , 0, . . . , 0) , 1 :S i :S n, then the map L:�= l ai Xi -+ L:� l ai ei is a surjective isomorphism . 0
}
Definition 1 . 5 . 2 LINEAR ISOMETRY
Let A : X -+ Y be a linear map . (a) NORM IS OMORPHIC Let X and Y be normed spaces. We say that A is N ORM PRESERVING if (using the same symbol for norm in each space)
I I Ax l 1 = Il x l l
for every X E X j w e also say that A is a LINEAR ISOMETRY o r a N O RM ISOMORPHISM. If A is onto, then X and Y are called LINEARLY ISOM ETRIC or N O RM ISOMORPHIC or NORM ISOMORPHS , and we write X e:: Y . (b) INNER PRO D UCT ISOMORPHIC If X and Y are inner product spaces and
( Ax, Ay) = (x, y)
for all x , y E X , then A goes by the names INNER PRODUCT PRESERVIN G , INNER PRO DUCT ISOMORPHISM, and UNITARY O PERATOR. If A is onto, then X and Y are ISOMORPHIC AS INNER PRODUCT SPACES or UNITARILY EQUIVALEN T. 0 Norm preserving and inn er produ ct preserving are equivalent on inner product spaces.
1.5.3 EQUIVALENCES Let A : X -+ Y be a lin ear m ap of the inner produ ct space X into the inner produ ct space Y. Let d be the norm- derived metric: d (x , y) = I l x - Y I I . Then the following are equivalent: For all x and y , (a) A is an isometry: d (x , y) = d (Ax, Ay) . (b) A is norm preserving: I I x l l = I I Ax l i . ( c ) A is inn er produ ct preserving: (x , y) = (Ax , Ay) . Proof. If A is an isometry, then
d (Ax , 0) = I I Ax l 1 = II Ax - 0 1 1
=
d (x, 0) = I l x - 0 1 1 = I l x l l ,
so (a) implies (b) . We argue that (b) => (c) for complex spaces only. If A preserves norms, then by the polarization identity (Exercise 1 .3-3) ,
4 (x, y)
I I x + Yl 1 2 - l l x - y l 1 2 + i I l x + i y l1 2 - i Il x iy l 1 2 II Ax + Ay l l 2 - I I Ax - Ay l 1 2 + i I I Ax + iAyl 1 2 - i I I Ax - iAy l 1 2 4 ( Ax, Ay) , _
26
1 . Metric and Normed Spaces
so A preserves inner products. The reverse implications are straightforward. o
For any n E N, there is essentially only one n-dimensional real or complex inner product space. 1 .5.4 FINITE- DIMENSIONAL INNER PRO DUCT SPACES Let n E N . Any real or complex n-dimensional inner produ ct space is isomorphic as an inner produ ct space to real or complex 12 (n), respectively.
}
P roof. By 1 .4.4, X has an orthonormal basis {X l , X 2 , . . . , x n . Consider the linear map
Kn , X = :L� l a ; x; ( al , a 2 , . . . , a n ) . By the previous result , it suffices to show that A preserves norms: A:
X
--
Is there essentially (up to linear isometry) only one n-dimensional n o rm ed space? No. Indeed, loo (2) is not linearly isometric to 12 (2) , a point we ask the reader to prove in Exercise 3 . But n-dimensional normed spaces can be identified in a weaker sense: Any two n-dimensional normed spaces over the same field are lin early homeomorphic (Definition 2.9.5) . This nontrivial result of Exercise 2 .9-5 follows from the fact that all norms are "equivalent" in a certain sense, discussed in Section 2.9, on an n-dimensional space (Exercise 2 .9-3).
Exercises 1 . 5
1 . INJECTIVE LINEAR MAPS Let X and Y be linear spaces and let A : X ---4 Y be linear .
=
=
(a) Show that AO O. (Write 0 as 0 + 0.) (b) Show that A is injective if and only if Ax 0 implies X = O. (Write 0 as AO.) 2. ISOMETRIES O F THE PLANE R2 Show that any inner product space isomorphism of 12 (2) onto itself may be represented by a matrix multiplication = Ax where A is of the form
y
( COS
(} - sin (} sin (} cos (}
) ( or
cos (} sin (}
sin (} - cos (}
)
.
Deduce from this that the only linear isometries of 1 2 (2) are rotations through an angle (}, reflection about a line making an angle (} /2 with the x-axis, or compositions of these . If we drop "linear ," there are other distance-preserving maps-translations , for example .
1 . Metric and Normed Spaces
3.
(2) = (R2 , 1 1 · 1 1 2 ) is not linearly isometric to £00 (2) 2 (R , 1 1 ' 11 (0 ) ' 4. Show that £2 (N) is inner product isomorphic to £2 (Z) . Show that real £2
27 =
5 . FINITE-DIMENSIONAL INNER PRODUCT SPACES Show that any n dimensional inner product space X over K is inner product isomor phic to K n . 6 . LINEAR ISOMETRIES O F £2 ( n ) Let
n
E N.
(a) Show that a linear isometry of the real space £2 ( n ) onto £2 ( n ) maps any basis of orthonormal vectors into another basis of or thonormal vectors . (b) Let A : £2 n ) � £2 n ) be a surjective linear map . Show that A is an isometry if and only if the columns of the matrix associated with A are an orthonormal set in K n .
(
(
C [a, b) Let C [a, b) be as in Example 1.2.3. (a) Show that C[O, 1] is linearly isometric to C[l, 2]. (b) Show that the map x (t) x( l + t 3 ) is a linear isometry of C[1, 2] onto C[2, 9] . (c) If h [c, d) � [a, b] is continuous, must the map x x h map C [a, b] linearly isometrically onto C [c , d)?
7. LINE AR ISOMETRIES O F
1--+
:
�
0
8 . Let s denote the linear space (with operations as in £2 ) of all summable sequences (an ) normed by l I (an) 1 I = s UP k 1 2:7= 1 ai .
l
(a) Show that s is linearly isometric to the normed space ( c, 11 . 1 1 (0 ) of ai) E all convergent sequences under the map (an) E s 1--+
c.
x of s into the space ( co , 1 1 - 1 1(0 ) = O} , 11 . 1 1(0 ) of all null sequences is a contin
(b) Show that the identity map x
( {(an) E /{OO an :
�
( 2:?= 1
1--+
uous linear map but not an isometry.
Hints
2.
First note that reflection of a point with polar coordinates ( r, 8) about a line through the origin making an angle with the x-axis takes the point into a point with polar coordinates ( r, 8) . I n Cartesian coordinates, therefore, this amounts to the following multiplication:
cp
( COS 2cp2cp sin
2cp
2cp ) ( x ) ' 2cp y
sin - cos
-
1 . Metric and Normed Spaces
28
(�), (� )
and the vector
(�)
)
a2 b2
Consider the effect of the isometry
on the unit vectors
, which is of length .j2, and solve the
resulting equations for a I , a 2 , bl , and b 2 .
3. Use the polarization identity of Exercise 1 .3-3 .
Holder and Minkowski Inequalities ; Lp and lp Spaces
1 .6
We prove the Holder inequalities in 1 .6 .2. We use them to prove the Minkowski inequalities in 1 .6 .3. The Minkowski inequalities tell us that the and spaces are normed spaces. First, we need the following numerical fact.
fp
1 .6 . 1 Let
aP
b9
-- + - . p q P roof. Let
a,
b � O. If p > 1 and q are such that
1+1
-
-
p
q
=
Lp
1 , then ab
b. (b)
The case
b > a.
represents the area between the curve and the x-axis and area between the curve and the y - axis
.
In each case
A2
Al
represents the
1 . Metric and Normed Spaces
29
aP - 1 > b and aP - 1 S; b, we see that the desired A simple way to see the validity of 1 .6 . 1 for p = 2 is as follows: (a - b) 2 = a 2 - 2ab + b 2 � 0 , so a 2 + b2 ' ab S; "2 "2
and considering the cases result holds . 0
The Holder inequality tells us when a certain sum of products is less than or equal to something like the product of the sums. 1 = 1, + p q
1 .. 1 .6.2 THE HOLDER INEQU ALITIES For p > 1 and q such that
-
aj , bj , j = 1 , 2, . . . , n ,
the following assertions hold: (a) For complex numbers
-
(an) and (bn) of complex numbers such that E�l l ai /P < q it follows that E� l l a i l l bil < and ( E� l I bil )
(b) For sequen ces 00 and
j. (c) CONVERG ENCE ON PRODUCT SPACES Let i = 1 , 2 , . . . , k, be metric spaces and let the Cartesian product X k carry x X2 X one of the metrics =
U)
(Xi , di ) , Xl
d
.
•
X
.
[Lik= l di (Xi , Yi )P] l ip l :S p < oo
maxi di (X i , Yi ) (2 .2) of Example 1 . 1 .4. By essentially the same argument as in (b) , with d (Xn, x) in place of Il xn - x1 I 2 ' it follows that if a sequence X n ( x n ( 1 ) , Xn (2) , . . . , Xn (k)) E Xl X2 . . . Xk , n E N , converges t o X E X l X2 . . . Xk , then it converges componentwise, i .e . , Xn X i n Xl X2 . . . Xk limn xn U) = X (j) for j = 1 , 2 , . . . , k . We establish the converse by replacing I XnU) - xU) 1 by dj (XnU), xU)) in equation ( 2 1 ) Consequently, for any metric space Y , a map Xl X2 . . . Xk I: Y Y (It (y) , . . . , !k (y)) i s continuous a t a point Y if and only if each Ii i s continuous a t y. Thus, =
x
-)0
.
X
X
X
X
X
X
X
x
===>
.
---t
X
1--+
X
x
we can paste new continuous functions together from old ones . The notion of componentwise convergence is generalized to pointwise convergence in Example 2.2.7(b) . (d) PROJECTIONS O N 1 :S P :S 00 , ARE CONTINUOUS The process of extracting the jth component of the n-tuple or sequence ( x n , as we did in (b) and (c) , generalizes the familiar notion of projecting a vector in R 2 onto the x- or y-axis. For X = 1 :S P :S 00 , and a positive integer j , we define the PROJECTIO N O PERATOR
fp ,
)
fp ,
Pj :
fp
( xn
)
---t
1--+
K,
Xj .
fp
By (b) , each of these projections is continuous. Since ( n ) is a metric subspace of the Pj are continuous on ( n ) as well, for any n E NU {oo } .
fp ,
fp
2 . Analysis
43
(e) CONTINUITY OF A LINEAR MAP Generally, continuity at one point has nothing to do with continuity at other points . To illustrate the intimate terms on which points of continuity and discontinuity may be, consider the map : [0 , 1] ...... [0 , 1] defined by
x
, x( t ) = { O1/qforfort irrational t = plq (lowest terms) , p, q E Z.
.
This function is continuous at each irrational number and discontinuous at each rational. Linear maps : X ...... Y between normed spaces X and Y are a different story. If A is continuous at (or any other point) then it is continuous everywhere-in fact, is uniformly continuous (see Section 2.7) : If < 6 ==::} < f (f, 6 > then < 6 ==::} = < f. 0
A
II x ll
II x - y ll
0
II Ax ll
0) ,
II A (x - y) 1I II Ax - Ay ll
For the sake of future use in Fourier analysis, let us compare some kinds [a , b] . of convergence in
L2
Example 2 . 2 . 7 CONVERGENCE IN
L 2 [a , b]
(a) MEAN CONVERGENCE Convergence x n ...... x in 11 ·/ / in the space L 2 [a , b] of Example 1 . 1 .8 is called CON VERG ENCE IN THE MEA2 N ; we denote this special limit as l . i.m.n xn = x . Convergence in the mean is obviously equivalent to
Xn ...... x (xn)
As we show next , it is possible that in the mean even though f+ (t) at every point t. (b) MEAN =/==} POINTWISE A sequence of real- or complex-valued functions with a common domain T converges P O INTWISE to the function as sequences of numbers for each t E T. The sequence if from 1] below converges to 0 in the mean but not pointwise:
Xn (t) x
x xn(t) ...... x(t) L 2 [0,
xn(t) = { 0,1 - nt , t0 >� lit �n. l in , . Since x n (O) = 1 for every n, it is clear that X n f+ 0 pointwise. On the other hand, 2 � t Jo / xn(t) - 0; dt = 3 n ...... O . For t E [0 , 1] ' the analogue of the projection map Pj on ip ( n ) of Example 2.2.6( d) is the evaluation map i : L 2 [0, 1 ] ...... K, x ....... x(t). Unlike the Pj ,
44
2. Analysis
which are all continuous, i is discontinuous when = 0, since = (0) == 1 . 0 respect to but
t
6 (xn) Xn
11 · 11 2
Xn
-->
0 with
Example 2.2.8 UNIFORM CON VERGENCE
Let X be any vector space of numerical-valued functions on a common domain T. Assume also that each function is bounded , alb eit not necessarily by the same bound. Norm X by the UNIFORM N ORM
II x li oo = sup { I x ( t ) 1 : t E T} . This norm induces the max metric of Exercise 1 . 1- 1 1 . Convergence with is called UNIFORM convergence. The space [a , b] of K respect to valued continuous functions of Example 1 .2.3 is a special case. Uniform convergence is a relatively strong form of convergence, as we show in (a) and (b) below. Some reasons for the imp ortance of uniform convergence in [a, b) are that many properties of the individual terms of a sequence persist into the limit. For example , suppose that for each n E N , the function is defined on the closed interval [a, b] . --> uniformly in the sense that --> 0, then the following hold.
11 · 11 00
C
C
Xn If Xn x •
Il xn - x ll oo
x.
Xn
CONTINUOUS FUN CTIONS If each is continuous, then so is This fact provides a good way to construct continuous functions with special properties such as continuous, nowhere differentiable functions or space-filling curves. The idea is to get the "exotic" function as the uniform limit of functions that almost have the exotic property. To generate a continuous, nowhere differentiable function, for example , create a uniformly convergent sequence of continuous functions are not differentiable at progressively more points. such that the (cf. Section 4.3 for an example.)
(x n )
Xn
•
Xn
TERMWISE INTEGRATION If each is a function in Lda , b] , then so is Moreover, the sequence can be integrated term by term in the sense that --> [A much more general result about termwise integration is the LEBESGUE D O MINATED CON VERG ENCE THEO REM (Stromberg 198 1 , p . 268, Rudin 1 976 , p . 305, or Natanson 196 1 , p. 149 ) : For a sequence of real-valued measurable func tions from L 1 (R) and a measurable function (R) such that (t) a.e . , if --> pointwise a.e ., then -->
x.
J: xn(t) dt J: x(t) dt.
(Xn)
I Xn (t) 1 :S y J�oo x(t) dt.
•
Xn
x
y E Ll J�oo Xn(t) dt
TERMWISE DIFF ERENTIATIO N Pointwise limits of differentiable func tions need not be continuous, let alone differentiable . However, if each converges pointwise to on has a continuous derivative (a, b) , and the derived sequence converges uniformly to then is differentiable and = thus, -->
Xn x
x�, Xn (x�) x' y; x� x'.
y,
x
2. Analysis
(a) UNIFORM CON VERGENCE
==::}
45
POINTWISE CONVERG ENCE For any
P E [a, b], I Xn(P) - x(p) 1 :S sup { I xn(t) - x(t) 1 : t E [a, b] } = II xn - x lloo ,
so uniform convergence implies poin twise convergence. To see that point wise convergence does not imply uniform convergence, consider the se n quence from C[O, 1] ; converges pointwise to the = function = 0 for # 1, = 1 . But cannot converge uniformly to because is discontinuous. As another example of such behavior , consider the sequence E C (R) , = 0 for < n and = 1 for 2: n . Clearly, this sequence converges t o 0 pointwise (just wait .until n exceeds any particular t) but not uniformly, because there are always points at which = 1 no matter how large n is. (b) UNIFORM CONVERGENCE ==::} MEAN CON VERGENCE For E [a, b] , -> 0 ==::}
un (t) t n , E N , u t u(l) u Yn(t) Yn
(un) (un) t
u
Yn (t)
t
t Yn (t) Xn , x L2 II xn x lloo II x n - x II; = lb I Xn (t) - x(t) 1 2 dt :S ( b - a) II x n - x II !:, ---> O. The idea of a clust er point of a sequence (xn) in a metric space is a 0
weaker notion than that of limit of a sequence. Definition 2.2.9 CLU STER POINT
x
(xn)
We say that is a CLUSTER P O INT of in a metric space if every neighborhood of contains for infinitely many values of n; we often phrase this latter condition as is FREQUENTLY in any neighborhood of
x.
x
Xn Xn
0
A sequence has at most one limit , but it may have many cluster poin ts . Example 2.2.10 CLU STER POINTS
(a) A limit of a sequence is a cluster point. (b) The sequence 0 , 1 , 0 , 1 , . . . has two cluster points : 0 and 1 . (c) The rational numbers Q may b e written out as a sequence Any neighborhood of any real number contains infinitely many rational 0 numbers; therefore , every real number is a cluster point of
(xn).
x
(x n).
There is a sequential way to characterize cluster points .
x
2.2. 1 1 CLU STER POINTS AND SUBSEQUEN CES In any metric space is a cluster point if and only su ch that has a subsequ ence
Xnk x. ->
of(xn)
X nk x,
if (xn) x
(x nk )
(xn). B(x, 11k), k E N ,
Proof. If Conversely, -> then obviously is a cluster point of let be a cluster point of Each ball contains for infinitely many values of n. Choose the smallest value of n for which 1/2) contains a point such E 1) and call it nl . Likewise such that nk > nk - l for each 0 that > n l ; choose
x
Xn B (x, n2
(x n ).
Xnk E B(x, 11k)
B(x,
Xn
Xn2 k.
2. Analysis
46
Definition 2.2.12 SERIES
A series E n e X n of vectors from a normed space X is said to CON VERGE to x X if the sequence Sn = � l Xi converges to X . We write ne X n = x. For a two-sided sequence ( X n ) ne Z or bisequence we say Xj = x , m , n Note that we use ne Z X n = x if limm n oo 0 not
E N E Ej= - m '
E N
Ej= - n .
E
,
-+
E N.
Ej= - m
Exercises 2 . 2 1 . CONVERGENCE IMPLIES NORM CON VERGENCE For any sequence ( X n ) in a normed space, show that X n -+ x � I I xn l l -+ I l x l l (x E X ) .
2. Prove the equivalences of 2.2.2.
3 . JOINT CONTINUITY Properties ( a ) and ( b ) below are known as J O INT CONTINUITY of a metric and inner product , respectively.
( a) (b )
In a metric space ( X, d) show that if X n
d(xn , Yn ) -+ d(x , y).
-+
x and Yn -+
In an inner product space X, show that if X n then ( X , Yn) -+ (x ,
n
y).
-+
y, then
x and Yn -+ y,
4. (-0 CONTINU ITY Consider metric spaces (S, d) and (T, d') and x : S -+ T. Show that for any convergent sequence Sn -+ S of points from S, x(sn ) -+ x(s) if and only if for all ( > 0, there exists 0 > ° such that d' (x(s) , x( t )) < ( for all t such that d(s , t) < o.
5 . Prove that the function of Example 2.2.6 ( e ) is continuous on the irrationals.
, k,
6 . PROJECTIONS ARE CONTINUOUS Let (Xi , di) , i = 1 , 2, . be metric spaces and endow X l x X2 X X Xk with the metric d oo of equation ( 2.2 ) . Analogously to what was done for ip ( n ) in Example 2.2.6 ( d ) , we define the projection map Pj ( 1 � j � on X l x X2 X X X k as the map that sends ( X l , X 2 , . . . , X k) into Show that each projection Pj , 1 � j � is continuous. .
.
.
k)
•
.
.
k,
.
•
Xj .
7. MAPS O N PRO DUCTS Let X, Y, and Z be metric spaces. Let X carry the metric doo of equation ( 2.2 ) .
( a)
x
Y
If f : X x Y -+ Z is continuous at (xo , Yo ) , show that the map ( defined on X ) x ...- f (x, is continuous at X o . ( b ) Show that the converse is false .
yo)
2. Analysis
47
( e n)
be an orthonormal sequence in 8. UNIQUE REPRESENTATION Let the inner product space X and let be a sequence of scalars. Show that if = 0 , then each = O. Conclude that for any sequence ( of scalars, = implies that = for all i.
(an) ai L i eN ai e i bn) Li e N ai ei Li eN bi ei
a i bi
9 . CONTINUITY O F LINEAR MAPS
(a) Let X and Y be normed spaces. Use the result of Example 2.2.6( e) to show that a linear map A : X ---+ Y is continu ous if and only if A is bounded on the unit ball U of X , i.e . , E U } is bounded. (b) Let D denote the linear subspace of continuously differen tiable functions of (C Use the result of (a) to show that the differentiation operator x � x , is discontinuous.
{ II A ull : u [0, 1]
10.
[0, 1] , 11 ' 11 (0 ) '
11 . 11 (0) denote the sup-normed space of null sequences of scalars. FOR q > p, fp fq. For alI I :::; p < 00 , show that fp co ; show also that for q > p, fp fq . Show that the identity maps x x of (fp , II · l i p ) into (co , 11 . 11 (0) and of (fp , II· li p ) into (fq , 1I ·l Iq) are continuous linear maps that
Let (co , (a) (b)
C
C
C
�
are not isometries.
11. 12.
NON ZERO O N A NEIGHBORHOOD Let X and Y be normed spaces and let f : X ---+ Y be any map . Show that if f is continuous and f(x) 1= then f is nonzero on an open ball about x .
0
UNIFORM CONVEXITY A normed space X is called UNIFO RMLY CON V EX if for any sequences x and of unit vectors If you think of � as the midp oint of such that II � " ---+ the line segment connecting the points and on the surface of the unit ball U , then the only way for the midpoints to approach the surface of U is for the points and to approach each other.
II xn - Ynll ---+ 0
( n) (Yn) X n Yn
1.
Xn
Yn
(a) Show that any inner product space is uniformly convex. (b) Show that none of the spaces ( n ) , :::; p :::; 00 , n uniformly convex.
fp
13.
1
E N , are
ANTIP ODAL POI NTS This exercise demonstrates some of the power of continuity.
(a) Points on a sphere that lie at opposite ends of a diameter are called ANTIPODAL. Assuming that temperature varies continu ously on the surface of the earth , show that there are antipodal
48
2. Analysis
points on any great circle of the earth at which the temperatures are identical. Temp erature was chosen arbitrarily here; we could have chosen any other continuous variable such as altitude above sea level or magnitude of wind velocity at a given time . This is a special case of the Borsuk- Ulam antipodal point theorem. ( b ) Consider a two-dimensional pancake ( the interior of a plane con vex set ) in the plane and a knife placed at an arbitrary angle . Show that the knife can be translated ( its angle with the x-axis does not change) to a place where it bisects the pancake into regions of equal area.
Hints 3 ( b ) . Subtract and add (xn , y) in I {xn , Yn ) - {x, y) l , and use the triangle and Cauchy-Schwarz inequalities. 4. Assuming the (-6 condition , it is easy to deduce the sequential for mulation. Prove the converse by proving its contrapositive, that if the (-6 condition does not hold, then neither does the sequential condition. The failure of the (-6 condition means that there is some ( > ° such that for all 6 > 0, there exists S6 such that d (S6 , s) < 6 but d' (X(S6 ) , x( s )) � L Therefore , for each n there is Sn for which d (sn , s) < lin but d' (x(sn ) , X(S)) � ( . 5 . Show that the denominators of rational numbers close to a particular irrational become arbitrarily large .
7 ( b ) . Let x (s , t) = stl (s 2 + t 2 ) if (s, t ) =P (0, 0) , and x (O, O) = 0; consider x (s, s) for s =P 0. 9 (a) . Look at the proof of 2.3.3. ( b ) The functions t n , n E the unit ball of D [0, 1 ] .
N , belong to
1 3 . Recall the intermediate value theorem for continuous functions:
For any [a, b) C R, any continuous map x : [a, b) ---+ R and any c such that x (a) ::; c ::; x(b) there is some d, a ::; d ::; b, such that x(d) = c. ( a) Let T = T «(}) denote temperature where () is a central angle of the earth . Note that T (O) T (211') . Extend T periodically, and consider T «(}) - T «(} - 11') . (b ) The area of the pancake lying on one side of the knife varies continuously between ° and the whole area as the knife is translated.
=
2. Analysis
2.3
49
B ounded Sets
The generalization of the notion of a bounded subset of R2 is very imp or tant in normed spaces because of its connection to continuity ( 2.3.3 ) for linear maps. A subset S of a metric space X is called BO UNDED if it is contained in a sufficiently large ball B(x, r) about some x X of radius r > 0. In a normed space, a set 8 is bounded if and only if it is contained in a ball B(O, r) about the origin: If y 8 C B(x, r) , then by the triangle inequality, l I y l l ::; l I y - x ll + l l x l l ::; r + l l x l l ; therefore 8 C B(x , r) ==> S C B(O, r + l l x l l ) ·
E
E
Example 2.3.1 BO UNDED SETS
( a) Finite subsets S Take
= {Xl > . . . , xn } of any metric space (X, d) are bounded.
Then S C B(X 1 ' r) . ( b ) Any subset of a bounded set is bounded. ( c ) The finite union of the bounded subsets 81 , . . . , 8n of a normed space is bounded. For each i 1 , 2, . . . , n, assume that 8i C B(O, rd . Let r max: 1 $ j $n rj . Then U7= 1 8i C U7= 1 B(O, ri ) C B(O , r) . ( d ) The terms of a convergent sequence Xn -> X in any metric space constitute a bounded set by the following argumen t. For some integer N , { X N+ 1 , . . . } C B(x, 1 ) , so the tail {X N + 1 , " ' } of the sequence is bounded ; the first part {Xl , . . . , X N } is bounded because it is finite. The terms of the sequence comprise the union of these two sets. Boundedness does not imply convergence, however: Consider the sequence 0, 1 , 0 , 1 , . . . ( e ) Any trivial metric space ( Example 1 . 1 .6) is bounded. (f) Any metric space (X, d) can be re-metrized in such a way as to be bounded using the metric d' dl ( 1 + d) of Exercise 2.9-7 ( a ) ; moreover, (X, d) and (X, d') have the same convergent sequences . 0
=
=
=
2.3.2 BOUNDED SE Q U ENCE CHARACTERIZATI ON If each denumera ble subset of a subset B of a normed spa ce X is bounded, then B is boun ded.
EN
E
Proof. If B is unbounded, then for every n there exists Xn B such that I Ixn l l � n. Thus, any unbounded set B has an unbounded denumerable subset . 0
Continuous functions need not map bounded sets into bounded sets , since the map X : ( 0 , 1 ) -> R, t � l i t , is continuous. Generally, maps that take bounded sets into bounded sets are called BOUN DED MAPS . As noted above , continuous maps need not be bounded , but continuous lin ear maps are . In fact , bounded is interchangeable with con tinuous for linear maps .
50
2. Analysis
A
2.3.3 CONTINUITY AND BOUNDEDNESS A lin ear map : X -4 Y of the normed space X into the normed space Y (a) is bounded if a n d only if it is bounded on the closed unit ball U of X; (b) is continu ous if and only if it is a bounded m ap. Proof. (a) B is bounded if and only if B is contained in rU for some For any > 0, (rU) (U) , so
r
A
= rA
sup I I A (rU) 1 I �
rk
¢}
sup II A (U) I I
� k (k > 0) .
r > O. (2 .3)
(b) If A is continuous, there exists 6 > 0 such that I l x l l � 6 => II Ax l l � 1 . For any x E U, 1 1 6x l l � 6. Therefore II A (6x) 1 1 � 1 and I I Ax l l � 1/6. Conversely, if sup I I A ( u ) 1 I k for some 0, and { 0 then sup II A ( f U) II � f It follows that A is continuous at 0 and therefore continuous by Example 2.2.6(e) . 0
k>
�
>
Boundedness is one measure of smallness of a set. In the Euclidean spaces it is the only one that is needed. In other metric spaces the stronger notion of total boundedn ess or the notion of compactness is needed, a topic we return to in Section 2.8 .
.e2 ( n ) , n
E N,
Exercises 2 . 3 1 . I n the space C [0, 1] of continuous functions on [0, 1] with sup norm show that the set I I x l l oo = sup { I x ( t ) 1 : t
E [0, In,
{ x E C[O, 1] 11 I x(t ) 1 :
is unbounded.
dt � 1
}
2 . Identify the bounded linear subspaces of a normed space. 3 . SUMS AND PROD UCTS Let (X, 1 1 · 1 1 ) be a normed space.
=
(a) SUMS If A and B are bounded subsets of X, show that A + B {x + Y : x E A, y B} is bounded. (b) CARTESIAN PRODUCTS Let ( Y, I I · I I ' ) be a normed space. Show that l I (x, Y ) l I oo max: ( lI x ll , lI y lI ') defines a norm on the Carte sian product X x Y (cf. Section 2.9) . If A is a bounded subset of X and B a bounded subset of Y, show that x B is a bounded subset of X x Y . (c) PROJECTIONS With X x Y as i n (b) , show that a subset B of X x Y is bounded if and only if each of its projections PI (B) and P2 (B) ( "projection" as in Exercise 2.2-7) is bounded.
E
=
A
2 . Analysis 4. DIAMETER The D IAMETER d space (X, d) is
51
(A) of a nonempty subset A of a metric
= sup {d (x , y) : X , y E A } if the set {d (x, y) : x, y E A} is bounded, d (A) = 00 otherwise. Show that a set is bounded if and only if it has finite diameter and that d(A) = 0 if and only if A is a singleton. d (A)
5 . SOME CONTINU OUS LINEAR MAPS Here are some applications of 2.3.3.
(a) For any fixed vector y from an inner product space X , the map x 1-----+ (x , y) is continuous. (Use the Cauchy-Schwarz inequality.) Thus, the map f : £2 � K, (an ) 1-----+ L ne N an/n i s continuous. (b) Consider the subspace cp of "finite sequences" (all but a finite number of entries are 0) of the inner product space £2 . Show that the restriction of the map f of part (a) to cp is continuous.
6 . BOUNDED NESS AND NULL SEQUENCES
(a) Show that a subset B of a normed space X is bounded if and only if for any sequence (Xn ) from B and any sequence (an ) of scalars that tends to 0, anXn � o . (b) Let X and Y b e normed spaces. Show that a linear map : X � Y is continuous if and only if it maps null sequences Xn � 0 into bounded sequences.
A
7. CONTINUOUS "HOMO GENEO US" IMAGE OF A BOUNDED SET We already know that the continuous linear image of a bounded subset of a normed space is bounded (2.3.3). This result is a little stronger. For normed spaces X and Y, let f : X � Y be continuous and such that for some fixed r > 0, f (ax) a r f (x) for all a > 0 and x E X . Show that f maps bounded sets into bounded sets.
=
Hints 1 . Functions with small integrals can have large peaks.
6 . (a) If B is bounded and {xn } C B , then there exists an r > 0 such that I I xn II ::; r for all n. Thus, I l anxn I I ::; I an I r � O. Conversely, if B is unbounded, it must contain a sequence (Xn) for which I I xn l l � n for each n. (b) If A is continuous, then it maps null sequences into null sequences. Conversely, if A is discontinuous, then there exists
52
2 . Analysis
a sequence ( Yn ) and a number r > 0 such that II Yn l 1 � 1 / n 2 , but I I A Yn l l � dor every n. Thus, Xn = nYn -+ 0, but I I Axn l 1 = n I I A Yn l l � n r , so (Axn ) is unbounded. 7. Show that f (0) = O . Note the result of Exercise 6(a) and consider an -+ 0 and f l an I 1 / r xn .
(
2.4
)
Closure and Closed Sets
Readers of this book surely know the arithmetic of the real numbers addition, subtraction, etc. Right? How exa ctly do you compute V2 + V3? Or 1re? The key word is exa ctly-no approximations. The fact is that most people, even very well educated people, know the same arithmetic as a well educated person of the seventeenth century. There is no need, for practical purposes, to know how to compute things like V2 + .J3 exactly. This is possible because numbers like V2 can be approximated by rational numbers (numbers that can be written as the ratio of integers) to any desired degree of accuracy. How many kinds of functions can you name?-p olynomials, trigono metric functions, logarithms , exponentials, . . . What else? As these are all continuous-infinitely differentiable , in fact-surely there are many others. As with numbers, we can approximate the more complicated functions by the relatively simple ones just mentioned. This section introduces a frame work for understanding the phenomena of approximation in a more general context. Definition 2.4.1 ADHERENCE POINTS , CLO SURE , AND DENSITY
Let S be a subset of the metric space (X, d) . •
A point x is called an ADHERENCE POINT of S if every open ball B ( x , r) , r > 0, meets (has nonempty intersection with) S.
•
The collection cl S, or 5, of all adherence points of S is the CLOSURE of S. If S cl S then S is called CLOSED .
•
=
If cl S
= X then S is DENSE in X .
0
As there are no points adherent to 0 , the null set is closed-how could any set have nonempty intersection with 0? It is also clear that the whole space X is closed. For any subset S of X , cl S is closed, since an open ball B of positive radius about any point x of cl( cl S) must contain a point of cl S; therefore , B must also contain a point of S, and it follows that x cl S.
E In any metric space (X , d) , x E cl S if
2 .4.2 CLO SURE AND SEQU ENCES and only if there is a sequence (xn ) of points of S that converges to x .
2. Analysis
x n x,
53
x contains almost all of the Xn so x E cl S, then B(x, l In) contains a
Proof. If --+ then every ball about every such ball meets S. Conversely, if point of S for every n . 0
Xn
Example 2 .4.3 CLO SURES
X.
C
(a) Let S be a subset of a metric space Then S cl S, since any ball about contains x. An imp ortant consequence of this trivial observation is that to show a set to be closed, it suffices to show that cl S S. (b) The closure of an open interval R is the closed interval b] . and let r be positive. The closure (c) Let X be a normed space, let of an open ball is the closed unit ball any closed ball is a closed set. To see that = cl ) and let let w cl be a sequence from that converges to w. Since < for every n , the continuity of the norm implies that
x
C (a, b).C [a, x E X, B(x, r) C(x, r) ; C(x, r) C (x, r) B(x, r ), E B(x, r B(x, r) II xn - x II r
(xn )
(2 .4) II xn - x II = I l li� Xn - x ii = I I w - x II ::; r , so w E C(x, r); therefore cl B(x, r) C C(x, r ). Conversely, if l l w - x l l = r, then Yn = X + ( 1 - ; ) (w - x) E B(x, r ) for every n , and Yn w , i.e . , W E cl B(x, r). (d) If S is a nonempty subset of an inner product space X , the set S 1. = {x E S x S} is called the O RTHOGO NAL COMPLEMENT or O R THO COMPLEMENT of S. S 1. is pronounced "S perp." It is always closed : If (xn ) is a sequence from S1. and Xn --+ x, then for any E S and n E N , 0 = (xn, ) --+ (x, s ) so ( x, s) = O. 0
li�
--+
:
1.
s
s
The following result is easy to prove and very useful.
X
--+ Y is 2.4.4 If X and Y are norm ed spaces and A : lin ear map , then the null space of A, A - I (0) , is closed.
a
con tinu ous
x E cl A- 1 (0) , then there exists a sequence (xn) from A- I (0) such Xn x. Since A is continuous, Ax = lim n Axn = limn O = O. 0
Proof. If --+ that
Example 2.4.5 DENSE SETS
(a) THE RATIO NALS The closure of the set of rational numbers Q in R is R by the following argument. As there is a rational number between any two reals, every open interval - l I n , + l In) about a real number contains a rational number Clearly, --+ A similar argument shows that Q x Q is dense in R2 . (b) POLYNO MIALS IN By the Stone- Weierstrass theorem [see Section 4 . 1 7] the closure of the set of polynomials on 0 1 ] is the space 1] of K-valued continuous functions on 1] when it carries the uniform norm 1 1 1 1 00 of Example
(x xn. C [a, b]
C[O,
1.2.3.
x Xn x. [0,
x
[,
2. Analysis
54
(c) DENSE SUBSPACES O F Lp [- 1I" , 1I"J , P � 1 [Natanson 1 96 1 , pp. 1 72, 200] The linear spans of the following sets are dense in Lp [-1I", 11"] . •
• •
{ eint
E Z} Ani kelement of the linear span of { e int : n E Z} is of the form 2:::: � = - n a k e t and is called a TRIG ONOMETRIC P O LYNOMIAL. SINES AND COSINES sin nt , cos (n - 1 ) t , n E N . :n
CONTINUOUS FUNCTIONS C[-1I", 1I"] .
Definition 2.4.6 STEP FUN CTIO NS O N R Consider real numbers
Co < C l < . . . < Cn ·
A fun ction x : R --+ R that is constant on each of the int ervals ( Ci ' Ci + d , i = 0, - 1 , is defined arbitrarily o n the Ci , a n d vanishes for t < Co and t > Cn is called a STEP ( or SIMPLE ) FUNCTION ON R. (d) STEP FUN CTIONS DENSE IN Lp SPACES , 1 :S p < 00 The collection of step functions defined on R is dense in Lp (R) , 1 :S p < 00 [Stromberg . . . , n
1981 , p. 342] . The fact that the step functions on [a , b] are dense in Lp [a , b] follows as an immediate corollary. More generally, suppose that T is a space with a positive measure p, defined on a u-algebra of subsets of T. Lp (T, p,) , 1 :S p denotes the linear space of K-valued pth power summable functions x on T, i.e., p,-measurable functions x : T --+ K such I normed by I l x l l p = [ IT I x l P dp, ] / P . (There is a brief that IT I x l P dp, discussion of Lp (T, p,) at the end of Section 1 .6. As noted in Section 1 .6 , if T = R or [a , b] and p, is Lebesgue measure, we write just Lp [a , b] or Lp (R) , respectively.) Let A I " ' " An be pairwise disjoint p,-measurable subsets of T such that U7= 1 A; = T. In this context , a STEP (or SIMPLE ) FUN CTION O N T is a linear combination x = 2:::: 7= 1 a lA " a K , = 1 , 2 , . . . , n , of . For 1 characteristic functions l A , :S p the step functions x on T such that p, (coz x) are dense in Lp (T, p,) [Rudin 1 974, p. 70] . (e) CONTINUOUS FUNCTIONS WITH COMPACT SUPPORT Cc (R) A func tion x : R --+ R that vanishes outside some closed interval is said to be of COMPACT SUPPORT . The linear space of continuous functions on R with compact support is denoted by Cc (R) . For alI I :S p Cc (R) C Lp (R) , and Cc (R) is dense in Lp (R) [Stromberg 198 1 , p . 342.] . 0
< 00, < 00,
< 00
iE
i < 00,
i
< 00,
Density results such as the density of Cc (R) in Lp (R) are very useful. As one illustration , consider the proof of 2.8.9 on "continuity in the mean." As is evident from Example 2.4.5, a linear subspace of a normed space need not be closed-it could be dense. The closure of a linear subspace M is again a linear subspace, however: If (xn ) and ( Yn ) are sequences from M that converge , respectively, to X , Y cl M , then for any scalar + Yn --+ ax + Y by 2.2.3; thus, cl M is closed with respect to the formation of linear combinations. A type of subspace that is always closed the orthogonal complement-is discussed in Section 3.2.
a, aXn
E
2. Analysis
55
Example 2.4.5 is about approximating exotic functions by simpler ones . If a metric space X has a countable (i.e . , denumerable or finite) dense sub set , we say that X is SEPARABLE. Thus R is separable because Q C Rj so are C[-1I", 1I"] and L 2 [0, 211"] because they contain, respectively, the de numerable dense subspaces of polynomials and trigonometric polynomials with rational coefficients. Example 2.4.7 SEPARABLE SPACES
( a) The space l2 of square-summable sequences ( 1 . 1 .7) is separable. Why? Consider the linear subspace cp of all "finite" sequences, sequences that are o except for a finite number of entries. Let x E l2 . For each n E N , let Xn be the sequence whose first n entries are the same as x , 0 thereafter. Clearly, X n -+ x, and cp is seen to be dense in l2 (real or complex) . If we restrict the entries in cp to be rational or Gaussian rational (i.e., of the form a + bi where a and b are rational) , it is easy to see that this countable subset is dense in l2 as well. (b) NON SEPARABLE Separability is a measure of smallness of a space. If a space is big enough to have an uncountable number of pairwise disjoint open balls, then it is not separable , because each such ball would have to contain at least one distinct point of any dense subset. Consider the space loo of all bounded sequences with sup norm 1 1 · 1 1 00 (cf. Exercise 1 . 1l 1 (b) ) . The subset X of sequences that have only O 's and l 's as entries is an uncountable set , since it is in 1-1 correspondence with binary decimals , and binary decimals are in 1- 1 correspondence with the real numbers between o and 1. Clearly, distinct elements of are of distance 1 apart. The open spheres B(x, 1 /2) about the points x of are therefore mutually disj oint, and it follows that X is not separable . What does this say about the larger space loo ? Though we do not prove it, any subspace of a separable metric space must be separable . Therefore , loo is not separable . 0
X
X
Exercises 2 . 4
X
1 . Let S b e a subset of a metric space and let x E S. I s it true that the closure of the set difference S - { x } is S?
2. Let X and Y be metric spaces and let I, 9 : X -+ Y be continuous functions. Show that if I = 9 on D then I = 9 on cl D .
e X,
3. DENSE SUBSETS O F DENSE SUBSETS Show that if A is dense in B, then A is dense in cl B. In particular if A is dense in B and B is dense in the metric space then A is dense in Application: The subspace Cc (R) of continuous functions on R with compact support is dense in Lp (R) , 1 :::; p 00 (Example 2.4.5) . Therefore , to show
X
0 , is an open set . For y B(x , r) , r ' = r - d(x , y) is positive. Therefore , for any z B (y, r ' ) ,
E
d(z , x) ::; d(z , y) + d (y, x)
< r - d(x , y) + d(y, x) = r,
E
and the result follows from 2.5 . 1 . The open balls B (x, r) , r > 0 , are called the BASIC O PEN SETS of X . (b) X and 0 are open. (c) UNIONS AND INTERSECTIONS Any union and any finite intersection of open sets is open. The statement about unions is easy. As for finite intersections, if G 1 ,G 2 , , Gn are open and x n 7= 1 Gi then there exist positive numbers rl ,r 2 , . . . , rn such that B(x, ri ) C Gi for each i. Let r = mini ri o Then B(x, r) C n 7= 1 Gi . Infinite intersections of open sets need not be open: n : l ( - lln, lin) = {O} , which is not open since it is too small to contain an open ball in R. (d) INTERI OR For any G C X, the INTERIOR OF G, the set •
int G = {x
•
E
•
EG
:
B(x, r)
C
G, for some r > O}
is open. A point of int G is called an INTERIOR POINT of G . I f x E int G, then B(x, r) C G for some r > O. Therefore , by ( a) , each point of B(x, r) is also in int G; therefore , int G is an open set. Indeed, G is open if and only if G = int G. If G is open , then for each x G, there exists rx > 0 such that B(x , rx ) C G. Therefore G= (2 . 5) B(x , rx ) .
E
U
xEG
2. Analysis
59
Thus, the open balls are the building blocks of any open set , hence the name BASIC open sets. In a normed space the representation of equation (2.5) is even simpler , since B (x, =x+ 1). 0
rx )
rx B(O,
Definition 2.5.3 EXTERIOR
The EXTERIOR of a subset A of a metric space, ext A , is int CA. 0 Nonempty sets can have empty interior : Consider the rationals Q C R, for example . Sets A that are so sparsely distributed that int(cl A) = 0 are called RARE or NOWHERE DENSE. The Hilbert cube of Exercise 5 is an example of a rare set. The totality T of open subsets of a metric space Xis called the TOPOLO GY on X. A reason for interest in the topology is that convergence can be characterized in terms of open sets without reference to the metric : For any open set G to which x belongs,
x n --+ x if and only if X n
E G eventually.
Exercises 2 . 5 1 . CLO SED V S . OPEN The notions closed and open are dual concepts, not opposites : Failure to be open does not imply closed, nor vice versa. Sets can be both (X and 0, for example) or neither ([0 , 1 ) C R, for example) . Those that are both open and closed are called CLOPEN . Show that in any ultrametric space (Exercises 1 . 1-13 and 2.2-4) every ball, closed or open, is clopen. 2. PRO DU CTS O F OPEN SETS For any two open subsets U and V of the metric spaces X and Y , respectively, show that U x V is an open subset of X X Y when it carries the max metric doc of Example 1 . 1 .4 (or any other of the metrics dp , 1 � p < 00 , of Example 1 . 1 .4, for that mat ter ) . 3 . CONTINUITY For metric spaces X and Y , show that I : X --+ Y is continuous if and only if
(
(
a) I maps closures into closures: For any subset A of X , I cl A) cl / (A) . (b) I takes open sets back into open sets: For any open subset G Y, 1 - 1 (G) is open.
B.
C
C
4. INTERIOR If A C B, then show that int A C int In a normed space show that for any A C int cA = c int A for any scalar c =1= 0 .
X
X,
60
2. Analysis
5. HILBERT CUBE The subset C = { (an ) £2 : I an :::; of the space £2 of Example is called the HILB ERT CUBE. (a) Show that C is closed. (b) Show that C has empty interior , i . e . , C is a rare set .
E
1.1.7
I 1 In}
Hints 5 . (a) Show that if
x rI. C, then x rI. cl C.
(b) Show that any ball B(x , r) about any point x E C also meets the complement , i.e., that there exists a sequence from B(x, r) such that > for some in particular, choose > 2/r.
I b k l 11k
2.6
(bn) k
k;
Completeness
(X, d),
In a metric space if Xn - x , then the Xn are eventually close to are close to each Since points (xn and xm ) close to the same point x. other by the triangle inequality
(x)
it follows that for any
i
> 0, d(xn, x m )
Xn 1.. X m and II xn l l = 1 for each n N ) in an inner product space is Cauchy.
E
2. SUMS AND SCALAR MULTIPLES If (an ) is a Cauchy sequence of scalars and (xn ) and ( Yn ) are Cauchy sequences of vectors from a normed space, show that (anxn ) and (xn + Yn ) are Cauchy. 3. ONE-DIMENSIONAL SUBSPACES COMP LETE Show that a one-dimen sional subspace of any normed space is complete , indeed that it is of the form Kx for some vector x and is therefore linearly isometric to
4.
K.
INFINITE- DIMENSIONAL TRIANGLE INE QUALITY For any summable sequence (xn) in a normed space, show that II L neN Xn I ::; L n e N I I xn l l · (Allow for the possibility L neN I I xn ll = 00 .)
5 . If (x n ) is an orthonormal sequence in an inner product space X and (a n ) E £ 2 , show that Sn = L �= l a i X i is a Cauchy sequence. Thus, if X is a Hilbert space, then any such series LieN a i X i converges. 6 . A COMPLETE SPACE Show that the linear span
M = {a cos t + b sin t a, b E C } :
o f {cos t , sin t } i n the Hilbert space L 2 [O, 211"] is complete. 7. B [a, b] COMPLETE Let B [a , b] denote the space of all bounded scalar valued functions on the closed interval [a , b] (or any set T) with point wise operations and sup norm (as in Example 1 .2.3) . Show that B [a, b] is complete .
11 · 11 00
8.
(C
11·11 2 )
C
INCOMP LETE Let (R) denote the subspace of contin (R) , uous square-summable functions of the Hilbert space L 2 (R) . Show that (R) is not a Hilbert space by showing that for any a 0,
C
>
I t I ::; a, It I > a, i s a non convergent Cauchy sequence; indeed, Xn -+ 1 [ - 4.41 , the de cidedly discontinuous characteristic function of [ - a, a] . 9 . CAUCHY SE QUENCES IN ULTRAMETRIC SPACES In an ultrametric space (X, d) (Exercises 1 . 1-13 and 2 . 1 -4) the Cauchy criterion sim plifies considerably : Show that a sequence (xn) from X is Cauchy
66
2. Analysis
d (X n, Xn + l)
if and only if -+ O . For contrast , show that there is a sequence of real numbers such that x -+ 0 that does not converge.
(X n )
IXn+ l - n l
1 0 . BAIRE NULL SPACE COMPLETE Show that the ultrametric Baire null space of Exercise 1 . 1- 1 3 is complete. 1 1 . CAU CHY IMPLIES BOUNDED Show that a Cauchy sequence in any metric space is bounded. 12. 13.
X
COMPLETE IF AND ONLY IF U COMP LETE Show that a normed space is complete if and only if its unit ball U is complete .
X
fp COMP LETE
For 1
::; p ::; 00 , show that fp is a Banach space.
14. CON VERGENT SUBSEQ UEN CE In any metric space, if a subsequence of a Cauchy sequence converges, then the sequence must converge to the same limit . 1 5 . WHEN CLO SED IMPLIES COMPLETE Show that a closed subset of a complete space is complete. What about the converse? i .e . , does complete imply closed?
Fn) n n EN Fn =
of closed subsets of R 1 6 . Show that there exists a nested sequence ( 0 and a nested whose diameters do not go to 0 for which sequence of nonclosed sets whose diameters go to zero with empty intersection.
Hints 1 1 . Let f = 1 and consider the sequence in two parts, an early part and a tail.
Xn)
p
1 3 . For < 00, if ( is Cauchy, show that for any fixed j , the sequence of jth components (j) ) is Cauchy. For 00, use the fact that Cauchy sequences are bounded.
(X n
p=
14. Closeness is a transitive relation .
2.7
Uniform Continuity
Let and T, be metric spaces. A function x : S -+ T is U NIFO RMLY S, CONTINUOUS on S) if for any c > 0 there is 6 > 0 such that for all s ,
(S, d)
(
( d')
d(s, s ' ) < 6
s' E
==::}
d' ( x ( s ) , x ( s ' ))
0 , the uniform continuity of
x implies the existence of a 6 > 0 such that for all s, s' E D , d(s, s') < 6 d' (x(s), x(s')) < f . Now consider any two points s, s' in cl D such that d(s, s') < 6/3 and sequences (Sn) and (s � ) from D such that S n s and s� s'. By the ==>
triangle inequality, for sufficiently large n ,
-+
-+
d(sn , s�) � d(sn, s) + d (s, s' ) + d(s', s�) < 6.
Since metrics are jointly continuous (Exercise 2.2-3(a) ) , it follows that
d' ( x ( s), x( s' )) = limn d' (x( sn), x( s� )) � f .
The uniqueness of x follows from the fact that if two continuous functions agree on a set , then they agree on its closure (Exercise 2 .4-2) . An imp ortant application is the following. If : D -+ Y is a continuous linear map defined on a dense linear subspace of a normed space into a Banach space Y , then since A is uniformly continuous, A has a continuous extension A to Indeed,the extension is lin ear, since for E D such that -+ and -+ and E K,
A
X. .A x y, a Xn Yn lim .A (ax + y) n A (axn + Yn) = a.Ax + .Ay.
X xn , Yn
=
Exercises 2 . 7 When Cartesian products of metric spaces are considered in the exercises below , they may carry any of the metrics of equation (2.2) of Section 2.2 (cf. Example 1 . 1 .4) , since any of the produce the same convergent sequences (2.9.3 ) . It is usually simplest to endow the product with the max metric
dp dp
doc .
1.
RESTRI CTIONS AND COMP OSITES Show that restrictions and com posites of uniformly continuous maps are uniformly continuous.
2. MAPS INTO PRO DUCTS Let X, Y, and Z be metric spaces. Show that a map I----+of X into Y x Z is (a) continuous at x if and only if f and 9 are continuous at x. (b) uniformly continuous if and only if f and 9 are uniformly contin uous.
x
(f (x) , g (x))
3 . METRICS UNIFO RMLY CONTINUOUS For any metric space (X, show that is a uniformly continuous map of X x X into R.
d
d)
2. Analysis
69
4. PROJECTIONS UNIFORMLY CONTINU OUS Show that the proj ection operators of Example 2.2.6( d) are uniformly continuous. 5 . DISTA N CE UNIFO RMLY CONTINUOUS Let A be a nonempty subset of a metric space (X, d) and let d (x , A) = inf {d (x, a) : a E A } denote the distance from x to A of Exercise 2.4-16. Show that the map x 1-----+ d ( x , is uniformly con tin uous.
A)
6 . EVEN-ODD Assume that the differentiable function f has an inte grable derivative f' . Show that is even on ( -r, r ) if and only if its derivative f' is odd there. Show that is odd if and only if f (0) = ° and f' is even.
f
f
Hints 2. Use 2.2.6( c) for the continuity and the definition of uniform continuity for the uniform continuity. 3. Note that I d (x, y) - d (x ' , yl ) 1 ::; d (x , x ' ) + d (y, y' ) .
4 . Projections are linear and continuous.
2.8
Compactness
The idea of compactness embodies two concepts: smallness and neatness . As we shall see, closed intervals [a, b] are compact, but half-open intervals ( a , b] are not. The half-open interval is even smaller than the closed interval but the fuzzy edge at the left endpoint disqualifies it. Unbounded sets are never compact . There are several ways to characterize compactness, but as with continuity, the sequential way is usually the best one to use in normed spaces. We first need the notion of an (-net. Definition 2.8.1 {- NETS
With { > 0, a subset E of a metric space (X, d) is called an {-NET for X if for any x E X, there exists y E such that d (x , y) ::; (. Equivalently, E is an {-net for X if and only if for any ( > 0 , X is "covered" by the closed balls C (x, ( ) in the sense that X = UXEE (x, ( ) . 0
E
C
Note that a dense subset of a metric space is an (-net for any positive (. The integers are a ( 1/2)-net for R. The finite set {O, lin , 2/n , . . . , nln} Spaces constitutes a lin-net for the closed interval [0, 1] for any n E such as [0 , 1] in which finite {-nets exist for every ( > ° are of special imp ortance.
N.
70
2. Analysis
Definition 2.8.2 TOTAL BOUNDEDNESS
A metric space (X, d) is called TOTALLY BOUNDED (or PRE COMPACT ) if it has a finite {-net for any { ° or, equivalently, a finite number of closed (or open) balls of radius f cover X for any positive f . 0
>
[0, 1]
We observed above that is totally bounded; so is any bounded closed (or open) interval. R is not totally bounded, and neither is any unbounded metric space (X, by the following argument. If X is totally bounded then it is covered by a finite number of bounded sets, so it must be bounded. Bounded sets need not be totally bounded, as we shall see after 2.8.7.
d),
M
2 . 8 . 3 SUBSETS AND CLOSURES For a n y subset of a totally bounded metric space (X, and cl M are totally bounded.
d), M
X and consider the subset { X I , X 2 , . . . , x n } = {x E E : d (x, M) < f/2} , where d (x, M) is the distance inf {d (x, m) m E M } from x to M. Choose arbitrary points { m l , m 2 , . . . , mn } from M such that d (xj , mj ) < f/2 for each j. For any point m E M, d(m, x) :::; f/4 for some x E E. Therefore , x = Xj E { X l , X 2 , . . . , xn } , and d (m, mj ) :::; d (m, xj ) + d (xj , mj ) :::; 0, x rB B l (0, r). 11 · 11 1' 11·11 2
x
Definition 2 . 9 . 1 STRONGER NORMS
If 11 · 11 1 and 11·11 are norms on the same vector space X and 1 1 · l l cconvergence X n x implies I2H l 2-convergence X n x, then we say that 11 · 11 1 is STRO NGER than 11 · 11 2 or that 11·11 2 is WEAKER than 11 · 11 1 ; we write 11 · 11 1 � 11 · 11 2 . Clearly, 11 · 11 1 � 11 · 11 2 1 1 · 1 1 1 � 1 1 · 11 z ; as we prove below (2.9.2) , this is essentially the only way in which one norm can be stronger than another . 11 · lIp � 11 · 11 00 on K nl forp any n E N and lalI I :s p < 00 Let (ad E K n . Since l aj I = ( I aj n / :s ( 2:7= 1 ( lad P )) / p for each j , it follows that (2.8) 11 · 11 00 :s 11·ll p ' 1 :s p < 00 . Therefore, 11 · l l p is stronger than 11 · 11 00 . 11·11 1 � 11 · l b CUI U2 for some c > O. Next, we show that 11 · 11 1 � 11 · 11 2 implies that a positive multiple of the closed unit ball Ul = {x E X : II x ll l � I} determined by 11 · 11 1 is contained in the closed unit -+
-+
0
=>
•
•
=>
C
0
77
2 . Analysis
ball U2 determined by 1 1 · 1 1 2 ' By the definition of continuity, 1 1 · 1 1 1 !': 1 1 · 1 1 2 means that the identity map
X
1----+
X.
is continuous. Since I is linear , it is continuous if and only if i t is continuous at (Example 2.2.6(e) ) . Therefore, the continuity of I implies that CUI C I - I (U2 ) = U2 for some c > by the { - o formulation of continuity at (Exercise 2 .2-4) . 0
0
0
0
We collect these observations together in the following proposition. 2.9.2 CHARACTERIZATIO N O F STRO NGER NORMS Let 1 1 · 1 1 1 and 1 1 · 1 1 2 be norms on the vector space X . Then the following statements are equivalent: (a) 1 1 · 1 1 z :5 1 1 · 1 1 1 ; (b) CUI C U2 for som e c > 0 ; ( c ) c 1 1 · 1 1 2 � 1 1 · 1 1 1 for some c 0 ; (d) Every 1 1 · 1 1 2 - open ball is 1 1 · l l l - open.
>
Proof. We have already seen that (a) implies (b) . If (b) holds and x =P 0,
then
O.
which yields (c) , the result being trivially true for x = That (c) implies (a) is clear , since c I Ixn - x l 1 2 � I I xn - x I I I ' Let Bl = {x E X : I I x l l l < I } and B 2 = {x E X : I I x l 1 2 < It is easy to see that Bl is the interior , int Ul , of Ul ; similarly, B 2 = int U2 . We show below that (b) implies that B 2 can be written as a union of 1 1 · l l l -open balls. Since any 1 1 · 1 1 2 -open ball about x E X is of the form x + rB 2 , for some r > 0, it will follow that every 1 1 · I I z -open ball can be written as a union of 1 I · l I copen balls and is therefore 1 I · l I copen. By (b) , CUI C U2 for some c > so (Exercise 2.5-4)
I}.
0,
int cUl = c Bl
C
int U2 = B 2 .
Since B 2 is 1 I ' 1 I 2 -open, it can be written as a union of 1 I · 1 I 2 -open balls about each of its points by Example 2.5.2(d) : For each x E B 2 , there exists rx > such that
0
Since x + rx cBl e x + rx B 2 for all x , it follows that (2.9)
78
2 . Analysis
Therefore , B 2 is 1 I · l I copen, and we see that (b) implies (d) . Last, we show that (d) implies (b) . With = 0 and reading equation (2 .9) from right to left , it follows that for k = cro ,
x
Now take closures. 0
implies that 1 I · l I l -Cauchy sequences are By (c) it is clear that 1 1 · 1 1 2 � 1 I · 1 1 2 -Cauchy sequences . In Exercise 2 . 1-2 we noted that p < q implies in By 2.9 .2(b ) , C for p < q . By this observation and (2.8) , it i t follows that � follows that for 1 S; P S; q S; 00 . � We say that 1 1 · 1 1 1 and 1 1 · 1 1 2 are EQUIVALENT, 1 1 · 1 1 1 '" 1 1 · 1 1 2 ' if each is stronger than the other , in other words if they determine exactly the same class of convergent sequences. Geometrically, this means that each unit ball may be shrunk so as to fit into the other one. By 2.9 .2(a,c) (twice) it follows if and only if there are positive constants such that that 1 1 · 1 1 1 '"
1I·lIq
1I·l h
Up Uq iq.
1I·lIp 1I · lIq 11·lIp
a, b
1I·l b
(2. 10) The roles of 1 1 · 1 1 1 and (2. 10) is equivalent to
1I·lb may be interchanged in the above expression : (l/b) 1 I ·lb S; 1 1 · 1 1 1 S; ( l/a ) 1 1 · 1 1 2 '
so it makes no difference whether one says 1 1 · l i I '" 1 1 · 1 1 2 or 1 1 · 1 1 2 '" 1 1 · 1 1 1 ' Another way of describing equivalence is that there exist b > 0 , such that < ll.:!k < b . - 11·111 -
a,
a
Il x lip =
(�n l ai lP ) lip
S;
(n
II x ll � ) l /p = n l /p II x l i oo , 1 S; p < 00 ,
so 1 · I � 1 1 · 1 1 00 ' Combined with Inequality (2.8) , we have the following result .
I lp
2.9.3
Kn . Kn .
1I · lIp
Kn , n
EQUIVALENCE OF p- NORM S O N on E N, 1 1 · 1 1 00 for all 1 S; P S; 00 ; by the evident transitivity of equivalence of norms, it follows that all p-norms are equivalent on '"
This is a special case of the fact that all norms are equivalent on any finite-dimensional space (Exercise 3) . Even though p-norms are equivalent in the finite-dimensional case, they are not in the infinite-dimensional case.
2. Analysis
79
Example 2.9.4 INEQUIVALENCE O F p- NORMS
(a) For q > p � 1, 1 I · l l p + 1 1 · l l q on £p . it follows that As observed after 2.9.2, for q > p � 1, C in belongs to £p C £ , and it is a proper inclusion : In particular, = £g , but not to £2 . If we restrict 1 I · l I g to £2 , it is not equivalent to 1 1 · 1 1 2 since the sequence with kth entry
q
(xn )
Up Uq £q; l x (n - / 2 )
x n (k) = { k - l / 2 kk >� n,n, 0,
'
nEN
is 1 I · l I g -Cauchy but not 1 I · 1 1 2 -Cauchy. (b) 1 I · l l p + 1 1 · 1 1 00 on Co for any 1 � p < 00. that are eventually 0 , and of sequences from sequences . Observe that
K
cp C
£p
C Co ·
As in the finite-dimensional case, 1 1 · 1 1 00 ::5 1 I · l I p on for any p �
1,
£p
( n l1/p ) E
Co
but
( n l1/p )
Co
Let cp denote the set the space of all null
£p for any p � 1 . Also ,
� £p ,
so is a proper subset of Co . Note that (cp, I I · l I oo ) is dense in ( co , 1 1 - 1 1 (0 ) ' since any point of Co is the limit of the sequence obtained by taking to be those of for � n and 0 for > the components of Since 1 1 . 1 1 (0 ) : therefore (£p , 1 1 . 1 1 (0 ) is not (cp, 1 1 . 1 1 (0 ) is dense in ( co , 1 1 - 1 1 (0 ) ' so is closed in ( c o , 1 1 - 1 1 (0 ) ' Since complete subspaces must be closed , it follows is a that (£p , 1 1 . 1 1(0 ) is not complete as a subspace of ( co , 1 1 - 1 1 (0 ) ' Since Banach space with respect to 1 I ' l I p , 1 I · l I p is not equivalent to 1 1 · 1 1 00 . (c) 1 I · l I p + 1 1 · 1 1 00 on Consider the space (Example 1 .2 . 3) and the norms
x X n
x(£ , k (xn ) p
k
n.
£p
C [a, b] .
I I x l l oo = sup I
C [a, b]
x [a, b] 1 and I I x l ip =
(lb l x(t) IP dt) l /p
Since
it follows that
x
II l i p
� (b - a) l / P I x l oo ,
so 1 1 · 1 1 00 is stronger than 1 I · l I p ' But they are not equivalent: The sequence = from 0 , 1 ] is 1 I · l l p -convergent to 0 for any � p < 00 , but does not converge uniformly to O. 0
Xn tn (n E N )
C[
1
In Example 1 . 1 .3 we considered the notion of isometry of metric spaces. The notion of "homeomorphism" is similar, but weaker. If two metric spaces
80
2 . Analysis
X and Y are isometric under an isometry I : X Y , then a sequence ( x n ) converges in X if and only if (f(xn ) ) converges in Y . If a bijection I has only -+
this convergence-preserving property, then I is called a HOMEO MORPHISM. A homeomorphism I, in other words, is a bicontinuous ( f and 1 - 1 are is a homeomorphism of continuous) bijection . The map I-> (1 + R onto the open interval ( - 1 , 1) that is not an isometry. In Section 1 .5 we discussed lin ear isometry of normed spaces. If two normed spaces are linearly isometric, they are called "geometrically" the same; if they are linearly homeomorphic (defined below) , they are called "topologically" the same.
x
Ix l)
xl
Definition 2.9.5 LINEAR HOMEOMORPHISM
X
A X
Let and Y be normed spaces. A bicontinuous linear bijection : -+ Y is called a LINEAR H OMEO MORPHISM . If a linear homeomorphism exists between and Y , we say that X and Y are LINEARLY HOMEOMORPHI C .
X
o
Exrunple 2.9.6 LINEAR HOMEOMORPHISMS
(a) A linear isometry is a linear homeomorphism. The converse is false : the map I-> 2t is a linear homeomorphism of R onto R that does not preserve distance . (b) Linear homeomorphisms preserve completeness . (Proof? Consider the linear image of a Cauchy sequence.) , (c) If "' I I · I I 2 then the identity map I : -+ I-> is a linear homeomorphism. Hence , by 2.9.3, for any n E N , the spaces fp ( n ) and ( n ) are linearly homeomorphic for all p :s q 00 . ( d ) I t follows from Example 2.9.4 that none o f the identity maps I-> below are linear homeomorphisms.
t
x x,
(X, II · II l )
11 · 11 1
fq
(X, 11 . 11 2 ) 1 ::; ::; x x
(fq , II · II q ) for q > p � I , ( Co, II · 11 00 ) , ( C [a, b] ' II · IIp ) . O Exercises 2 . 9
1.
(X, II · II ). 11 · 11 11 · 11 00 .
Write Let {X 1 , X 2 , . . . , Xn } be a basis for a normed space E as for appropriate E I 0, of 0, B (0, 7'i ) n (Xi + Mi ) = 0 . It follows that there must be some ri > ° such that , for all m E Mi , Il x i + m il � ri . Let X = 2.:7= 1 aj xj , where aj E K , j = 1 , 2 , . . . , n . For any nonzero scalar ai , n n a· Il x ll = L= aj Xj = l ail X i + '" L...J ...L Xj � l a d ri � l a il m in ri j l j = l,j ;t i ai so I 2.:i= I aj X I � (mini ri ) Il xi l ( b ) The converse is true. Let B be a Hamel base for X. For X E X write X = 2.:7= 1 a i X i for appropriate X i E B and scalars a i . Each of the following defines a norm on X: Il x ll l = 2.: 7= 1 l a i l and Il x ll oo = maxi l ai l · If X is infinite-dimensional, show that 11 · 11 1 1:. 11·11 00 . 4. ( a) . Let {X l , X 2 , . . . , Xn} be a basis for X and let 11 · 11 00 be as in Ex ercise 1 . By Exercise 3 , we may assume that X carries 11 · 11 00 . Let {X l , X , . . . , Xn} be a basis for X, so that for any X E X, X 2.:7= 1 2ai X i , for appropriate scalars ai . Then . Z
j
OO"
'
2. Analysis
83
Now use the criterion for continuity of a linear map of Exercise 2.2-9 . (b) . If dim X = 00 , construct a linear functional that i s unbounded on the closed unit ball.
d' is a metric , note that for all x, y, E X , d(x, y) d(x, y) 1 + d(x, y) � 1 + d(x, y) + d(y, ) To see that it is equivalent to d, note that the function t tf (1 + t), t > 0 , has a unique inverse.
7 . (a) . In order to show that
z
"
z
�
2 . 10
Direct Sums
Y
Can two normed spaces X and be combined to yield a new normed space? Can two inner product spaces be combined to get a new inner product space? Moreover, can the combining be done in such a way that we do not lose the original spaces entirely? Specifically, can the combination be formed so that it contains copies of the original spaces? We considered the analogous question for metric spaces (X, in Example 1 . 1 .4. The new space was the Cartesian product X x metrized by any of the metrics
d l ), (Y, d2 ) Y
or
doo (x, y) = max [dl (x, y) , d2 (x, y)] .
y
In each case the original space X is recoverable as (isometric to) X x { } for any fixed in For linear spaces X and over the same field K , we make X x into a vector space by defining addition and scalar multiplication as follows: For E X, E and a E K,
y Y.
Xl , X2
Y
Y
Y2 , Yl Y, Y
$Y
$Y
So equipped, X X is denoted by X or X (ext) and called the EXTERNAL ( ALGEBRAIC ) DIRECT SUM of X and Y. If (X, { " ' } l )and (Y, ( are inner product spaces, the EXTERNAL INNER PRODUCT DIRECT E X SUM is X equipped with the following inner product: For and E
'}2 ) $Y Yl , Y2 Y,
"
Xl , x2
Y
The original spaces X and may be recovered as inner product isomorphs and of X x { O } ( = X {O}) and {O} x respectively. If (X,
$
Y,
11 · 11 1 )
(Y, 11-I1 2 )
84
2. Analysis
are normed spaces, we have choices about how to norm X and y E Y , norms are defined by
x
Y : For
(2.11)
or any of
(1.6.3
xEX (2.12)
1.6-2)
The Minkowski inequalities and Exercise validate the triangle inequality in each case. It makes little difference which norm is selected , since the 00 , are equivalent ; in other words, �p< q
11·lI p , 1
(X
Ef)
Y,
�
1I·lIp )
is linearly homeomorphic to
x x.
(X
Ef)
(
Y,
11 · 11 ) q
1I · ll p ) (2.11)
under the identity map 1---7 Thus, although the spaces X Ef) Y, are obviously not linearly isometric as p varies, they do have the same conver gent sequences . X Ef) Y equipped with any of the norms of equation or i s called the EXTERNAL TOPOLOGICAL DIRECT SUM of X and Y; other aliases are DIRECT PRO DUCT and TOPOLO GICAL PRO D U CT. Any finite number Xl , X , Xk of normed spaces is easily accommo dated by the same technique; ED := l Xi denotes the direct sum in this case. c) , a We endow with one of the norms 00 . As in p sequence n E N , from ED := l Xi converges = to if and only if � i) for each no matter what norm 00 , is chosen for ED := l Xi . It follows from this observation � p that
(2.12)
2,
•
.
.
11 · lIp ' 1 � � x Xn (xn(1), n(2), . . . , xn(k)), x Xn (i) x( 1�i�k 1I·ll p , 1 � • •
•
2.2.6(
The external direct sum of Banach spaces is a Banach space.
.
{O} x . . for all j .
x
{O}
x
Xj
x
{O}
x . . . x
{ O } is a closed subspace of ED := l Xi
of ED := l Xi onto Xj The projection maps = are continuous for each j; indeed, they are uniformly continuous linear maps .
Pj [(X l , X 2 , . . . , Xn)] Xj N
Now suppose that M and are linear subspaces of a normed space X . For appropriate M and N (i.e . , big enough M and N) , can w e synthesize X from M and N by this technique of pasting spaces together? As we shall see, it is easy to put X back together algebraically but it may not be possible to recover X as a normed space from component subspaces . More specifically, the norms p 00, on M x N may not yield a space that is even linearly homeomorphic to X, let alone linearly isometric to it. If M n N = {O} and X = M + N in the sense that any vector in X can be written as
11 · lIp , 1 � �
x
x = m + n for
m
E M, n E N,
85
2. Analysis
M
N,
M
N
then X is called the INTERNAL DIRECT SUM of and and and are called ALGEBRAIC COMPLEMENTS of each other. Sometimes this is written X
= M EEl N (int) .
It can be shown that every subspace M has an algebraic complement to a Hamel base B' for the whole space; a extend a Hamel base B for complement N will be the linear span of the "new" basis vectors , the basis vectors of B ' not in B. Furthermore, every algebraic complement of M has the same dimension , and this dimension is called the CODIMENSION of M, denoted by codim M. Not only can every in X be written in the form for some and but the and are unique: if
M
x = m+n
then
x = M EEl N m E M n E N, m x = m + n = m' + n',
n
m - m' = n' - n E M n N = {OJ .
Algebraically, there is no difference between external and internal direct sum. If X N (int) , then consider
= M EEl
M' = M {OJ
N' = {OJ N. It now follows that X is linearly isomorphic to M' EEl N' (ext) : The map M' EEl N' (ext) ---. M EEl N (int) , m E M, n E N, (2.13) ((m, O) , (O, n)) m + n, x
x
and
�
is a linear isomorphism . For X N (int) , since the representation N, is unique, the PROJECTION of X ONTO M
= M EEl
x = m + n, m E M, n E Pl Pl : X = M EEl N (int) ---. M, x=m+n m, is a well-defined linear map; so, of course, is P2 x = n, the PROJECTION of X O NTO N. Unlike projections on external direct sums, however, proj ec �
tions on internal direct sums do not have to be continuous. For example , there is no continuous projection of (bounded sequences) onto Co (null sequences) . (The argument can be found in Narici and Beckenstein p . 87.) Let (int ) . Suppose (ext) is normed by one of the norms :::; p :::; 00 , of equations and and consider the map (with "s" for "sum" )
£00
X = M EEl N 11 · lIp , 1 S:
M EEl N (2.11)
M EEl N (ext) ---. X, (m, n) m + n. �
1985,
(2.12)
(2.14)
It is routine to verify that S is a linear bijection ; it is continuous because
86
2. Analysis
2.2.6(
by Example c) , and the continuity of addition in a normed space But it need not be bicontinuous, i.e . , its inverse
(2.2.3).
need not be continuous, something we say a little more about in
2.10.1.
2 . 1 0 . 1 WHEN 8 IS A LINEAR HOMEOMORPHISM 8 is a lin ear bijective homeomorphism if and only if one of the projections and is contin uous.
PI
P2
Proof.
Since 8 is a continuous linear bijection, the question concerns only the continuity of the map 8 - 1 , the map � For any metric spaces X, Y, and Z, a map � (f 9 of X into Y x Z (with any of the usual product metrics) is continuous at a point x if and only if f and 9 are continuous at x, since by Example c) ,
x (PI X, P2 X) . (x) , (x)) 2.2.6 ( (f (x n ) , 9 (xn)) � (f (x) , 9 (x)) I (xn) � I (x) and 9 (xn) � 9 (x) . Therefore, in particular, x (PIX, P2 x) is continuous if and only if PI and P2 are continuous. But one of PI and P2 is continuous if and only if the other one is, since 8 = PI + P2 , 8 is continuous, and the difference of X
�
�
continuous maps is continuous.
0
Definition 2 . 10.2 TOP OLO GICAL DIRECT SUM
The normed space X is the TOPOLOGICAL DIRECT SUM of the subspaces M and N, X = M (top) , if the map S of is a linear homeomor phism. In this case and are called TOP OLOGICAL C OMPLEMENTS (or TOPOLO GICAL SUPPLEMENTS ) of each other, and M (and are said to be TOP OLOGICALLY COMPLEMENTED . 0
EEl N M
N
(2.14)
N)
PI
is If M and N are TOPOLO GICAL COMPLEMENTS of each other , then continuous and N = so is closed. Thus, a necessary condition for a subspace to possess a topological complement is that it be closed. There are "uncomplemented" closed subspaces , however. For example (N arici and Beckenstein 1985, pp. the closed subspace Co of null sequences has no topological complement in the Banach space of Exercise 1 . 1- 1 1(b) . Thus, if N is any algebraic complement of in the map 8 : Co x N � ( m , n ) � m + n , cannot be bicontinuous. Three imp ortant special cases in which complements do exist are the following:
Pl- l (O), N 86-88),
too ,
•
•
too Co too ,
In any normed space X , if f : X � K is a nontrivial continuous linear form, then 1 - 1 has a topological complement . (N arici and Beckenstein 1 985, p. 8 9.)
(0)
N
If M and are closed algebraically complementary subspaces of a Banach space X , then they are topological complements . This follows
2. Analysis
87
from the open mapping theorem-which implies that a continuous linear bijection between Banach spaces is bicontinuous-applied to the map S. For a closed subspace M of a Hilbert space X, things really simplify : X = M EEl M a topic w e return t o i n Section 3.2. If X , Y, and W are normed spaces, and A : X - W and B : Y - W •
l. ,
are continuous linear maps , then
A $ B : X $ Y (ext) (x, y)
�
1---+
W
Ax + By
determines a continuous linear map called the direct sum of A and
B.
Exercises 2 . 1 0 In the exercises below, direct sums are endowed with any of the norms of equation (2 . 1 1 ) or (2. 12) . 1 . n-DIMENSIONAL HILBERT SPACE Suppose X is an n-dimensional Hilbert space, and that , is an orthonormal basis of vectors for Show that is isomorphic as an inner product space to the inner product space EEl f= 1 (ext) , where denotes the linear span of for = 1 , 2 , . . . , n .
{X l , X 2 , x n } X. X [X i ] [X i] KX i X i i 2 . I s the real space Roo (2) = (R2 , 11 - 11 (0 ) linearly isometric to ( R x {O}) EEl ({O} x R) (ext)? (If the direct sum carries the max norm 11 · 11 00 of equation (2. 1 1 ) , yes; otherwise, no.) .
•
•
3 . D IRECT SUM OF COMPLETE SPACES
X M
X$Y
(a) If and Y are Hilb ert or Banach spaces, show that (ext) is a Hilb ert or Banach space, respectively. (b) If and N are complete orthogonal subs paces of an inner prod uct space show that M + N is complete.
X, 4. Let Xl , X2 , Xn be normed spaces. Show that: (a) {O} x . . . x {O} x Xj {O} x . . . {O} is a closed subspace of EB �= 1 Xi (ext) for all j. (b) The projection maps Pj (X I , X 2 , , X n) = Xj of EB �= I Xi (ext) onto Xj are uniformly continuous linear maps. 5. D IRECT SUM O F SEPARABLE SPACES Let X l , X2 , . . . , Xn be sepa rable normed spaces. Show that EB� 1 Xi (ext) is separable . •
•
•
,
x
x
•
.
•
88
2.
Analysis
6. If M and N are orthogonal subspaces of an inner product space X , show that any vector x in M + N = {m + n m E M, n E N} has a unique representation as x = m + n ( i.e . , the m E M and n E N are :
unique) .
7 . If M and show that
N are closed, orthogonal subspaces of a Hilbert space X , M + N is closed.
Hints
1. Consider the map ( X l , X 2 , . , xn) X l + X 2 + . . . + X n ; cf. 1.5.4. 7. For w E cl (M + N), choose Xn E M and Yn E N such that x n + Yn w. Use the Pythagorean theorem to show that (x n) and ( Yn ) are . .
1---+
�
Cauchy sequences.
3
B ases The classical subject of Fourier series is about approximating periodic func tions by sines and cosines, specifically, about expressing an arbitrary peri odic function as an infinite series of sines and cosines. (Any function that vanishes outside some interval can be viewed as a periodic function on by merely extending it periodically.) The sines and cosines are the "basic" periodic functions in terms of which we express all others. To use a chemi cal analogy, the sines and cosines are the atoms; the other functions are the molecules. Unlike the physical situation, however, there can be other atoms, other functions, that can serve as the "basic" functions just as effectively as sines and cosines. We consider in this chapter a more general setting in which to view de composition into elemental comp onents . Specifically, we investigate how to write a vector as an infinite series of "basic" vectors in Hilb ert and Ba nach spaces. We show (3.4.8) that , in any separable Hilbert space there is an orthonormal sequence such that any E X can be written as Such a sequence is called an O RTHONORMAL = BA SIS for (The collection of sines and cosines (suitably normalized) constitutes an orthonormal base for the Hilbert space the HAAR FUNCTIONS of Section 3.5 are an orthonormal basis for The ex istence of a canonical way to represent an arbitrary element is very useful of any information. For example , the representation = in a separable Hilb ert space makes it possible to "digitize X," to view E as a sequence Put another way (3 .4.9) , any infinite dimensional separable Hilbert space is linearly isometric to Thus, one has a choice: one can, for example , view as a function space or as the sequence space We cannot do as well in separable Banach spaces . Suppose that is a sequence in an infinite-dimensional B anach space uniquely If every E can be written as = with the determined, then is called a SCHAUDER BASE (or BASIS ) for Exam The ples? Any orthonormal base is a Schauder base: take = Haar system mentioned above is a Schauder base for each of the ::; p < 00 . Does every separable Banach space have a Schauder base? This was a long-standing open question known as THE BASIS PRO BLEM . We have j ust noted that the spaces have Schauder bases; the function spaces (D) of all analytic functions on the disk D = E C : < I } , such = [J JD < 00, normed by that J JD for p = 00 , have Schauder bases; so does for ::; p < 00 , and by (C In fact, all the separable Banach spaces known before 1 973
R
(xn)
x En eN (x, Xn) xn · X.
x
X.
X « ( x, x n} ).
x X
X,
x
(xn)
L 2 [0, 271"]; L 2 [0, 1].) x En eN ( x, xn) X n £2 .
L 2 [0, 1]
£2 . (xn) x X (x n )
x En e N anxn,
an X. an (x, xn). Lp [O, 1],
{z
Izl I I (z W dxdy] l i p ,
(xn)
1
Hp
Lp
I I (zW dxdy 1 11·11 00 [0, 1] , 11 ' 1 1 00 ) '
I
11 / 11
90
3. Bases
had Schauder bases. These facts , together with the knowledge that every separable Banach space is linearly isometric to a closed linear subspace of [0 , 1] , led to the consensus that all separable Banach spaces possessed a Schauder base. But they do not . The solution came not from looking "big" -at larger and more exotic spaces-but to look "small, " inside Co . En flo ( 1973) showed that certain subspaces of and for 2 < p < 00 do not have Schauder bases. An improved version of Enflo's argument may be found in Lindenstrauss and Tzafriri 1 977; it is only for the decidedly more advanced (intrepid?) reader.
C
(co, 11 · 11 00 )
3.1
ip
Best Approximation
When we speak of approximation, as in "a real number x can be approx imated by rational numbers," we mean that there are rational numbers that are arbitrarily close to it. This implies that there is a sequence of rational numbers converging to x-choose a rational X n from the inter val - lin, + lin) for each n E But there is no sequence of vectors in the plane that converges to the point z = ( 0 , 0 , 1 ) . The vector in that comes closest to is the origin 0 (0, 0, 0). It happens that the distance from z to 0 is the same as the (perpendicular) distance from to
(x n)
(x
N.
x R2
R2 R2 :
z
=
z
d (z, 0) = d (z, R2 ) = inf { li z - x II : x E R2 } .
We take the minimal distance requirement imation in a normed space.
as
the criterion for best approx
Definition 3 . 1 . 1 BEST ApPROXIMATION
X
x
Let M be a linear subspace of the normed space and let be a vector not in M. We say that E M is a BEST APPROXIMATION TO X FRO M M if = inf : E We call the DISTANCE from x to M. As an in!, Ilx - mil 0 for all
mo
II x mo ll d(x, M) m E M.
{ ll x m il m M} = d(x, M). II x mo ll :::; -
Best approximations do not always exist-What is the best rational ap proximation to Vi? If they do exist , they need not be unique. What is the best approximation to the origin from the surface E = I } of the unit ball in R3 ? (cf. also Exercise 2 and Example 3.2 . 6) . If a linear subspace of a normed space is such that there exist unique approxi mations to any then is called CHEBYSHEV . We provide examples of Chebyshev subspaces in 3 . 1 . 2 and 3 . 1 .3. For Chebyshev subspaces M , we can speak of the best approximation to E from M.
{x R3 : I lx ll
M
x E X,
M
X
x X
3 . 1 . 2 FIN ITE-DIMENSIONAL SUBSPACES O F INNER PRO D U CT SPACES Let { X l , . . be an orthonormal set in an inner produ ct spa ce Then .
, xn}
X.
3. Bases
91
the best approxim a tion to any x E X from M = [X l , X 2 , . . . , Xn) is given by L:7= 1 ( x, Xi ) X i · Proof. Consider X = (a, b , ) E R3 and let e l = ( 1 , 0 , 0) , e = (0, 1 , 0) and e 3 = (0, 0 , 1 ) . Note that a = (x, e l ) and b = (x, e 2 ) and that2 x - a e l - b e 2 .l R2 . The distance from x to R2 is given by c
{X l , . . . , xn} M
This idea extends to general inner product spaces. Let be an orthonormal set in an inner product space X and let denote the linear span of Finding the best approximation of from is equivalent to finding scalars such that
M
[X l " ' " xn) {X l , " " xn }.
{a I , . . . , an} n X - 2: ai X ; i= l
X
ai = (x, X i ) . First, a, ) - a (X- l , X ) + ali----II x I1 22 - (x, X l---Il x ll - (X, X l ) (X, x r) + (X, X l ) (X, X l ) X l ) - a (X l , X) + a li (X, 2 II x I1 - I (x, x r) 1 2 + I (x, X l ) - a l 2 . The last expression is clearly minimized when a = ( X, X l ) ' When aX l is replaced by L:� 1 a i X i , we get more terms like the ones in the expression is minimized . We show that it is minimized when the observe that for any scalar
-Ii
Ii
above. We get
2 n n n X - 2: ai X i = II x I1 2 - 2: I (x, x i ) 1 2 + 2: I (x, Xi ) - ai l 2 . i= l Clearly, the best approximation to x is obtained by setting a i = (x, X i ) for each i. The best approximation to x by vectors in the linear span M = [X l , X 2 , . . . , xn) is therefore the "sum of its projections" (x, X i ) X i on the subspaces [X i ): n rna = 2: (X, X i ) X i . 0 ;=1 Let {X l , " " x n } be an orthogonal subset of an inner product space X and let X E X. Then the vector X - L: ?= l (x, X i ) X i is orthogonal to each Xi , for i = 1 , 2, . . . , n. Therefore , it is orthogonal to the linear span M = [X l , X 2 , . . . , xn). This suggests that in more general circumstances the best approximation to x by a vector rn a E M is an rn a such that X - rna .1 M. We show in 3 . 1 .3 that a unique best approximation to any X E X from a complete (linear, as usual) subspace M of an inner product space X ;=1
92
3. Bases
exists no matter what the dimension of M is. Since a closed subspace of a complete space is complete , below applies to closed subspaces of Hilbert spaces, and this is its most frequent application. The conclusion of the theorem fails for closed subspaces of incomplete spaces (see Exercise and Example
3.1.3
3.2.6).
2
M X x E X. mo E M x,
3 . 1 . 3 BEST ApPROXIMATION FRO M COMP LETE SU BSPACES Let be a complete lin ear su bspace of the inner produ ct space and let Th en (a) there is a uniqu e best approximation i. e., a unaqu e to E such that
mo M
II x - mo ll = inf { ll x - m il : m E M} = d(x, M); (b) x - mo M and mo is the only point in M such that x - m o .1 M. Proof. (a) Let d = d(x, M). We want a vector m o E M such that Il x - m o ll = d. We create a sequence of vectors mn E M that almost have this property, then take a limit. Since d(x, M) is an infimum, for each n E N there exists mn E M such that d � lI x - mn ll < d + 1/n. (3.1) Since the m n are all close t o x, they are close t o each other; indeed (mn) is Cauchy. To verify this, consider a "parallelogram" with sides x - mn and x - m k and diagonals 2x - mn - m k and m k - mn , the sum and the difference. It follows from the parallelogram law (1.4.5) that 2 11 x - mn ll 2 + 2 11 x - m k l1 2 = 11 2x - mn - m k l1 2 + II m k - mn ll 2 • Since M is a linear subspace, t (mn + m k ) E M so, for any n and k, .1
II x - mn ll < d + l/n, I I x - mn ll 2 < d2 + 2d/n + 1/n22 . For c > 0 we 2d/n < c/2 and 1/n < c/2, so that 2 2 n n, k � N , d II x - mn ll � + c lI (mk - mn) 1I 2 � ' (2 d2 + 2c) + (2d2 + 2 c ) - 4d2 = 4c . It follows that (mn) is Cauchy. Since M is complete , there exists mo E M such that mn � mo. By the continuity of the norm it follows from inequality (3.1) that II x - mo ll = d. Suppose m E M also satisfies II x - m il = d. Since t (m o + m) E M, the parallelogram law implies that 2 11 x - mo l1 2 + 2 11 x - m l1 2 - 11 2x - mo - m ll 2 11 m - mo ll 2 = 2d2 + 2d2 _ 4 1 I x - t (mo + m) 11 2 � 4d2 - 4d2 = 0, Since can choose N to simultaneously satisfy for � N. Therefore , for
93
3. Bases
m m o. m
from which it follows that = (b) For any nonzero point of M and any nonzero scalar
B y expanding the left-hand term and using the fact that follows, for all nonzero t, that
I t I 2 II m il 2 - 2 Ret- (x - m o , m» 0 Suppose that (x - m o , m ) =p 0 and let t = (m, x - mo)
real. The above inequality becomes
t,
II x - mo ll = d, it
8,
where
8
=p 0 is
which implies that
8 2 ! ! m ll 2 - 28 > 0 for all real nonzero 8. For 8 = 1 , then II m ll 2 > 2 for any m E M , which is imp ossible . We conclude that (x - mo, m ) = 0, i.e . , that x - mo M . Finally, suppose that m' E M also has the property that x - m' M . Then, for any m E M, (x - mo, m) = 0 = (x - m' , m ) . This implies that (m' - mo, m ) = 0 for every m E M. Therefore, m' - m o m' - mo; this yields the desired uniqueness of mo . 0 1..
1..
1..
More generally, 3 . 1 .3(a) is true for closed convex subsets of uniformly convex Banach spaces (see Exercise for the definition of uniformly convex and Exercise 1 of this section; see Narici and Beckenstein 1 985 , p . 363, ( 1 6 . 1 . 5) , for the more general result) . Hilbert spaces are a special kind of uniformly convex space.
2.2-12
Exercises 3 . 1 1 . UNIQUE VECTO R O F MINIMAL NORM This generalizes the "linear subspace" of 3 . 1 .3 to "convex set" (defined in Exercise 1 .3-8g) . For any complete convex subset of an inner product space
(a) (b)
K X: There is a unique vector w E K of smallest norm, i .e . , such that !! w ll = d (0, K) = inf { ! ! x ll : x E K }. (This is the best approxi mation to 0 from I Let Show = / that E and --+ from above. Thus, M) If m E is such that show that = < which is contradic tory.
a l:i EN 2 - i bi , an = a � k, an = 0 k. X k (2 k (2 k 1)) (an) - (bn). X k M II x - x kll l a l d (x, � il a l . = (an ) M x - m il � l a l , ll:i EN 2 - bi / II ll: iE N 2 -i (bi - ai ) 1 ::; l:i EN 2 - i I bi - ail l a l , (bn)
3.2
Orthogonal Complements and the Proj ection Theorem
2.10,
X
As discussed in Section a normed space may possess a closed linear subspace M that has no topological complemen t, i.e., no subspace for which X = Ef) (top) . It follows from 3 . 2 .4 that a closed subspace of a Hilbert space is "complemented ."
M N
N
3. Bases
95
Definition 3 . 2 . 1 ORTHO G O NAL COMPLEMENTS
S S1. = {x S
Recall that if is a nonempty subset of an inner product space X , the set E : x 1. is called the ORTHOGONAL CO MPLEMENT or O RTHO CO MPLEMENT of S. is pronounced "S perp." •
•
•
Clearly, other.
S} S1.
0
{O} and the whole space X are orthocomplements of each {O} x R R2 . R3
Rx {O} {(O, On x R i s the R2 . In £2 (Example 1 . 1 .7) the orthocomplement of the subspace M of sequences whose first three entries are 0 is the subspace of sequences whose entries are 0 after the first three . This idea extends to L 2 [0 , 211"] : The orthocomplement of the subspace M of functions that vanish on The " y-axis" is the orthocomplement of the "x-axis" o r any subset thereof i n Likewise, the z-axis orthocomplement in of any subset of the plane
[0, 11"] consists of the functions that vanish on [11", 211"] .
The most basic properties of orthocomplements are the following. 3.2.2 PRO PERTIES O F ORTH O CO MPLEMENTS For any subsets S and of an inner produ ct space X : (a) i s a closed lin ear subspace. (b) C (c) ===? C (d)
S1. S S1.1. . S e T T1. S1. . (S1.1.) 1. = S1. .
T
S1.
Proof. We prove only (a) to illustrate the technique. That is a linear subspace follows immediately from the linearity of the inner product in its first argument : For any scalar a, any x , y and z E
E S1. S, { ax + y , z} = a { x , z} + {y , z} = O. To see that S1. is closed, let ( xn ) be a sequence from S 1. such that Xn x E cl S1. . Then for any z E S, by the continuity of the inner product ->
(Example 2 .2 . 5( b) ) ,
The next result sharpens the result that the only vector perpendicular to every vector is
O.
S
3.2.3 ORTHO C OMPLEMENT OF A DENSE SET If is a dense subset of the inner p roduct space X , then = (The converse is true, too, if X is a Hilbert space (3.2.5).)
S1. {O}.
96
3. Bases
Proof.
y
Let x E 81. . For any r > 0, since 8 is dense, there exists E 8 such that I I x < r. By the Pythagorean relation (equation ( 1 . 12) of Section 1 .4) , -
y ll
Since r is arbitrary, this implies that x = o .
0
The following imp ortant result is a corollary to 3 . 1 .3. 3.2.4 T HE PROJECTION THEOREM If M is a complete subspace of the inner produ ct spa ce, X then: (a) X = M $ M1. (top) . (b) M = M 1. 1. . Most often , we apply this to a elosed subspace M of a Hilbert space.
Proof.
(a) Clearly, M n M1. = {OJ . Let x E X , and let rn a E M be the best approximation to x from M of 3 . 1 .3. Since x E M 1. and -
x=
rna
rna + (x rna) , -
=
M and M 1. are algebraic complements. To finish the proof that X M $ M1. (top) (Definition 2. 10.2) we use the criterion of 2 . 1 0 . 1 and show that the projection on M along M 1. is continuous. In other words, we show that
X i = rni + ni E M $ M1.
�
X = rn + n
This follows from the Pythagorean relation
(b) By 3.2.2(b) , M
C
==>
rni
� rn .
M 1. 1. . Now suppose that x E M 1. 1. and write x =
rn + rn' E M $ M1. by (a) . Since x and rn each belong to the linear subspace M 1.1. it follows that rn ' = x rn E M 1.1. , well. Since rn' E M 1. n M 1.1. , it follows that rn' = O . 0
-
as
The completeness of X is vital in 3.2.5, as shown by Example 3.2.6. 3.2.5 ORTHO CO MPLEMENTS IN HILBERT SPACES space M of a Hilb ert space X (a) M 1. 1. el M . (b) M 1. = { O J if and only if M is dense i n X .
For any lin ear sub
=
Proof.
(a) Since M C M 1.1. and M 1. 1. is elosed, i t follows that e l M C M 1.1. . Since el M is complete , by the projection theorem we may decompose the Hilbert space M 1. 1. = el M $ (el M) 1. , where (el M) 1. is computed within M 1. 1. . Since M C el M, it follows that (el M) 1. C M1. . If x E (el M) 1. , then x E M 1. n M1.1. = {OJ, and M1.1. = el M $ (el M) 1. collapses to M 1. 1. el M .
=
3. Bases
( b) We have already proved the "if" assertion M.L = {o} then l M = M.L.L = {O} ol = X . 0
97
in 3.2.3. Conversely, if
e
In 3 . 1 .3 we showed that if M is a complete subspace of an inner product space then given any x there is a unique best approximation rno E M to x. If M is not complete, the result can fail , even for closed subspaces . If, for example, M M .L.L and x M.L ol - M, then there is no best approximation to x from M by the following argument . As shown in 3 . 1 .3, any such best approximation rn o M is such that x - rno M.L . Since x and rno belong to Mol.L , it follows that x - rno M.Lol . This leads to the contradictory result x - rno = Closed subspaces M such that M f. M.L.L are given in Example 3.2.6 and Exercise 9. The following example demonstrates the necessity of having a Hilbert space in 3 . 1 .3, 3 .2.4, and 3.2.5.
X,
E X, f.
E
E
E
E
0.
Example 3.2.6 No BEST ApPROXIMATION
X
There is an incomplete inner product space which has a closed proper subspace M such that (a) There is no best approximation in M to any x - M. (b) cl M M f. M.Lol . (c) M.L {O} but M is not dense in
EX
= =
X. 2 Discussion . Consider z = (1/i ) E £2 and let X be the subspace of £ 2 of sequences that are eventually 0. Since is dense in £2 ( the truncated sequences of any in £ 2 belong to and converge to x in £ 2 ), it is incom plete. Since the inner product is linear in the first argument, the nontrivial cp
x
0 , choose N such that l lYn - y ll < € / 3 for n � N. By hypothesis YN = L k E N (YN , Xk ) Xk , so there exists M such that n
L (YN , Xi) x ; - YN i= 1
< €/3 for n � M.
By the triangle inequality,
I L�= l (y , Xi) Xi - L�= l (YN , Xi) Xi I + II L�= l (YN , Xi) Xi - YN I I + l l YN - Y I I · (3. 12) By the orthonormality of the Xi and the Bessel inequality of 3 .3 . 1 we have I L�= l (y, Xi) Xi - y l n
0 there ex ists 8 > 0 such that for any n E N , for all E such that < < and < < 8, we
f: f'(t) dt
[a, b] a l b l ::; a 2 b2 ::; . . . ::; an bn ,
b
[a, b]
ai , bi [a, b] L:?= l (bi - a i )
168
4. Fou rier Series
2:7=1 I I (bi ) - I (a i ) 1
have < f. Since n could be 1 , absolutely contin uous functions must be uniformly continuous. We denote the set of absolutely continuous functions on by Prove the following:
[a, b] AC [a, b].
(a) PRODUCTS The product of absolutely continuous functions is absolutely continuous. (b) AC => U C Absolute continuity implies uniform continuity. (c) If satisfies a uniform Lipschitz condition, then is absolutely continuous. (d) If I has a bounded derivative on then is absolutely con tinuous on a , (e) AC => BV An absolutely continuous function is of bounded variation. (Note that this implies the existence of continuous functions that are not absolutely continuous such as x sin ( l /x) on By the remarks about functions of bounded variation after it follows that absolutely continuous functions have integrable derivatives: If is absolutely continuous on then is differentiable a.e. , and f' E The next result is in the converse direction . (f) INTEGRALS OF FUNCTIONS ARE ABSOLUTELY CONTINUOUS If then 9 (x) is absolutely continuous. (g) INTEGRATION BY PARTS [Stromberg 1 9 8 1 , p. By (e) and (f) of this exercise, absolutely continuous functions are dif ferentiable a.e . and have integrable derivatives. If and 9 are absolutely continuous on then
I
I
[a, b],
[ b].
(0, a).) 4 . 3.6
I
I
I E L'i [a, b],
Ll
I
[a, b],
LHa, b].
= f: I(t) dt
6.90, 323] I
[a, b], f: I (t) g ' (t) dt + f: f' ( t) 9 (t) dt = I ( b) 9 (b) - I ( a) 9 (a) .
Hints
(f (tn))
tn
f' (a + ) b.
a. a
1 . (a) . Show that is Cauchy whenever --+ Since exists, has a bounded derivative in r) for some < r < By Example 2.7. 1(d) it follows that is uniformly continuous in ( a , r) . Hence , by the discussion in Section 2.7, maps Cauchy sequences into Cauchy sequences. By the uniform continuity, if --+ and --+ limn then limn = (b) Define let --+ and apply the mean value theorem to on r] for some r >
I
I
(a,
I
I (tn) = I (Yn). I (a) I (a + ), tn a, 0. l(tn) [a, (a) . If I: (a + ) exists, show that I (a + ) I (a). Yn a,
2.
=
tn
a
4. Fourier Series
3. Consider the partitions [0 , 1] .
169
Pn = {O, 1/2n , 1 / (2n - 1) , . . . , 1/3, 1 / 2 , 1 } of
6 . (a) . Use the mean value theorem. (b) Use (a) o n each of the subin tervals. (c) Let I satisfy a uniform Lipschitz condition on b] . For . .< E [ , b] , [{
[a, Xn a ( I (X k + 1 ) - I (X k )) � (X k + 1 - X k ) . Xl < X2 < 7. (d) . For X < y , show that p ( y) - p ( x ) = V (I, X, y) - (f ( y) - I (x)) . Note that I ( y) - / (x) � V (j, a, x). 8 . (a) . an = t D" I(t) cos nt dt = n1" f:'" I (t) d (sin nt) . Integrating by parts , we get an = �; f:'" sin nt dl (t) . (b) Consider Example 4 . 1 .3. .
9 . (f) . Make use of the following property of the integral [Nat anson 196 1 , p. 148] : If I E b] then, for any > 0, there exists 6 > ° such that for any measurable set E C b] of measure less than 6, I (t) dt E l I
L'1 [a,
f
4.4
[a,
< f.
f
The Riemann-Leb esgue Lemma
We enlarge the class of functions for which we compute Fourier series from to in this section. For I E the products (t) sin nt l and If (t) cos nt l are each less than or equal to l l (t) l , so the integrals that define the Fourier coefficients still converge; therefore it makes sense to consider the Fourier series of such functions. One reason to consider the theory for functions instead of functions is that there are more functions: As noted in Exercise 1 , the space is a proper subset of b] for all > 1 ; indeed, generally, as p increases , the spaces decrease in size. We cannot really go beyond for if rf: 11"] , what sense could we make of the Fourier coefficients
L2 L1
L1
LH-1I", 1I"],
L2
p
II
L1 Lp [a, b] Lda, Lp [a, b] L 1 , I Ld-1I",
an = -11"1 1" f (t) cos nt dt? Another reason to enlarge the theory to L1 functions is that it is fairly easy to do. As noted in the Remark before Example 4.1 .4, the Fourier coefficients of a linear combination of L; functions are the corresponding _"
linear combination of the Fourier coefficients of the original functions. Since this result depends only on the linearity of the integral, it remains true for functions. Also, for functions the product Ig is in 71"] , since
L1
L2
2 41g = (f + g)
L1 [-7I",
-
(j - g) 2 E L1[-1I", 11"] ,
so we can now speak of the Fourier series of the product of two another convenience.
L2 functions,
170
4. Fourier Series
L1
Although we consider functions from now on , we do not consider con vergence in the norm. Rather, we consider various kinds of convergence: pointwise , uniform, (e, 1), and Abel convergence. Lagrange , who was so troubled by the erratic behavior of Fourier co efficients , would perhaps have been mollified by the Riemann-Leb esgue lemma 3 .3.2; it demonstrates at least that the Fourier coefficients of any go to 0, i.e . , E
L1
f L;[-7I", 71"] 11" an = 71"1 1 f ( t ) cos n t dt -
- 11"
11"
---+
O, and bn = -71"1 1-11" f (t) sin nt dt
---+
O. (4.2 1 )
Intuitively, as the oscillation of the sinusoids increases , it puts as much of the function's area above the axis as below ; we mention two more instances where the increasing oscillations of a kernel function drive an integral to 0 in 4.5.4 and Exercise 2( a) . For our development of the theory of Fourier series for functions, we generalize the Riemann-Lebesgue lemma to functions fE b] , -00 � � b � 00 , and to more general kernel functions than sinusoids; the key to 4.4 . 1 is the density of the step functions in mentioned in 2.4.5( d).
L'i LHa,
a
L'i [a, b] ,
4.4.1 THE GENERALIZED RIEMANN-LEBESGUE LEMMA For any f E -00 � ::; 00 , and any bounded measurable fun ction defined on R it follows that
Li:[a, b],
then
h
�a b
c
l c h (t ) dt
lim ! .... ± oo c 0
limoo
w-+
-+
0
( the averaging condition) ,
l b f (t) h (wt) dt = O . a
Remarks Sinusoids are bounded, measurable (indeed, continuous) func tions, and, using sin t (t) for example,
=h �I l c sin t dt l = I � (1 - cos C)I ::;
I� I
-+
It therefore follows from 4.4. 1 that for any f E lim
W -+ OO
0 as c -+ ±oo.
Li:[a, b],
l b f (t) coswt dt = lim lb f (t) sinwt dt = O. W -+ OO
a
a
L1
(4.22)
In particular, it follows that Fourier coefficients of functions ap proach Note , too, that oscillation is not the driving force for the conclusion of 4.4. 1 ; rather the averaging condition is. Clearly,
O.
h (t) = {
�:t , IItt II
> ::;
1, 1,
4. Fourier Series
171
h( )
satisfies the conditions of 4.4. 1 . As w - 00 , the graph of y = wt becomes a horizontal line the t-axis with a peak of 1 at t = 0 , it does not oscillate at all.
(
Proof.
b
)
I(
< a,
If a or is finite, define t ) to be 0 for t > b or t respectively, so that we can restrict attention to E L� (R) . Suppose that l [ c,d] is the characteristic function of C [0 , (0 ) . Then, letting x = wt ,
[e, d]
l1 ( )
100 l[c,d] (t ) h (wt ) dt
l
I
d
dw
-wI
o
By hypothesis ,
1
dw
() Thus, 1000 l[c, d] ( t ) h ( wt ) dt any step function g , J uW
0
1
h x dx - O and �
-
h wt dt
0 as w
l0
-
h ( x ) dx - -wl
cW
1
c
0
W
()
h x dx .
()
h x dx - O as w - oo.
00 . It follows by linearity that for
r oo 9 (t ) h (wt ) dt = O.
W -+ OO 10 lim
As noted in 2.4.5, the step functions are dense in L'i (R) . Hence, if C is a bound for h , then for any E L'i (R) and { > 0 , there exists a step function {/2C. For sufficiently large t - 9 t dt 9 such that {/2, so w , 1J000 9 t ) h wt dt
I l
(
10(00 I I) ( ) < l
I
( ) 1 � II I gil l
p � 1 , the exact opposite of what happens for the lp spaces. Does this remain true for infinite intervals? (b) There are functions I, g E L2[-7I", lr], such that Ig fi. L2[-7I", 71"]. 2. RIEMANN-LEBESGUE LEMMA VARIATIONS Let [a, b] be a closed in terval, and let I E Ll [a, b]. (a) Divide [a, b] into n subintervals of length (b - a) In, and define gn = ±1 on alternate subintervals . Show that for continuous I, limn co f: I (t) gn(t) dt = O. (You can use 4.4. 1 for this, but it is easy to give a more revealing elementary argument .) (b) OTHER ORTHONORMAL BASES Show that if ( gn) is a uniformly bounded (i .e . , for some M, I gn (t)1 � M for all t, and every ) countable orthonormal basis for L2[a, b], a, b finite, then nlim � oo ta I (t) gn(t) dt = O. 3 . ALTERNATIVE PRO OF OF RIEMANN-LEBESGUE Let I E Ll ( R) . (a) Show that lim6 o f�co I I (t + 6) - I (t)1 dt = O. (b) With g (w) = f�oo e iwt l (t) dt , without using 4.4. 1 , show that limw 9 (w ) = O. q
.....
n
.....
.....
o
Hints
L [a, b], split [a, b] into {t E [a, b] : I / (t)1 < I } U {t E : I / (t)1 � qPI}, and use the fact that I / (tW I / (tW on the latter . Since I / I � max { I , lf n , it follows that I /P I is integrable . Since q p � 1 , q p + for some O. Let 9 (t) = II (p + ) on
1 . (a) . For I E
[a, b]
>
>
c
c
>
�
c
(0, 1], 9 (0) = O. Then 9 E Lp [0, 1] but 9 fi. L q [0, 1]. The inclusion L q Lp for q � p does not hold on infinite intervals: Consider I (t) C
lit on [1 , 00) .
=
174
4. Fourier Series
2. (a) . Use the uniform continuity of f, and consider Riemann sums.
-g
3 . (b) . Since (w) = it follows that
f�oo eiw (t +1r /w ) I (t) dt = f�oo e iwt I (t - 7r/w) dt,
2g (w) = Therefore,
4.5
f�oo eiwt [I (t) I (t -
-
7r/w)]
dt.
Ig (w) 1 � � f�oo I I (t) - f (t - 7r/w ) 1 dt.
The Dirichlet and Fourier Kernels
This section paves the way for the pointwise convergence theorem 4.6.2 by providing certain ways to express the nth partial sum of a Fourier series (4.5. 1 ) . The device by which we accomplish this is the simpler way provided by equation (4.24) to express a sum of cosines as a ratio of two sines . In practice, Fourier series are always truncated to approximate a function as a finite sum of harmonics (t) = sin kt . cos k t Through the use of the nth DIRICHLET KERNEL
Bn
+ L � = l bk
ao/2+ L� = l a k
(4.23)
Bn
(see also equation (4.25) ) , we can theoretically calculate (t) in just one integration (4.5. 1 ) . Clearly, for n 1= 0, is an even 27r-periodic function . As illustrated in the figure below , (t) becomes more and more highly concentrated at t = 0 as n increases, becoming like a pulse 2n 1 units high on a shrinking pedestal of width 47r / (2n 1) . The shrinking base and increasing height are such that the area under the curve remains constant (Example 4.5.3) . (To sketch it, use the representation of equation (4.24) , and treat 1/ [sin (t/2)] as the envelope.)
Dn
Dn
+
+
The Dirichlet kernel for
n
=
3 and
n
=
7
4. Fourier Series
175
, Dn =
For t = 0 , ±21r, ±47r, . . . (t) 2n + 1 by substitution. To calculate # 0, we can avoid the summation by showing that for any n E N,
Dn (t) for t
sin ( n + t ) t , t # 0, ±21r, ±47r , . . . . sin (t/2) (4.24) To see this, add the equalities below for k = 1 , 2, . . . , n : _
1 + 2 cos t + 2 cos 2t + · · · + 2 cos nt -
sin
(k �) +
( �)
t - sin k -
t = 2 cos k t sin
�.
The sum on the left telescopes t o yield just the end terms:
which implies that
For t # 0, ±21r, ±47r, . . . , we can divide both sides by sin (t/2) to get equation (4 .24) . (An alternate derivation is suggested in Exercise 1 .) By I 'Hopital 's rule, for any 0, ±21r, ±47r, . . . ,
p=
. sin (n + l/2) t hm . sm ( t /2)
t -+ p
{
= 2n + 1 = Dn (p) .
Hence , the function defined as sin (n + l/2) t , t # 0, ±27r, ±47r, . . . , sin (t/2) otherwise, 2n + 1 ,
n E NU
{O} ,
(4.25)
Dn
is continuous and agrees with (t) as defined in equation (4.23) for all t. When we write in the sequel, we do so with the understanding that it has the value 2n + 1 when t = 0, ±21r, ±47r, . . .. Various expressions for the nth partial sum of a Fourier series are given in 4.5 . 1. As we shall see, as n -- 00, (t) behaves like Dirac 's 6 function in the integrals below , sieving out the value of the function f (t) when the argument of (t - ) is 0, i.e . , when = t; in other words, (t) -- f
Si:�:t;��)t
Dn
x
Dn x
Dn
Sn
(t) .
4.5 . 1 THE nTH PARTIAL SUM USIN G For any n E N U { O } , the n th partial sum of the Fourier series for f E L1 [-1r, 7r] is given by any of the convolution integrals below:
Sn
176
4. Fourier Series
-211'1 1_�� f(x) Dn (t - x) dx = -211'1 1_� f(x) Dn (x - t) dx, Sn (t) = (b) -211'1 1_�� f (t - x) Dn (x) dx = -211'1 1_� f {t + x) Dn (x) dx, � (c) 1 ior [f (t + x) + f (t - x)]Dn (x) dx. 211' Proof. The statements are all trivial for = 0, so we assume i- 0 in the arguments below . (a) The nth partial sum of the Fourier series of f is � Sn (t) = 211'1 1_ f (x) dx + � L:� = l cos kt I: f (x) coskx dx "" nk = l sin kt 1 � f (x) sin kx dx + 2:.11' L...-; [cos kt cos kX + sin kt sin kx] dx 1;: 1 � f (x) ( 21 + {; _� ) ) n ( 1 ;:1 1_� f (x) 2 + {; cos k (t - x) dx -211'1 1_�� f (x) Dn (t - x) dx -211'1 1_�� f (x) Dn (x - t) dx because Dn i s even. (b) With w = t - x, the penultimate equation becomes Sn (t) = -2111' I-t + � f (t - w) Dn (w) dw. t� Since f and Dn are each of period 211' , by 4.2.2 , the value of the integral remains the same as long as the range of integration is 211'; hence Sn (t) = 211'1 1_� f (t - w)Dn (w) dw. � With w = x - t, the equation Sn (t) = � 1 � f (x) Dn (t - x) dx becomes 211' _ � -t � Sn (t) = 211'1 1-� -t f (w + t) Dn (w) dw = 211'1 1_ � f (w + t) Dn (w) dw. � (c) We split the first integral 21 J::� f (t - x ) Dn (x) dx in (b) to get � 1 10 Sn (t) 211' _ f (t - w) Dn (w) dw + 211'1 ior f (t - w) Dn (w) dw. � (a)
n
=
n
4. Fourier Series
Now replace w by to get
177
-u in the first integral , and use the fact that Dn is even
sn (x) = 211"1 Jr [f (x + u) + f (x - u)] Dn (u) duo o We need the estimates of 4.5.2 to prove Dirichlet ' s theorem 4.6.2 about 0
the pointwise convergence of Fourier series.
4.5.2 Two UNIFO RM BOUNDS ON INTEGRALS (a) THE DIRICHLET KERNEL For any 0 :S :S b :S {O} , b sin n t t < 411". . a sm
NU
( + l /2) d ( t/2)
l
a
11" ,
a n d any n E
-
(b) THE CONTINU OUS FO URIER KERNEL For all real w and t , the qu a n tity cp (t , w ) = sin wt t for t 0, cp (O, w ) w , is called the CONTIN U O U S FOURIER KERNEL. We write sin wt t sometimes, but with t h e understan d ing that sin wt t is defined to be w when t O. For any 0 :S
0, and A k < 0, so for some E [0, 1] , 10a Dn(t) dt = Al +A2 + · · +Ak - l + cA k � Al +A2 + · ·+Ak - l � Al � 2 11". Consequently, for any ° � a � b � 1r, and any n E N u {O}, c
If k is even,
br
Ak
sin u duo A k = jk I 'II" -(-) u
The alternate in sign and decrease in absolute value-indeed, by a geometric argument as in a , It follows that � k , for each = ( 'II" Sl�. t for any n , 11", and therefore that
( ) I Ak I 1/ k E N. n dt < J E k = l Ak < Al o 1 '11" sin wt sin t 1 6 -< t d t - 0 t dt < 1r .
a
0
Example 4.5.3 CONSTANT AREA UNDER DIRICHLET KERN EL O N [0 , 11"] Sh ow that
1 '11" sin (.n + 1/2) t dt = 1 '11" Dn (t) dt = 11", for all n E NU {O} .
( t-/2) 0 Solution. Since fo'll" cos n t dt = ° for every n , o
sm
1o '11" Dn (t) dt = 1'11" (1 + 2 t cos kt ) dt = 1r. 0
k= l
0
4. Fourier Series
179
Dn
(t) = S i :�:t;Wt jumps up to 2n + 1 at t = 0, The Dirichlet kernel then stays at about constant amplitude between t = 7r / (n + 1 / 2 ) and t = 7r while becoming increasingly more oscillatory. We might expect that the oscillations of are enough to make
Dn
1,..,./.(n + l/ 2)
f (t)
Dn (t) dt
--->
0 as n ---> 00
f E Ll [- 7r, 7r] , and indeed this is what happens. 4.5.4 T HE RIEMANN -LEBESGUE PROPERTY O F Dn L e t Dn ( t ) th e Dirichlet kernel. For any f E Ll [ - 7r, 7r) , and r E (0, 7rJ , '" lim ! n f (t) Dn (t) dt = O.
for any
den ote
Proof. Since r > 0, then sin (t/2) � sin r/2 > 0 for all r S; t S; 7r. It follows r
f?j
E Ll [r, 7r) . Hence, by the Riemann-Lebesgue lemma 4.4. 1 , as that . sm t 2 ) n � oo ,
Definition 4.5.5 DISCRETE FO URIER KERNEL
Suppose we replace the sin (t/2) in the denominator of the Dirichlet kernel (t) = [sin (n + 1/2) t) / sin (t /2) by t /2, i.e . , consider
Dn
sin (n + l/2) t sin (n + l/2) t instead of t/2 sin (t/2) . . N U {O} , we call the contmuous function � n (t) =
sin (n + 1/2) t t/2 for t '# 0, �n (0) = 2n + 1, the DIS CRETE FOURIER KERNEL of order n . sin (n + 1/2) t . sin (n + 1/2) t . IS we mean that If we WrIte mstead of t/2 /2 defined to be 2n + 1 at = O. 0 For n E
t
t
�n ,
�n 2; �7, �7
The discrete Fourier kernel provides a very good approximation to the Dirichlet kernel for It I < and are sketched below. Eventually, (t) = sin the discrete Fourier kernel / (t/2) decreases more rapidly than but and are essentially indistinguishable until about = 2. (To sketch treat 1/ (t/2) as the envelope.)
Dn
D7,
�7 D7 �n (t),
D7 (7.5t)
t
4 . Fourier Series
The Dirichlet kernel
The Fourier kernel
Dr (t)
and
D7 (t)
7 (t)
r ( t)
=
=
�in 7 . 5 t sm(t/2}
sin 7 . 5t t/2
4. Fourier Series
181
AnotherFourier observation concerning the behavior similarityasbetween thein integral Dirichlets . The and discrete kernels concerns their kernels essentialby content of 4.Fourier 5 . 6 and kernel 4. 5 . 9 isas that you can replace the Dirichlet kernel the discrete n Let 1 E L1: [0, 7r] 4.5.6 sin (t/2) CAN BE REPLACED BY t/2 as n and r E (0, 71"]. Then, assuming that the limits exist, limn for I (t) sins�: �:;)2)t dt limn for I (t) sin (nt;21 /2) t dt. (4. 2 6) �i; -+ 00 .
=
sin(
In addition, the discrete Fourier kernel
-+ 00
/ 2)t
LEBESGUE PROPERTY : 11
has the ''RIEMA NN
o.
l� 1" 1 (t) sin ( nt721 /2) t dt Proof. By two applications of l' H 6pital's rule, lim..... o (_ sin1 t - �)t = Therefore, 9 (t) � / is continuous everywhere if we define it to beis the0 atproduct t = O. Itoffollows that 9 isfunction integrableandanda bounded boundedintegrable on [0, 71"] . function, Since Ig an integrable Igwithis integrable on [0, 71"] . Thus, by the Riemann-Lebesgue lemma 4.4 .1 h (t) sin ( n + 1 /2) t , l� 1 " [I (t) Cin (lt/2) - t;2 ) ] sin (n + 1 /2)t dt 0 , or, assuming "that the limits exist, + 1 /2) dt = limn 1" 1 (t) sin (n + 1/2) t dt. limn 1 1 (t) sinsm�n (t/2) t/2 By an identical" argument + l/2) dt = limn " l (t) sin(n + 1 /2)t dt. limn 1r l (t) sinsm�n (t/2) t/2 1r By 4. 5 . 4 , however, limn Ir" 1 (t) :(;(i/N dt 0, and the desired result follows. For what 1 do the limits of equation (4. 2 6) of 4. 5 . 6 exis t? What is the limit? For certain 1 (piecewise smooth, for one type) the limi t is 7r 1 (0 + ) . This fact is the essence ofthese the pointwise convergence theorems 4. 6 . 2 andon 4.integrals. 6 . 6 . BeforeIn dealing we consider questions, we consider some variants with infinite series, wesomething allow absolutely convergent and nonabsolutely convergent ones. We do similar for integrals for more flexibility. =
o.
t
=
s i n( /2) - t 2
=
=
0
0
0
si
i
)
=
182
4. Fourier Series
-+ , pv , a < b 1a -6 1-+6a 1-+ a 6 1a b we say thatconvergence. f is integrable on [a, b], we mean that f: I f (t)1 dt < i. e-If. , absolute If f is integrable, f+ (t) = max:(f (t) , 0), and f (t) = min (f (t) , 0), then f = f+ -f - , and f: f(t) dt f: f+(t) dt f: F(t) dt. If we abandon absolute convergence and demand only that f be integrable on [a, c] for all c < b and that li c-+ f: exist, then we represent the limit as fa-+ b • Similarly, b and -+ = lim d = lim 1 1-+ba c-+ a c 1-+ a 6 c-+ a , d--+ 1c b kernel sin tit is not The distinction is important. For example, the Fourier in L'i (R+) since (4.48) ) r oo I sint t l dt ( eqUation of Section 4.9 ' Jo but the DIRICHLET INTEGRAL - 00 sin t 1o t dt 11"/2 (4. 5 . 8 (b )) . If f is integrable on [a, b], then 1a b f(t) dt = 1a -+ b f(t) dt = 1-+ba f(t) dt. For c E ( a, b) , the CAU CHY PRINCIPAL VALUE of f: is defined to be ( -€ PV { b lim r + 1 ) . Ja €-+ o Ja c+6 € By this convention, even though neither f'::' r 3 dt nor r:o t-3 dt exists, °O -€ PV 1 t-3 dt = lim ( 1 r 3 dt + J oo r3 dt ) = lim ( -� + -4) = 0, €-+o 2f 2f €_ o - 00 - 00 € whereas if we take 2f in the second integral, we get -1 --1 ) ' hm. ( e-O 2f 2 + 2 (2f) 2 sob, then the symmetric approach is crucial. Likewise, if a Cl < C2 . . . < Cn ::; Definition 4.5.7 IMPRO PER INTEGRALS
,
,
- 00
::;
::;
00 .
00 ,
=
-
m
6
= 00
=
=
= - 00
::;
0, and choose r E (0, ) such that f (u) - f (0 + ) < 0/2 on (0, r] . Now split I; : sinwu .!.7r r [f (u) - f ( o + ) ) u du io -+
s
Since integrable, [f (u) - f (0+ )) /u is integrable on ( r, ) and it follows from thef is Riemann-Lebesgue lemma 4.4. 1 that I I: I < 0/2 for sufficiently + large w. Since (u) ) is f -f ( 0 an increasing, nonnegative function of u, and sinwu on value [0, r) when defined to be w at (hence integrable /utheis continuous there), second mean theorem (4.6.4) implies that for some a E (0, ) r .!. r [f (u) - f ( O + ) ) sinwu du '!' [f ( ) - f (O + )) l sinwu du o u 7r 7r io U Since I I: sin"w" dul -s 7r by 4.5.2( b ) , it follows that s ,
°
r
,
=
r
a
The exact same steps as beforeconvergence lead to equations (4.32)-(4.34) , and ul timately to Jordan' s pointwise theorem 4.6.6. 4.6.6 POINTWISE CON VERG ENCE FOR BY If f is the 27r-periodic exten sion of a function of bounded variation on [ - 7r , 7r) ( which implies that f is integrable on [ -7r , 7r) ) , then for any t, its Fourier series converges to
� (J (C) + f (t + )) . The twentieth century has seen a dramatic enlargement in the class of functions f whose Fourier series converge pointwise to f. Carleson [1966) proved with no smoothness assumptions that the Fourier series of any 1 E L 2 [-7r, 7r) converges pointwise a. e . to f (t) . Hunt [1968) extended Carleson ' s result tocanallbe1 found E L [- 7r , 7rJ, 1 < < 00. Proofs of Carleson' s and Hunt' s results in Mozzochi 1971 and J!Ilrsboe and Mejlbro 1982. In Fejer showed (4.15.3) that for any f E Ld-7r, 7r) at any point t at 1904 which 1 (t - ) and 1 (t + ) exist, the Fourier series for f is (C, 1) summable to [/ (t - ) + f (t + )) /2. p
p
4 . Fourier Series
195
Exercises 4 . 6
1. S U2 M S O F INVERSE SQUARES Prove Euler's formula: l:n E N 1/ n 2 11" / 6 . 2 2. SUMS . . . = 11"O2F/ 8IN. VERSES O F SQUARES Show that 1 + 1/3 + 1 / 5 2 + 3. wise PIE CEWISE SMO OTHNESS Which of the following functions is piece smooth? (a) I(t) = { sin0, 1/t, t0 =< O.t :::; 1, tsin 1/t, t0 =< Ot . :::; 1, (b) I (t) = { 0, (c) I(t) = { 0,1, tt rational, irrational. { t2 0 :::; t < 1/2, 4. Let I(t) = 0, ' t = 1 /2 . Compute f' (t + ) , and f' (r) · -Ii, 1/2 < t 1. 5. periodic SAME FO URIER COEFFICIENTS If I and 9 are piecewise smooth 211" that have the same Fourier coefficients, then show that I (t)functions = 9 (t) at all t where they are both continuous. { 6. Let / (t) = 0,In(� /t) ' 0- 11"< t :::;t :::;1I" , O. . :::; (a) Show that:::; I satisfies the conditions of the Jordan theorem1I", 4. 6 .6 for-11" -11" a0 < 0 < b :::; i.e ., that I E BV [a, b) LH- 11"] for :::; a < < b :::; (b) Show that [I (u) + I ( - u)] /u f!. LHO, r] for any r 0, i. e ., that I does not satisfy the condition [f (t + u) + I (t - u) - 2/(t)] /u E L1[O, r] of Dini's test, 4. 5 .12, at t = O .
=
ODD
,
11" .
:::;
C
11" ,
>
7 . D E L A VALLEE-POUSSIN CRITERION F O R POINTWISE CON VERG EN C E
(
a) Let extended, let E R, and let I E AILH-(x,1I",t)1I"]=beI 211"+-periodically (x t) + I (x - t) - 28 , x, t E R. Fix x E R. If (x, t) t A 8
v
=
f; / (x, u) du
196
4. Fourier Series
is ofbj forbounded as a function0 oft t in0a, then closedshowinterval some avariation 0 , and (x, t) that [a, the Fourier series of f converges pointwise to s at x. 's (b) pointwise Use the deconvergence la Vallee-Poussin criterion of (a) to deduce Jordan of bounded theorem 4.6.6 for functions f variation. (c) Use the de la Vallee-Poussin criterion to deduce Dini's test 4 .5 . 12 . 8 . FIRST MEAN VALUE THEOREM Let 9 be continuous, and let f be increasing interval [a, bj. Prove that there exists c E [a, bj such that on the closed J: 9 (x) df (x) = 9 ( c) (f (b) - f (a)) . 9 . SE COND MEAN VALUE THEOREM I Let 9 be continuous, and let f be increasing on [a, bj. (a) Prove that there exists c E [a, bj such that >
v
->
as
->
J: f (x) dg {x) f (a) J: dg (x) + f { b) J: dg { x) . Show that there exists c E [a, bj such that J: f (x) 9 (x) dx = f (a) J: 9 (x) dx + f (b) J: 9 (x) dx. =
(b)
increason ing function on the closed interval [a, b],Letand9 belet afcontinuous be nonnegative [a, bj . (a) Prove 4.6 .4, namely, that there exists a point c E (a, b) such that J: f (x) 9 (x) dx = f (b) J: g{x) dx. ( b) If f is decreasing on [a, b], then show that there exists a point c E (a, b) such that J: f ( x ) g {x) dx = f (a) J: g (x) dx. (c) Show that (a) or (b) can fail if we omit the requirement f � o. 1 1 . SE COND MEAN VALUE THEOREM III Let 9 be continuous, and f be increasing of Exerciseson9 (theb ) , closed and 10.interval [a, bj. Prove the following extensions (a) Show- thatd, forthereanyexists real numbers and d such that c ::; f (a + ) ::; r E [a, bj such that 10 . SECOND MEAN VALUE THEOREM II
c
f (b ) ::; J: f {x) g (x) dx = c J: g (x) dx + d t g {x) dx.
4. Fourier Series
(b)
197
If f � 0, then show that there exists a point r E [a , b] such that J: I (x) g (x) dx = d J: g (x) dx.
Consider the trigonometric series (4.35) S (t) = � + L: an cos nt + L: bn sin nt, neN neN where are real sequences. Now consider the power senes ( an)n � o and (bn)
12. CONJU GATE SERIES
(
4 . 36 )
(a) Show that for
cn _- { aoan/2- , ibn , nn =� 01 , ' ,and z = eit , the real part of L: �= o cnzn is (t), and the imagi n nary part of L:�= o cnz is THE CONJUGATE SERIES O F (t), (t) = L: an sin nt - bn cos nt. n eN If the series in equation (4.35 ) is the Fourier series of 1 E s
s
0"
Li [ - 7I", 7I"] ,
so that
an = .; D,. I(t) cos nt dt, n E N u {O} ,
and bn = .; J�,. I(t) sin nt dt, n E N , then( , t),werespectively; will denote wethedenote series the(t)nthandpartial(t) sums by (f,of t) (f,andt) andf (f, t) by Sn (f, t) and O"n (f, t), respectively. (b) Let dn (t) = L: � = 1 sin kt, and note that dn is odd. Show that s
0"
0"
s
O"n (f, t) = -./ D,. I(x )dn (x - t) dx,
and also that
O"n (f, t ) = -,.l Io" ( f (t + x) - / (t - x)) dn (x) dx.
( c)
s
0"
Show that and
dn (t) -- cos t / 2-2 sinco(st(/n2+) 1 /2 }t dn (t) - 12-tancost /n2t + 2 · lli!..!!.1
( 4.37 )
( 4.38 )
( 4.39 ) ( 4.40 )
198
4. Fourier Series
(d) Show that there is some constant c such that "./2 Sl. � 2 n t dt -< e ln n . sm t
Jro
(e) Show that there is some constant k such that
� J�". l dn (t) 1 dt = � Jo"' l dn (t) 1 dt �
kin n . (f) pleted Prove thean earlier followingresultresultof Fejer due to(Zygmund Lukacs (1920) , which com 1959, p. 107) . The presentation here .follows Bary 1964. Let f E LH-7I", 7I"] be peri odically extended If has a jump discontinuity at t , and f (t + ) f + ( and f (t- ) exist, let d = f t ) - f (r). Then show that limn UnIn(t,n t) = -71"d .
Hints
4
2
The Fourier( cosseries for f (t) = t 2 on the closed interval [0 , 271"] is ; + 4� LmeN n 2nt ". sinn n t ) . Consider the expansion at t = 271" 2. By Example 4.1 .5, the Fourier series for f (t) = It I on [-71", 71"] is � ;. ( cost + '; + o; + . . .) . Now set t = O. 7. (a). The Fourier series of f converges at x to s if li![l J; D"f (x, t) sin(n� 1/2)t dt = 0 for some 0 < r < 71" by 4.5 . 10. Now, A f( x, t) - 8(tv(x,t)) 8t - v t 8v8t almost everywhere. Thus J; v (x, t) si n(n � 1/2)t dt J; � f (x, t) si n (n � 1/2)t dt in( n l/2)t + Jrro t 8v 8t s +t dt . Toof bounded treat the variation, first integralandJ;v v(x,(x,t)t)-+sin(0n�as1/ t2)t dt ,O.noteNowthatarguev(x ,ast) inis the proof of Jordan's pointwise convergence theorem 4.6.6. For the integral lr 8v sin (n + 1/2) t 1.
_
C 23t
C 25 t
+
u
-+
t ut o !:j
t
dt ,
note thatandsince v (x, t) is of bounded variation, �� exists almost every where is integrable. Now argue as in the proof of Dini's theorem, 4.5. 12.
4. Fourier Series
199
(b)(f . Since f is of bounded variation, �f (x, t) = f (x + t)+ f (x - t ) (x - ) + f (x + )] is of bounded variation in [a, b] , a 0, for any x E [-11", 11"] . It follows that v (x, t) is of bounded variation in an to the right oft = 0 (prove). Now show that interval lim t f; (f (x + t) + f (x - t) - f (x + ) - f (x - )) dt = 0 By(f the+ ) result of (a) , the Fourier series of f converges at x to (x) = (x + f (x - )) /2. ( c ) . If �f (x, t) /t E Ll [O , r], then h (x, t) f; A f C: . u ) du is abso lutely continuous on [0 , r], and therefore is of bounded variation there. Now consider >
t-O
s
=
v (x, t) = t f; ud� h (x, u) du h (x, t) - t f; h (x, u) duo
=
8.
In thisfunction hint and the one= cforonExercise that any USe the 9factdefined stant [a , b] and9(aany) Wefunction on [con a , b] f (x) c the Riemann-Stieltjes (x) is equal to ( g (b) 9 (a)). sup f9:(f(x) Let inf 9 ( [a, b]), Mintegral [a , b] ).dgThen -
m =
=
m (f (b) f (a)) ::; f: 9 (x) df (x) ::; M (f (b) f (a)) . Since the result and observe thatis trivial if f (b) f (a), suppose that f (b) :/; f ( a), f ( b ) �f ( a ) f: g (x) df (x) -
-
=
is an intermediate value of the continuous function 9 on [a , b]. 9. gration ( a). Integration by parts is justified by Exercise 4. 3 -9(g). Using inte by parts and Exercise 8, f: f (x) dg (x)
10.
f (b) g (b) - f (a) g (a) - g (c) (f (b) - f (a)) f (a) f: dg (x) + f (b) f: dg (x) . ( b ) . Consider the continuous function h (x) f: g (t) dt, and USe ( a) . ( a) . Let w (x) a < x ::; b, and let w (a) 0, and USe Exerci se 9(b) withf (x)w inforplace of f. (c ) Let f (x) g (x) x on
=
=
=
== =
[-1, 1]. 1 l . ( a) .. Let w (x) = f (x) for a < x < b, w (a) c, and w (b) = d. Now use Exercise 9 (b) with w and g. Note that this result is true eVen for 9 E L l [ a, b]j cf. McShane 1947, p. 210. =
200
12.
4. Fourier Series
(b) . The result of equation (4.37) uses arguments like those of 4.5 . 1 . As for equation (4.38) , let in equation (4.37) . Then
w x-t -.,.11 r::;� t f( t + w)dn (w) dw -.,. f::1r f(t + w)dn (w) dw -.,.1;f�,'lt J(t + w)dn (w) dw - fo f( t + w)dn (w) dw.
=
w
Now let = - u in the first integral, and use the fact that (c) . See the hint to Exercise 4.5-6 .
dn is odd.
1::;= 1 sin ( 2k 1 ) t Si�� � t Hence /2 2 1::;= 1 fo'lt /2 sin ( 2k - 1 ) t dt J('Ito Si�smtnt dt �n L- k = l 2k 1 1 :::; �n L- k = l k1 :::; c In since 1:: ; = 1 1 j k is asymptotic to In in the sense that [1::;= 1 1 j kl / In 1 as � 00 (look at lower sums for In n l j ( ) fI t dt ) (e) . Since 1 - 2sin(t/2) 1 - - 1 tan t 2 tan(t/2) is 0 ( 1 ) for t E [0, 11"] , it follows from part (c) that 1 - cosnt dn (t) + sin2nt 2tan( /2) t - cosnt + 0 (1) ( 4.41) 2ll tan(t/2) cosnt + 0 ( . 1) 2sin(t/2) Thus i �2 ( nt/ 2) + 0 ( 1) , l -:-cosnt t /2) + 0 (1) s sm(t/2) 2sm( =
-
(d) . By Exercise 4.5-5 ,
.
n,
_
n
.
n
n
�
n
_
With
2 (n t /2 ) 0 1 ) dt + ( Jo('It I dn (t) 1 dt 1r Jo('It smsm(t/2) =
1.
w t j 2 this becomes 2 nw dt 0 ( 1 ) . /2 'It Si� ( 'It Jo sm w +
=
.
1.
Now use (d) . (f) . By assumption, f )-f ) as By equation (4.38) we have �
x 0+ .
4"
=
and it follows that 1. 'It
"2
(t + x
=
(t - x d + f ( x ) where f ( x ) 0 �
4. Fourier Series
Show first that
n n n ];0" dn(x) dx
lim , I
_
=
Observe that
l.
( 4.43)
n -kk",- I 0" - - L.. " dn ( x ) dX -- - � � nk = 1 ( cas -kk-" L.. k = 1 cas
Jfo
_
For m = (n - 1) /2 this becomes 2 1 + 1 + . . . + 1_
(
3
201
-
1)
k
.
_ ) 2m + l
which is asymptotic to
2 (ln m - t ln m) = ln m .
In turn, this is asymptotic to In n , which establishes (4.43) . Next , we show that lim , 1 ];0" ( 4.44) =
n n n dn(X) f (x) dx O. For any r > 0 there exists 0 > 0 such that I E (x) 1 < r for 0 < x < o. Hence , by (e) , there is some constant k' such that _
To consider f.s" By equation (4.39) �
x 1r.
dn(X) f (x) dx we must get a bound on dn (x) for 0 �
I dn (x) 1 � � for 0 < 0 � x < 1r. Since sin x � 2x/1r for 0 � x � 1r/2, it follows that I dn (x) 1 � 1r/x for O < o � x < 1r � 1r /0 for 0 < 0 � x < 1r. sin / 2
Therefore,
fo" dn(X) f (X) dx � f fo" If (X) 1
f
[-1r, 1r).
dx,
which is finite, since E L� This establishes equation (4.44) . From equations (4.42) , (4.43) , and (4.44) , it follows that lim �
n In n
=
-
.4.
"
.
202
4. Fourier Series
4.7
Uniform Convergence
Since sines and cosines are continuous, if a trigonometric series converges uniformly, its limit must be continuous. If a function f is discontinuous, its Fourier series cannot converge uniformly, so this is an easy way to make up convergent series that do not converge uniformly. Knowing when a se ries converges uniformly is imp ortant, because it enables termwise integra tion and with more hypotheses termwise differentiation . By Dirichlet 's pointwise convergence theorem 4.6 .2 we know that the Fourier series of a periodic, piecewise smooth, continuous function f converges pointwise to f everywhere. We can do much better than pointwise convergence, how ever. We show in 4.7.4 that the Fourier series of such a function converges uniformly. We show in Example 4 . 1 1 . 1 that not every convergent trigonometric senes sin + cos +
(
)
�
L an nt L bn nt neN n eN is a Fourier series. ( L nEN sin ntl In n is the counterexample .) Theorems
4.7 . 1 and 4.7.2 provide sufficient conditions for a convergent trigonometric series to be a Fourier series.
4.7.1 UNIFO RMLY CON VERGENT TRIG ONOMETRIC SERIES ARE FO URIER SERIES If the trigonometric series
�o + L an cos nt + L bn sin nt n EN neN converges uniformly on [-11", 11"] , then the limit is a (211"-periodic) continu ous function whose Fourier coefficients are an and bn .
(t)
o / Ln EN an nt L n EN bn nt
Proof. Clearly, 9 = a 2+ sin is continu cos + ous because it is the uniform limit of a series of continuous functions. Since it converges uniformly, given c: > 0, there exists a positive integer N such that for m � N ,
k b e a nonnegative integer, and consider the series 9 (t) cos kt �o cos kt + L an cos nt cos kt + L bn sin nt cos k t . ( 4.45 ) nEN n EN
for all t E [-11", 11"] . Let
=
Since
Ig (t) cos kt -
Sm
(t) cos kt l
::;
I g (t) - Sm (t) 1
0, the continuity of / enables us to choose r > 0 such that 1 ( 1 (t + x) - / (t)) 1 < f and 1 - (I (t) - / (t - x)) 1 < f for t E [a + r, b - r] and 0 ::; x ::; r' . lt follows that I f;' � [sin ( + 1/2) x]A / (t, x) dx l is uniformly small for all t E [a + r, b - r] for sufficiently small r. For fixed r, show that 1 1� � [sin ( + 1/2) x] A/ (t, x) dx l is uniformly small for t E [a + r, b - r] when n is sufficiently large .
1 . Assume that
n
n
4. Fourier Series
4.8
207
The Gibbs Phenomenon
By 4.7.4(b) , the Fourier series of a piecewise smooth function f converges uniformly to f on any closed subinterval not containing a point of discon tinuity of f. How does the series behave on intervals containing points of discontinuity? In an 1899 letter to Nature (vol. 59, p. the American mathematician Josiah Willard Gibbs, famous for the development of vector notation among other things, considered the function
606)
h (t) = with Fourier series
{
-1r - t -1r ::; t < 0, 0,1r -2 t t = 0, 2 ' 0 < t < 1r, L
n EN
h
sin n t n
.
The function and the seventh partial sum in the figure below.
S7
of the series above are shown
of immediately to the right of 0 is a bit higher than The peak hfact,(0+) = 1r /2; on the left , the minimum of is below h (0 - ) = -1r /2. In i + (. 0 9)1r = i + ( . 0 9 ) (h (0+) - h (0 - )) , that is, overshoots h (0+) by about 9% of the total j ump 1r. A similar thing happens to the left of O. Not only that, this overshoot persists even as 00, despite the pointwise convergence of n (t) to h (t) throughout [-1r, 1r] Unknown to Gibbs, Henry Wilbraham had made similar observa . tions 5 1 years earlier ( Camb. and Dublin Math. 3 ( 1 848) , p . 198) . The American mathematician Maxime Bacher went considerably farther than Gibbs or Wilbraham in 1906 (Annals of Math. 7) . He showed that this 8 7 m ax
87
87
S 7 m ax �
87
n
-->
S
J.
overshoot of Fourier series by 9% of the total jump is a general property
208
4. Fourier Series
of Fourier series in the vicinity of a jump discontinuity. Thus, even though Gibbs's statement was not the first, not accompanied by any proof, and dealt only with a special case, this quirk of nature is universally known today as the GIBBS PHEN OMENON . Such are the vagaries of history. Consider what happens to a step function.
-4
4
The Gibbs Phenomenon
4.8.1 GIBBS PHENO M ENON FOR A STEP FUN CTIO N
Suppose a
n� 2 -
-
1
11"
= 00 .
( 4.48 )
213
4. Fourier Series
Hence, the terms o n the right of inequality (4.47) g o t o infinity as n � 00, and the argument i s complete.
4.10
Termwise Integration
2:n EN fn of integrable functions with limit f defined on [a, b],
For general series the closed interval if
L fn = f uniformly, then
n EN
la b nLEN fn (t) dt = nLEN lab fn(t) dt . f (t) t/2 [
But consider the 211"-periodic extension of = on -11" , 11"] . It is inte grable , and of bounded variation on [-11", 11"] . Therefore , by Jordan's theorem 4.6.6, for any (-11", 11") ,
tE
t=
L nEN
( _ 1 )n + 1
sin n .
( 4.49)
The series converges to 0 for t = - 11" , 11" . For what sides from 0 to to get
t can we integrate both
2
n
t
t
t2 = 4
( - 1 t + 1 ( 1 - cos t) ? L 2 nEN n n
Since the series of equation (4.49) has a discontinuous limit, it certainly does not converge uniformly on [-11", 11"] . It is therefore not clear that termwise integration is permissible . In a case like this where the limit of the series is known, and the function is easily integrated, the ability to integrate term by term is just a curiosity. But suppose that all we know about a function is its Fourier series. Then the ability to integrate termwise is much more imp ortant. In the case of Fourier series, termwise integration abets convergence be cause it puts n 's in the denominators of the terms in the series (differentia tion puts n's in the numerators) . We prove in 4.10.1 that the Fourier series of any f Lt [-1I", 1I"] may be integrated term by term; not only that , but the termwise integrated series converges uniformly and is the Fourier series of the integrated function. Before we prove 4.10 . 1 , recall a few things about absolute continuity that were first mentioned in Exercise 4.3-9:
E
1. A function f defined on the closed interval [a, b] is ABS OLUTELY CON TINUOUS on [a, b] if for every > 0 there exists 6 > 0 such that for any n E N , for all a i , b i E [a, b] such that a 1 < b 1 � a 2 < b 2 � . . . � an < bn and 2:7=1 (bi - ai ) < 6, we have 2:� l l f (bi ) - f (ad l < f. (
214
4. Fourier Series
n
Since could be 1 , absolutely continuous functions must be uniformly continuous.
2. An absolutely continuous function is of bounded variation. Since any function 1 of bounded variation has an integrable derivative f' [see the remarks after 4.3.6] , if 1 is absolutely continuous, then f' Li [a, b].
E
3. If 1 E L'i [a, b], then g is absolutely continuous, and = J: g' = 1 a.e. [Exercise 4.3-9(f) , Natanson 196 1 , pp. 252-3] .
I(t) dt
(x)
4. 10.1 TERMWISE INTEGRATED SERIES CON VERGE UNIFORMLY
E LH-7r, 7r] have the Fourier series �o + L: an cos nt + L: bn sin nt , n
27r-periodic extension of 1
eN
Let .the
neN
which may or may not converge pointwise. Then: bn converges. (This (a) The sine coefficients bn are such that L: n N e n has an interesting application in Section 4. 1 1 .) (b) The Fourier series for J: 1 is obtained by termwise integration of the Fourier series for f, namely,
(t) dt
an . - bn cos n x ) , ao-x2 + L: bnn + L: ( -smnx n n eN n eN n and it converges uniformly to J: 1 (t) dt on R. (c) For any finite interval [a, b], l b I(t) dt = �o (b - a) + L: [� (sin nb - sin na) - � (cos nb neN -
Proof. By Observation
-
3 before the theorem,
-
cos
na)] .
(4.50)
F (x) lx (/ (t) - �) dt is absolutely continuous. Therefore , F E BV[-7r, 7r] LH-7r, 7r] . It is 27r periodic, since F (x + 2 7r) = Jro (/(t) - �o ) dt + lx+ 21< (/ (t) - �o ) dt F (x) + 1 1< (I (t) - �o ) dt [4. 2 .2] F (x) + . l� 1 (t) dt - 7rao = F (x) . =
C
1
0 mono tonically decreases to 0 and Il:�=l Xi 1 � for all then l: neN X n Yn "n cos(22 n-l)t converges. U se thOIS to prove t h a t L.J eN ( n - l) converges pomtwise for all t =P ±h, k E N. [Hint: Show that
M
n
k'fl
cos (2 k -
n,
•
1) t = s��i�n/ for t =P ±h, k E Nu {O} .]
5. Let f (t) = - In 12 sin (t/2) 1 . Show that :
fE
(a) LH- 7I" , 71"] . (b) - ln I2 sin (t/2 ) 1 = cos t + � + � + . . . for t ( c) Use (b) to show that In 2 1 t + i- � + . . . . (d) Use (b) again to show that
c 2 t cO 3 t = -
In 1 2 cos U) 1 = cos t for t
=P 2 (k + 1 ) 71", k E Z .
co; 2t + c�3t -
=P 2h, k E Z . •
.
.
218
4. Fourier Series
t
(e) Use (b) , and (c) to prove that for 0 < < 7r,
'" cos (2 k + l )t - .=.l In (tan t/2) 2 ki..J=0 2 k + l 00
-
•
6 . Use 4.10.1 to show that
sin n t - t n ln n
--
and
r
12
oo
In
n+l
dt t In t
dt = lim In (ln t) I � = oo . I t n t b _ oo
The series L: n > 2 1/(n ln n) diverges. Therefore L:n > 2 sin nt/ ln n is not Fourier series. -0
a
As an indication of the delicacy involved here we mention (Stromberg 1 9 8 1 , p.516) that even though L: n > 2 sin nt/ In n is not a Fourier series, L:n > 2 cos nt/ In n is! In 1875 P. du B ois-Reymond showed that if a trigono metric series converges to a Riemann-integrable function f in (- 71" , 71") , then the series is the Fourier series for I n 1 9 12 de la ValIee-Poussin improved the result to Lebesgue-integrable functions:
f.
If a trigonometric series converges pointwise everywhere to a finite-valued function, then the series is the Fourier series of the function .
Ll
(The series L: n > 2 sin ntl In n is therefore not integrable on ( - 71" , 71" ) . ) For the du Bois-Re ymond-de la ValIee-Poussin theorem, see Natanson 1961 , vol. II, p . 52, Bary 1964, pp. 326 and 382; or Rees, Shah, and Stanojevic 1 9 8 1 , pp. 209-2 12. For series with monotonically decreasing coefficients, we have (Rees, Shah , and Stanojevic 198 1 , p. 217) the following result . 4 . 1 1 . 2 If (an ) is a sequence of posahve numbers that decreases mon oton ically to 0, and
f (t) = �o + I: an cos nt , and n eN
9 (t) =
I: an sin nt ,
neN then the cosine series is a Fourier series if and only if f E L l [- 71" , 71"] . The analogous statement holds for g .
The fact that not every everywhere-convergent trigonometric series i s the Fourier series of its sum was considered to be something of a defect of the Lebesgue integral, and this motivated Denjoy to generalize the Lebesgue integral to the "Denjoy" integral. Once again , Fourier series motivated mathematicians to expand the concept of integral.
4 . Fourier Series
4.12
221
Termwise Differentiation
By equation (4.49) of Section 4.10, the Fourier series for the periodic ex tension of I (t) = t on [-11", 11"] is
2
L n eN
( _ l) n + l sin nt . n
Termwise differentiation yields
2
(-It + 1 cos nt. L neN
Since cos nt f+ ° for any t E [-11", 11"] as n --+ 00 , the latter series does not converge, so termwise differentiation fails. This I, however, has disconti nuities at odd multiplies of 11". Sufficiently smooth functions have termwise differentiable Fourier series. 4.12.1 TERMWISE DIFFERENTIATION Let f be a continuous, piecewise smooth 211"-periodic function on all of R with Fourier series
�o + L an cos nt + L bn sin nt. neN neN
If f' is piecewis e smooth, then the series can be differentiated term by term to yield the following pointwise convergent series at every point t: I'
(t + ) + I' (t - ) = L (nbn cos nt - nan sin nt) . 2 neN
Proof. By the uniform convergence theorem 4.7.4 for piecewise smooth func tions, the Fourier series for I converges uniformly and absolutely to I (t) on [-11", 11"] . The cosine coefficients of f' are
Co = -11"1 j 1r1r I' (t) dt = -11"1 [J (1I") - f (-1I")] = 0, -
and for n � 1 ,
=
I-; j-1r1r f' 1
-; f (t) cos nt n bn
·
1 1r- 1r
(t) cos nt dt +
; j1r f (t) sin nt dt - 1r
dn - nan .
Similarly, the sine coefficients of f' are given by = The Fourier nb (cos nt - nan sin nt) of f' therefore converges pointwise series by the pointwise convergence theorem 4.6 .2. 0 to (I' (t+ ) + I'
Ln eN n (t )) /2
222
4. Fourier Series
The rate of convergence of a series is directly related to how rapidly the terms of the series approach In order to compare rates of convergence, the following terminology is helpful.
O.
Definition 4.12.2 SMALLER ORD ER
9
9
We say that (n) is of SMALLER ORDER than h (n) , or (n) goes to 0 0 , and we write o ( h ) ; "g o ( h)" faster than h (n) , if limn-+oo " is pronounced g is little oh of h." 0
���� =
9=
= an = /' .
Let / be a piecewise smooth continuous 27r-periodic function with Fourier coefficients an and bn . We saw in the proof of 4.12.1 that dn l n and cn l n , where cn , and dn are the Fourier coefficients of By the Riemann-Leb esgue lemma 4.4. 1, � 0, and dn � Hence , for a piecewise smooth continuous periodic function, nan = a n i (lin) � In other words, � 0 faster than lin; similarly, b � 0 faster than lin as well; equivalently, the Fourier coefficients of a piecewise smooth continuous periodic function are of smaller order than lin. (More generally, this is true for absolutely continuous functions; see Exercise 4.12-3.) If / is smoother, then its Fourier coefficients approach 0 even more quickly:
bn
=
O.
Cn
n
an
O.
4.12.3 SMO OTHN ESS AND SPEED O F CONVERGENCE Let / be a con tinuous 27r-periodic function with Fourier coefficients an and bn. If for k E N , /' , . . . , /( k - l ) are continuous and /(k) is piecewise continu ous, then nkan 0 and nk bn 0 as n � 00. �
�
Exercises 4 . 1 2 1 . Assume that /, /' E L�;[-7r, 7r] with
J::'". / (t) dt = O. Show that
2. SQUARE SUMMABILITY IMPLIES ABSOLUTE CON VERGENCE If a sequence cn goes to 0 faster than 1/ n 2 , i .e . , limn 1/�2 0 , show that
E n eN Cn converges absolutely.
=
3. MAG NITUDE OF FOURIER COEFFICIENTS Let / be a 27r-periodic function. (a) If / is absolutely continuous with Fourier coefficients an and bn , show that = o ( l / n ) and bn = o ( l / n ) . (b) Let If /(k - l ) is absolutely continuous, then show that an (link) , and bn (link) .
an k E N.
=0
=0
4. Fourier Series
4. BIG OH FOR FUN CTIONS For real-valued functions
I�I is bounded as t � a, f g.
(g),
223
f and g, if
where a could b e ±oo , then w e write = 0 and say f i s "big oh" of or f is AT MOST OF ORDER If limx--+ a then w e write (as � ) Show that :
g,
f '" Lg
x a.
f (x) / g (x) = L,
(a) f 0 as t � if and only if is bounded in a neighborhood of a . (b) sin t 0 as t � for any c) sin t '" t as t � O. t 2! + t 4 4! + 0 (t6) as t � O. (d) cos t ", (e) sin t - l/t = 0 on the interval (0, 1r/2) .
=
(
(1) = (1)
a
a
1 - 2/
1/
f
(1)
a.
/
(xn),
( Yn ) n '" L Yn
5 . BIG OH FOR SEQUENCES For sequences of real num and bers we say =0 if � J( for n � N , and say Xn is "big oh" of If limn (as � ) = then we write X Show that:
Yn '
Xn
( Yn) X n / Yn xn/ Yn L,
x a.
=
(1) (x n ) is bounded . n = ( Yn), 2:7= 1 X i = 0 (2:7=1 Yi ) . 2:7; 1 1/i = In n + 0 ( 1) . Suppose that Xn � 0 for all E N . Then 2: n E N Xn converges if and only if 2:�= 1 X i = 0 ( 1 ) .
(a) X n 0 if and only if (b) If X 0 then (c) (d)
n
Hints 1 . Use Parseval's identity 3.3 . 1 and the comments after the termwise differentiation theorem 4.12. 1 .
f
f' f'(x) dx
3 . If i s absolutely continuous, then exists almost everywhere on [-1r, 1rj ; define f' (t) = 0 at points t where the derivative does not exist . Then f ( - 1r) + f" for t E [-1r , 1rj . Integrate by parts as in the proof of 4 . 1 2 . 1 .
f (t) =
224
4. Fourier Series
4 . 13
Dido 's Dilemma
One measure of the size of a parcel of land is how long it takes to walk around i t . This method avoids the problem of the calculation of the area of an irregular shape. But figures of the same perimeter may enclose very different areas. As Vergil tells it in the A eneid (retold in Henry Purcell 's opera Dido and A eneas) when the Phoenician princess Dido fled her mur derous brother Pygmalion's rampage in Tyre (Lebanon) , she and a few of the faithful arrived on the North African coast circa 900 B . C . in the area that later became Carthage. The stingy local monarch, King Jarbas, said he would allow her to buy as much land as the skin of an ox could surround. Her minions set to work and cut an ox hide into the thinnest strips they could. Having decided that one side would be the Mediterranean coast , in what shape should the "cord" be strung so as to enclose the most area? One of the last scholars of the Neo-Platonic school at Athens, Proclus, described such problems in about 450 A . D . in his commentary to Euclid's first book. He called them isoperimetric problems: Of all curves of a given length, find the shape that encloses the most area. Zenodorus's thoughts on the subject (ca. 100 B.C.) included the following: 1 . Of all polygons with n sides and the same perimeter, the regular polygon encloses the most area.
2. Of regular polygons with the same perimeter, the more angles, the greater the enclosed area.
3. The circle encloses a greater area than any polygon of the same perimeter. It is not known what shape Dido chose, but Carthage did prosper for a time , and she was its Queen. Her love life did not fare as well. When Vergil's hero Aeneas stopped in Carthage, she fell in love with him to Jupiter's intense displeasure . Jupiter told Aeneas to leave . Aeneas came out all right, and subsequently founded Rome. Dido committed suicide. Though Zenodorus had the right idea, his proofs were incorrect . In 1 836 Jakob Steiner thought he had solved the problem, had proved the circle to enclose more area than any curve of the same length. He argued as follows: 1 . Suppose a closed curve all curves of length
L.
C of length L encloses the most area among
2. Then the enclosed area must b e convex (the line joining any two points in the region must lie wholly within the region) .
A
B
C
3 . If two points and are chosen on that produce arcs of equal length, then the straight line bisects the area.
AB
4. The arcs of part 3 must be semicircles.
4. Fourier Series
225
Dirichlet demurred. The problem lay in assuming that a solution existed . Steiner had only proved that if s u c h a c u r v e C exists, then must be a circle. Its existence must be established for the argument to be valid. Some years later Weierstrass did prove that a circle enclosed more area than any other curve of the same perimeter. Our interest is in A . Hurwitz 's 1902 solution to the problem using Fourier series. Suppose that a closed curve is defined by
C
C
x = x (s) , y = Y (8) ,
0�
8 � L,
x t / L, L. x (t) y (t) (bn) (dn ), n N, (ndn, , n N,
where 8 denotes arc length along the curve , and and y are continuous functions with piecewise smooth derivatives. Let = 27r8 so that varies between 0 and 27r as 8 varies between 0 and Suppose the Fourier cosine coefficients for the 27r-periodic extensions of and are ) and denote their respectively ; let n E and E respective sine coefficients. By 4.12. 1 , the Fourier coefficients for and are , - n cn ) respectively. Since ) and E
N U {O}, dy/dt (nbn -nan 2 2 ( �� ) + (*) = 1 , i t follows that ( dxdt ) 2 + ddty 2
(cn),
t (an
dx/dt
()
By the previous equality and Parseval's identity 3 .3 . 1 ( see also Exercise 4 . 1-10 ) ,
( �)') dt ; t (e:)' n 2 ( a� b� c� d�) . +
Ln E N
+
+
+
Using the version of Parseval's identity of 3.3.4(f ) , we can compute the enclosed area as an inner product :
A
Consequently,
L 2 -47rA = 27r 2 L [(nan - dn) 2 + (nbn + cn) 2 + ( n 2 - 1 ) (c� + dn) 2 ] 2:: O. nN ( 4.55 ) Thus, no matter what C is, the upper bound for A is L 2 / 47r. This fact , L2 A - 47r ' is known as the IS OPERIMETRIC INEQ UALITY . Of course, if C is a circle of E
o Xn x (A),
Lk>O S k rk
L k>O s k r�
r
Ln>o x n = x (A) if and only if Ln >o xnrn < n L n�o xnr x jor all 0 < r < 1 . Example 4.14.9 Show that 1 - 1 + 1 - . . . = � (A) . Solution . Note that the partial sums Sn of the series are 1 , 0, 1 , 0, . . . Let (rn )n �O be a sequence of positive numbers such that rn 1 - . Then '" S k rnk Vn � k>O'" rn2 k - '" (rn2 ) k �k>O � k>O 1- � 1 0 · 1 + r� 2 4. 14.8 ABEL SUMMABILITY = 00, and limr-+ l -
-
--+
�
--
-
(C, I)
We saw in Examples 4.14.2, and 4. 14.9 that the and A-sums of are each 1/2. These are special cases of 4.14. 1 0 , which says that the 1) sum must be the same as the A-sum. The converse is false : the series 1 - 2+ 3 4 + · . . = � and this series i s not 1 ) summable by Example 4. 14.3.
LnEN (-It + 1 (C, 4.14.10
x (A) .
-
(A),
(C,
(C, 1 ) IMPLIES ABEL If L n � o Xn = x (C, I), then Ln�o Xn =
4. Fourier Series
=
233
P roof. Let S n L � = o S k , and let O"n = L � = o sk i (n + 1) denote the nth Cesaro sum of Ln> o x n . First we show that Ln>o x n r n converges for any r E (0, 1). Since O"n x, there must be some > 0 such that 100n i � for (n + 1) r n all n E N. The radius of convergence of the power series Ln>o (n + 1) . . IS gIven b y I'Imn (n + 2) = 1 . Th erefiore ,
---+
I MM
M
1
M M
(4.56 ) L: ( n + 1) O"n rn , 0 < r < 1 , n�O i s convergent , since i t i s dominated b y the series Ln > (n + 1 ) M r n . M ul tiply the series in equation (4.56) by ( 1 - r ) , and set -O"_ l = O. Then n � ( n + 1) O"n rn ( 1 - r) � �n>O �n�O (n + 1) O"n r - Ln> o (n + 1) O"n rn + 1 = � [ (n + 1) O"n - nO"n - d r n �n>O L: n o [ L� = O S k - L �:� S k r n - s n rn . L:n�O If w e multiply b y ( 1 - r ) again , we get n n ( 1 - r )2 � ( 1 - r) � �n� O (n + 1) O"n r �n>O sn r n+I n � �n>O S n r �n>O Sn r -- � (4.57) � - (s n - sn_ I } ;n �n>O - x n rn , L:n�O so L n>o x n r n converges. Now we have to show that it converges to the (C, 1) llmit x limn n ' As we ask the reader to prove in Exercise 4 , ( 1 - r)2 L: ( n + 1) rn 1, (4.58) n�O 0
]
:
=
0"
=
so Therefore, by equation (4.57) ,
( 1 - r )2 L: ( n + I ) ( O"n - x ) rn = L: x n rn - X .
n�O
n�O Given > 0 there exists an integer p such that 100n - xl < £/2 for n Now split Ln�o ( n + 1) ( O"n - x ) rn into two parts : p- l n � (1 - r) 2 � � n�O x n rn - x �n _- O (n + I ) ( O"n - x ) r + n + ( 1 - r)2 � �n� p ( n + l) ( O"n - x ) r . £
�
p.
4. Fourier Series
234
We can make the first term less than c/2 by taking r sufficiently close to 1 . The second is small because I Un - x l < c/2 for n �
p:
<
2 c "2 (equation (4.58) . 0
It follows from 4.14.6 and 4.14.10 that summability implies Abel summa bility, which is another way to prove the Abel continuity theorem mentioned above. As mentioned before 4. 14. 10, the converse of 4.14.10 is false . The converses of the regularity statements are obviously false as well. Sometimes it is possible to get a partial converse, something like "Abel summability + another condition" implies summability. Theorems of this type are known as TAUBERIAN theorems after A. Tauber, who proved such a result relating Abel summability to ordinary summability.
Exercises 4 . 1 4 1 . With notation for Un as in Example 4.14. 1 , verify that the nth ( C, I ) sum of Ln e N X n is n n n S i L (1 - i -n 1 ) X i = .1n ;L (n + 1 - i) X i . Un .1n ;L ;=1 =1 =1
=
=
2. Show that if 1 - 2 + 3 - 4 + . . . 3. Show that 1 - 2 + 3 - 4 + . .
.
= S (P), then S = t .
= t (A).
4. For 0 < r < 1 , show that (1 - r) 2 L k � O (k + 1) r k
= 1.
Hints 3. Show that
1 - 2r + 3r 2 - 4r3 + . . . = � ( 1 + r)
(ordinary con vergence).
4. By standard results about power series, Ln >o (n + 1) rn converges for I rl < 1 liffin n� l Now multiply the series by r and subtract .
=
.
4. Fourier Series
4.15
235
Fejer Theory
We discussed in Section 4.9 that there are functions f E L1 [-1I", 1I"] whose Fourier series diverge everywhere. Even so (4. 15.3) , the Fourier series of any f E L1 [-1I" , 11" is (G, summable to [f at any point +f exist . The Hungarian mathematician Leopold and f where f Fejer ( 1880-1959) made this remarkable discovery in 1904, and the body of related results is known as Fejir theory. (We are told that Fejer is pro nounced "fay-ha.") Lebesgue (4. 18.5) proved that the Fourier series of any f E L1 [-1I", 11"] is ( G, 1) summable to f almost everywhere. To get Fejer 's theorem, we proceed in a manner analogous to that used to get Dirichlet 's theorem on pointwise convergence 4.6.2.
t
(t - )
(t - )
1) (t + )
]
(t + )] j2
(t)
1 . We use trigonometry to find a simple expression for the average of the first n + 1 Dirichlet kernels (equation (4.61) and Theorem 4. 1 5 . 1 ) . The average of the first n + KERNEL, denoted by
Fn (t) .
1 Dirichlet kernels i s called the FEJ ER 1)
2 . Then we use information about the Fejer kernel to evaluate. the (G, partial sums o f equation (4.60) i n 4.15.3. I n particular, the Fejer kernel has a selector property identical to that of the Dirichlet kernel (cf. 4.6 . 1 , and 4.5.6) . For any 211'-periodically extended f E L1 [-1I', 11'] for which f exists,
O'n
(t + )
lim -.!.. 211'
n
r f (t + x) Fn (x)
10
dx = f (2t+ ) .
Do (t) = 1 and sin (n + l/2) t , n E N , (4.59) L.J 2 cos k t = Dn (t) = 1 + � sin (t/2) k= l where the latter fraction is defined to be 2n + 1 when t is an integer multiple Recall (Section 4.5) the Dirichlet kernels:
of 211". By 4.5 . 1(b) we can express the kth partial sum of the Fourier series (of the 211'- periodic extension) of f E L1 [-1I", 11"] as
S k (t ) = 211"1 l_1r f (t + x) Dk (x) dx, k E NU {O} . 1r The ( G, 1) partial sum, the average n of the first n + 1 (starting at n = 0) partial sums Sn , is therefore 1 l 1r 1 En Dk (x) dx. ( 4 .60) f (t + x) O'n (t) = 2 n + 1 k=O The bracketed term, the average of the first n+ 1 Dirichlet kernels, is known as the nth FEJER KERNEL Fn (t). For n � 0 we have the following options (note that Fo (t) = 1) for expressing Fn (t): 0'
11'
- 1r
[
--
1
236
4. Fourier Series
1 ", n Dk (t) n +1 1 � nk = o sin (k + l /2) t ( 4 . 6 1) ", = n + 1 � k O sin (t/2) where si:f�t;') t is defined to be 2k + 1 when t is an integer multiple of 21r. Since Dk (21rm) = 2k + 1 for any E Z and any k � 0, it is easy to Fn (t)
m
verify that
Fn (2 1r) = n + 1 for any n E N and any E Z . Using the Fejer kernel, w e can rewrite equation ( 4 60 ) as (4.62) un (t) = 21r1 Lr1f f (t + x) Fn (x) dx. Since the Dirichlet kernels D k are even, so is every Fn. Consequently, we can split I::,.. into I� ,.. + 10"' , replace x by - x in I� ,.. , and rewrite equation 4.62 as (4.63) un(t) = 21r1 10 '" [! (t + x) + f (t - x)] Fn (x) dx. m
m
.
As with the Dirichlet kernel of equation (4.59) , a little trigonometry enables . . · · m us to CIrcumvent t h e addlhon
l /2)t . " L.,.. nk = O sinsin( k (+t /2)
4.15.1 ANO THER EXP RESSION FOR THE FEJER KERN EL and 21rZ, sin 2 l ) sin 2 (t/2)
t r;.
n
For E Nu {O}
Fn (t) = (n +[(n + 1) (t/2)] .
P roof. Since
and sin 2 0 to to get
n
2
2 sin or
2 sin (t/2) sin(k
+ 1/2)t = cos kt - cos (k + 1) t
= 1 - cos 20, we can calculate the telescoping sum from k = 0
(t/2) t sin(k + 1/2)t = 1 - cos (n + 1) t = 2 sin 2 (n � 1) t , k=O · 2 (n + 1) t n sm 2 L sin(k + 1/2)t = . t k=O sm 2
4. Fourier Series
Therefore , using equation
237
(4 .6 1 ) ,
n 1 ( n + 1) 1sm· (t /2) 2:sink 2=O(nsin+ (kl ) +t /21/2 )t ( n + l ) sin (t /2) sin (t/2) . 0
Fn (t)
------�--�
.
The Fejer kernels look a lot like the Dirichlet kernels (which can be negative) . They are highly concentrated around 0, rising to (n + 1) on a shrinking base of width 1r/ (n + Their effect as a kernel in an integral is to sieve out the value of the function at = 0 as n -+ 00 , something that we formalize in Fejer's theorem on pointwise convergence.
4
-4
1) . (4.15.3)
t
-2
2
The Fej er kernel for
4.15. )
n
=
t
3 and
4
n
=
7
Fn (t)
We show in 2 (c that the narrowing width of is sufficiently offset by the increasing height to keep the area under the curve constant . 4.15.2
PROPERTIES O F THE FEn �R KERNEL
Fn (t) = {
and
( (n + l ) /2 ) 2 , (n + l) (t/2) 1
(n + 1) ,
sin
sin
t
The Fejir kernels
Fo (t) = 1
, n � 1,
have the following properties: For every n E N , ( a) is an even 21r-periodic function; (b) � 0 for all t, and -+ 0 uniformly outside [ -r, r) for all o < :S 1r ; 1r (c) =2 (t) 21r. r
FFn (t) Fn (t) n J:1r Fn (t) dt 1 Fn dt =
238
4. Fourier Series
Proof. The result of (a) is obvious, as is the nonnegativity assertion of (b ) . As to the uniform convergence outside any interval about the origin , let r be positive. For any such that 0 < r � � 11",
t
as
n
-+
1 (
Fn ( t ) = (n + 1)
It I
sin
\n + 1) t/ 2
sm (t/2)
)2 - (n +1 1) (_sm._1_r/2 )2 -+0
O . 1f ( c ) I Fp (t) dt = 271". 1f The n-dimensional Fejer kernel Hp has quite similar characteristics : 4.20.2 T H E n-DIMENSIONAL FEJER KERNEL Hp Let Hp (t) = II Fp (tj) , t = (t 1 , . . . , tn) E Rn , n, p E N . (a)
�
�
n
j=l
p E N: Hp is 271"-periodic in each variable; (b) Hp (t) � 0 for all t E Rn , a n d Hp ( t ) 0 as p 00 uniformly outside Ir = [-r, rr for all r > 0; (c) 1 Hp (t) dt = (2 71" t . The results of (a) and the first part of (b) are obvious. The uniform convergence to 0 outside of Ir follows by noting that if t E I - Ir , then for at least one coordinate t i , I t i I � r; (c) follows b y splitting II into the product of n integrals. Part (a) of 4.20.3 is the n-dimensional analogue of Fejer's theorem 4 . 15.3(b). 4.20.3 THE n-DIMENSIONAL FEJER THEOREM L e t n , p E N , let I = [-71", 7I"] n , let I den ote the 271"-periodic extension of f E L'i (1) , a n d let (t) = (2!t 1 Hp (x) I(x + t ) dx, x, t E Rn (equation (4.75)) be the n th (C, 1) partial sum of the Fourier series for f. Then: (a) If I is continu ous on I, then II I - u l l � O. (b) If f E L1 (I) , then 11 1 - up ll 1 o . p oo For every
(a)
�
Up
�
�
260
4. Fourier Series
I
Proof. ( a) Since is continuous on the compact set I, I is uniformly con tinuous there ( 2.8.8 ) . Since all norms are equivalent on Rn , for > 0 there exists r > 0 such that for every E l , < t / 2 for all such I + ::; r, i.e. , for all E Ir = [-r, r]n . Now, for p E N , that
t
t
I /(t x) - (t )1
x
x II x l i oo l up (t ) - I (t) 1 = (2!f 1 1 Hp (x) I /(t + x) - I (t)1 dx l ·
Split II into IIr + II -I r . The first integral is small by the choice of r, and 4.20.2 ( c ) :
·r ( 27rf 1Ir
1
Hp (x) I /(t + x) - l (t) 1 dx ::; (27r1 f 1rIr Hp (x) 2" dx < 2" . 1. The second is small because Hp (t) -+ 0 uniformly p -+ 00 on 1 - Ir (4.20.2 ( b )) and I(t + x ) - I (t) is bounded (1 / (t + x) - I (t) 1 ::; 2 11 / 11 (0 ) . ( b) Observe that t
E
as
III - up l 1
l l up (t) - I (t) 1 dt < 1 ( 2�r 1 1 Hp (x) ( J (t + x) - I (t)) dx l dt < ( 2!f 1 Hp (x) dx 1 1 / (t + x) - /(t)1 dt. With 9 ( x ) = II I / ( t + x) - I (t) 1 dt, this is ( 2!f 1 Hp (x) g (x) dx.
x,
Since 9 is continuous on I ( Exercise 1), 27r-periodic in each argument of and 9 ( 0 ) = 0, the expression above represents the pth (e, 1) sum of the Fourier series for 9 evaluated at = O. By ( a) , therefore,
t ( 2!f 1 Hp (x) 9 (x) dx Hence I I - up l 1 1 -+ o. 0
-+
0 as p
-+
00 .
Some sources that go beyond this glimpse of multiple Fourier series are Nikolsky 1977, Hobson 1957, and Zygmund 1959. Exercises 4 . 20
1.
9t
CONTINUITY IN THE MEAN Show that the function of 4.20.3 ( b ) is continuous. ( Hint: The idea is that for I E Ll (/) , if is close to then the translated function It = I + t) is close to It o = I + more precisely, that the map E l ....... It E Ll (/) is continuous. Use
t
(x
to, (x t o ),
4. Fourier Series
26 1
the fact that for any I E L1 ( I) and > 0 there exists a continuous function h on I such that III h il l = II II (x) - h (x) 1 dx < Then, for t, to E I, note that -
I g (t)
-
9 (to) 1
Finally, add and subtract
::;
i
II I I (x + t) - I (x + to) 1
L
dx.
h (x + t) and h (x + to).) [Hint: See 2.8.9.]
5 T he Fo urier Transform NOTE. In this chapter, unless otherwise indicated, all functions are complex-valued functions of a real variable.
t , f, and k, the function g (w) = L f (t)k(t , W) dt for some set E is called an INTEGRA.L TRANSFORM of f with KERNEL k (t, w) of the trans form . By "transforming" both sides of certain equations, we can sometimes Given integrable functions of
(
)
convert them into simpler ones-differential equations to algebraic equa tions, for example. Assuming that we can solve the transformed equation, then we convert its solution back into the original situation, this latter pro cess being called "inversion." The practicality ( ease, doability ) of inversion is the raison d 'etre of the transform method: If you cannot invert with reasonable ease, why do it? Up till now we have considered Fourier series representations of functions that vanish ( either naturally or by the brute force of truncation ) outside a finite interval, and of periodic functions. Assuming that
f
7r n J 211" f(t)e -i t dt is known for every n we recover f through L j ( n ) ei n t some kind of convergence . n EZ Now suppose that f is defined for all real t, and w e cannot truncate i t t o a finite interval for some reason. For f (R) , instead of integrating from j(n) = �
- 7r
E N,
(
-11"
to
11" ,
we integrate over R:
)
E L1
(5 . 1)
f.
The latter expression is called the FOURIER TRANSFORM of Rather than from a series, we seek to recover from the integral ( the inversion formula ) . (w) -1
2
f 1 00 f e,w t dw - 00 �
11"
.
As we shall see, the inversion formula holds for a large class of functions
f E L1 (R).
264
5. The Fourier Transform
We recast some things that we did for Fourier series into exponential form in Sections 5.1-5.3. The reason for this is to motivate analogous re sults for the Fourier transform, which we introduce in Section 5.4. Guided principally by the quest for analogues of results for Fourier series, we in vestigate the Fourier transform in this chapter. In Section 5.18 we consider the Fourier transform of functions I E L 2 (R) . This is the most symmetric theory. The map I 1-+ j is practically a linear isometry of L 2 (R) onto L 2 (R); it misses being an isometry by a factor of V21r: 2 V21r 1l / 1l 2 (see Plancherel's theorem 5.19.3) .
1I �1
=
The Finite Fourier Transform
5.1
p.
Convention We normalize Lebesgue measure taking p./27r. Thus, for I, E L 2 [7r, 7r] ,
9
in Sections
5.1-5.3 by
(f, g) = 217r 1 '" I (t) 9 (t) dt, and 11 / 11 22 = 27r1 1 '" I I (t) 1 2 dt. For I E L 1 [7r, 7r] , 11 / 11 1 = 2� f�". 1 1 (t) 1 dt. Any trigonometric series �o + L an cos nt + L bn sin nt , t E R, n eN neN where the (an) n �o and (bn) n � l are complex sequences may be converted into exponential form L: n eZ cn e i n t by taking bo = 0 , and for all n � 0, -
_ ".
-
-
_ ".
(5.2)
i
To convert from exponential to trigonometric form, use an = Cn + C - n and bn = (cn - c - n ) . Note, too, that for every n � 0, Sn (t) = ,",� n+ L:�= 1 ika k cos kt + L:� = 1 bk sin kt = wk= - n Ck e t ,
We say that L: n eZ Cn e i n t converges (somehow) to s ( t ) if the symmetric partial sums Un (t) = L: �= - n Ck e ik t converge to s (t) as n The expo nential form of the Fourier series of I E L 1 [-7r, 7r] is given by � 00 .
I(t)e - in t dt. 1'" L i{n) ein t , where j(n) = � 27r - ". n eZ
The relationships between the j (n) and
(5.3)
2 an = 7r1 1 '" I(t) cos nt dt, n � 0, and bn = 7r1 1 I(t) cos nt dt, n � 1 , -
_ ".
-
0
".
5. The Fourier Transform
5.
265
( t
are as in equation ( 2 ) . We call the bisequence j (n) of complex E Z Fourier coefficients of f the FINITE FOURIER TRANSFORM ( FINITE FT ) of Ij we say "finite" because the domain of integration [ -71", 71"] is finite, in contrast to the Fourier transform of equation of Section 5.4 in which we employ (Sometimes we get less formal , and call just j(n) the finite Fourier transform rather than j (n) n ) By making changes EZ of variable, i t i s routine t o verify that the finite Fourier transform has the following properties.
(5. 15)
f�oo .
( ) .
5 . 1 . 1 FINITE FOURIER TRANSFORM BASICS If have: function finite FT
I E Ld - 7I", 7I"] ,
then we
f (t) f (-t)
j(n) j C -n) I (t - a ) e - i an fen) eik t l (t) j(n - k)
(t)
j
j(-n)
Products are more complicated. 5 . 1 .2 PRODUCTS OF L 2 FUNCTIONS (cf. Exercise 4.1-13) Let f, E
9
L 2 [-7I", 71"]
have the Fourier transfo rms j and g, respective/yo Then the finite FT of E at any k E Z is given by
Ig L I [-7I", 7I"]
fg (k) = Proof.
B y 5 . 1 . 1 , i t follows that
fg
L j(n) g (k - n) . n EZ
--
g et) = g (-n) and g e i k t = g (- (n - k)) = g ( k - n ) . Since L 2 [- 7I", 7I"] is a Hilbert space, we may use Parseval's identity 3.3.4(f) to get 1 I(t)g (t) e - ik t dt fg (k) 271" 1 11" 1 11" I(t)g (t)eikt dt 271" 1Ln EZ11" j(n) g (k - n) . .=..
--
- 11"
Definition 5 . 1 . 3 THE CIRCLE GROUP T
0
: =
The points of the unit circle T {z E C I z l I} of the complex plane form a multiplicative group known as the CIRCLE GROUP , and the map R/271"Z (R modulo 271") =
t
--+
I--->
266
5. The Fourier Transform
is a group isomorphism of the additive group R/211"Z onto T . From now on we view 211"-periodic complex-valued functions E Lt [ - 1I", 1I"] or L 2 [ - 1I", 1I"] as having T as their domain, and denote L 1 [-11", 11"] and L 2 [- 1I", 11"] by L 1 (T) and L 2 (T) , respectively. 0 Since { e i n t /.J27r n E Z } is an orthonormal base for the Hilbert space L 2 (T) of square-summable, complex-valued functions on T, it follows from the general theory of Hilbert spaces (3.4.9) that the map
/
:
(5 .4) is a surjective inner product isomorphism ; in particular, every square summable bisequence (an) of complex numbers is the Fourier transform of some I E L 2 (T) , and
This equality is frequently called PARSEVAL'S IDENTITY (compare it to the version in 5.17.2, where the domain is R rather than T) . For any I E L 1 (T) , by the Riemann-Lebesgue lemma 4.4.1, le n) 0 as n ±oo . Thus, associated with each E L1 (T) is a null bisequence = 1 To discuss what happens for L1 functions, recall that the space Co (Z) (or j ust co ) of all complex bisequences (a n) n eZ such that limn-d oc an = 0 with the sup-norm 11 · lloc is a Banach space. Now consider the process of converting L 1 functions into bisequences j. Definition 5 . 1 .4 FINITE FOURIER MAPPING F1 The map
/
(1 (n))
->
->
/
F1 :
L 1 (T) /
� �
is called the FINITE FOURIER MAPPING. 0
co � ) ,
/,
(5 .5)
We show in 5 . 1 . 5 that the properties of the finite Fourier mapping F1 are similar to those of the map F2 of (5 .4) . 5.1.5 T H E FINITE FOURIER MAPPING F1 The finite Founer mapping Fi is lin ear, continuous, and injective, i. e., il j = 0 lor every n, then 1 = 0 a. e., or, equivalently, il j(n) g (n) lor every n E N, then I = 9 a. e.
(n)
=
P roof.
The linearity is clear. As to the continuity of F1 , note that
l i 0,
. -sm aw sm aw . on a, a] , show that I [- a,a] = 2-- = w w/2 .
-
Solution .
f
a . -e - zwt I [ - a ' a] (w) = -a
dt = e
i aw - e -i aw .
ZW
=
2
sin aw
--
W
.
By the argument after inequality (4.47) of Section 4.9, sin w/w rf. L 1 (R). Thus, for a = 1 (actually, for any a f. 0) , here is another instance of a . nonintegrable transform. 0
282
5. The Fourier Transform
The hat function k (t) = ( 1 - It I ) 1 [ - 1,1 ] (t) of the next example plays an imp ortant role in Section 5 . 1 2 , where we discuss (5. 12.3) the Fourier transform analogue of Lebesgue ' s theorem namely, that the Fourier series of any 1 E L 1 [-71'", 71'"] is Cesino summable to 1 a.e . For that reason, the hat function is also known as the CESARO KERNEL.
4 . 1 8.5 ,
Example 5.5.5 THE HAT FUN CTIO N Show that
k (t) = (1 - It I) 1 [ - 1,1] (t ) Sol ution .
Since k is even,
2 2
2
.0
(1 -
-2
0
( : - C � I:
Integrating by parts , we get =
Si W
s wt
2 - ( 1 - cos w) w2 . W -2 sm 2 - 0
w4
( sin�/t2) ) 2
ill k(t) cos wt dt 1 1 t) cos1 wt dt sm w 1 t cos wt dt . -w
k (w)
k (w )
f---+
-
�2 (1 - C OS W )
)
2.
A rare case of symmetry between function and transform is exhibited
next.
Example 5.5.6 GAU SSIAN TO GAUSSIAN Sh ow that
I (t) =
_1_ e - t 2 /2 -$
f---+
e - w 2 /2 •
Solution . Since power series converge uniformly within all circles of conver gence, and termwise integration is valid for uniformly convergent series,
j (w)
=
=
5. The Fourier Transform
The integral 8(b))
283
f�oo e -t 2 / 2 tn dt is 0 if n is odd; if n = 2m , then (see Exercise
Thus,
i(w)
,", 00
_ iw) 2n (2n)!n ( L...tn = o (2n) ! n!2
Exercises 5 . 5 Find the Fourier transforms of the functions in Exercises 1 - 5; Ix denotes the characteristic function of the set
X.
(t)
f (t) = e - a 1t l , a > O . 2 . f (t) = (1 - t 2 ) 1[ - 1,1 ] (t) . 3. f (t) = e - a 2 t 2 , a > O . 4. f (t) = e 2it l [ _1 , 1J (t) . 0 < t < 1, 5. f (t) = -1, -1 < t < 0, 0, I t I > 1. 6 . For f E L l (R), express the Fourier transform of f (at - b ) , a, b E R, 1 . THE ABEL KERNEL
{ I,
in terms of
i(w) .
7 . REAL-VALUED FUNCTIONS : REFLECTIONS 1-+ CONJUGATES Show that is real-valued if and only if
f E L l (R)
i { -w) = i (w).
8 . THE GAMMA FUNCTION Like the Fourier transform, the gamma function is defined by an improp er integral dependent upon a parameter: 00 r (x) = x > o.
1
e -t tx- 1 dt,
One reason for interest in the gamma function is that for n E N, r (n + = n! I t provides a continuous extension of the factorial function and a way to compute factorials.
1)
284
5. The Fourier Transform
(a) Show that
(b)
_1_ ] 00 e -t 2 / 2 t 2m dt = 2m r (m + � ) . 2 � - 00 .,fi (Hint: Let y = t 2 12.) Show that r ( m + 1 ) _ .,fi (2m) ! 2" - m ! 4m ' m 2: 1 .
Hence , using (a) ,
1 --
�
(2m) ! . ]-0000 e _ t2/ 2t 2m dt - -m!2m _
r + t) = the fact that r ( t ) = Vi· )
( Hint:
r
(
m
2m2- 1
(
m
-
t) .
Then use induction, and
Answers
2al (w 2 + a 2 ) 3. f (w) = ( Vila) e - w 2 /4a 2 . 4. i (w) = 2 sin (w - 2) I (w - 2) . 1 . i(w) =
5.6
•
The Fourier Transform and Residues
Recall from Definition 4.5.7 some of the variants of the notion of improp er integral, namely and 00 . If I is -00 � 2: integrable , I E Ll b), then I, I exist and are I, equal to I. The method of residues can be helpful in evaluating Fourier transforms as well as some extensions of the Fourier transform of I such e - w I (t) dt instead of I (t) dt . First , we summarize some as basic facts . If I is analytic throughout a neighborhood of except at itself, then is called an isolated singularity off At an isolated singularity, I may be represented by a Laurent series
f:
a b� fa-+b , f� a ' f'::, PVf: , -+b (a, fa f!" a f':: PV f:
I,
PV f: i t
f: e - iwt
Zo
Zo
Zo
" an (z - zo) n + b 1 + b2 2 + . . . , 0 I z - zo l < r, z L..J ( Z - Zo ) (z - zo) n� O
0 , where
I
I
(z) dz and bn 1 (z) n dz 1 an = .1 1 + 1 27rZ (z - zo) - + 1 27rZ (z - zot C
= -.
C
5. The Fourier Transform
285
Zo
for any closed contour C around in the positive (counterclockwise) sense , provided that I is analytic on C and the region (which satisfies some de cent topological conditions, such as no holes) it contains except at The coefficient of the first negative power
Zo .
1
b 1 = Res (f (z) , zo) = -. 2 1n
zo o
f I (z) dz
le
is called the residue of I at The RESIDUE THEOREM states that contour may be evaluated by summing the residues inside the contour integrals (the integral is 0 if there are no singularities inside) and multiplying by 2 71" i.
Ie
RESIDUE THEOREM If C is a closed contour within and on which a function I is analytic except for a finite number of (per force isolated) singular po m ts Z l , . . within C, and R l ' · . · ' Rn are the residues of I at those points , then
. , Zn
Zo
If I is analytic everywhere within and on C, and is any point interior to the region enclosed by C, then we get CAUCHY'S
INTEGRAL FORMULA
1e zJ-(z)Zo dz = (
)
2 71"zl .
(zo) .
The following result also converts the computation of an integral into summing residues. It is a limiting case of the residue theorem taken on semicircles with diameters on the real axis as the radius goes to 00; the value of the function on the curved part is eventually o. If I Ll (R) , then e- i wt PV e- iw t = so the theorem provides a way to calculate i ( w ) .
I (t) dt I�oo
I�oo
J (t) dt,
E
5.6.1 RESIDUE THEOREM FOR THE UPPER AND LOWER HALF PLANES Suppose that I ( ) is (a) analytic in the open upper half plane H = {z : Im z > O} except for a finite number of poles at Z I , . . . , Zn E H, (b) analytic on the real axis R = {z 1m z = O} except for a finite number 01 simple poles at rl , . . . , rm , and (c) f r l m z > 0, II ( ) 1 ::; M/ l z l k when I z l > R o , where M and k > 1 are · constants such that II (z) 1 -+ 0 as I z l in the upper half plane. Then, for w > 0, z
:
o
z
PV
i: eiwt J (t) dt
-+ 00
=
286
5. The Fourier Transform
(z) eizw, ) denotes the residue of I (z) eizw at and pv l-°O00 eiwt I (t) dt = l I(t)e - iwt dt denotes the Cauchy principal value of f� eiw t I (t) (see Definitio n The corresponding result for the lower half plane is also valid, but this time we get some minus signs: For w - 2'11"i "nm (J (z) eizw , ) _'II"i " (J (z) eizw , ) where Res (J
Zk
Zk ,
R
lim
R-+ oo
-r
4· 5. 7).
oo
< 0,
L..t k = 1 Res
Zk
L..t k = 1 Res
Tk .
This technique enables us to evaluate certain Fourier transforms. Example 5.6.2
Show that the Fourier transform of
I I
I(t) = t 2 + a2 ' a > is f(w) = �'II" e -aw . I E L l (R), P V 1 e - iw tl(t) dt = 1 e -iw tl (t) dt = J(w) . : :2 ) ( I z + I(z) ia, I(z) a > I I (z) 1 2/ =I z l 2 I z l > J2a. ;zwia _ e2-;aw , e z w e; (J (z) , ia) z + I z= i a a . 00 ezwtl (t) dt = 2'11"i ( e -aw ) _ e -aw w > 2za a 1- 00 w t2�a2 dt = � � , 00 1 t 2 + a 2 dt = w -w > 00 e-iwt I (t) dt ?!.. e aw. 1- 00 a 1
0,
Sol ution . Note first that
�
so
( z ) = 1/ a2 , 0 . In the upper half plane , the only pole of is at and is analytic for Im z 0, so conditions (a) and (b) of 5 .6 . 1 are satisfied . Moreover, ::; for Since Let
Res
=
•
it follows that
-.-
Since f
tan - 1
0.
holds for this equality also 'II"
= 0:
1
If
< 0, then
�.
- 00
0 , and therefore
=
Finally, since
'II"
for
=
287
5. The Fourier Transform
we have
i (w)
= 1-0000 e - ;wt f (t) dt = �a e - a l w l . - lwt (t) dt. e f a f: e - iwt ( ) dt 5.23. 0
�
.
In the next example we do not compute f (w) , but PV b f Some authors consider PV f t as an "extended Fourier trans form," a topic we return to briefly in Section Example
5.6.3 Show that for a > 0 , PV
1 ":' e- I.Wt
dt+ a2 ) = a2 ( 1 - e -a1w I ) 2 (t t - 00 - 1ri -
P roof. The function
f (z)
1
- z (z 2 + a 2 )
---;--;::-::7"
satisfies the conditions of 5.6 . 1 and has simple poles at Moreover, Res (!
Res Therefore ,
)
±ia and
z
= o.
-
1
( )
21r i � 2"
1ri
Note, too, that
=
e;zw , ia) = z (zei+zwia I z =ia e2-:: ' eizw (! z e i z w , 0 ) = 2 2 z + a I z=o - a 2 · ( _e-aw ) + ( a2 ) o. a (I e-aw )
(z)
while
z
sgn w .
1
1ri
( 5 . 18 )
for w >
-
I: eiwt (t) dt : _ a ( 1 e-aw) . = < 0, (5.18) (5.18) 1-0000 e -iwt (t) dt = a2 (1 _ eaw ) . PV
f
- i
Since f is odd, equation holds for w O. For w can replace w by -w in equation to get PV It follows that PV
I: e - iw t f (t) dt
-w >
1r i
f
- :z
a (1 -
e-a1w l )
sgn w .
0
0 , so
we
288
5. The Fourier Transform
We showed in Example 5.6.2 that I (z ) =
1
z
2 + a2
�
I I
_11"a e -a w = I�(w) , a >
O.
Contrast this with the almost symmetric result of Example 5.6 .4. Note , too, that even though does not have an integrable transform (Example 5 .5 . 3) , does.
e-1 t / e-t U (t)
Example 5.6.4 THE ABEL KERNEL
O. Show that for any
a E R,
e -1 at l
e - 1 a1 t Ue -(t)1 at l U f (t)I a l ++ iw)/ (-t)
�
The A bel kernel is / (t) =
2 1a l a2 + w 2
e- a 1 t l , a >
(t) ( I a l - iw).
= denote the unit step function. By Example 5.5.3, 1 = � 1/ 1/ ( . By 5.5 . 1(a) , / = which will not affect the except at = Now , transform (the functions are equal almost everywhere) , so we can use the linearity of computing transforms to get Sol ution . Let
(-t) t 0,
�
e -1 at l
1-+
e1 a 1 t U (-t)
2 1a l 1 1 al l + iw + l a l - iw = a 2 + w 2
.
0
Exercises 5 . 6 1 . Find the Fourier transforms of
t - �t + 2 · [Ans. i (w) = 1 . [Ans. 1 (w) = 11" (b) / (t) 2 2 ( t 2 + 1) 1 ( c) 1 (t) = t4 + 1 . Find PV f�oo 1 (t) dt for w > 0, where (a) I ( t ) =
2
=
2.
�
1I"e - 1w l - i i w .J _ e-1w l + Iw l ) (cos w
sn )
.J
(1
eiwt
I (t) = } . [A ns. 7ri.J t-1 (b) I ( t ) = t (t 2 + 1) " For a > 0 and any constant b, find the Fourier transforms of cos bt � 11" I + I I) J (a) -- . [Ans. / (w) = - ( I 2a + t 2 2a (a)
3.
e-a w- b e-a w +b
.
289
5. The Fourier Transform
(b)
sin bt . a2 + t 2
4. Show that the function I (t ) = cos� ". t is in and find j. [Hint: To find j, integrate c�:�"':z around the rectangle with base [ - R, R] and upper vertices at ( - R, -R + i) , (R, R + i) . The only pole of the function within the enclosed region is a simple pole at i/2 , and the residue of the function there is
L 1 (R),
Thus, by 5 .6 . 1 , R
e- iw t
L R cosh 1rt
e - iw ( i/ 2 )
ew/ 2
sinh 1r (i/2)
1rZ
1 coshe-iw(1r (RR+iy)+ iy) dy + IR- R coshe -iw(Hi) dt 1r ( t + i)
dt + Io +
10 1
ew/2 e- iw ( - R+iy) . . . dy = 21ricos h 1r ( - R + zy) 1rZ
Io1 I1° ()
go to 0, which implies that and As R -+ 00 , the integrals w/ e eW j(w ) (1 + ) = 2 2 or j w = cOs h W/ ) " ] 2
5.7
(
The Fourier Map
In equation (5 .5) of Section 5 . 1 we considered the finite Fourier mapping
F1 : L 1 (T) I
---+
f----+
Co
LZ) ,
I.
We now investigate its continuous analogue, the FOURIER MAPPING
IE
L 1 (R)
f----+
j (w ) =
I: I(t ) c iw t dt ,
F1 ,
also commonly called the FOURIER TRANSFORM. After selecting an appro priate analog of Co (Z) as codomain , we show in 5 .7.2 that the Fourier map is a linear , continuous map . For I E (T) , lim lnl-+oo j(n) = 0 , by the Riemann-Lebesgue lemma 4.4 . 1 ; the Riemann-Lebesgue lemma also implies that for any I E liml w l -+oo j (w) = 0. We single out this latter property for further consider ation.
L1
L 1 (R),
Definition 5.7.1 FUN CTIONS THAT VANISH AT INFINITY ; COMPACT SUP and Cc If lis a real- or complex-valued function defined on such that lim I (t) = 0,
o (R),
PORT ; C
R
(R)
t -++oo
290
5. The Fourier Transform
ff
then is said to VANISH AT +00, with a similar convention for VANISHES AT -00 . If vanishes at +00 and at -00 , then we say that VANISHES AT 00 . The collection of continuous functions that vanish a t infinity with sup norm is denoted by Co ( and is a Banach space-the subscript 0 denotes that the functions decay to 0 as I � 00 . Functions that vanish at infinity need not decay to 0 rapidly enough to be integrable : Consider = Functions defined on that vanish outside a closed (finite) interval are said to have COMPACT SUPPORT. stands for time and has compact support , then we say that is TIME-LIMITED . For E L 1 if j(w) has compact support, we say that is BAND-LIMITED . Clearly, any function with compact support vanishes at infinity. The subset of Co of functions with compact support is denoted by Cc It happens that 0 Co is the completion of (Cc (Rudin p.
11 11 00
f
R)
It
1/Jt2+I.
(R)
R
f f 1ft
(R) , 11 11 00 )
f (t) (R), f f(t)
( R). 1974, 72).
(R)
We have already noted that the Fourier transform i of any function f E L 1 ( ) vanishes at 00 ; we show in our next result that Fourier transforms i are uniformly continuous and that if two functions are close to each other as members of L1 ( ) then their transforms are close to each other as members of Co This property is useful when we want to approximate a function that is difficult in some way (e.g . , hard to integrate) by one with more manageable properties.
R
(R).
R,
5.7.2 T HE F OURIER
M AP The Fourier map F:
R
L1 ( )
f
is a continuous linear map, and for any tinuous.
f
Proof. By the Riemann-Lebesgue lemma
4.4 . 1, for any
R , i is uniformly con
E L1 ( )
i (w) = I: f(t)e -iwt dt
f
R,
E L1 ( )
(R), we must show that i is con
vanishes at infinity. To see that i E Co tinuous. To that end, for arbitrary E
w , h R, consider
Hence
i (w + h) - i (w) = I: f(t ) e- iw t ( e- iht - 1) l i (w + h) - i (w) 1 ::;
dt . [: I f(t) l l e -iht 1 1 dt . -
w.
Note that the term on the right does not depend on So, if we can show that this limit is 0 as � 0, uniform continuity follows. To complete
h
5. The Fourier Transform
(t) l hn In (t) = f (t) (e- ihn t i: I(t)e- iwt (e-i h n t 1) dt = 1-0000 n [J(t)e - iwt (e -i hnt - 1)] dt = (hn ) h l i (w h) - i (w) 1 = i I f l oo :::; I I f l l w E R, l i(w) 1 i: I f(t)e -iwt I dt = I fl l ·
291
4.4.2
the argument , we use Lebesgue's dominated convergence theorem on passage to the limit under the integral sign. Since 1 1 :::; , the integrand is dominated by an integrable function. For any sequence --+ 0 , 1) --+ O . Therefore ,
2 11
I/(t) l l e - iht
-
-
limn
-
O.
lim
Since
is an arbitrary null sequence, it follows that lim .....
O
+
0,
and is seen to be uniformly continuous. As an integral operator, the linearity of F is obvious . The continuity of F, that F p follows from the observation that for any 0
:::;
5.12.5
We shall see in and injective but not surjective.
5.12 . 6, respectively, that the Fourier map is
Exercises 5 . 7
f, fn L l (R) R, "in - �I oo
1 . UNIFORM CONTINUITY O F THE FOU RI E R MAP F If E and --+ --+ 0 , show that uniformly on i.e . ,
I /(Hint: n f il l -
--+
O.
in i l in (w) - i(w) 1 :::; i: I /n (t)
-
f(t) 1 dt . )
Actually, whenever a lin ear map is continuous, it is uniformly contin uous; this is a special case.
5.8
Convolution on R
By analogy with what we did on the unit circle in Definition 5 .2 . 1 for 9E we define the CON VOLUTION of E Ll to be
I,
L l (T)
T
f, 9 (R) I*g(t) = i: f (t - X)9 (X) dX.
5. The Fourier Transform
292
2
Note that this time we omit the factor 1/ 1r in front of the integral. The existence of the integral is j ustified by the following argument :
I (t - x) 9 (x) i s integrable on f�oo I (t - x) g (x) dx is defined for almost
I t follows b y the Tonelli theorem 4 . 1 9 .2 that by Fubini's theorem 4 . 1 9 . 1 , every Moreover,
R2 j
t.
1: IU
II I gi l l *
oo [l: f(x) cosw (t - X) dX] dw (5.20)
In this section we develop conditions under which the FOURIER INTEGRAL FORMULA
or its equivalent
f (t) = 1 -> 00 [a (w) coswt + b (w) sinwt] dw,
(5 . 2 1)
(w) = ;:1 1-0000 f(t) cos wt dt and b (w) = ;:1 1-0000 f(t) sin wt dt . (You might want to look back at equations (5. 1 1 ) and ( 5 . 12 ) of Section 5 .4.) The version of the Fourier integral formula in equation (5.21) is especially where
a
useful if f is even or odd. In Section 4.5 we obtained the following criteria for recovery of a function from its Fourier series. 4.5.9 CRITERION FOR CONVERGENCE The Fourier series of the 21r-periodic function 1r] converges at a point if and only if there exists r (0, such that
t fE E Ll1r][-1r, sin ( + 1/2) u d lim .!. r [f (t + u) + f (t _ u)] 1r io u u n
n
296
5. The Fourier Transform
exists; the limit Fourierifseries for 1exists, at t. the limit is the value of the sum of the 4.series 5 .10 ofCONVERGENCE T O A PARTICULAR VALUE The Fourier 1 E L 1 [-11", 11"] converges to c at a pointthet if211"and-periodic only iffunction there exists r E (0, 11"] such that r [/( ) ( ) 2 ] sin( n + 1 j2)u d 1 1 u = 0. Imn 11" t + u + I t - u - c U 1·
0
T
4.some 5 .12rDINI'S TES If for the 211"-periodic function 1 E L1 [-1I", 11"] , E (0, 11"] , and some fixed t, 1 (t + u) + 1 (t - u) - 2c ;:....., --'-..:... u '-...: --'- '--- E L 1 [0 , r] , then the Fourier series for 1 converges to at t. The effectseriesof 4.in5 .terms 9 is toofconvert the integral nth partial sum of a Fourier the Dirichlet kernelforDthe n (t) = L:�= - n eilet to an integral with analog the discrete Fourier kernel
0,
sinwu du = o. hm. -1r1 10r [f (t + u) + 1 (t - u) - 2c] -w--+oo U Proof. If 00 � 1--+ [1: 1 (x) cosw ( t - x) dX] dw then by 5 . 1 0 . 1 there exists a positive number r such. that smwu du. . r [f (t + u) + /(t - u)] -C1r = lun w--+oo u l c
= c ,
0
(5 . 22)
�
298
5. The Fourier Transform
hm. l r 2c s. Uwu du = C1r.
By 4 . 5 . 8 (b), Therefore,
W -+ OO
0
m --
hm. lr [/ (t + u) + /(t - u) - 2c] �nwuu du = 0.
W -+ OO
--
0
Conversely, if equation (5. 22) holds, then r [! (t + u) + I (t - u)] -sinwu du C7r = hIllw u . -+oo l = � l -+oo [l: /( x )cosw (t - X ) dX] dw by 5.10 . l . 0 A condition that suffices Iffor theE L1recovery of I (t) from the Fourier integral formula is differentiability. (R) is differentiable at t, and C = I (t), I then l (t + u) + / (tu - u) - 2/(t) = I(t + u)u - /(t) + I(t - u)u - / (t) isEquation a bounded-hence integrable-function of in [0 , r] forlemma some4.4.r >l . O . (5. 2 2) then follows from the Riemann-Lebesgue Finally, there is the continuous analogue of Dini's integrability test. o
u
ANALO GUE OF DIN I 'S THEOREM For ICorollary E L dR) ,5il. 10.3 ICONTINUOUS (t + u) + I (t - u) - 2c E L1 [0 , r] (5. 23) u lor some r > 0, then the Fourier integral lormula
c. (The most interesting case is where c = I(t).) Proof. By the Riemann-Lebesgue lemma 4.4. 1 , equation (5. 22) holds, and the result follows from 5.10. 2 . 0 As notedderivative in 4.5 .13in( Lipschitz' s testabout ) condition (5. 2 3) is satisfied if I has a bounded some interval t or, more generally, if I satisfies a Lipschitz condition of order I I (t + u) - I (t) 1 b l u l a for l u i r, where b, and r are positive constants; if so, then I (t + u) + I (t - u) - 21 (t) E L [0 , r] . By Dini 's theorem 5.10.3, this yields the following corollary. converges to
a,
a,
::;
"-'----'--=--.:'-----'---'---'--'u
L 1 (b , 00 ) ,
� [ 00 .6.1 (t, x) d lim x = o. 0 ..... oo 7rr 1b x2
r
5.12.4 INVERSION WHEN 1 E (R) If I and 1 are integrable on R, then at any Lebesgue point t of I, in particular at any point t of continuity of I, 1 e·w t dw. I (t) = 2 7r 00 f (w)
L1
J
OO
-
�
.
5. The Fourier Transform
313
The integrability of j implies that I: j(w) eiwt dt = s istherefore finite. By the regularity of (C, 1) summability (5.11. 2 (b)), and 5.12.3, f (t) limr ....00 2� J:r ( 1 - I� I) eiwt j(w) dw 1 211" 1- 00 ezwt f (w) dw. 0 Now we can easily demonstrate the following result. Proof.
oo
•
�
Th e Fourier m ap F : L l (R) -- Co LR) , f f,
5.12.5 INJECTIVITY OF THE FOURIER MAP
of 5. 7. 2 is injective. Proof.
1--+
By 5.12. 3 ,
f (t) = 2� I: eiw t j(w) dw (C, l ) a . e . , so if j (w) = 0 , then f (t) = 0 a. e . Since the Fourier map is linear, this . implies that if j(w) = g (w) a. e . then f = 9 a. e . or that the Fourier map F is injective. 0
Supposegivenwe want to solve an integral equation of the convolution type, namely, g , k E Ll (R), find f E L l (R) such that I: k (t - x) f (x) dx = 9 (t) for all real t . By the convolutions-to-products theorem 5.8 .2, (w) . k (w ) 1 (w) = 9 (w) or j (w) = � k (w) exists hmap, E L l (R) such that h (w) = 11k (w), then, by the injectivity ofIf there the Fourier j (w) = It (w) g (w) = ';;-g (w) , so f (t) = I: h (t - x) 9 (x) dx. OfUsing course,thethebounded probleminverse is thattheorem, in manywecases, h does not exist. proved after 5.1.5 that the finite Fourier map Fl : L l (T) -+ Co (Z) is not onto. The same holds for the Fourier maD F.
314
5. The Fourier Transform
5.12.6 THE FOURIER MAP F : L1 ( R)
---+
Co ( R) Is NOT ONTO .
Proof. We exhibit a g E Co (R) that is not the Fourier transform of any
I E L 1 (R) . First , note that I is odd if and only if j is odd : By the definition of j, if I is odd, then j is odd; by 5 . 1 2.3, if j is odd, then I is odd. Let e denote the base of the natural logarithm. We will show that if f E L l ( R )
fe a
is odd , then
j ew ) W
dw < 2 '11" 1 1 / 1 1 1 for any
a > e,
E
then produce an odd function 9 Co ( R) that does not satisfy this inequal ity to conclude that 9 cannot be the Fourier transform of any ( R) . First note that if I E ( R ) is odd, then
I E L1 L1 OO I(t) sinwt dt = -2i (XJ I(t ) sin wt dt . j ew ) _i Jo -00 For any a > e , j ew ) /w is continuous on. fe , a], so we can invoke Fubini's
j
=
theorem 4 . 1 9 . 1 to interchange the order of integration as follows:
00 -2i le a �1 1n[0 I(t) sinwt dt dw oo -2i 10[ I(t) Je[ a sinwwt dw dt .
r j cw) dw
1e
w
t
With u = w , this becomes
By 4.5.2(b) , for all 0
for all
::;
00 sin u duo - 2i 1o I(t) dt le a t -
t
c < d, I t si: u du l
0, independent of t, such that h ( x ) :::; f.X for all 0 :::; x :::; 6.]
l
-
5.13
)
(
Convergence Assistance
Many integrable functions have nonintegrable Fourier transforms. The func tion I (t) e - t U (t), for example, has 1/ (1 + iw) as its Fourier transform (Example 5.5.3 ) ; the magnitude of the transform is I/Vl + w 2 1/ lw l for large I w I . 1 is not integrable. If 1 is not integrable, then =
as
�
� 211" 1-0000 f (w) e1wt dw
1 -
•
may not exist , and we cannot recover I (t) from it. Nevertheless, regardless of the integrability of 1, if we massage the integrand a little and consider (C, 1 ) summability, then (5. 12.3)
100- eiw t 1(w) dw (C, l ) a.e. 00 J ( 1 - I� I) eiwt 1(w) dw a. e . 2� :r
1 2 limr - oo
f (t)
11"
� (f
limr_oo 211"
where with k (t)
= ( 1 - I t I ) 1 [ - 1 , 1] ,
*
Kr ) (t) a.e.,
rW/2 ] Kr (w) = rk (rw) = r [ sin(rw/2) �
2
Somehow the factor
makes the integral converge. We will investigate other kernels k (w) in this section such that 1 lim r_oo 2 11"
jr k ( W ) e1w t f� (w) dw = / (t) a.e. .
-
-r
r
318
5. The Fourier Transform
In particular, in 5.13.4 we generalize the Fejer-Lebesgue inversion theorem to one in which (1 - I Ir) is replaced by a more general "convergence factor." Let us examine more closely the properties of the hat function
wl
k (t) = (1 - It I) 1 [ - 1 , 1) (t) ,
also known given by
as
the CES A RO KERNEL. By Example 5.5.5, its transform is
k (w) = ( sin��t 2) r
(w)
w
Since k is continuous on [- 1 , 1] and bounded by 4 1 2 for I integrable. Moreover, by 5 . 12 .2 ( c) ( with r = 1 ) ,
00 1 k (w) dw = l. 1r 00
1 2
w l > 1 , it is
-
Obviously, k (w) is even . We abstract these properties of the Cesaro kernel now . Definition 5.13.1 SUMMABILITY KERNELS
k
A real-valued, even function E L 1 ( R) is called a SUMMABILITY KERNEL if its Fourier transform satisfies
k (w)
100
1 k (w) 2 1r - 00
dw = l.
0
Obviously, the Cesaro kernel is a summability kernel. Here are two other examples. Example 5.13.2 GAUSSIAN AND ABEL KERNELS
( a) The ABEL KERNEL is By Example 5.6 .4, its Fourier transform is
2 �k (w) = -12·
+w
Since J�oo 12 a:::2 2 1r , is a summability kernel. + 2 ( b) By Example 5.5.6, the Fourier transform of e - t / 2 is V2iie - w 2 /2 . Therefore, replacing by .;
rw/2 ] I 0 , its transform is dominated by the decreasing function 4/w2 there . By 5 .12.4, if f and 1 are integrable, then 1 f (t ) = 211"
1
00
- 00
w
f (w) e , t dw a . e . �
•
But when is f E
L l (R) ? Our next result provides a criterion. 5.13.5 CRITERION FOR INTEGRABILITY OF i Let f E L l (R) be such that for some M, and some positive d, I f (t) 1 � M a. e. in [-d, d] . If f ? 0, then fE L l (R) . P roof. Consider the Abel summability kernel k (t) = By Example 5.6.4, k ( w) = 2/ (1 + w 2) . Let r be positive, and let (x) = r k ( rx ) . By
5.13.3(a) ,
1tl . eKr
324
5. The Fourier Transform
Since k is even, with a change of variable we have
( / (x) * rk (rx)) (t)
i: I (t - x) rk (rx) dx i: I (t + x) rk (rx) dx
i: I (t + � ) k (x) dx.
With t = 0, we get
i: j(w) e - 1w l /r dw I: I (�) k (x) dx. =
J�oo into r::d + f�d + frO;; · Since I I (t) 1 S; M a.e. for I t I S; d and k � 0, d J:r d l (�) k (x) dx S; M i: k (x) dx = 21rM. Since 2x/(1 + x 2 ) is bounded on R, xk (x) 1 +2xx 2 S; C for some constant C > 0, say. For x � 0, Split
�
=
With
w = x/r, this becomes C 00 C d J[ I I (w) 1 dw S; d 11 / 11 1 '
d
Likewise, for some positive constant C ' ,
Therefore, there is some constant f{ , independent of r, such that
The monotone convergence theorem [Stromberg 1981, Theorem 6 . 20, p . 266] asserts that if (In) is a n increasing sequence of real-valued functions
5. The Fourier Transform
In
325
I In e- / w / / a e- / w l / b,
integrable on R such that liIDn J�oo (t) dt < 00 and ( t ) is defined to be limn In (t) a.e., then I E L 1 (R) and J�oo I ( t ) dt = limn J�oo (t) dt . the Since � 0 by hypothesis, and a < b implies � monotone convergence theorem permits us to make the following inter change:
j(w)
I: j(w) dw = r�� i: j(w) e - /w l /r dw
� I O. Show that for any f E L l (R), =
g rt), r t) rg gr ( ( 1 2� (f * gr ) - f i l l
-+
0 as r -+ 00 .
21"
g (t) dt
[Hint: By hypothesis,
2� f�oo gr (t) dt = 2� f�oo rg ( rt) dt = 1 . Now, 2� (f * gr ) (t ) - f (t) 21" f�oo [f (t - x) - f (t)) rg (rx) dx,
so
f�oo f�oo I f (t - x) - f (t) 1 r I g (rx) 1 dx dt 1 21" (f * gr ) - f il l =:S 2211,.". f� w (x) r l g (rx)l dx, where w ( x) = f�oo I f (t - x) - f(t) 1 dt. Since f E L l (R), by con tinuity in the mean 2.8.9 ( namely, that I l f (t + h) - f (t) 1 1 -+ 0 as h -+ 0 ), w ( x) 0 as O. Given ( > 0, there exists 6 > 0 such that I x l 6 implies 0 < w (x) < ( ( 2 11" ) / I l g 1 1 ' Hence 1 2� (f * gr ) - f il l :S I + J, :S
-+
x -+
where
J
21" �xl � 6 w (x) r lg (rx) 1 dx and = 21" �x l>6 w (x) r l g (rx) 1 dx. Let rx = y. For an upper bound c for w (x), I J I :S 2� �YI > r6 Ig (y) I dy -+ 0 as r -+ 00 . 1=
Next,
5. NORM CONVERGENCE AND SUMMABILITY KERNELS Let k be a summability kernel . Show that for any f E L l (R),
1 2� f (x) (rx) - f il l ----. 0 as r -+ uk
00
5. The Fourier Transform
327
1 ( 1 00 ) f (x) rk (rx) (t) = f (w) k (w l r) e , w t dw. * 2w 2w 1-
and
00
�
.
�
[Hint: Use the preceding exercise with 9 = k . Then use
6. Show that
5. 1 3.3(a).]
2 /4 cos xt dx = e - t 2 . [Hint: Use equation (5. 36) with k (t) = e-t 2 ; k (w) = .../ie -w 2 / 4. ] 1
.../i
[ 00 e -x
Jo
7. COMPOSITION
( a) Use the convolutions-to-products theorem and 5.12.4 on inver
sion when 1 E L 1 (R), then 1 2w
L 1 (R) to prove that if f, g , and lu belong to
00 1_00oo f�(w) u (w) e,wt dw = U * g) (t) = 1_oo f (t - x) g (x) dx .
for almost every t E R. ( b ) Use ( a) to show that for
a, b > 0, 00 dx w 2 2 1- 00 (x 2 + a ) (x + b2 ) ab (a + b) ' [Hint: Let f (t) = a > and g (t) = b > O. Then an d 9 ( w ) - 2 Now use ( a) and take t = INTEGRAL EQUATION For a > and f E L 1 (R), solve the
e- a 1 t l , 0 0.]
8 . AN
integral equation
00
1- 00
�
-
e- b1 t l , b2 +bw 2
0
1 f (t - x) f (x) dx = -a 2+ t-2 '
[Hint: Take the Fourier transform of both sides , and use the convolution to to-products theorem and Example . 6 . 2 ( [ 4 ] = (wi a) get l (w) 2 = so l w) (.../il y'ii) Then use 5 . 12.4 on inversion when 1 E L 1 (R) to get f (t) = 2 fi ( )
(
e - a 1 w I /2 .
5 a 2 ! e -a 1 w l , a2�4t2 .]
e- a 1w l )
328
5. The Fourier Transform
9. AN INTEGRAL EQUATION Assuming that integral equation
1 0000 a2 f+(y)(t _dyy) 2 = b2 +1 t 2 ' 0 -
[Ans. f (t) :b ( b _ba)f+ t 2 ') =
10. AN INTEGRAL EQUATION Assuming that integral equation
[A ns. f (t)
=
f E L l (R), solve the 0 , j(w) = 1: e- bt2 e - iw t dt.
Solution . Let
Since
tf E Ll (R) , it follows from 5 . 1 5 .3 and an integration by parts that dj - i 1 00 e - bt 2 t e - iwt dt dw -w 1- 0000 e - bt 2 e -iwt dt 2b - 00 -w -u f (w) . �
Solving the differential equation, we conclude that Since 00 � = e - bt 2 , b - 00 it follows that
f (O)
1
�
Example 5.15.8 - iw ( 1 +4w 2 )2 .
dt = Ii-
Show that the Fourier transform of f (t)
Sol ution . By Example 5 . 6 .4,
By 5 . 1 5.3,
j(w) = j(O) e - w2 /4b .
te - I t I
2 . e - I t l l---+ --1 + w2 .
d 2 = -4iw . dw 1 + w 2 ( 1 + w 2 ) 2
1---+ z - --
0
340
5. The Fourier Transform
Exercises 5 . 1 5
I E L 1 (R) show that f (t) 1-+ / ( -w). For any I, 9 E L 1 (R), if I' E L 1 (R), show that f 9 is differentiable and (J g)' = f' g. [Hint: Use 5 . 15.2 to differentiate under the
1 . CONJUGATES For 2.
*
*
*
integral sign.]
3. LINEAR COMBINATIO NS OF DERIVATIV ES As in 5 . 1 5 . 1 assume that is absolutely continuous in every closed finite interval, 9 E that g(k) E for k = 0, 1 , 2 , . . . , n , and that the g(k) vanish at infinity. For any constants k = 0, 1, 2, . . show that
L 1 (R)
L 1 (R)
ak ,
.
, n,
4. Find the Fourier transforms of (a) (b)
t 2 e-t 2 • t e - a 1 t l , a > 0.
5. MODULATIO N Find the Fourier transform of for 6.
j I E L 1 (R). For f E L l (R), show that (a) f ( a t - b ) 1-+ ! l j(�) e - i ': b . 1
I (t) sin at in terms of
(b) Use the result of (a) and Exercise 5.5-5 to obtain the Fourier transform of the characteristic function l[c,dJ of the closed inter val [c, d].
7. FORM O F ODD TRANSFORM Show that for f then = - 2i
1 00 I(t) sin wt dt.
j (w)
5 . 16
E
L l (R), if j is odd,
Fourier Sine and Cosine Transforms
We consider only real-valued functions in this section. Let The Fourier transform
I E Ll (R). j(w)
i: I(t) e -iw t dt
i: I(t)
cos
wt dt - i i: I(t) sin wt dt
5. The Fourier Transform
341
simplifies to
00 00 i (w) = 1 I(t) cos wt dt = 2 10 I(t) cos wt dt (5 .39) - 00 if I is even. Likewise, if I is odd, we retain only the sine integral. Many functions I (t) of interest are CAUSAL in the sense that they are 0 prior to a certain t; for example , a function I such that I (t) = 0 for t < 0 This gives us some choices for Fourier transforms of such functions: we can define on all of R by extending it as an even or an odd function. This leads us to consider the following transforms; they play an imp ortant role in signal analysis, and heat flow problems.
I
Definition 5.16.1 SINE AND COSINE TRANSFORMS
I E Ll (R) , ic (w) I E Ll (R+ )
Let R+ denote the nonnegative real numbers; obviously, if then ( R+ ) . The Fourier COSINE TRAN SFORM of
I E LI
Fc ! (w) = ic (w) = l oo I(t) cos wt dt; the Fourier SINE TRAN SFORM is (w) is
IS
F. / (w) = is (w) = l oo I(t) sin wt dt.
Clearly, ic is even and is simple observation is that for any
(w)
0
(w) is odd for any I E Ll (R+ ) . Another I E Ll (R) , if I is even, i (w ) = 2ic � ) , (5 .40) -2il. (w) , if I is odd. . Any function I E Ll (R) can be written as the sum of an even and an odd function, I = Ie + 10 ' where Ie (t) = I (t) +2 I ( - t) and 10 (t) = I (t) -2I (-t) .
{
Thus, by equation (5.40) ,
I = Ie + 10
Therefore,
f---+
i = Ie + 10
2Fc (fe) - 2iF. (fo ) .
342
5. The Fourier Transform
As integral operators, the sine and cosine transforms are obviously linear . Suppose that 1 (R) is such that E (R+ ) ; then we can differ entiate under the defining integral of the transforms with respect to by 5 . 15.2 to get
E L2
tf L1
w
Is (w) = Fe (tf (t)) and fe (w) = -Fs (tf (t)) . - I
_ I
Some other elementary properties are the following:
5.16.2 SINE-CO SINE TRANSFORM BASICS For f E L1 (R + ) and any a > 0; (a) SCALE ( DILATION )
( �) � 1.(b) SHIFT. and
Fe
Fe [f (at)] (w) = �1 fe ( �w ) , and Fs [J (at)] (w) = �
[J (t + a) + f ( I t - a l )] (w) 2!c (w) cos aw =
[J ( It - a I ) - f (t + a)] (w) = 2 1. (w) sin aw. (c) MOD ULATIO N For any scalar b 1 Fe [J (t) cos bt] (w) = "2 [fe (w + b) + fe (w - b) ] Fs
�
and
Fe [1
�
(t) sin bt] (w) = � [1. (w + b) - 1. (w - b) ] .
The analogous results hold for Fs .
Proof. Since (a) requires j ust a change of variable, we prove only (b) and (c) . (b) SHIFT Let 9 denote the even extension of to R. Then
f Fe [g (t + a) + g (t - a)] (w) = 1 00 9 (t + a) coswt dt o + 1 00 9 (t - a) cos wt dt. Let u = t + a in the first integral and u = t - a in the second. The integrals
become
100 g (u) cos w (u - a) du + 1: g (u) cos w (u + a) duo
Expand the cosine in each, and split the second integral into get
oo to
I� a + Io
5. The Fourier Transform Fc [g (t
+ a) + 9 (t - a)] (w) cos wa
1a00
+ cos wa + cos wa which equals
0
g (u) cos wu du + sin wa
1a 00
343
0
g (u) sin wu du
L a g (u) cos wu du - sin wa L a g (u) sin wu du
1 00
g (u) cos wu du - sin wa
fooo a = fo
100
g (u) sin wu du,
= f�oo
Since 9 is even, 9 (u) cos wu du 9 (u) cos wu du and 9 (u) sin wu du - 9 (u) sin wu du so we can write the displayed ex pression as
f� a
cos wa
[L:
]
g (u) cos wU du +sin wa
The second term is
[l
[(100 1a -100 ) +
]
9 (u) Sin w u du .
1°O 9 (U) Sin wu du] = 0. oo Similarly, since f�oo 9 (u) cos wu du = (f� oo + fo ) 9 (u) cos w u du , the first sin wa
°O
g (u) sin w u du -
term can be rearranged to yield
Fc [g (t + a) + 9 (t - a)] (w) =
2!c (w) cos aw .
Since a > 0, for t - a 2 0, we have 9 (t + a) 1 (t + a) and 9 (t - a) 1 (t - a) ; 9 (t - a) = 1 (a - t) if t - a < o. Thus, for t > 0, 9 (t + a) + 9 (t - a)
Therefore , Fc
(g (t + a) +
9
(t - a»(w)
=
= 1 (t - a) + 1 ( It - a D . = Fc [I (t + a) + 1 ( It - a l )] (w ) = 2!c (w) cos aw , for a > O .
The second shift formula follows similarly. (c) MO DULATI O N For any b E R,
Jro oooo f (t) cos (w + b) t dt
!c (w + b) =
1
f (t) cos wt cos bt dt -
Fc [I (t) cos btl (w)
-
Fs
1 00 f (t) sin wt sin bt dt
[! (t) sin bt] (w) .
=
344
5. The Fourier Transform
Replacing
b by -b yields lc (w - b) = Fe [! (t) cos bt] (w) + Fs [I (t) sin btl (w) .
Adding,
[/ (t) cos bt] (w) = � [!c (w + b) + !c (w - b) ] . The expression for Fe [I (t) sin btl (w) is obtained similarly. 0 Suppose that I E Li (R+ ) , and we extend I to e E L'i (R) as an even function. As noted in equation ( 5 .40) , e(w) = 2!c (w). If I is of bounded variation on [a , b], 0 � a < b, and is continuous at t E ( a , b), then I (t) can Fe
00 e(w) eitw dw 1 211' - 00 1 211' 00- 00 / (w) ez tw dw 1 1 -PV !c (w ) wt dw 00 ; 1 00 lc (w) cos wt dw. -
be recovered by the inversion theorem 5 .9 . 1 : -pv
1
I (t)
- pv
1
11'
�
.
2 e
cos
....
(5.41 )
a
A similar result holds for the Fourier sine transform. We considered Fourier transforms of derivatives in 5.1 5 . 1 . In particular, if L l (R) is twice differentiable and and are integrable and vanish at infinity, then f" (t) 1-+ What about Fourier sine and cosine transforms of derivatives? To simplify the argument, we assume slightly stronger conditions than are necessary.
IE
(iw) 2 !(w).
I
I'
5.16.3 COSINE T RAN SFO RM OF DERIVATIVES Suppose that I E L'i (R+ ) is twice differentiable, that each derivative is integrable, and that limt ..... I (t) 0 = limt ..... f' (t). Then
=
oo
oo
Fe
(f" (t)) (w) = -w 2 !c (w) - I' (0) .
Proof. Integrating by parts, we obtain Fe
(I" (t)) (w)
r oo f" (t) cos wt dt -f' (O) + w Jo
Another integration by parts yields
10 f'(t)sin wtdt. 00
-,' (0) - w 2 �1o °O I (t) coswt dt -f' (0) - w 2 Ie (w) . 0
5 . The Fourier Transform
f', . . . , f( iv ) are integrable ill f, . , f vanish at + 00, then Fc ( f( i V ) (t) ) (w) = w4 ic (w) + w 2 f' (0) - /' " (0) , etc.
A very similar argument shows that if ..
345
and
Analogous results hold for the Fourier sine transform. In particular (same assumptions about integrable derivatives that vanish at as above) , Fs
and
Fa
( I" (t)) (w) = - w 2 1. (w) + wf (O) ,
+ 00
(f( iV ) (t)) (w) = w 4 1. (w) - w 3 f (0) + w /" (0) .
iw;
The Fourier transform of an integral is obtained by dividing by division by w and switching from sine to cosine transform or cosine to sine transform accomplish this for the sine and cosine transforms.
5.16.4 TRAN SFORMS O F INTEGRALS If f, belong to L1 (R+ ), then for w f- 0, Fc
J; f(u) du,
and
!too f{u) du
(1 00 f{u) dU) (w) = �1. (w)
Fs ( It f{ U) dU ) (w) = � ic (w) . Proof. We prove only the first statement . Since !too f( u) du is integrable and
and the cosine is bounded, their product is integrable, and we can switch the order of integration as follows: Fc
(!too f{u) du) (w)
l
100 [1 00 f(U) dU cos wt dt 100 [lU cos wt dt f( u) du
1 100 w-1 0 sin wuf(u) du w-fs (w) . 0 �
We illustrate the use of these general principles in some examples. Example 5.16.5 EXP ONENTIAL I transforms of e - a t , > are
a 0,
Fc ( e- a t ) (w) =
W
2
a + a2
Show that the Fourier cosine and sine and Fs (e- a t ) (w) =
W
w + a2 . 2
346
5. The Fourier Transform
P roof. By Example 5.5.3
e -at U (t) _1 a +_zw._ , 1------+
i .e . ,
1 00 e -at cos wt dt - i 100 e- at sin wt dt 1 a + iw a - zw a2 + w2 · Example 5.16.6 GAUSSIAN Fa (k e- t 2/2) ( w) = w e -w 2/2 . By Example 5 .5.6 we know that the Fourier transform of e-t2 /2 is e- w 2 / 2 . Since e-t2 /2 is even, e -w 2/ 2 = 2 Fc ( vh e -t2/2) (w ) . Differentiating Fe (J;,.. e -t2/2) (w) under the integral s ig n (justify) with respect to w, we get Fa (.Jk:. e - t2 /2 ) (w) = w e - w 2/2 . 0 1, o t ::; a, , a > Example 5.16.7 RE CTANGULAR PULSE Let f (t) = { 0 , t > a, O . Then, for w =1= 0 , . n aw . 0 a cos wt dt = -lc ( w) lo w Example 5.16.8 FIRST MOMENTS O F EXP ONENTIALS For a > 0 show that the Fourier cosine and sine transforms of t e-at are, respectively, a2 _ w2 and 2aw . (w 2 + a2) 2 (w 2 + a2) In Example 5 . 1 6 . 5 we saw that for a > 0 , a os -at d c e wt = --::-_---=t Fe (e- at) (w) = 100 + w2 a2 o and Fa (e- at) (w) = 1 00 e -at si n w t dt = w2 w+ a 2 . 0
::;
Sl
-----,,-2
Solution .
o
5. The Fourier Transform
347
Fe under the integral with respect to w, we get 00 sm w t dt = - 2aw = ( -Fa -1 (w 2 a 2 ) 2
Differentiating
a
Differentiating
te - at ·
t e - at) .
+
Fa, it follows that
For a, b > 0, show that the
Example 5 . 16 . 9 SHIFTED EXP ONENTIALS
Fourier cosine transform of e- al t - bl is
a ( 2 cos w w 2 + a2
b
- e -ab )
.
Solution . By Example 5.1 6.5,
By the linearity of the cosine transform,
Fe
(e - a(tH) )
(w) =
e - ab
a . 2 w + a2
Hence , by the shifting property 5 . 16 .2(b) ,
W 2 ci a2 cos bw - e -aeb-awb2)+. a2 a
2a
w 2 + a2
( 2 co bw s
-
Example 5 . 1 6 . 1 0 MODULATED EXP ONENTIALS cosine transform of cos a > E R, is
e - at
bt,
0, b
0
Show that the Fourier
Solution . By Example 5 . 1 6 . 5 , the cosine transform of
e-at
is
By the modulation property 5 . 1 6 .2( c) , the effect on the cosine transform of of multiplying by cos is to replace
e-at
bt
348
5. The Fourier Transform
Le. , 1 Fe ( e -at cos bt ) (w) = 2
[ (w + b)a2 + a2 + (w - b)a2 + a2 ] .
0
The next example illustrates the utility of the differentiation property for computing Fourier sine or cosine transforms. Example 5 . 1 6 . 1 1 EXP ONENTIAL I I
Use 5. 1 6 . 3 to show that the Fourier
a > 0, is
cosine transform of I ( t ) = e-at ,
Fe {e -a t) (w ) = Proof. Let I
(t) = e- at . Since f" ( t ) =
a w 2 + a2 .
a 2 e- at = a 2 f (t ) ,
it follows from 5 . 1 6 .3 that
a 2 lc (w ) = Fe (t" (t)) (w) = f' (0) - w 2 lc (w) . Since f' (0) = -a, a 2 I�e (w) - a - w 2 I�e (w) , -
_
(w 2 + a2 ) Ie� (w) = a or Ie� (w) = w 2 +a a2 . 0 We computed the sine transform of f ( t ) = e - at , a > 0, as is (w) [wi (w 2 + a 2 ) ] in 5 . 1 6.5 . So, by 5 . 1 6 .4 on transforms of integrals, w = 1 . Fe ( 1 00 e- au du ) (w) = ! is (w) = ! 2 w w w + a2 w 2 + a2
it follows that
t
In other words,
( a ) (w) = w 2 +1 a2 '
Fe ! e-at or
a w2 + a Recall from Example 5 .6.2 that for a > 0, Fe (e-at) (w) = 1 I ( t ) = __
t2 + a2
�
2 .
0
�e - a lw l .
a
Since I is even, it follows that
lc (w) = .!.. 2a e -a1 wl , w > O .
=
5. The Fourier Transform
Exercises 5 . 1 6 In all the exercises
1.
f
E
L1 (R
349
+) .
MORE ON MODULATION Extend the modulation result 5 . I 6 .2(c) follows: For scalars a > 0 and any show that:
b,
Fe (f (at) cos bt) (w) = 21a [ic ( �) + ic ( W � b ) ] . (b) Fe (f (at) sin bt) (w) = 1a [1. (�) - 1. (W� b) ] . 2 (c) Fs (f (at) cos bt) (w) = 1a [1. (�) + 1. (w�b) ] . 2 (d) Fs (f (at) sin bt) (w) = ;; [ ic (�) - ic (W�b) ] .
as
(a)
2. Use the mo dulation property of Exercise I above to find , for a > 0 and any
b:
Fe ( e- a t sin bt) . Fs ( e-at cos bt) . Find Fe (1�t 2 ) . (a) (b)
3.
4. Assuming that the integrals all exist, show by differentiating under the integral sign (justify) that
Fe (-t 2 f (t) ) (w) = fl/ (W) . (b) Fs (_t 2 f (t) ) (w ) = fll (w) . n ( c ) Fe ( - I t t 2 n f (t)) (w) = jJ 2 ) (w) . n (d) Fs ( - I t t 2n f (t)) (w) = j} 2 ) (w) . (e) Fe ( - It t 2n + 1 f (t)) (w) = j} 2 n + l ) (w) . 5 . Assuming that f' E L1 ( R+ ) and that limt _HlO f (t) = 0 , show that Fs (f' (t)) (w) = -wic (w) . t 6 . Find fo...... oo si�a coswt dt, a > O. (a)
7. Under appropriate smoothness and uniqueness assumptions, use the Fourier cosine transform to solve f" = 0, 2: 0, where f' (0) = (0) 1. Use the Fourier sine transform to solve the same problem.
f =
(t) - f (t)
t
350
5. The Fourier Transform
E Ll (R+ ) , prove that 2lc (W ) ge (w) = Fe [100 l (u) [g (t + u) + g ( l t - u l )] dU] (w) ,
8 . CONVO LUTIO N PRO PERTY For 9
2!s (w) gs (w) . [Hint:
and derive a similar formula for Extend and consider the convolution 9 as even functions and
Ie
ge ,
I and
Take the Fourier transform of both sides of this equation, and use the fact that Ie = = Finally, show that and
(w) 2lc (w) ge (w) 2ge (w). I: ge (t - u) Ie (u) du = 1 00 I (u) (g (t + u) + 9 I t - u l ) du.]
9. INVERSION In equation ( 5.41 ) we noted an inversion result for even
I E L'i (R+ ) is of bounded variation on the closed [a, a < b, and is continuous at t E (a, b), then I (t) = 2- Jo Ie� (w ) cos wt dw. Now suppose that I � 0 decreases on R+ , vanishes at +00, and is integrable on ( 0, a) for all a > O. Show that at any t > 0 : ( a) HI (t - ) + I (t + )] = � J':ooo [ Jo-+ oo I (u) coswu du] coswt dw. ( b ) H/ (t - ) + / (t + )] = � Jo-+ oo [ Jo--+ oo I (u) sin w u du] sin w t dw . [Hint: ( a) By the second mean value theorem Exercise 4.6- 1 0 ( b ) , for all 0 � c < d there is some E (c, d) such that I t I (u) cos wu du l = I / (c) J: cos wu dul � 2/�e) , W > O. This "Cauchy condition" guarantees that Jo--+ oo I (u) cos wu du is uni formly convergent for 0 < a � w � b. Therefore , for any t > 0 , I: [10--+ 00 I(u) coswu du] coswt dw 10-+ 00 I( u) I: cos wu cos wt dw du t 10--+ 00 I(u) (;: cosw (t - u) + cosw (t + u) dw du s a (t u ) I(u) si n tb(t-u 2 Jo u ) m t - u- ] du Si n a (t+ U) ] du o + 2 J(--+o 00 I(u) [ si n tb(t+U) t +u +u extensions: If interval b] , 0 � 7r
-+00
7r
r
1 ( -+ 00 1
_
_
5 . The Fourier Transform
351
Consider any one of these last four integrals for arbitrary 0 < c < d. For example , by the second mean value theorem, Exercise 4.6- 10(b) again , for some s E d) ,
(c,
l Ied I( U ) sin :£t;u ) I du = f(c) l IeS sm :�;u ) 1 du ::; KI (c) d for some constant since I fe s m :£tu- u) I du is bounded (4.5.2(b) ) . K,
K
f (c)
B y hypothesis, for f > 0, < f / for sufficiently large each of the four integrals of the form
I f-· oo f (u) si n :£t; u) I du,
c. Thus,
l Ie--+OO f (u) si n :itu- u) I du, etc.
is less than f for sufficiently large c and all 0 < a ::; selector property of the Fourier kernel 4.6.5,
b. Using the
b�� � I I; 1 (u) si n:£t; u) I du = � [ I (r ) + 1 (t + ) ] ,
while by the Riemann-Lebesgue lemma 4.4. 1 ,
.1'" 1 Jrco f (u) si n t+b(t+u u ) du l = 0 blim oo --+
and
(b) The proof is virtually the same as for (a) . In this case, however, since for d > c > 0, there is some instead of we can write d) such that a E
(c,
fo--+ oo
r:ooo ,
I fed f (u) sin wu du l
0, is
/ E L 1 (R) n L 2 (R), it follows from the Parseval identity 5 . 1 7 .2(a) that j E L 2 (R), and II / II� = 2� I � I: , j (w) = 2 sin aw/w . Since
1 00 ( 2 sin aw ) 2 dw la dt = 2a, w 211' - 00 -a or sin aw ) 2 100 ( -- 00 w dw - lI'a . The special case a = 1 yields sin w ) 2 1-00 ( -:;dw = 11'. 0 �
that is,
=
_
Example 5 . 1 7.4
00
looo ( : ) Si W
Show thai
4
dw =
i'
Sol ution . Consider the Fourier transform (Example 5 .5 .5)
i (w) =
(
)
sin w/2 2 w/2
(t) = - I t I )
L 1 (R) L 2 (R), it j L 2 (R), and 1 � 1 00 ( sin w/2 ) 4 / dw = ( 1 - l t l ) 2 dt = � . 3 w/2 211' - 00 -1
of the Cesaro kernel f 1 [ - 1 , 1 ] ' Since / E (1 follows from Parseval's identity 5 . 1 7 .2(a) that E
n
By a change of variables, it follows that
1: ( : r Si w
Example 5 . 1 7.5
dw
=
Show thai
� or lo oo ( Si:w ) 4 dw = i' [ 00 dw !:. .
Jo
(1 + w2)
2
_
0
4
= e - I t l is i (w ) 2 2/ ( 1 + w ) by Example 5 .6.4. Since / E L 1 ( R ) n L 2 (R), it follows from Parseval's equality 5 . 1 7 .2(a) that i E L 2 (R), and � 100 ( �) 2 1 00 e - 2 lt l
Sol ution . The Fourier transform of the Abel kernel / (t)
211'
- 00
1 +w
dw =
- 00
dt = 1 . 0
=
5. The Fourier Transform
355
sin3 w dw dw = 311" . S Jo w 3 Solution . Let f (t) = 1[ - 1 , 1 ] and 9 (t) = ( 1 - I t /2 1 ) 1[ - 2 , 2]. By Examples 5.2sinw/w, 5 .4 , andand5.5.5g(w)the= Fourier functions are i(w) = 2 w/wtransforms 2 . Both f ofandthese 2 sin 9 belong to L 1 (R)nL 2 (R). Therefore, it follows from Parseval's identity 5.17 .2(b) that Example 5 . 1 7.6
Show t h at
rOO
1 (1 - l t /21 ) dt = � 1 °O 2sinw ( 2sin2 w ) dw. 211" - 00 W w 2 1 Therefore, 2 1 1 (1 - t/2) dt = -32 = -11"2 1-00 sin3w W dw, 00 or 1 00 sin3w w dw = 3411" . 0 - 00 3 Thus,
-1
o
3-
Exercises 5 . 1 7
1. Generalize Example 5.17.4 by showing that for [Hint: Take f (t) = ( I - It I /1' ) 1 [- r , r] (t).]
l'
> 0,
a,
2. Generalize Example 5.17 .5 by showing that for b > 0 , 2 b ( + b) [Hint: Take f (t) = e- a1tl and 9 (t) = e- b 1 t l , and use Parse val's iden tity 5. 1 7 . 2 (b).] 3. Show that: sin5 1'w w = --11511"1'4 lor >- 0 . (a) 1o 00 --w5 384 sin6 1'w dw = -1 1 11"1'5 for >- 0 . (b) 1 00 -w6 40 a
d
o
l'
l'
l'
a
'
356
5.18
5. The Fourier Transform
The L2 Theory
For finite intervals (Exercise 4.4- 1 ) :
Lp (R)
[a, b], the bigger gets, the smaller Lp [a, bj becomes q > p L q [a, bj Lp [a , bj . =?
p
C
The spaces do not descend as p increases. For example , with U denoting the unit step function,
I (t) = � U (t - 1) belongs to L 2 (R), but not to L 1 (R). Hence, even though a function I E L 2 (R), the Fourier transform integral I: I(t) e -iwt dt I L 2 (R),
need not exist . To define a Fourier transform for functions E we must therefore proceed differently. The idea of the Fourier transform for functions is classic analysis: We sneak up on it . We choose a dense subset X of norm (dense in the , we on which the Fourier transform is defined. Then , for E select a sequence from such that fn --+ and define to be limn In · Problems? Is well-defined? If E , does this new agree with what we get by the usual method of calculation? First , for the dense subset of (R) on which the Fourier transform is already defined, the obvious choice is but is it dense in )? It is, because contains the continuous functions n Cc with compact support, and Cc is dense in Some other imp ortant dense subsets are considered in Exercise 5 . 1 9- l . For let
L2
L 2 (R) L2 I L 2 (R) I, j (In) X j I L 1 (R) n L 2 (R) j L2 L 1 (R) n L 2 (R), L 2 (R L 1 (R) L 2 (R) (R) (R) L 2 (R). I E L 2 (R), In (t) = I (t) l[ -n ,n ] (t) , n E N . As functions with compact support, each of the In belong to L 1 (R) L 2 (R). Since In - I E L 2 (R) , it is also clear that I l /n - 1 11 2 o. 11 - 11 2 )
--+
n
We show next the crucial fact that the sequence verges to a function in (R) .
(In) of transforms con
The sequence
constructed above
L2
5 . 1 8 . 1 CON VERG ENCE O F
(Tn)
converges to a function in L 2 (R) .
(Tn)
5. The Fourier Transform Proof. Since each In belongs to
357
L 1 (R) n L 2 (R), each in is in L 2 (R) by
(In)
Parseval's identity 5.17.2(a) . We will show that is Cauchy in the Hilbert space L 2 (R). Since 1m--::-In = 1:. - In, by Parseval's identity,
For m > n ,
and this approaches 0 as m , n -+ 00 . 0 We now define the FOURIER TRANSFO RM j of I 11 1 1 2-limit j = lim n In.
E L 2 (R)
to be the (5.42)
Defined this way, j is also called the PLAN CHEREL TRAN SFORM . To em phasize that the limit is computed with respect to 1 1 · l b , we denote it by l.i.m . for "limit in the mean." Thus, j = l.i.m.n
i- n f (t) e- iw t dt. n
Since 1 I · l b -limits are unique only in the sense of equality almost everywhere, it is important to note that this defines only j (w ) almost everywhere. This quirk occasionally causes some difficulties . There is nothing special about the sequence (fn ) of truncated f's. Any sequence from L 1 (R) nL 2 (R) that converges to f may be used to compute j as the following result demonstrates. 5 . 18.2 WELL-DEFINED If (fn ) and ( g n ) are sequences from L 1 (R) n L 2 (R) that converge to f E L 2 (R), i. e., l.i.m .n In = l.i.m .n gn , then
l.i.m-n In = l.i.m .n g,;.
Proof. By Parse val's identity 5.17.2(a) ,
Since
358
5. The Fourier Transform
it follows that
1 I y,;- - Tnt O. Therefore, l.i.m.n Tn = l.i.m .n y,;-. ---+
0
To See that this mode of computation caU SeS no change in the Fourier transform for functions E calculated in the usual way, consider I(t) e - iw t dt . =
j
I L 1 (R) n L2 (R),
j(w)
I:
In (t) (t)I1[- n,In jfor(t) all, nnEENN. Byandthealmost dominated convergence theo every w E R, n :::; OO j(w) = limn j- 00 In ( t ) e - iwt dt.
=I Let rem 4.4.2, since
This being a pointwise limi t , i t says that Tn ---+ pointwise a.e. A s we note in Exercise 2, mean convergence of a sequence of functions implies the convergence of a subsequence pointwise a.e. to the same limit . Since any subsequence of a pointwise convergent sequence converges to the original limit, it follows that We get the same transform a.e. by either method of computation. Passage to the limit enables us to get versions of many results for functions. Theorem 5 . 18.3 illustrates the procedure. Note that 5 . 1 8 .3(a) establishes the continuity of the linear map 1-+ of into indeed, Were it not for the 27r, it would establish an isometry.
j
L2
L1
I
j
L2
L2 ;
5.18.3 PARSEVAL'S IDENTITIES FOR L 2 FUN CTIO NS (a) If I E L 2 (R), then I / I � = 2� I �I: . (b) If I, 9 E L 2 (R) , then I: I(t)g (t) dt = 2� I: j(w) g (w) dw. (a) Let In (t) = I (t) 1[ - n , n j (t), n E N . Since l j - l 1 I Tn 2 0 , it follows from the second triangle inequality that O . Since l 1 2 1 12 IITn � In E L 1 (R) n L2 (R), it follows from Parseval's identity, 5 . 17.2(a) , that Proof.
---+
---+
Hence, by continuity of the norm,
h�. l /nl1 22 = 1 / 1 22 = j�· 27r1 1 _In 1 22 = 27r1 I 11� 1 22 '
5. The Fourier Transform
359
1.4-3,
(b) By Exercise we can express an inner product by means of the polarization identity; hence , by (a) ,
(f, g )
=
I: I (t) g(t) dt � (II I + g ll� - II I - g ll� + i II I + ig ll� - i II I - ig ll�) + i l i + i9 1 l : - i l i - i g l l : ) �12�/ �( l I i) + g l1: 1- 00Ii i �- gi l :211" \
I, g = 2 11"
- 00
I (w) ( g (w)) dw.
0
Ll
Another basic property of the Fourier transform of functions that survives for functions is the change of roof property of 5 .5 . 1 ( e) .
L2
5. 1 8.4 CHANGE O F RO OF FOR L2 FUN CTIONS If I, g E L 2 (R) then I: l (t) Y(t) dt = i: f(i)g (t ) dt.
n E N , let fn (t) = f (t) 1[-n,n ] (t) , n E N , and gn (t) = g (t) 1 [-n,n ] (t) , n E N . Since II g - gn ll 2 --+ 0, it follows from the Holder inequality (1.6.2) that for Proof. For
I i: j;. (t) gn (t) dt - I: J::. (t) g (t) dt l
any m ,
I,
[1(:) ] . By
I E L 2 (R), and let h (w) = the reciprocity theorem 5 1 9 1 , 1 = (1/271") h or 1 = (1/271") h = h/271". is surjective. To see why, consider .
.
L 9 2 L2
The Fourier transform for functions is also injective by the inversion theorem for if I and are functions with (almost everywhere) equal transforms and then
5.19.2,
j g, I (t) = l.i.m.n 2� I: j(w) e iw t dw = 9 (t) . The aggregate of these results about the L 2 Fourier transform is usually
called Plancherel's theorem.
5 . The Fourier Transform 5 . 1 9.3 PLAN CHEREL ' S THEOREM
5. 18,
The
L2
363
Fourier transform of Section
F : L 2 (R) f
j(w ) = Li.m ·n 1: f(t)e -iw t dt,
where
has the following properties. (a) F is a continu ous linear bijection (5.18.3 , and
5.19.1). 5 .18.3 ) (j, y) = 2 7r {!, g) or l � 1 2 = .)2; lI fI 1 2 • (c) (5.18. 2 ) If f E L l (R) n L 2 (R), then the Fourier transform j is the same whether computed as an L l function or an L 2 function. (d) ( Inversion 5.19. 2 ) n 1 it f (t) = Li . m .n � 27r - n j (w ) e w dw . Now you can see why the Fourier transform f E L l (R) is frequently defined 1 1 00 f(t)e - l.w t dt. f (w ) = V2i -00 This sacrifice for symmetry places a 1/V2i in front of the integral in the inversion theorems instead of 1/ 27r and makes the Fourier map F for L 2 functions a linear isometry-hence a Hilbert space isomorphism by 1.5.3. (b) (Parseval' s identities
as
�
Since surjective linear isometries of complex inner product spaces are called unitary operators, this version is sometimes referred to as the UNITARY FOURIER TRANSFORM of functions. For this formulation we have
L2
n � 1- n f(t)e - iw t dt j(w ) = l.i.m.n v27r n � - n j (w ) eiw t dw . f (t) = l.i.m .n v27r
and
1
Exercises 5 . 1 9
1.
S f E N U {O},
Let (R) denote L. Schwartz 's space of infinitely differentiable func tions on R that are RA PIDLY DECREASIN G in the sense that for all
m, n
I t m JC n ) (t) 1 < ER
sup
t
00 ,
where the superscript ( n ) denotes the nth derivative . The subspace C S (R) of infinitely differentiable functions on R that vanish
D (R)
364
5. The Fourier Transform
outside some closed (bounded) interval is dense in for p ?: 1 ; therefore, is dense in for p ?: 1 , too (see, for example, Szmydt 1977) . Since is dense in we could use in place of n in the definition of the Fourier transform for functions. Show that :
S (R) Lp (R) S (R) L 1 (R) L 2 (R)
L2
Lp (R)
L2 (R),
S (R)
I E S(R),2 then tm f(n ) (t) -+ 0 I t l -+ indeed, I tm I(n ) (t)1 Kj ( 1 + t ) for some constant K . ( b ) It follows from (a) that tm f( n ) (t) belongs to L 1 (R) for all m, E NU {O} . By 5.15.3 on derivatives of transforms, it follows that i is infinitely differentiable , and dn j(w)n = / 00 (- it t f(t) e - iwt dt. dw - 00 Show that i E S (R). [Hint: Consider I(m) (t) E L 1 ( R); f(m ) (t) -+ 0 as I t I -+ 00. Thus, by 5.15.1 on the transform of a derivative, the Fourier transform of I(m) (t) is (iw)m i(w), and (a) If
as
-+
so
f(t) = -211!' 1-0000 f� (w) e'.Wt dw, e -iyt dy. ] f (t) / 00 j(-y) 2 =
(d) For
- 00
I, 9 E S ( R) , verify that :
1!'
5. The Fourier Transform
II / I I � =
I I
365
�:· I (t) (t) dt 2� f�oo l (w) yew) dw. f� oo [Hint: Use Parseval ' s identity 5.17.2.] (e ) Prove that if I, E S (R) , then 1 * E S (R) . [Hint: Since I, E (R) , it follows from (b) that i, y E S (R) . It is straightforward to verify that i · y E S (R) . Now 1· y i.
ii.
21"
9
=
9 9 S
9
=
2.
r:;g E S (R) . It follows from the inversion theorem for I such that 1 E L 1 (R) , 5.12.4, that 1 * 9 E S (R).] (f) Use the dense set S (R) of L 2 (R) to define the Fourier transform i of I E L 2 (R) , and then establish Plancherel's theorem 5 . 19 . 3.
INVERSION AND DERIVATIVES Let L 2 (R) . Show that
IE
i be
the Fourier transform of
I� (w) = -ddw 1-0000 I(t) 1 - zte. - iwt dt a.e. and I(t) = -211r dtd 1-0000 I(w) e;wtzw. - 1 dw a.e . [Hint: Let In (t) = I (t) l[- n ,n ] (t). Then Tn (w) = I: In (t) e - iw t dt and f; f�oox Inn (t)(t) ee -- iiww tt dwdt dwdt f�00oo fo I 1 - e -itx f- oo In (t) it dt. Since inner products ( t , g) are generally continuous ( Example 2.2.5) and ( 1 - e -iw t ) / iw E L 2 (R) , conclude that l x Tn dw = lim 100 I - �- itx dt, lim n o (w) n n (t) zt or lx 1 00 I (t) - �-itx dt , i (w) dw o - 00 which implies d 1 00 I(t) 1 - e. - itx dt a.e . I� (x) = dx 00 zt �
1
- 00
=
-
1
d
The second part follows by a similar argument and Plancherel 's the orem 5 . 1 9 .3 d) .
( ]
366
5. The Fourier Transform
!.:: j
3. THE UNITARY FOURIER TRAN SFORM Let n = l.i.m.n f(t) e - iw t v 21r - n be the unitary Fourier transform of f E L 2 (R) discussed at the end of this section. Its inverse is given by, for 9 E L 2 (R) , n e i w t dw . (t) = l.i.m. n 9 v 21r - n Show that = 1 , the identity map = and that . L L onto of (R) f (R) 1-+ 2 2
Gf (w)
(G - 1 g ) Gf (t) (G - 1 f) (-t)
f
dt
!.:: j
(w) G4
4. SINE AND COSINE TRANSFO RMS In Section 5 . 1 6 we defined the Fourier sine and cosine transforms lc for E L1 (R+ ) and 1. (real-valued) . Now let f E L2 (R+ ) , define = f ( t ) l [ O,n ] , and let = Li .m .n rn f ( ) cos = l.i.m.n [Tnt,
(w)
Fe (w)
(w) f fn (t) t wt dt
[Me
fn
where denotes the usual Fourier cosine transform of E Ll ( R + ) n L 2 (R + ) . Show that the limit i n the mean exists and defines as an element of L 2 (R+ ) (hence a.e.) . Do the corresponding develop ment for the Fourier sine transform Fs In particular, show that :
(w) L It f (t) SIntt dt . t �1t Iooo Fe (w) s, :w t dw . Fs (w) d: It f (t) 1 - c�s wt dt . (t) �1t It Fs (w) l - c;s wt dw.
Fe (w)
(w).
= ( a) Fe (b) f ( ) = = (c) = (d) f
[Hint: Apply the corresponding results for the Fourier transform of f E L 2 (R) in the case where is even, and then odd. ]
f
5 . 20
Pointwise Inversion and Surnmability
f
Note that a complex-valued function is of BOUNDED VARIATIO N if and only if the real and imaginary parts of f are of bounded variation. In 5 .20.2 and 5 .20.3 we get analogues for L 2 functions of the following theorems. 5 . 9 . 1 If f E L1 (R) is of bounded variation in some neighbor hood [t - s, t + s] , s > of E R, then
0, t
f {t - ) + f(t+ )
2
1 ei tw dw PV I�oo 2 1r P limP-+OO l 2 ,.. I- p f e i tw dw .
j(w )
�
(w)
5. The Fourier Transform
367
5 . 12.3 (Fejer-Lebesgue theorem) At any Lebesgue point t of E L r (R), � 1: eiw t [ (w ) dw (C, I) I (t) 2 -d- l r ( 1 - tir ) eiwt [(w ) dw limr .... oo 1r -r = limr .... oo � (f * J{r ) (t) 2 where k (t) denotes the Cesaro kernel (1 - It I) 1 [ - 1 , 1] (t), and I 0,
[
e - iw (x+t) d - r Sin r (x + t) /2 r ( x + t ) /2
w
_
(w) e iw x dw
ei w t e iwx
r
As in the proof of 5 . 1 2 . 1 , it follows that fo r
l ( 1'1)
jw
e iwt ( ) dw , r > O.
� I: ( I� I ) J... j ( 1'1)
9 (x)
0
]2
T ( X + t) , - Hr _
/
370
5. The Fourier Transform
Kr ( u) =
( ru/2) [r (u / 2) 2 ]
where r sin 2 I kernel of Section Thus,
5.12.
denotes the continuous rth Fejer
2 r(x+t l 1 Kr (x + t) . -1r4 sinr (x + 2t) 2 = 2 1r Hence , by equation (5.46), for I E L 2 (R) , r � J Sr (t) 21r ( 1 - �r2 )r eiw t !(w ) dw 1 00 I (x) -4 sin ( -x2 + t2l dx - 00 oo 1r r (-x + t) 1 21r J 00 l (x) Kr (-x + t) dx.
9 (t) =
-r
(5.47)
-
(5.28) 5.12 I E L 1 (R) 1 1 00 1 Sr (t) = 2 1r (f * Kr ) (t) . 2 1r 00 l (x) Kr (-x + t) dx = Since I (x) I (1 + I x l ) E LdR), analogously to how we obtained 5.20.2 we can make some minor adjustments in the proof of the Fejer-Lebesgue inversion theorem 5.12.3 for L 1 functions to conclude that Sr (t) -- I (t) at every Lebesgue point t of f. In summary: 5.20.3 FEn�R-LEBESGUE INVERSION FOR L2 FUNCTIONS If f E L 2 (R), then at any Lebesgue point t E R of I, Iw l ) I� (w ) ezw. t dw = lim -1 (f * Kp ) (t) . I (t) = plim -1 P ( 1 - p ..... oo 2 1r ..... oo 2 1r I_p P In other words, we have arrived at the same point we did in equation of Section for , that -
Exercises 5 . 20
1.
Lp
CONVOLUTION BETWEEN L 1 AND FUN CTIO NS For and 9 1 < p < 00 , show that for almost every fu-n ction w = (x belongs to ,
E Lp (R), (y) I - y) g ( y)
f E L l (R) x E R, the
L 1 (R)
and
(f * g) (x) = J�oo I (x - y) 9 (y) dy belongs to Lp (R) , and also that II I * g ll p � 11/11 1 II g ll p ' where lip + l / q = 1. [Hint: Let h E L q (R), so that gh E L 1 (R) by the Holder inequality (1.6.2). In
order to evaluate
J�oo J�oo I I (x - y) 9 (y) h (x) 1 dx dy
5. The Fourier Transform
by Tonelli's theorem 4.19.2, with
u = x - y,
371
consider
by Holder's inequality with v = - u ) . This is clearly equal # 0 for (R) such that h to . Now choose any R. Then we see by the above that is finite a.e . , so (R) . It is also clear that the linear functional
(x) g (x h E Lq (x) II f ll 1 11 g ll p Il h ll q xE f�ooo I f (x - y) g (y) I dy f (x - y) g (y) E L l
C, F : L q (R) ) (x) h (x) dx, h (f � g * f oo is bounded, and Il F ll q :S Il f l1 1 1l g ll p . Since the continuous dual of L q (R) is Lp (R), it follows that f * g E Lp (R) and II I * g ll p :S 11 1 11 1 II g l i p . ] 2. A CON VOLUTIO N THEOREM Let I E L 1 (R) and g E L 2 (R) . Show that h = 1 * g E L 2 (R) , and that h (w) = j(w) g (w) . [Hint: By the preceding exercise with p = 2 , it follows that h E L 2 (R) . For gn (t) = g (t) I [ - n,n] (t), II g - gn l1 2 ---+ 0, and l.i.m .ng,;- = g. Now let hn = f * gn. Since gn and hn belong to L 1 (R) n L 2 (R) , -+
1-+
it follows that
-;;;, = jgn .
By the preceding exercise,
Therefore , by the Parseval identity 5 . 1 8 . 3 ,
Since
I � E Co (R) , I� i s bounded b y some number M , and 1 -;;;, - jg l 2
I jg,;- - jg ll
:S M 11 g,;- - gl1 22 ---+ 0 as n ---+ 00 .
=
Combining these results, we get
a.e .]
II h - jg ll 2
L2
=
0 so
h (w) = j (w) g (w)
3 . SUMMABILITY KERN ELS AND FUN CTIO NS Here is a version of 5 . 1 3 . 4 on summability kernels for ( R) be functions. Let k such that k (R) n (R) is an even function with
E L1
L2
1 2 11'
00
L2
L oo k (w) dw
f, E L 2
�
=
1.
372
5. The Fourier Transform
[{rex) = rk (rx), r > 0 , show that 11 (1/27r) (f [{r) - 1 11 2 -+ 0 r -+ and
With as
*
00 ,
[ Hint: By Exercise 5 . 19- 1 and 5 . 1 8 .4 we have
J�oo 1 (t + y) [{r ( y) dy 2� J� 1 (t - y ) [{r ( y ) dy 2�� (f oo [{r ) (t) a.e . , 2 *
where 2�
(f [{r ) E L 2 (R) by Exercise 1 . The fact that *
follows from Exercise 5 . 1 4- 1 .]
5.21
A S ampling Theorem
(tn), E N ,
If you know infinitely many values 1 n of a function, do you know I ? Of course not. There are obviously infinitely many distinct ways of connecting the points so that a function is defined by the ensuing graph. Nevertheless, there are many ways in which a denumerable amount of information about a function suffices to determine it: If 1 is a sufficiently smooth integrable periodic function, for example, and you know the Fourier coefficients ; or if 1 is analytic in a region and you know all its derivatives at some point Zo Thus, there is some precedent to believe that if you know something about the character of 1 (integrable , periodic, analytic) as well as a denumerable amount of information, then 1 can be reconstituted. In an intriguing application of transform theory we show in 5 .2 1 . 1 that if 1 (R) is band-limited in the sense of Definition that vanishes outside some interval [- K , K] , then 1 can be recovered from knowledge of the values 1 K ) , by means of
(tn , / {tn))
E D.
j ew)
E L2
D,
L2
5.7.1,
(n7r/ ( n7r ) sin ( [{t - n7r) . 1 (t) = '" 1 L..J n EZ [{ Kt - n7r
Theorems like this-recovering 1 from certain o f its values-are known as sampling theorems (also interpolation theory) , and they go back a long way. The result of this section was essentially first discovered by Cauchy in 1841 , then rediscovered by Whittaker in 1 9 15 . Shannon made remarkable applications of sampling theory to communication theory ( 1 948 ) . Beutler 's 1961 article is of interest in this evolving and imp ortant area.
5. The Fourier Transform
373
5.21 . 1 SAMPLING THEOREM If 1 E L 2 (R) is continuous and band-limited, > K for some constant K, then 1 is determined by its
i (w) = 0 for I w l
values at a discrete set of points:
sin �f{t f (t) = L 1 (n�) l\ t - mr 1\
mr
)
nEZ
.
Proof. Since i (w) is band-limited (of compact support) , i E L 1 (R) n L 2 (R) , and by the inversion theorem for L 2 functions ( 5 . 1 9 . 2) , 1 n � 1.m 1 (t) n 211" - n I (w ) eiwt dw (5 .48) K 1 ! (w) e iwt dw a.e. (i E L I (R) ) . 2 11" .
l.
.
j
-
j
-K
Since J!:K i (w) e iwt dw defines a continuous function of t (see the proof of 5.7.2 that i (w) is continuous) , and since 1 is continuous, equation (5 .48) holds for all t . Since i E L 2 [-K , KJ , we can expand it in a complex Fourier series
i (w) = where
K
�
.
�
( ;;) = ;1 ( ;; ) .
By equation (5.48 ) , however, 211"
so
(5 .49)
n EZ
2\ j
dn = 1 ? } 1 -
L dn e - in rr w /K ,
-K
i ( w ) e i n rrw/ K dw .
j-KK I (w) e,wn rr/K dw =
dn = { 2 11" 1 2
1
(n1l" -) , K
Substituting the Fourier series of equation (5 .49) into equation (5.48) yields 1
(t)
j
1 K � . 1 (w ) e, wt dw 2 11" - K 1 / � - wt (inner product) 1, e , 211" \ dn e- i n rr w / K , e -iwt . /" 211" \ L...- n E Z
�
.
)
)
374
5. The Fourier Transform
Since the inner product is continuous (Example 2.2.5(b) ) ,
f (t)
d ( e -i m r w /K , e -i wt ) J.-211" " L...t n EZ n K 1 __ " n EZ f (�) j- e iw (K t -mr)/K dw K 2K L...t K ) ( e i w( K t - n 1r) /K K n1l" 1 f [( i ([(tn1l")- n1l") I -K 2[( L n EZ K n1l" ([(t sin ) ( " f . L...t n E Z
K
0
K t - n1l"
The quantity aT = 11"/ [( is called the NYQUIST INTERVAL , its reciprocal v = [(/11" is the NYQ UIST RATE , and the result is often written in terms of [( [(
" L...t
them as
naT) f (t) = f (naT) sin (t (t--naT) .
nEZ
fE
f
Suppose that L 2 (R) is time-limited rather than band-limited: (t) = It I > K for some real number K . Then f E Ll (R) n L 2 (R), and it is easy to modify the preceding argumen t to show that is determined by its values at = n1l" / K, n E
o for
i (w)
Z.
w
Another Approach
There is another way to view the sampling theorem that is j ust irre sistible. Consider the functions 11" -i n1r w /K l [ -K , K] ( ) e gn (W )
- - /{
w.
_
Then
n .l.m ·n 211r j n U-n (W ) e iwt uW J.- j K � e -i n1rw /K e i wt dw 211" /{
Un (t)
I.
J
-K
sin (Kt - n1l") /{t - n1l"
In other words, the quantities si nk����1r ) are j ust the inverse Fourier trans forms of the g.;. Since the e -i n 1r w /K are an orthogonal basis for L 2 [- K, K] , it follows from Plancherel's theorem 5 . 19.3 that the sin �-t -n n1r) 1r constitute an
k
orthogonal basis for B = E L 2 (R) = 0 for > K . The sam pling theorem then gives a pointwise expansion for the continuous members of B in terms of this basis as well as 1 1 1 1 2 -convergence.
{f
: i(w)
Iw l
}
5. The Fourier Transform
5 . 22
375
The Mellin Transform
f (t) e - iwt. 1 t8 - f )
The Fourier transform involves integrating The Mellin trans form is concerned with integrals of ( t , t E e , i.e . , with "moments" of I t resembles the Fourier transform in some respects, and because it involves moments , it has many applications in mechanics and statistics. For those familiar with the idea, we mention that the Mellin transform of is the same as the bilateral Laplace transform of
f.
f (t)
f (e - t ) .
Definition 5.22 . 1 THE MELLIN TRANSFORM
f and the complex number s are such that (M f) ( s) = f (t) t s - 1 dt exists, then M f is the MELLIN TRANSFORM of f. More generally, if f is integrable over any finite interval ( a , b), 0 < a < b, and f (t) t s - 1 dt (M f) ( s) = converges for some complex number s r + iv , we take M f to be the MELLIN TRANSFORM of f. As equation (5 .5 1 ) below illustrates , M f can be expressed in terms of If the function
100
1 -+0 00 -+
0
=
Fourier transforms. For sufficiently well behaved functions, we have a fairly symmetric inversion formula.
5.22.2 INVERSION OF MELLIN TRANSFORM Let s = r iv denote a complex variable, let f be of bounded variation in a neighborhood of t E R, and suppose that r - 1 f ( ) E L 1 ( R+ ) for some r E R. For s such that -
x
x
(5.50)
exists,
Proof. With
f (t-) ; f (t + ) = -1 .
t = eY, and B (Mf) (r
2
= r -
- iv
iv)
lr+ia (Mf)
11"1 a-+ oo r - i a
=
lim
(s )
r s dB.
(r as above) in equation ( 5 . 50) , we have (5.51) 1: f( eY)erYe-iy'IJ dy,
which i s the Fourier transform w ( v ) of w bounded variation in a neighborhood of =
(y) = f(eY )ery . Since f i s of t eY, it follows by 5 .9 . 1 that �pv1°O w� ( v ) eiy'IJ dv = w (y- ) + w (y+ )
2
11"
- 00
2
'
376
5. The Fourier Transform
or
J... PV 21r
j oo00 M f ( r
_
-
iv )
e iy v dv = W (Y - ) +2 W (y+ ) .
e - ry , we obtain J...21r pv joo00 Mf ( r zv ) e - (r - iv )Y dv = e _ry w (y- ) +2 w (y+ ) . Since t = eY and s = r - iv , � j r +i a lim (M f) (s) r S ds = f (r ) +2 f (t + ) . a_ oo 21rZ r - i a 5.22.3 MELLIN TRANSFORM BASICS Assuming that the integrals exist for f and for any complex numbers a and b, (a) LINEARITY M (af + bg) = a Mf + bMg. (b) SCALING For a > 0, 1 [Mf (at)] (s) = a s (Mf) (s) . Multiplying both sides by
_
.
-
0
g:
(c) TRANSLATION For any complex number
a,
[M W f (t))] (s) = (Mf) (s + a) . Proof. We use It below, rather than Io--+ oo ; the same argument is valid in
either case. (b) Consider the Mellin transform
( M f (at)) (s) = 1 f (at) tS - l dt . 00
With W =
(c)
Now
at this becomes (M f (at)) (s) (Mt a f (t)) (s)
1
ws - 1 f ( w ) dw s-1 a a f a s (Mf) (s) . 00
--
r OO t a f (t) tS - l dt
Jo 1 oo ta+ s-1f (t) dt
(M f) ( + a) . (Mf) (s) = j --+0 OO f (t) t S- l dt. --+ s
0
5. The Fourier Transform
With 9
(y) = I
( e Y ) , this becomes
(MI) (s) =
377
L�:g(y) eSYdY.
If h is integrable over every finite interval and the improp er integral exists, it is customary to call
T (s) = 1 -+-00 h (t) e - s t dt -+ 00 the BILATERAL LAPLACE TRAN SFORM of h.
B y contrast, the usual (one-sided) Laplace transform of a function defined on (0 , 00) and integrable over every finite interval is
h
(ef) (s) = 1-+ 00 h (t) - s t dt. e
Exercises 5 . 22
1.
I( ) =
Find the Mellin transform of t t a u (t - e) , where U is the unit step function, e is a real constant , and a is a complex number. [ A n . s) for Re s < Re (s) _ ea + s
s - a.] (Mf) = f(a + 2 . Let (M f) ( + i v) be in L1 (R) as a function of v and also be a func tion of bounded variation as a function of v in some neighborhood of v = Show that if 1 lim J,r!".zai a ( Mf) (s) t -S ds, s = r + i v, l (t) = 2 . a-+oo r r
y.
11"1
then
-+ oo
alim ft,a l ( t ) tr + iv - 1 dt =
t [(M f) ( + iy+ ) + (M f) (r + i - )] .
y
r
[Hint: As in the proof of 5 .22 .2, use 5 .9 . 1 , and make an appropriate change of variable.]
a
3. Let I W ) , > 0 , and let e < d. Show that (a) (b)
(1 ft ) I (1 f t) be integrable over all (e, d) , °
0, let I (t) but I (t) sin wt ( R+ ) .
L l (R+ )
E Ll
=
( lit) e -a t , t >
O. Th e n
I fI.
Proof. B y Example 5 . 1 6 . 5 , the Fourier sine transform of e - at is given by
Fa (e - a t ) (w) = Jo[00 e- a t sin wt dt = w 2 w+ a 2 .
Integrate both sides of this equation with respect to a from a to 00 , then interchange the order of integration on the left side (justify) to get
1o 00 _e- at
_
t
or
sin wt dt =
1o 00 e - at
-- sin wt dt t
1a00
w -:_-:-2 da 2 w +a
1f' a - - tan - 1 - ' 2 w
=
w tan - 1 - . a
It is common to see this called the Fourier transform of I (t) to see it written as (w) tan - 1 (wla ) . 0
i =
= e - at It and
We leave the details of the following example to Exercise 3 .
379
5. The Fourier Transform Example 5.23.2 For f (t) = ( 1 / ..Ji) f (t) cos wt (j:. L 1 ( R + ) , but
r-+ oo cos wt dt
10
..ji
=
,
we have
t > 0,
rrr- , V�
w>
f (j:. L 1 (R+ )
and
o.
Exercises 5 . 23 1 . ALTERN ATIVE ARGUMENT FOR EXAMPLE 5 . 2 3 . 1 . Show that 1 4' SIn . wt dt e-t- tan - a ' a > 0 , Jor oo W
by differentiating under the integral sign. [Hint: Let 9 (w ) =
e-4' . Jor oo t SIn wt dt .
Differentiation with respect t o w then yields
g' (w)
foooa e- a t cos wt dt a 2 +w 2 ,
which implies that 9 (w) = tan - 1 (w / a) + C where C is an arbitrary constant . Since 9 (0) = 0 and tan - 1 0 = 0, we get C = O. Thus , 9 (w) = tan - 1 (w/a) .J 2. Use the result of Example 5 . 6 . 3 to find fooo t(�i2n+,:t2) dt for a >
O.
3 . Show that fo-+ oo C (�t dt = ,.fE, w > O. [Hint: Let 9 (z) = e-Z z a - 1 , 0 < < 1 , which is analytic for Re z 2: 0 , Im z 2: 0, except at z = O . We integrate 9 around the contour C below
a
c.
00
380
5. The Fou rier Transform
Izl Ie e - Z z a - l dz = z
z
from r to R where 0 < r < R, then along = R from 0 ::; arg ::; 'Ir/2, then along the imaginary axis from iR to ir, then over =r from arg z = 'Ir/2 to arg O . Now 0 by the Cauchy integral theorem. Since � 0 as � 0, it follows that the integral along the small circular arc approaches 0 as r � 0 (See, e.g., Sansone and Gerretson 1 960 , p . 130) . Along the large circular arc , the integral becomes
z= zg (z)
and
0, and
r
Jo
and
1-00
a = t to get
2
.... oo cos wt dt = f1r V 2::
Vi
1 .... 00 --
4. Use the previous exercise to find, for t > 0 and 0
o
cos wt d -t an d t1-a 0
< a < 1,
sin wt d t. t1-a
6
T he D iscret e and Fast Fo urier Transfo rms The Fast Fourier transform-the most valuable numerical algo rithm of our lifetime . -G. Strang, 1 993 For some functions I, it is impractical (indeed, well-nigh imp ossible) to evaluate the Fourier transform
j(w)
=
1: I(t)e -iwt dt.
I n such a case w e truncate the range of integration t o an interval [a , b] and then approximate the integral for j (w) by a finite sum such as
j (w)
�
N- l
L I (t k ) e - iwtk D.. t .
k =O
This latter sum is called the discrete Fou rier transform D I of /, and it is very useful indeed . As it happens, f can be computed as a matrix product. Certain terms and patterns recur in the matrix. A method that exploits this recurJ,'ence is called the fast Fourier transform algorithm. The efficacy of the fast Fourier transform is prodigious; it can, for example , reduce a billion computations to about a million-to less than a thousandth of the original number! It is one of the major technological breakthroughs of the twentieth century. One thing that Fourier series and Fourier transforms have in common is that the functions on which they operate are defined on groups-the circle group T , R, or Rn . When we discuss the discrete Fourier transform we consider a different group as the common domain, but this time it is finite (the integers modulo an integer) . The crucial thing is the ability to recover the function from its transform.
D
Z
6.1
The Discret e Fourier Transform
As our first approach to the discrete Fourier transform, we treat it as an approximation. The cases of greatest interest concerning the Fourier trans form occur when I Ll (R) is not a familiar function but a complicated
E
384
6. The Discrete and Fast Fourier Transforms
signal, one such that
i (w ) =
1: f(t) e- iw t dt
cannot be evaluated in closed form. For sufficiently large a
< 0 and b > 0 ,
i s a good approximation t o i( w ) for any E (R) . To approximate this integral, we sample the signal at a finite number of equally spaced times =a O. Let
f L1
to
tN - 1
b-a D.. t = � and t k = a + kD.. t ,
Then the approximation tf; of i is given by
k
=
0, 1 , 2 , . . . , N.
" N=o- 1 f (t k ) e - iw t k D..t L....J k N kb N e - iw a " L....J k = O1 f (t k ) e -iw ( - a)/ D.. t .
( )
tf; w
(6. 1)
We further take the time duration [a, b] into account by focusing attention on the points ( frequencies )
Wn = -b27rn -a' where n is an integer. At these points
N- 1 e - i aw n L f (t k ) e -i 27r n k /N D.. t . k=O Neglect multiplication by the constant e - i aw n D.. t and focus attention on the N-periodic function Df Z C, N- 1 Df (n) = L f (t k ) e - i 27rn k /N , n E Z , k=O tf; (w n ) =
:
-+
or ( the same thing )
N- 1 Df (n) = E f (t k ) w - n k where w = e 27ri /N , n E Z . k=O
( 6 . 2)
f
Let us view things from a "discrete" perspective . Forget that was de fined on As we are dealing with the values of at only a finite number of
R.
f
6. The Discrete and Fast Fourier Transforms
385
points , suppose that f is defined on { O , I, . . . , N - I } . In other words, view f as an n-tuple of complex numbers: f = (f (0) , f ( I ) , . . . , f (N - I ) ) E C N . Or consider f as defined on the cyclic group of integers modulo the positive integer N , Z N = Z/ (N) (Z modulo N) where (N)
and
f:
ZN k + (N)
--->
�
c,
=
{kN : k E Z } ,
f (k) .
Another possibility is to view f as the N-periodic function defined on Z by taking f ( k + nN) = f (k) on Z , k = O, I , 2, . . . , N - I , n E Z ,
which is quite analogous to the 27r-periodic functions of Chapter 4. Definition 6.1.1 DISCRETE FO URIER TRANSFORM
Suppose that the finite set ZN is equipped with the discrete measure (all subsets are measurable and the measure of any subset is the number of elements in it) . Since Z N is finite, any function defined on it is integrable ; thus, Ll (Z N ) = L 2 (Z N ) = C N , the collection of all functions f : Z N -+ C . (a) The DIS CRETE FOURIER TRANSFORM (D FT ) of f : Z N -+ C is Df (n) =
N- l
L f ( k) e - 27rikn/N , n E ZN .
k =O
We depart from standard notation here in the use of D f, i being the common designation. As DJ is N-periodic, we can view it, too, as defined on Z N . (b) DISCRETE FO URIER TRAN SFORM OPERATOR The map
is called the DIS CRETE FO URIER TRANSFORM MAP or OPERATOR or (un fortunately) also the discrete Fourier transform. 0 The DFT in Matrix Form We have defined the DFT of f as
Df (n) =
N- l
L f (k) w - n k ,
k =O
where w
=
e 27ri /N .
386
6. The Discrete and Fast Fourier Transforms
In order to achieve a certain symmetry between function and transform (see the inversion theorem 6.2.1) , two commonly used variations of the definition of the DFT use 1/ N or I/Vii as normalizing factors : DI ( n ) =
and
� L l ( k) e - 27ri k n/N N- l k =O
1 DI ( n ) = __
N- l
L 1 ( k) e - 2 7ri k n/ N .
Vii k = O
(As we mentioned earlier, some authors define the Fourier transform with a 1/$ in front of the integral for j(w) for reasons of symmetry.) Another variant is to replace Z N by the cyclic group of Nth roots of unity. We could also view 1 and D 1 as vectors 1=
(
1 (0) : I (N - 1)
)
and DI =
(
DI (O) : DI (N - 1 )
)
.
With
MN =
1 1 1
1 e - l · 2 7r i/ N e - 2 . 2 7r i/ N
1 e - 2 · 2 7r i/ N e - 2· 2 . 2 7r i/ N
1 e - (N- l ). 27r i/ N e - 2 (N - 1 ) 2 7r i/ N
1
e- (N - l )· 2 7r i / N
e - 2 ·(N- l).2 7ri /N
e - (N- l)' . 27r i / N
(6.3) then DI = MN f. MN is also called the NTH ORDER D FT matrix. With 7r N , w = e2 i/
MN =
1 1 1
1 ii; ii; 2
1 ii;2 ii; 4
1 ii;N- l ii; N- 2
1
ii;N- l
ii;N- 2
ii;
As mentioned above , the DFT is sometimes defined so that MN /Vii is obtained, thereby becoming a unitary operator . Since the DFT map (Definition 6 . 1 . 1) is a matrix product , it is clearly a linear operator. The analogs of some of the basic properties of (w) of 5 .5 . 1 are proved in 6 . 1 .2.
j
6 . 1 . 2 DISCRETE FO URIER TRANSFO RM BASICS For 1 : Z N � C , with as the independent variable of f in each case, and any (fi x ed) j E Z,
n
6. The Discrete and Fast Fourier Transforms
(a) TIME SHIFT
Df (n) e -27rijn /N j
1-+
EXP ON ENTIAL MULTIPL ICATION
f (n) e i 2 7rjn / N
f (n - j)
387 1---+
D f (n - j ) ; (c) MODULATION f ( n) cos 21rjn/N 1-+ � [Df (n - j) + Df (n + j)] . Proof. (a) Time Shift. Let 9 (n) = f (n - j) . Then N- l N- l n N ik f (k - j) e - 2 7r ik n / N . / 7r -2 e Dg (n) = L 9 (k) L k=O k=O With m = k -j the latter sum becomes E ;;::�j j f ( m ) e -27r i ( m +j ) n / N . Split "", N l - j . (b) FREQUEN CY SHIFTING
1-+
=
t h e sum
Since
L.J m=- - j
mto
N - l -j +L m=L-j m=O -1
f is N-periodic, -1 m +j ) n / N NL- l f ( m) e - 2 7ri (m +j ) n / N . i ( 7r -2 e ( f ) m m=L- j m=N -j =
Thus
Dg (n)
1 N- l j ", L..J m- = -l J. f ( m) e - 27r i ( m+j )n / N + ", L..J m N=o - fj ( ) e - 2 7ri (m +j ) n /N m nN ", L..J mN =N - J. f ( m ) e -2 7ri ( m+j ) n / N + ", L..J m =-lo f ( m ) e - 2 7r i ( +j ) / (DfL(n)::e�-f27r(imjn)/Ne-2. 7rimn /N) e - 27rij n /N (b) Frequ ency shift. The DFT of f (n) e i 27rjn / N is E f=-OI f (k) e 27rij k /N e -27rik n /N E f;OI f (k) e -27r ik (n - j ) fN f (n m
=
D - j) .
( c) Modulation. Since f (n) cos 21rjn/ N = � ( f (n) e 27r ij n / N + f (n) e - 27r inj / N ) , it follows from (b) that the DFT of f (n) cos 21rj n/ is � Df (n - j) + � Df ( n + j) . N
0
Some examples are in order.
388
6. The Discrete and Fast Fourier Transforms
I : Z4 --+ C be 1 at (1, 2, 2, 1).
Example 6. 1 .3 D F T O F CONSTANT FUN CTION L et 01 1 and 9 =
n = 0, 1, 2, 3: f = (1, 1, 1, 1) . Find the DFT Sol ution . In this case
(1, 2, 2, 1), then Dg = (6 , - 1 - i, 0, -1 + i) . Example 6. 1 .4 Find the DFT 1 9 = (1, i, i 2 , i3 ) = (1, i, -1, -i) . Solution . We can use the frequency shift property 6 . 1 . 2( b ) to find the DFT. If we take 9 =
0
0
With f the constant function of Example 6 . 1 . 3,
9 (k) it follows that
= 1 ( k) e 21fik/ 4 = 1 (k) ik , k = 0, 1, 2, 3,
Dg (0) = Df (0 - 1) = Df (3) = 0, Dg ( l ) = DI (I - I) = DI (O) = 4, Dg (2) = DI (2 - 1) = DI (I) = O, Dg (3) = Df (3 - 1) = DI (2) = 0. Example 6. 1 .5 Let h : Z C be d efined by taking h (k) = 9 (k - 1) 4 2 where 9 = ( l , i, i , i3 ) = ( l , i, - I , -i) is as in Example 6. 1.4. Find th e DFT ol g . Solution . Instead of computing the DFT directly as in Example 6 .1 . 3, we use the time shift property 6.1.2(a): Dh (n) = Dg (n) e -21f i n / 4, n = 0, 1, 2, 3. 0
--+
Hence
Dh = (0, -4i, 0, 0) .
0
Exercises 6 . 1
DI denotes the discrete Fourier transform of f. 1. CONJ U G ATES Let 1 : ZN R.
In the exercises below,
--+
6. The Discrete and Fast Fourier Transforms
DI(n) = DI ( -n) N - n.) D
2.
RO OTS OF UNITY For a positive integer N , let that
"-n" in Z N
w = e 2 1Ti / N . Show
= 1. k w = w - k w N+ k = wN - k for any integer k . ( c ) 1 + w + w2 + . . . + wN- 1 = 0 , N > 1 . C b e given by 1(1) = 1 and I(n) = 0 for n =P 1 . Let I : ZS (a) ww (b)
3.
n.
(a) Show that for all (Note that can be taken to be (b) Show that if 1 is even, then 1 is real and even.
389
=
�
(a) Find Df.
= 1(2) = 1 , I(n) 0 for n =P 1, 2. e - an , for some constant a . Show that 4. Consider I : ZN C, n N DI(n) = 1 -1e-- ae--.. a2 1Tn N 5 . Let N = 4 and let I : Z 4 � C be given by 1 = (0, 1 , 1 , 1) . Find Df. 6. TRAN SFORM OF I (n) = n Find Df for the following 1 : Z N C . Show that for any N, when f(k) = k , k = 0 , 1 , . . . , N - 1 , ) DI(n) = -N/2 + 2 (i1N-sincos(2(2 nn/ N/ N)) (b) Find DI for 1( 1 ) �
=
1---+
.
/
�
7r
7r
7. DFT OF A CONSTANT As a generalization of Example 6 . 1 .3 find the D F T DI of � C where 1 for all
I : ZN
I(n) =
n.
8. VANDERMONDE MATRIX The Vandermonde matrix is 1
Zo z� V (zo, . . . , zN-d = ZoN - 1
1
ZN l Zh _1ZNN_- 1l
(zo , . . . , zN - I ) = I1 ��i�lO .j < i (Zi - Zj ). Show that : MN = V (1 , w , w 2 , . . . , W N - 1 ) where w = e 2 1Ti N . det MN =P O .
Its determinant det V ( a) (b)
1
Z1 z 12 Z 1N - 1
/
390
6. The Discrete and Fast Fourier Transforms
Hints 2 . (c) . Note that 1 + x + X 2 + . . . + X N- 1 = ( 1 - X N ) / ( 1 - x) . 6 . Use the fact that
which equals - N/ ( 1 - r ) if rN = 1 . Then use this result with r = w- n = e - 2 1r in/ N and simplify.
6.2
The Inversion Theorem for the DFT
By an "inversion theorem" we mean a way to recover f from its discrete Fourier transform. We consider one in 6 .2. 1 . Some other avenues to inver sion (same result, different proofs) are considered in equations ( 6 . 5) , (6 .7) , (6 .9) , and (6 . 1 1 ) . Note that the space L 1 (Z N ) of all functions f : Z N � e with respect to the inner product { f, g} =
N- 1
L f( k )g ( k ) ,
k =O
f, g
E L 1 (Z N ) ,
is just e N with its usual inner product and is therefore a Hilbert space. Our first inversion theorem 6 .2. 1 formally resembles the inversion theo rem for the Fourier transform 5 . 12.4. Before proving it, we need the elemen tary observation that l + x + x 2 + . . ' + X N- 1 = (1 - x N ) / (1 - x) for x i- I ; it follows that for any Nth root of unity w = e 2 1r i k / N , k = 1 , 2, . . . , N - 1 , 1 + w + w2 + . . . + wN- 1 6.2.1 INVERSION THEOREM For f : Z N
Df(n) =
�
=
O.
e, w = e2 1r i/ N , with DFT
N- 1
N- 1
k=O
k =O
L f(k)e - 2 1ri k n/ N = L f(k) w - k n ,
we have f(n)
=
� NL- 1 Df(k)e 2 1ri k n/N = � N-L1 Df(k) w k n . k =O
(6 .4)
k =O
39 1
6. The Discrete and Fast Fourier Transforms Proof.
N1 kn N L -Ol Df(k)w ) �� ( N 1 N L k = O LJ. =-O f(j)w - kj W n k ) N- 1 ( N - l N LJ. = O f(j) L k =O W ( n - j ) k . If n = j, then E :';Ol w ( n - j ) k = N , while if # j, then E�;Ol w ( k - j ) n = 0 N1 N L k =-O Df(k) e 2 1ri k n / N
.1..
.1..
.1..
.1..
n
by equation (6 .4) . Hence
N- l L Df(k) e 2 1rik n / N = �N f(n)N = f(n). N k=O
�
0
( 6 . 5)
It follows from the invertibility of the DFT that it is a linear bijection. 6.2.2 DFT MAP IS A LINEAR BI�ECTION The discret e Fourier transfo rm map (Definition 6. 1 . 1)
is a lin ear bijection.
W = e 2 1r i / N and consider the functions Wj : ZN C with m wj (m) wj for m,j = 0, 1, ... , N - 1 , or , in vector notation , Wj ( m ) = ( 1 , , -W2j , . . . , ;;-;;(W N - l )i ) . (6 .6) Lemma 6.2.3 AN O RTHONORMAL BASIS FOR L 2 ( ZN ) = L 1 ( Z N ) Th e functions {Wj lIN j = 0, 1 , . . . , N - 1 } form an orthonormal basis for L 1 ( ZN ) (see Section 3 .4 for the definition of orthonormal basis). P roof. Since L 1 ( Z N ) is N-dimensional , it suffices to show that the Wj /.,fN Next, let =
-+
=i OUT
:
are orthonormal. First, note that
( .,fN Wj , ..jN1 Wk ) 1 N�- l Wj (n)w-k (n) 1
Obviously,
=
N
N- l k ' n 1 w ( -J) . = N
�
(l/N) E �;Ol w( k - j )n = 1 if k = j, and by equation (6.4) , N- l ( l/N) L w ( k - j ) n = 0 if k # j. n= O 0
392
6. The Discrete and Fast Fourier Transforms
{ Wj / VN : j = 0, . . . , N - 1 } is an orthonormal basis for L 1 ( Z N ) , so is the set of conjugates {wj/VN : j = O , . . . , N - l } . Thus, any I E L 1 ( ZN ) can be written N- 1 1 1 (6.7) Wj . 1 = r; I, VN Wj VN Since ", N-1o I(m) -j;; wj (m) L....J Nm = ", -1 / ( m ) 1 w -mj L... .J =O TN '" W - l ( m) 1 e - 2 1r i mj / N , L....J m =o / TN Since
)
(
( I, J& Wj ) J& DI(j)
it follows that
=
, j
= 0, 1 , . . . , N - 1 ,
( 6 . 8)
DI ( ) VN (I, -j;; Wj) I Dl ( ) I E L 2 [-11", 11"] , e: = j(n) e in t , I (t) = L I, ve: 2 11" v 2 71" nLE Z nEZ where j(n) = 2� J�1r I(t) e - i n t dt is the usual Fourier coefficient of I · Since DI(n) = L �;Ol l(k) e -2 1r i n k / N , with W k (n) = e -2 1r ik n / N , we can write DI(n) L:�Ol I( k )wn( k ) wn ( k ) . VN ",N-l ) I( k =o L....J k ..IN Consequently, since {wd VN j = 0 , . . . , N - I} is an orthonormal basis for L 2 ( Zn ) = L 1 ( Z N ) by 6 .2 . 3, we have
and times the coefficient in equation ( 6.7) . Thus, j is may be written as a linear combination in which the scalars are mul tiples of j , analogous to the Fourier series expansion of a function
)
(
:
or
I(n)
=
N
1
(DI, wn) � L :�Ol DI(k)wn (k)
( 6 . 9)
6. The Discrete and Fast Fourier Transforms
393
which is another way to prove the inversion theorem 6 .2 . l . To emphasize the analogy with the Parseval identities for Fourier trans forms ( 5 . 1 8 . 3 ) , we denote by each being equal to in 6.2 .4.
eN ,
L 1 (ZN ) L 2 ( ZN ),
f, 9 E L 2 ( ZN ). Then: l l (a) l: :: f(k)g(k) = � l: ::O D f(k)Dg(k) . Ol (b) l: ::o l f(k) 1 2 = � l::: l ID f(k) 12 . O
6.2.4 PARSEVAL'S IDENTITIES Let
Proof. (a) By equation ( 6 . 9 ) ,
l:::: f(n)g(n)
�N - l ( 1 �N-l Df(k)wn (k) ) ( 1 �N - l Dg(j)wn (j) L..t n = o N L..t k =O N L..ti = o N - l -1 � N - l --� N - l Dg (j) L..t k =O Df(k) N1 � L..t n =o wn(k)wn(j) N L..ti =o -1 �N - l �N - l Df(k)Dg(j)8 ki =o =o k L..t N L..t i -- . 1 �N- l Df(j)Dg(j) N L..ti = o (b) Follows immediately from (a) by setting 9 = f. Another way to obtain the inversion theorem utilizes the inverse of MN . Recall that with W = e 2 1r i / N ,'Df = MN f, where -
0
MN =
1 1 1 1
1
iiJ
iiJ 2
iiJN- l iiJ N - 2
iiJ
( 6 . 10)
L 1 ( ZN ) ( 6.2.3 ) (l/VN) MN (Wk (j)) = (w ki ) , or
wdVN, i = 0 ,
Th e orthonormal basis vectors are the rows of the unitary matrix (with k as the row variable) ,
Clearly ,
iiJN1 - 1 iiJN -2
iiJ12 iiJ4 .
. . , N - 1 , for =
Wk (j) = wkj = Wj (k). Thus MN = Mfv , its transpose .
394
6. The Discrete and Fast Fourier Transforms
With
A = (aij ) we denote the adjoint A* of A by A* = (aj ; ) . Since the
w;fVN are orthonormal,
N N
it follows that
N o o
0 N 0
0 0 N
o o o
o
0
0
N
= NI
where I is the x identity matrix. Thus ( l /N) MN DI MN I, it follows that
=
Nl .
M-
1 = MN l D I = � MN (DJ) .
· S mce (6 . 1 1 )
Thus, we have a third way of obtaining the inverse of the DFT.
Exercises 6 . 2 1 . EVEN TRANSFORM Prove the converse of Exercise 6 . 1- 1 (b) : Show that if E L and D is even, then is real and even.
I I I l ( ZN ) 2. ON D 2 I For any I E L 1 (Z N ), use the inversion theorem 6 . 2 . 1 show that for any n E Z N , D [DIl (n) = NI (-n).
to
3 . Using the inversion theorem 6.2 . 1 o r equation (6 .9) o r (6. 1 1 ) , calculate I from in Examples 6 . 1 .4 and 6 . 1 .5.
DI
I = 2:f=-Ol a/wj /VN,
Wj is as in 6.2.3. N, j = 0, . . . , N -
4. PERIO D O GRAMS Let where The PERIOD O G RAM of I at the frequencies Aj 2n-j / 1 , is the set o f values I(Aj ) =
=
l aj 1 2 . ( a) Show that l aj l 2 = 1:1 1 2: �=-01 l(k) e -2 1f ij k /N I 2 = 1:1 I DI(j) 1 2 . (b) Show that 1111 1; = 2:f=�l I( Aj ) .
The periodogram plays an imp ortant role in stationary discrete Lz-processes because I(Aj ) can be expressed in terms of the sample covariance function of the process.
N N matrix is ROW CIRCULANT if it is of the form CN - 1 Co C l C2 C CN - 2 CN - l o C1 CN -3 , CN - I ) = CN -2 CN - l Co C = circ(co , C l , C l Cz C3 Co
5. An
x
·
·
·
6. The Discrete and Fast Fourier Transforms
395
Successive rows are obtained by shifting the elements of the row above one place to the right with wraparound at the last column . Since the columns can be obtained in the same way, we can just as well say that the matrix is COLUMN CIRCULANT:
C = circ
CI C2 Co CN- I Co CI CN - 2 CN - I Co
Co CN- I CN-2
CI
CI
C2
CN- I CN - 2 CN- 3 Co
C3
Since it does not matter , we simply use the term CIRCULANT. Davis 1979 discusses circulant matrices in detail. (a) Let S be the SHIFT MATRIX circ(O, 1 , 0, . . . , 0) and let A be an N x N matrix. Show that AS shifts the columns of A one place to the right, and SA shifts the rows upward one place with wraparound at the edges. (b) Show that if C is any circulant matrix, then CS = SC.
1 and (c) Show that the characteristic polynomial of S is ).N consequently its roots are w, . . . , w N - 1 where, w = e 2 7r i / N . (d) Show that any N x N circulant matrix C can be written in the form C = Col + CI S + C2 S2 + . . . + CN_ 1 SN - 1 , -
where I stands for the identity matrix. (e) Let FN = -iN MN , where
1
MN =
1 1
1
iiJ iiJ2
1
iiJ 2 iiJ4
1 iiJN - I iiJ N - 2
1
iiJ N - l iiJ N - 2
iiJ
is the DFT matrix of equation ( 6 . 10) . Show that FN diagonalizes S in the sense that FN SF'N = D,
where D is a diagonal matrix whose entries are 1 , W, w 2 , . . . , wN- 1 down the main diagonal. FiV stands for the adjoint of FN (which in this case is j ust the conj ugate, since MN is symmetric) . (f) Show that FN Si FiV = Di for all positive integers j. (g) Show that FN diagonalizes any N x N circulant matrix C.
6. The Discrete and Fast Fourier Transforms
396
6.
Let FN be
as
in Exercise 5 .
(a) Show that
(FN ) 2 = (F;:. ) 2 =
1 0 0 0 0 0 0 0 0
0 0 0 1 1 0
0 1 0
0 0
(b) Show that (F;:') 4 = (FN )4 = I. (c) Show that the distinct eigenvalues of FN are
h
± 1 , ±i.
7. CHARACTERS Let : Z N --+ C be N-periodic, such that I h (k) 1 = 1 for all k , and additive: ( x + y ) = ( x ) + ( y) for all x and y. Such an is called a CHARACTER . Show that for any character
h
(a) (b) (c)
h
h
h
h:
h (O) = 1 . h ( -k ) = h (k) - 1 for all k . h (k) N = 1 for all k.
Hints
1. 5.
Use Exercise
6.1-1 ( b ) and
6.2. 1.
(f) Use (e) . (g) B y (d)-(f) ,
FN CF;:'
c 1 FN F;:' + C2 FN SFN + . . . + cN FN SN - 1 FN = C1 l + C2 D + . . . + cN DN - 1 .
For more detail, see Barnett
6.3
the inversion theorem
1990.
Cyclic Convolution
The imp ortant thing about convolution is the property that the Fourier transform of a convolution offunctions from L 1 (T) or L 1 (R) is the product of the transforms (see 5.2. 3 and 5.8.2 ) , i.e . , = 19. To see what happens for the DFT, we first define a notion of convolution of functions f, 9 E L 1 (ZN ) . We show in 6. 3.3 that the DFT of a convolution is the product of the DFTs.
r;-g
6. The Discrete and Fast Fourier Transforms
CYCLIC CONVOLUTION
Definition 6.3.1
f
The DISCRETE (CYCLIC) CONVOLUTION, * 9 of at each E by
k ZN
N- l f * g (k) = "L. f(j)g(k - j) . j =O
f, 9 E L l ( ZN ) is defined ( 6 . 12)
0
The basic properties of the cyclic convolution are listed in 6.3 .2 and
6.3.2 CONVOLUTION BASICS L e t (a) f * g = g * f; (b) f * * = ( f * * (c) (f * g ) = ( f ) * ( for any ( d) 1 * + = 1 * 9 + 1 *
f, g , h E L 1 ( ZN ).
a
397
(g h) a (g h)
g ) h; ag)
h.
6.3.3.
Th en:
constant a ;
Proof. Since this i s so straightforward, w e prove only part (a) . By definition,
,,\,jN - l l(j)g(k - j) � =o ,,\, - ( N - l) I(k - )g (m) � m=O
k - N+ l I(k - m ) g ( m) "L.m=k ,,\, N - l I(k - m) g (m) . 0 �m =O 6.3.3 CONVOLUTIONS To PRODUCTS For I, g E L 1 (Z N ), the DFT D (f * g ) (n) is e q a l to DI(n)Dg (n) for all n. m
u
Proof.
N1 "L. k =-O 1 * g (k) e - 27r ik n / N ,,\, N - l 1 * g (k)w k n (w = e 2 7ri /N ) � k= O ,,\, N - 1 ( ,,\,jN - 1 l(j) g (k - j ) -Wk n �k = O � =O ) ( 6 . 13 ) N- 1 N l n k "L.J. =O l(j ) ( "L. k=O g ( k - j)-W ) ,,\,N - l f U )-WJ. n ( ,,\, N -0l g (k - j)-W( k -J. )n � k= �j = O ) = D 1 ( n) Dg (n) . 0 We noted for f E L l (R) and L l (T) that there does not exist an identity for the convolution operator (see Exercise 5.2-10 for L l (T) and at the D (f * g ) (n)
398
6. The Discrete and Fast Fourier Transforms
beginning of Section 5.14 for Ll (R)). This is not the case for L 1 (Z N ) . The function e ( k ) = (1 , 0, . . . , 0) ( 6. 14) satisfies 1 * e = I for all
I E L 1 (ZN ) . This fact is used in the exercises.
Exercises 6 . 3
1.
Complete the proof of 6.3.2.
(a) determine 1 * f. (b) For g (k) = k , k = O , I , . . . , N - l , find / * g.
We say that I E Ll (Z N ) is INVERTIBLE if there exists 9 E L l (Z N ) such that 1 * 9 = e.
3 . INVERTIBLE ELEMENTS IN Ll (Z N ) (a) Show that vanishes.
I
E L 1 (Z N ) i s invertible if and only if
DI never
I
(b) For noninvertible E L 1 (Z N ) , find all 9 such that 1 * 9 = o. (c) We say that 1 =P 0 is a divisor 01 0 when there exists 9 =P 0 such that 1 * 9 = o. Show that I is a divisor of 0 if and only if I is not invertible . (d) For invertible I, show that = for every
[DI - l ] (n) I/DI (n)
4. POWERS A N D THE (a) Let I : Z 2 where e =
DI.
�
n.
D FT
I
C be given by = (0, 1) . Show that 1 * 1 = e, the identity for convolution . Also, determine
( 1 , 0),
I
(b) Let I : Zg � C be given by I = (0, 0, 1 ) . Show that * 1 = e, where e = ( 1 , 0, 0), the identity for convolution. Determine (c) Generalize the results of ( a) and (b) for 1 : Z n � C is given by 1 = (0, . . . , 0, 0, 1 ). (d) In general, characterize all 9 : Zn � C such that the n-fold convolution 9 * 9 * . . . * 9 = e in terms of the DFT Dg .
DI.
6. The Discrete and Fast Fourier Transforms
399
Hints
3.
(a) . First note that e(n) = 1 for all n . Then use 6.3.3 on D (J * g ) = DfDg . (b) Find f, g E L 1 (Z N ) with (DfDg) (n) = Df(n)Dg ( n) = 0 for all while f :j:. 0 and 9 :j:. O.
n
4.
(d) . Use the convolution to product theorem
6.4
Fast Fourier Transform for N
6.3.3. =
2k
Modern signal processing requires the ability to process huge numbers of bits rapidly. A color TV picture, for example , requires about eight million-a megabyte-per second. To evaluate the discrete Fourier trans form D f = MN f, f E e N , requires N 2 multiplications and N (N 1) additions-a formidable thicket of computations for large N . The FAST FOURIER TRANSFORM (FFT) algorithm for N = 2 k reduces the N 2 multi plications to something proportional to N log 2 N . For example (see 6 4 . 3 ) , for N = 2 1 = 32, 768, it reduces -
5
.
approximately one-thousandth of the multiplications. Using it makes it effectively possible to force the flow of a fire hose through a garden hose , as in forcing a video signal through a telephone line , for example . Though the applications to image processing, optics, geology, etc . , are contemporary, the idea of the FFT is not . Indeed , Heideman et al. [ 19 84] trace the notion back to Gauss. They mention that he expressed it in a clumsy notation and that his work on the subject was published only posthumously. The mo dern development begins with the seminal article of Cooley and Tukey [ 1965] . In this section we investigate the FFT technique known by the various aliases INDEX REPRESENTATIO N , RADIX-SPLITTING , or TIME D ECIMATION . There is a symmetric approach-FREQUENCY DECIMATIO N -in which the roles of rows and columns are exchanged . This symmetry is related to the symmetry of the DFT matrix MN of equation ( 6.3) of Section 6.1. The approach is a classic-split a large process into groups of smaller (half as many at each stage) subprocesses. We count how many steps the FFT procedure takes in 6.4.3. To discuss the process we need the notion of even and odd parts of a function .
6. The Discrete and Fast Fourier Transforms
400
Ie and 10 of I E e N For I E e N and N even we define I" e v en " and l "oM' to be Ie = ( 1 (0) , / (2) , . . . , I ( N - 2)) and 10 = ( 1 (1), / (3), . , I ( N - 1 )) .
Definition 6.4. 1 EVEN AND ODD PARTS
..
The FFT algorithm consists of repeatedly decomposing a function
e N into even and odd parts as follows:
lee /
I
/
Ie
'\.
!
/
leo
!
'\.
1
'\.
loe
!
!
Ieee leeo
10
'\.
leoe
leoo
loee
100
0
IE
( 6.15)
loeo
We compute the DFT of functions like lee o e and then synthesize D I from them. Consider what happens for N = 4 in the following example . Example 6.4.2 USIN G
For
I E e4 ,
DIe
( (
AND
D lo
DI
)( ) )
TO GET
we compute its DFT directly as
DI =
1 1 1 1
1 1 1 -i - 1 i -1 1 - 1 - 1 -i i
1 (0) 1 ( 1) 1 (2) 1 (3)
(6 . 16) 2 (1) [/(0) + 1 ( )] + [/ + 1 (3)] [/(0) - 1 (2) ] - i [/ ( 1) - 1 (3)] . [/ (0) + 1 (2)] - [/ ( 1 ) + 1 (3)] [/ (0) - 1 (2)] + i [ /(1) - 1 (3)] Note that each row of DI has 1 (0) , I (2)-the terms of Ie-and the terms of 10 , I (1) and I (3) . Rewriting the matrix equality of equation ( 6 . 16 ) as _
four equalities , we get
DI (0) DI ( I) For
= = =
[/ (0) + 1 (2) ] + [/ ( 1 ) + 1 (3) ] , [/ (0) - / (2) ] - i [f ( 1 ) - / (3) ] ,
Ie = ( 1 (0) , / (2)) and 10 = ( 1 ( 1 ) , / (3)) , D Ie = M2 !e =
( � !1 ) ( ���� ) ( ���� � �g� ) =
(6. 17)
40 1
6. The Discrete and Fast Fourier Transforms
and
D 10 = Mdo = Thus
(6.17)
( � �1 ) ( � gj ) = ( � gj � � ��j ) .
DIe (0) = 1 (0) + 1 ( 2) , DIe (1) = 1 (0) - 1 (2) , becomes
DI ( O) DI ( l )
etc . , and equation
DIe (0) + D lo (0) , DIe ( 1 ) - iD lo (1) ,
(6 . 18)
Thus, knowing DIe and D lo permits us to calculate Df. Equation (6.18) is a special case of the butterfly equations (see equations (6.25)-(6.27)) . 0 In 6 .4.3 we count how many multiplications are needed to compute
DI .
6.4.3 FFT FOR N = 2 k For N = 2 k and I E e N , the number of multi plications re quired to compute DI is 2N log 2 N = 21:+ 1 . k. Proof. We prove this b y induction o n
DI
��
k. For k = 1
and
I E e2 ,
Md = -
_
) ( � ��j ) 1 (0) + (1) , 1 ( ) - 1 (1) ) �1
0
I
which involves 4 multiplications or 2 . 2 1 log 2 2. Assume that the result holds for N = 2 k - 1 , i.e., that 2 · 2 k- 1 1 0g 2 2 k - 1 multiplications are required for N = 2 k- 1 , and let N = 2 k . With q as a variable, let Wq = e2 1f i/ q , so that the DFT of I is
N- 1 DI ( n) = L I (k) wN" , n = 0, 1 , 2, . . . , N - 1 . (6 . 19) O = k Divide k by 2 and n by N /2 and write each in terms of a quotient ( ko and no, respectively) and a remainder (k 1 and nd: k = 2ko + k 1 , n = (N/2) no + n1 .
Since k and n vary between 0 and N - 1 , the possibilities for quotients and remainders are ko = 0, 1 , . . . , N/2 - 1 , k 1 = 0, 1 , no = 0, 1 , and n 1 = 0, . . . , N/2 - 1 . Hence
402
6. The Discrete and Fast Fourier Transforms
w� k o n o
(WN WN raised to any multiple of N is 1), we can . 1 N/ 2- 1 n k o o � DI ( n ) = w L I ( 2 ko + k d vl{/2) k , n o w;t 0 n ' WN L k , = O k o =O or, by setting k l successively to 0 and 1 , wNNk o n o L Nk /2= O- 1 I (2k0 ) w2Nk o n , DI(n) o N/2 - 1 ,,\, N/2 n o k n n (6.20) n Nk ( o o w + N k (2 I 0 + 1 ) wN ) w2N o , wN' . L...t k o = O Since w'jy WN/ 2 , we can rewrite the w;t° n , in each summand as w�/�' to get Since = 1 or rewrite equation (6 1 9 ) as
k ' '' ' ,
=
=
DI(n)
WN ( N/ 2)n o WN Il , WN Il• This simplifies w NNk o n o L Nk o/=2O- 1 I (2 k0 ) W-k N/2 o n , (6.22) N/2 - 1 1 (2 ko + 1) W k o n , , + WN Il L N / 2 ko=O
l
Replace ( N/2 ) no + n by the second summation:
DI ( n ) no = 0, 1 , n l
=
=
n;
then
=
0, 1 , . . . , N/2 - 1 . The terms
N/2 - 1 N/2- 1 n k o , I (2 k o + 1 ) WN/2 k o n , and o W ) (2k L 1 N/ 2 L k o =O k o =O
are the rows of
D ie = MN/ 2
( ) 1 (0) 1 (2) :
and
I (N - 2) so we can rewrite equation ( 6.22) as
D lo
=
MN/ 2
( )
( 6.23)
/ ( 1) 1 (3)
:
'
I (N - l)
Now let us count what is needed to get the values of DI ( n ) for every To get
n.
6. The Discrete and Fast Fourier Transforms
403
for every nl we need 2 (N/2) log 2 N/2 = N log 2 N/2 multiplications by the induction hypothesis . We need the same number to calculate D fo (nd, for a total of 2N log 2 N/2. For a fixed n, we need two multiplications (by w�ko n o and WNIl) to get Df (n) . Thus, for n = 0, 1 , . . . , N - 1 , we need 2N more multiplications, bringing the count to Remark In calculating the number of multiplications in 6.4.3 to be 2N log 2 N, we counted multiplications by 1 , so that
requires 4 operations. If we do not count multiplications by 1 , we get ( N/2 ) log 2 N multiplications, one-fourth of 2N log 2 N. The result also appears in this form. The key to the FFT algorithm is equation (6.24) , which expresses terms of Dfe and Dfo . With no = 0 and 1 , respectively, equation yields what are known as the BUTTERFLY RELATIO NSHIPS
Df(n)
=
Dfe (n) + wNIlDfo ( n), O :S n :S N/2 - 1
Df in (6 .24) (6.25)
and
Df(N/2 + n)
= w�ko Dfe (n) + WNN/ 2 + n Dfo(n),
O :S n :S N/2 - 1 . (6.26)
1 and wN/ 2 -1 , this simplifies to = Dfe (n) + wN"Dfo (n) , O :S n :S N/2 - 1, Df(n) Df ( N/2 + n) Dfe (n) - wNl1Dfo ( n) , O :S n :S N/2 - 1 . ( 6.27) The FFT algorithm continues to split fe and fo into even and odd parts fee , feo , foe , foo , etc . , until we get two-dimensional vectors . Then we synthesize D f from the DFTs of the parts . We outline the procedure for N = 8 . For f = ( 1 ( 0 ) , / (1) , . . . , / (7)) E C8, (6.28) Since w�ko =
=
=
we have
fe = ( 1 (0) , f(2) , f(4) , f(6)) and fo = ( 1 ( 1 ) , f(3), f(5), f(7)) . (6.29) Split fe and fo into its 2-dimensional even and odd components : fe e = ( 1 (0), f(4)) , and foe = ( 1 (1) , / (5)) , (6.30) fo o = ( 1 (3) , f( 7 )) . feo ( 1 (2) , f(6)) , Now compute Dfee = Mzfe e and Dfe o = Mzfeo and use equations (6 .27) to obtain D fe . Similarly, use D fo e and D foo in equations (6.27) to obtain D fo . Finally, use D fe and Dfo in equations (6 .2 7) to obtain D f. We present an actual calculation in Example 6.4.4. =
404
6. The Discrete and Fast Fourier Transforms
Example 6.4.4 A D FT CALCULATIO N FOR Calculation . Apply M2 =
(�
�1
)
I E C8 .
to
lee = ( 1 (0) , / (4)) , and loe = ( 1 ( 1 ) , / ( 5)) , 100 = ( 1 (3) , / ( 7 )) , . leo = ( 1 (2) , / (6)) , to obtain
-
D lee = D le o
_
�
1 (0) + 1 (4) 1 (0) - 1 (4) 1 ( 2) + 1 (6) 1 (2) - 1 (6)
)
'
and
'
-
-
Dloe = D loo
_
�
1 ( 1) 1 ( 1) 1 (3) 1( 3)
Now use equations ( 6.27) to compute Dle and Dlo :
= Dlee (n) + � D leo (n) , 0 Dle (n) Dle ( n + 2) = D lee (n) � D leo (n) , 0
( (-
D Ie _
Similarly,
( 1 ( 0) + 1 (4) ) + ( 1 (2) + 1 (6) ) ( 1 (0) - 1 ( 4) ) - i ( l( 2) - 1 (6)) ( 1 (0) + 1 (4) ) - ( 1 (2) + 1 (6)) ( 1 (0) - 1 (4) ) + i ( l (2) - 1 (6))
( 1 ( 1 ) + 1 (5)) + ( 1 (3) + 1 (7)) ( 11 ) - 1 (5)) - i ( l (3) - 1 (7)) Dlo ( 1 ( 1 ) + 1 (5)) - ( 1 (3) + 1 ( 7 )) ( 1 ( 1 ) - 1 (5)) + i ( l (3) - 1 ( 7 ) ) Finally, w e use equations (6 .27 ) t o compute DI : _
= Dle (n) + WS'l Dlo (n) , 0 D I (n) D I(n + 4) = Dle (n) - WS'l D lo (n) , 0
S; S;
+ 1 (5) - 1 (5) + 1 ( 7) - 1 (7)
S; S;
S; S;
n n
) .)
), .
1, 1.
.
n n
S; S;
3, 3.
With W8 = e2 7r i/ 8 in equations (6 .27 ) we get D 1 =
[/ (0) + 1 (4)] + [/ ( 2) + 1 (6)] 1 (0) - 1 ( 4) - i [/ ( 2) - 1 (6 )] 1 (0) + 1 ( 4) - [/( 2) + 1 (6) ] 1 (0) - 1 (4) + i [/ ( 2) - 1 (6)] 1 (0) + 1 (4) + [/ (2) + 1 (6) ] 1 (0) - 1 (4) - i [/ (2) - 1 (6) ] 1 (0) + 1 ( 4 ) - [/ ( 2) + 1 (6)] 1 (0) - 1 (4) + i [/ (2 ) - 1 (6) ]
+ [/ ( 1 ) + 1 (5)] + [ / (3) + 1 (7) ] + e - 2 7r i/ 8 ([I ( 1 ) - 1 ( 5) ] - i [/ (3) - 1 ( 7 )] ) + e - 4 7r i/ 8 ( [/( 1 ) + 1 (5) ] - [/ (3) + 1 (7) ] ) + e - 67r i !8 ([/ ( 1 ) - 1 (5)] + i [ / (3) - 1 (7)] ) - ([1 ( 1 ) + 1 (5)] + [/ (3) + / (7) ] ) _ e - 27ri/ 8 ([I ( 1 ) - 1 (5) ] - i [ / (3) - 1 ( 7 ) ]) _ e - 4 7r i/ 8 ( [/( 1 ) + 1 (5) ] - [/ (3) + 1 ( 7 ) ] ) _ e- 6 7r i / 8 ( [/( 1 ) - 1 ( 5) ] + i [ / (3) - 1 (7) ] ) (6 . 31 )
6. The Discrete and Fast Fou rier Transforms
405
Buneman's Algorithm
The pairs of terms in the rows of (6 .3 1 ) , ( / (0) , I (4)) , ( / (2) , I (6) ) , etc. , are the terms of lee. leo , loe and 100 ' respectively. Forget the I for a moment and just look at the arguments of I: 0 , 4, 2 , 6 , 1 , 5 , 3 , 7. The technique known as BUNEMAN 'S ALGORITHM permits us to calculate these arguments directly. It consists mainly of multiplying by 2 and adding 1: ( 0, 1 ) � (0, 2 ) � ( 1 , 3 ) co�n e (0 , 2, 1 , 3 ) . This yields the order of the terms in the rows of D I for I E C4 ( see Example 6 .4.2 ) . For N = 8, as in Example 6 .4.4, ( 0, 2 , 1 , 3 )
x 2 ( 0, 4, 2, 6 )
--+
+1
--+
( 1 , 5, 3 , 7
) combin e --+
( 0, 4, 2 , 6 , 1 , 5 , 3 , 7 ) .
Thus lee = ( / (0) , / (4)) , leo = ( / (2) , / (6)) , loe = ( / ( 1) , / (5 )) , and 100 = ( / (3) , / ( 7 ) ) . As a simple exercise, use the algorithm on ( 0 , 4, 2 , 6 , 1 , 5 , 3 , 7 ) to see what happens for N = 16. The numbers 0 , 1 , . . . , 7 written in binary form are 000, 00 1 , . . . , 1 1 1 . Buneman ' s algorithm transformed (0 , 1 , . . . , 7 ) into (0, 4, 2, 6 , 1 , 5 , 3 , 7 ) . In particular, the entry 1 in position 1 = 00 1 2 ( we begin the count at 0 , not 1) ended up in position 4 = 1 00 2 . The binary representation of its final location is the reverse of its original. This general phenomenon is known as BIT REVERSAL. For example , if N = 16, 1 ( 13 ) = 1 ( 1 1 0 1 2) will ultimately be in position 1011 2 = 1 1 .
Exercises 6 . 4 1 . Show that the column vector in equation (6 .31) is indeed equal to Ms/ . 2. ( a ) Use equations (6.27) to show that when multiplication by 1 is not counted, it is possible to evaluate D I in ( N/2 ) log 2 N steps. ( b ) With both ( N/2 ) log 2 N and 2N log 2 N as the number of re quired operations instead of N 2 , find the percentage savings in operations for N = 21°, 2 2 0 , 23° .
406
6. The Discrete and Fast Fourier Transforms
3 . Use Example
6.4.4 to obtain Df when f is given by f (n) =
{ n,I ,
O :S n :S 5 , 6 :S n :S 7. .
4. The fact that MN is symmetric and that M;;/ = (1/ N) MN ( Section 6 .2, equation ( 6.11 ) ) supports the statement that there is a process by which one obtains f from D f similar to that of Example 6 .4.4. Write the details of the process of Example 6.4.4 and use it to show that the inverse of D f for D f( n ) 1 for 0 :S n :S 7 is the multiplicative =
identity of the convolution operator
1 o e=
o o Hints 2. ( a) . Begin by determining that Md requires only one operation and proceed by induction. (b) Recall that MN f would be performed with N 2 operations.
6.5
The Fast Fourier Transform for N
=
RC
We consider the less dramatic improvement in calculation of the DFT for the case where N factors but is not a power of 2, in this section . Let N = RC, where r or C is not a power of 2. We can view the calcu lation of D f = MN f as the computation of N dot products-each row of D f is (row of MN) • f. The FFT of 6 .5.2 reduces the N dot products to R dot products of vectors of dimension C, or C dot products of vectors of dimension R; we illustrate in Example 6.5 . 1 . This reduction will not occur if N does not factor, however ( Exercise 2 ) . Example 6.5.1
REDUCTION WHEN
N
=
6
We group terms in M6f in two ways, by reversing the roles of 3 and 2 . Method 1. With w = e 2 1ri/ 6 and f (k) = for k = 0 , 1 , . . . , 5 ,
ak
6. The Discrete and Fast Fourier Transforms
M6 / =
I I I I I I
I I I3 I I
w w2 w3 wt WS
P
u
w wt I I w3 2 w I wt w3
wt w2 I wt w2
wS wt w3 w2 w
407
ao at a2 a3 a4 as
Thus
M6 /=
ao + at + a 2 + a 3 + a4 + as ao + atw + a 2 w2 + a 3w3 + a4wt + asws ao + alw 2 + a 2 wt + a 3 + a4w2 + aswt ao + alw3 + a 2 + a 3 w3 + a 4 + asw3 ao + alwt + a 2 w 2 + a3 + a4wt + as w 2 ao + atwS + a 2 wt + a3 ws + a4w2 + asw
( 6 .32 )
Since 6 = 2 · 3 , we group the terms in M6 / in two groups of three terms. In the first of the two groups we put those terms whose subscript divided by two has remainder O-namely, ao , a 2 and a4-in the second, we put those whose subscript divided by 2 has remainder I-at , as and as-as in
M/ 6
(ao + a 2 + a 4 ) + (at + a3 + a s ) (ao + +a 2 w2 + a4wt) + (at w + a 3 ws + asw5 ) (ao + a 2 wt + a4w2 ) + (at w2 + a 3 + aswt) (ao + a 2 + a4) + (alwS + a 3 w3 + asw3 ) (ao + a 2 w 2 + a4wt) + (at wt + as + asw2 ) (ao + +a 2 wt + a4w2 ) + (at wS + asw3 + asw)
=
( 6 .33 )
Since WS k = e - 2 1ri 6k / 6 = I for any integer k, we can factor the terms as below: o
M6 / =
(a o + a 2 +�) + (a1+as+a s ) I (ao + a 2w2 + a4wt ) + w(at + asw 2 + aswt) 2 (ao + a 2wt + a4w2 ) + w2 (al + as wt + asw 2 ) . 3 ( a o + a 2 + a4 ) + w3 ( a t + a3 + aS ) 4 (ao + a 2 w2 + a4wt) + wt(at + a 3 w2 + aswt ) 5 (ao + +a 2 wt + a 4w2 ) + w S (at + a 3wt + asw2 )
. ( 6 . 34 )
Group rows whose remainder is the same when divided by 3-namely rows and 3, I and 4, and 2 and 5. In rows 0 and 3 , note the recurrence of (ao + a 2 + a4 ) and ( al + a3 + as) . In rows and 4, (ao + a 2 w2 + a 4 wt) and (a t + asw2 + aswt) recur . Similarly, in rows 2 and 5 , (ao + a 2 wt + a 4w 2 ) and (al + a 3wt + as w 2 ) recur. Later, we make use of the fact the parenthesized sums need only be calculated once .
o
I
Method 2.
Now we consider an alternative grouping. We collect the terms of M6 / by grouping terms whose subscripts, divided by 3 , yield the same remainder:
408
6. The Discrete and Fast Fourier Transforms
( 0 , 3 ) , ( 1 , 4) and ( 2, 5 ) . Thus, (ao + a3 ) + (al + a4) + (a 2 + a 5) (ao + a3 w3 ) + (alw + a4tv4 ) + (a 2 w 2 + a5 w5 ) (ao + a3 ) + (al w 2 + a4w2 ) + (a 2 tv4 + a 5tv4 ) ( ao + a 3 w3 ) + (alw3 + a4) + (a 2 + a 5w3) (ao + a 3 ) + (al tv4 + a4tv4 ) + (a 2 w2 + a 5 w2 ) (ao + a3 w3 ) + (altuS + a4w2 ) + (a 2 tv4 + +a5w)
(6 .35)
(a O + a3 ) + ( a l + a4) + ( a 2 + a5) 1 (ao + a3 w3) + Weal + a4w3 ) + w2 (a 2 + a 5 w3 ) 2 ( a o + a3 ) + w2 (a1 + 84) + tv4 ( a 2 + a5 ) 3 (ao + a 3 w 3 ) + w3 (al + a4w3 ) + (a 2 + a 5 w3 ) 4 (a o + a3 ) + tv4 ( a l +84) + w2 (a 2 + a5) 5 (ao + a3 w3) + w5 (al + a4w3 ) + tv4(a 2 + +a5 w3)
(6 . 36)
o
=
Now consider pairs of rows whose remainders are the same when divided by 2; that is, rows 0, 2, and 4, and rows 1, 3, and 5. Note the recurrence of (ao + a3 ) , (al + a4 ) , and (a 2 + a 5 ) . 0 It is time to consider the general result.
IF
N = RG, the DFT MN I of any I E 6.5.2 FFT FACTOR FORM L l ( ZN ) can be computed with N ( R + G ) multiplications. Proof. For I : ZN
D/ ( M)
N- l =
-+
C, the DFT of I is
� I ( K ) wK M , M = 0, 1 , 2,
.
. . , N - 1 , w = e 2 1ri/ N . (6.37)
K =O Dividing the exponent K by G enables us to write K = Gk + k o , where k = 0, 1 , . . . , R
-
land ko = 0, 1 , . . . , G
-
1.
Similarly, dividing the row variable M by R, we get
M = Rn + n o , n Now,
WKM
=
= =
0, 1 , . . , G .
- I, no = 0 , 1 , . .
.
, R - 1.
W 1r } whose inverse is
7rt
n
E Z} is an the fact that
: n
7. Wavelets 00
of band-limited functions. By the Sampling theorem 5.21 . 1 , if
- n)) I(n) sin(7r(t I(t) = " L...J (7r t - n ) - 00
449
I E VO , then
( 1 I 1 I 2 -convergence)
{!pet - n) : n [1 Vi = cl [{ !pi , n = 2j/2 !p(2i t - n) : n E z}] , j E Z .
and the series converges pointwise as well. Thus, E Z } is an orthonormal basis for Vo . We take (with denoting linear span)
Thus,
= { I E L 2 (R) : j(w) = 0 , I w l > 2j 7r } . It is easy to see that { Vi j E Z} is an MRA with scaling function !pet) = (sin 7rt ) /ri. The increasing, separation, scaling, and orthogonal properties are obviously satisfied from the form of each Vi , That cl ( Ui � o Vi ) = L 2 {R) follows from the facts that the map I j is a bijection from L 2 {R) to itself, and Parseval's identity 1 I 1 2 = $ 1 1 /1 1 2 (Plancherel 's theorem 5 . 1 9 .3) , as we now show. Let I E L 2 {R). Then j E L 2 (R), and let t:. (w) = j(w) 1 [ - 2 " 11' , 2 " 11' ] (w). Then In E Vn and l - 1 0 as it:. � 2 n Therefore , II /n - 111 2 0 as n Vi
:
�
�
00 .
�
By the filter equality 7.4.5,
�
1-+
�
00 .
cp{w) = mcp (w/2) cp{w/2) = 1[ _11', .. ) (w) . Generally, mcp E L 2 (T), so m cp is the 27r-periodic extension of 1 [ - 11'/ 2 ,11' / 2 )' By the mother wavelet theorem 7.4.8,
�(w) =
=
e - i (w / 2+ 1I') m cp (w/ 2 + 7r) cp{w/ 2 ) a.e. e - iw /2 (1[ -2 11', - 1I') (w) + 1 ( 1I' , 24w))
is the transform of a mother wavelet. Hence we can take as the mother Shannon wavelet - sin 27r(t - 1 /2) . '!/J{t) = sin 7r(t - 1/2) 7r(t - 1 / 2 )
7.7
0
Riesz Bases and MRAs
!p {t - n E
« Vn) , !p)
If n) , Z , of the scaling is an MRA, then the translates function are an orthonormal basis for Vo . A kindred notion to that of or t h o n o rm a l basis is basis. When we substitute Riesz basis for orthonor mal basis in the definition of MRA, we get a RIESZ MRA Specif ically, we require that Z} be a Riesz basis for There is
Ri esz {h(t - n) : n E
((Vn) , !p): Va .
7. Wavelets
450
(( n) , h)
((Vn) , cp).
a close connection between Riesz MRAs V and MRAs In particular, if - n ) : n E Z} is a Riesz basis for then there exists a function cp E Vo such that t - n ) : n E Z} is an orthonormal basis for Vo (7.7. 8 ) .
{h(t
{cp(
Yo,
Definition 7 . 7 . 1 SCHAUDER BASIS
(xn )
A sequence is a SCHAUDER BASIS for a Banach space ( V, 1 1 1 1 ) if (1 1 1 1 convergence) where every E V can be written as = the scalars are uniquely determined . 0
x
x En EN anXn
an
Clearly, if V has a Schauder basis, then V must be separable : If V is =P 0 for finitely many n } is a count real, then E Q, able dense subset of V . In the complex case, the set E Q + iQ, =P 0 for finitely many n } is a countable dense subset of V .
an
{ E nEN anXn : an
{ E n EN anXn : an
an
Definition 7.7.2 EQUIVALENT SCHAUDER BASES
(x n )
(Yn) are called EQUIVALENT if they have the
Schauder bases and mutual convergence property:
L anXn converges L anYn converges. nEN nEN
0
Equivalence is characterized in 7.7.3. We leave its proof to Exercise 3 . 7.7.3 EQUIVALENT BASES Schauder bases (xn) and (Yn) for V are equiv alent if and only if there exists a bounded linear bijection T : V - V such that TXn = Yn for all n.
We discuss
Riesz bases below i n the context of separable Hilbert spaces.
Definition 7. 7.4 RIESZ BASIS
(X n )
A Schauder basis for the Hilbert space V is a RIESZ BASIS if it is 0 equivalent to an orthonormal basis
( Yn ).
Obviously, any orthonormal basis is a Riesz basis. There are many alter native characterizations of Riesz bases (see Young 1980 ) . The most imp or tant for our purposes is 7.7.5. Others are indicated in the exercises.
If V is a separable Hilbert space, then (X n ) is a Riesz basis if and only if each x E V can be expressed uniquely as x = E n EN anXn and there exist positive constants A and B , the RIESZ CONSTANTS, such that 7.7.5 RIESZ BASIS CHARACTERIZATION
I
l1
2 2 f A L l an l � anxn � B L l an l 2 n= l nEN n EN
for all x E v.
( 7 . 49 )
7. Wavelets
45 1
Proof. Assume that (xn) is a Riesz basis and write x E V uniquely as x = Ln EN anxn · Since (xn) is a Riesz basis, there exists an orthonormal basis ( Yn ) such that the map T defined by T ( Ln EN anXn) = L n EN an Yn is a bounded linear bijection of V onto V. Since I I Tx I I 2 ::; II T I I 2 11 x II 2 and ( Yn ) is an orthonormal basis, it follows that 2 L anYn = I an 1 2 = I I Tx II 2 ::; I IT I1 2 11 x l1 2 . nEN n EN Thus ( 1 / II T II 2 ) Ln EN I an 1 2 ::; II x 11 2 , which proves the left half of (7.49) .
L
The right half follows from a consequence of the bounded inverse theorem (Bachman and Narici 1966, p. 27 1 , Theorem 1 6 .6) , which says that a con tinuous linear bijection between Banach spaces is bicontinuous. In this case it means that T - 1 is bounded or T- 1
(Ln EN anyn) nLEN anXn
::; II T - 1 1 1
L an Yn .
nEN
(Xn) Ln EN anxn · (Yn) = Ln EN an Yn ; L n EN anYn 2 L n EN l an l ::; (I/A) I Ln EN anXn l 1 2 . II Tx II 2 Ln EN l an l 2 ::; (l/A) I ILnEN anXn l 1 2 . = = L n EN an Yn , an = Y = Ln EN an Yn
Conversely, suppose satisfies (7 .49) and that each x may be uniquely Since V is a separable Hilbert space, all or represented as thonormal bases are denumerable (3.4.8 and 3.4.9) . Let be an orthonor T is well-defined because of the mal basis for V, and let Tx uniqueness of the representation. The series converges because by equation (7 .49) , T is clearly lin ear ; it is bounded because = Clearly, T is injective, because if Tx 0 0 for then all n . To see that T is onto, we observe that if E Y, then < 00 . As a consequence of (7.49) , the sequence of finite sums ZN is a Cauchy sequence. Hence, x E V and 1 Tx It follows from (7 .49) that T- 1 is bounded. 0
Ln EN l an l 2 = L;;= anXn = Y = Ln EN an Yn .
= Ln EN anXn
We now extend the notion of MRA by substituting "Riesz basis" for "orthonormal basis" as a property of the scaling function. Because a Riesz basis is equivalent to an orthonormal basis and convergent series of orthonormal vectors converge unconditionally (3.6.2 ) , a Riesz basis i s also unconditional i n that sense . This j ustifies writing sums in material to come with indices belonging to Z, i.e . , as rather than
( Yn)
Ln EZ anXn
Ln EN anxn ·
Definition 7.7.6 RIESZ M RA
A RlEsz MULTIRESOLUTION ANALYSIS (RIESZ M R A ) is a sequence of closed subspaces Vi , j E Z, of (R) with the following properties:
L2
452
7. Wavelets
V- I Vo VI . . . . (b) (Density) cl Uj e Z V; = L 2 ( R) . (c) ( Separation ) nj e Z V; = {O} . (d) (S caling) f (t) E Vi if and only if f ( 2 t ) E V; + I . (e) (Riesz Basis) There exists a RIESZ S CALING FUNCTION h E Vo such that {h (t ) E Z} is a Riesz basis for Vo . (a) (Increasing)
- n
Vn , h
C
...
C
C
C
0
: n
Our first goal (7.7.8) is to show that if we start with a Riesz MRA ( ) , we can find a function cp such that {cp (t - n ) : n Z is an We begin by proving 7.7.7. orthonormal basis for
({Vn ) ,
Vo.
E Vo
E }
7.7.7 Let h) be a Riesz MRA. Let A and B be the Riesz con stants {7.49} of the Riesz basis { h ) E Z } for Let
(
L: n e z
I h (w + 2mr ) r )
1 / 2 , w E R. (tThen
- n
VA �
Vo. gh (w)
: n
gh (w) � VB a. e.
gh E L 2 (T) . P roof. Note that h (w + 2 mr ) is the Fourier transform of e - 2 mr t h (t) . The series by which gh is defined converges by Parseval's identity ( 5 . 1 8 .3) and (7.49) . Clearly, gh is 2 71"-periodic. To see that gh E L 2 (T) , consider [10 2 " 1 9h (w)1 2 dw = 1o 2,. L I h (w + 2 ) 1 2 dw ke Z 2" 2 L k e Z 10[ 1 h (w + 2k71") 1 dw. If we change variables in the preceding equation and replace w + 2k7l" by w , we obtain 1 2,. 1 9h (W ) 1 2 dw = Lk e Z 12h( 2k + 2),. I �h (w) 1 2 dw 1: Ih (w) 1 2 dw and
i
k71"
O
I h l : = 271" Il h ll � .
by Parseval's equality 5 . 18.3(a) . Thus
gh E L2 (T) .
7. Wavelets
453
In the notation of equation (7. 15) of Section 7.3, we write
(a k h e z
Let 5 . 1 9 .3 ,
ho,dt) = h (t - k) , k E Z. E £2 ( Z) and consider L keZ a k ho , k . By the Plancherel theorem
We write the above as
2
�
r ( 2n +2) 1r L a k e - ik W h (w) dw. = L 27r nE Z i2 n1r kEZ Replacing w by w + 2n7r in the previous integrals, we obtain 2
2
2 n ik ( ) L ak e - w +21r h (w + 27rn) dw kEZ 2
] [ (w) = L keZ a k e - ikw and gh (w) = LnE Z I h (w + 27rn) I 2 2 1 f 2 1r 2 (7.50) = 27r io lu (w) 1 g� (w) dw.
1/2
Then , with u
By hypothesis ,
2
2
A L l anl 2 :::; L an h ( t - n) nEZ nEZ 2
n EZ
'
454
7. Wavelets
Also, 2'11'" L ne Z Thus
l an l 2 = II Ln eZ an e - i nw l l � , where Il l b is the L 2 (T)-norm. 2
A lI u ll � S; 2'11'" L an h o , n n eZ
Using equation
2
(7.50), we obtain
or
(7.51 )
(7.51)
where the functions and norms in equation are L 2 (T) functions and norms. Choose n E Since {a kh eZ E £2 ( Z) is arbitrary, let be fixed and choose the scalars ak such that
N.
y
By some trigonometry,
(
1
2'11'" ( n +
)1)
1 Fn { y - w),
sin 2
{
1) (W - y) / 2 (w - y) / 2
+ 2 sin n
(7.52)
2'11'"
Fn(w) is the nth Fejer kernel (Section 4.15). Inserting this into equa (7.51), using the fact that Fn(w) is 2'11'"- periodic as well f�oo Fn(x) dx = (4.15.2), we obtain 1 j 1r Fn (y - w) g� {w) dx S; B. A S; 2
where tion 2'11'"
as
'II'"
Hence
- 1r
(Fn * gO (w) S; B. Now , g� E Ll (T) ; hence as in equation (5.8) of Section 5.3 and Lebesgue's pointwise convergence theorem 5.3.1, I f w e let
or
A S;
n
---+ 00 i n the previous inequality, w e obtain
A S; g� (w) S; B a.e . ,
VA S; gh
(w) S; Vii a.e.
0
7. Wavelets
455
(t - n) : n Z}
If {h E is an orthonormal basis for Va ,it follows from the Pythagorean theorem that
2 Thus, i n this case
g�
2 n EZ
A = B = 1 and
(w) = L I h (w + 2mr) 1 2 n EZ
=
1
a.e. (cf. 7.4 . 1 ) .
7.7.8 RIESZ BASIS T O ORTHONORMAL If { h (t - n) : n E Z} is a Riesz basis for the closed subspace Va of L 2 (R), then there exists I{J E Vo such that {I{J (t - n) : n E Z} is an orthonormal basis for Va . P roof. By 7.7.7,
� l /gh (w) � l/VA a.e., where 1/2 2 gh (w) = I h (w + 2mr) n EZ Since l/gh ( ) is bounded a.e., ip = high E L 2 (R), and therefore E L 2 (R). It is also clear ithat there exist (an) and (bn) in £ 2 ( Z) such that l / gh (w) = Ln EZ an e - nw and gh (w) = Ln EZ bn e - i nw ( 9 h (w ) E L 2 (T) ) . Thus an e - inw and h ew) = ip (w) bn e - i nw . ip (w) = h ew) n EZ n EZ 0 < l /VB
(L
1)
I{J
w
Hence
l{J(t) =
L Ln EZ anh(t - n)
L
and
which shows that I{J E Vo and cl [l{J(t Finally, we note that
h(t) =
Ln eZ bnl{J(t - n),
n) : n E Z] = Va .
2 L l ip(w + 2n1r) 1 2 = g� �w) L I h (w + 2n1r 1 1 a.e . , n eZ nEZ which shows that the family {1{J(t - n)} n EZ is an orthonormal family by 7 .4 . 1 . 0 =
The proof of 7.7.8 follows Hernandez and Weiss 1996 .
7. Wavelets
456
Exercises 7 . 7
(xn)
1 . COEFFICIENT FUN CTIONALS CONTINUOUS Let b e a Schauder basis for the Banach space V. Any E V can be written uniquely as = Show that the linear coefficient functionals = are continuous and in fact there exists M > 0 = such that 1 � � M for all n E
x
x E n e N anxn· In (x) ( a Xn) an in En e N n N. I I xn ll ll /nl l 2. Let (xn) and (Yn) be Schauder bases for the Banach space V. Show that the maps Tn ( E n e N a i x i ) = E �= 1 a i Yi , E N , are linear and n
continuous.
3 . EQUIVALENT SCHAUDER BASES Verify the assertion of 7.7.3 that Schauder bases and for V are equivalent if and only if there for exists a bounded linear bijection : V -+ V such that = all n .
(xn)
( Yn )
4. RELATIO NS BETWEEN (R) .
L2
T
TXn Yn
.e2 (Z) ELEMENTS AND h E L 2 (R)
Let h E
E keZ I h ( w + 2br) 1 2 converges almost everywhere. Let (an) E .e2 ( Z), and let h k (t) = h (t - k) for k E Z. Show that
(a) Show that (b)
o,
5 . INEQUALITY O N Uh EXTENDS TO Uh ( w ) =
.e2 ( Z)
(keZL Ih
(w
For h E
+ 2 k1 r) r
)
L 2 (R) and 1 /2
as in 7.7.7, choose A , B > 0 such that v'if � U h ( w ) � Vii a.e . Show that for it follows that E
(an) neZ .e2 ( Z),
2 2 A L l a kl � L a k h (t - k) n eZ keZ 2
6 . Show that the result in Exercise 5 generalizes the part of 7.4 . 1 that
L2
L keZ I h (w + 2 k1 r) 1 2
states that for h E (R) , if { h (t n ) : n E is orthonormal. -
Z}
= 1 a.e . , then
7. Wavelets
457
Hints 1. Norm the vector space W = ing
{(an) : 2: nEN anXn converges } by tak n sup L a i X i . lI (an) 1I = nEN i= 1 It can be easily verified that this is a norm and that W is a Banach space. Define T W V as T « a n )) = 2: n EN anxn. Clearly T is a linear bijection. By the continuity of the norm on V, T is also :
continuous:
�
n aX L II T « an)) 1I = L anXn :::; nsup EN i =1 i i nEN
Since a continuous linear bijection between Banach spaces has a con tinuous linear inverse (Bachman and Narici 1966 , p. 27 1 , Theorem 1 6 .6 ; this is sometimes called the open mapping theorem) , is continuous. With =
x 2:n EN anxn, I fn (x) 1 = l an l anxn ll = Il II Xn ll 2:7=1 a i X i 2:7;;11 a i xi l 1 = II Xn ll
T- 1
-
But
11 2:�= 1 a , x , - 2:�:11 a,x, II II x n ll
I fn (x) 1 :::; 2 ii T- 1 i i ll x l I / II xn ll · Thus, each fn is bounded, and II fn ll :::; 2 ii T - 1 i / lI xn ll . With M = 2 i i T- 1 ii , we have II xn ll ll fn ll :::; M, E N . Finally, 1 = fn(xn) :::; II x n ll ll fn ll :::; M, E N. n
n
458
7. Wavelets
Tn
2. Note that is a linear combination of coefficient functionals, each of which is continuous by Exercise l .
(xn) and ( Yn) are Schauder bases for the Banach space V T V --+ V is a bounded linear map such that TXn = E N . Assume that L n eN anXn converges and consider Yn L n eN an Yn · For m > n , IIL�= 1 a kYk - L � = 1 a kYk ll = IIL�=1 ak Tx k - L �= 1 a k Tx k ll l i T ( L�= 1 a k X k - L � = 1 ak x k ) 1I < II T II IIL�=1 a k X k - L � = 1 a k x kll · Since L n eN anXn converges, IIL �= 1 a k X k - L � = 1 a k x kll can be made arbitrarily small for sufficiently large n , and the result follows. Since T- 1 is bounded (open mapping theorem) , it follows similarly that if Ln eN anYn converges, then Ln eN anXn converges. Conversely, suppose that (xn) and ( Yn) are equivalent bases. Write x E V uniquely as x = Ln e N anxn· By hypothesis Y = L n eN an Yn converges, and we define T V V by taking T ( L n eN anXn) = L n eN an Yn · It is easy to verify that T is a well-defined linear bijec tion. To show that T is bounded, let Tn (L n e N anXn) = L �= 1 a i Yi be the bounded linear map of Exercise 2. Clearly, Tx = limn Tnx, and by the Banach-Steinhaus theorem (quoted in Chapter 4 as 4.9 . 1 , or Bachman and Narici 1 966, p . 25 1) it follows that T is bounded. 2 1 / 2 E L (R) by (a) Follows since ( w ) = (L keZ I h ( w + 2 h ) 1 ) 2 7.7.7; only the fact that h E L 2 (R) was used in the proof. (b) This was also proved in 7.7.7, assuming only that h E L 2 (R) .
3 . Suppose and that for all n
:
:
4.
--+
gh
5 . Use 4(b) and the hypothesis to obtain
2
2 < - 1 11" B
- 2 71"
6.
0
2
dt.
2 2 k i But J:1I" IL keZ a k e - t l dt = 271" L keZ l a k l 2 . If L keZ I h (w + 2 h ) 1 2 = 1 a.e . , then (w ) = L keZ I h (w + 2k7l") 1 2 = 1 a.e . Thus, with A = B = 1 in Exercise 5 , 2 gh
(a k h eZ E £2 ( Z ) .
2
for all This is equivalent to the family n being orthonormal .
E Z}
{h (t - n ) :
7. Wavelets
7.8
459
Franklin Wavelets
An example of a Riesz basis for the space Vo of piecewise linear continuous - n) : n E functions on the intervals [k , k + 1] , k E is given by = (1 where is the hat function (see Exercise 3) . We - 1 1) provide an elementary direct method of obtaining an orthonormal basis for Vo from the n : n E independent of the general metho d of conversion for Riesz bases of 7.7.8 (cf. Exercise 3).
h(t)
- It {h(t
-
-0.5
-,
Z,
l[ o, 2j (t) ) Z},
The Hat
Function
h
{h(t
Z}
2.5
(t)
=
(1 - I t - 1 1 ) 1 [0, 2]
A fact we shall need is the following. 7.S . 1
1
1
sin
2 "TrW = L n E Z (7rW + 7rn ) 2
a. e.
Proof. We know that the Fourier transform of the Haar scaling function (see Example 5 . 1 5 .5) , and that I [O,lj (t) is tP(w ) = n) : n E is orthonormal. Thus, by 7.4. 1 ,
g(t ) = {cp(t
e - iw /2 Si:ir
Z}
-
Now replace
W by 27rw .
4 sm· 2 "2W ", n ( +2n ". ) 2 a.e . L..." EZ w 1
=
0
Example 7.S.2 FRANKLIN WAVELETS
Let Vo be the set of piecewise linear continuous functions on the intervals [k , k + 1] for all k E Z, which are in L 2 (R). Any f E Vo may be written as
(see Exercise 3)
L a k h(t k) , kEZ h(t) = ( 1 - I t - 1 1 ) l[o, 2j (t) f(t) =
where
-
460
7. Wavelets
is the hat function. The collection Vj = {f E L 2 (R) : f 2 - i t E is an increasing ( Vj C Vj + l ) family of closed subspaces of L 2 (R) . Since {h t - n ) : n E Z } is clearly not orthogonal, h is not a scaling function. By Example 5.5.5 and 5.5. 1(b) , the Fourier transform of h is
) Vo l
(
it h (w) = e-iw ( si: 2
)
(
2 (7.53)
[Or observe that h( t ) = l[O, l](t) * l[O, l ](t) (see Exercise 1 ) .] Since {h(t - n ) : n E Z } spans Vo , any cp E Vo can be written
cp (t) =
In
L an h ( t
ne Z
)
- n .
the remainder of the argument, we obtain a way to choose the a k so that
{ cp ( t - n ) : n E Z } is an orthonormal base for Vo. By the continuity of the L 2 -Fourier transform map F (I) = i (5. 19 . 3) , (7.54) �(w) = a n e-iw n h (w) = pcp (w)h(w) n eZ wn where Pcp (w) = L neZ a n e -i . By 7.4. 1 , in order for {cp ( t - n ) : n E Z }
L
to be orthonormal, we must have
L I�(w + 2mr) 1 2 = 1 a.e.
ne Z Thus we want 1
= =
o r (note that
Since
h (w)
=
LneZ I�(w + 2 mr) 1 2 2 I pcp (w) 1 2 LneZ I h (w + 2mr) 1 a.e. ,
L neZ I h (w + 2 mr) 1 must be nonzero a.e .) 1 Ipcp (w) 1 2 = 2 a.e . 2mr) (w + 1 LneZ I h
(7.55)
2
e -iw
(Si:iF) 2 by equation (7.53) , '" sm[( w +2n7r){21 1 4 L.... n e Z I (w +2n ". ) / 2 w
. sm 4 2
Lne Z (w/2 + mr) 4 · 1
(7.56)
(7.57)
7 . Wavelets
Thus
46 1
(7. 5 8)
By 7.8 .1, Differentiating twice, we obtain (7.59) 2'Tr2 (sin2 'TrWsin+4 3'TrWcos2 'Trw ) = L ('TrW 6'Tr+ 2'Trn )4 . nEZ Replacing 'TrW by w/2 in equation (7.59) and using some basic trigonometry, 1 - G) sin 2 W /2 " 1 (7.60) � sin4 w/2 -- n�E Z (w/2 + n'Tr)4 a.e . By equation (7. 58), we see that we want -
- .,---'--
We choose
1 . PIP(w) = ( 1 - ( 3'2 ) sin2 w/2) - / 2 eSw • By equation (7.54), 2 . 2 w/2) - 1 / 2 [ sin2 W/22 ] (7.61) � (w) PIP (w)h� (W ) ( 1 - ( 3' ) sm (w/2) ' In terms of the filter mIP(w), by the filter equality 7. 4 . 5 , I{J
so
=
=
sm. 22w ( 1 - -sm 2 ' 2 w) - 1 / 2 -w 3 2 sin (w/2)2 ( 1 - -sm 2 . 2 w / 2) -1/ 2 3 (w/2)2 sin2 w (1 - (2/3)Sin2 2w/2) 1 / 2 4sin (w/2) 1 - (2/3)sin w 1/2 2 1 (2/3) sin w /2 cos2 ( �2 ) ( 1 - (2/3) sin2 w )
(7. 62)
462
7. Wavelets
By the mother wavelet theorem 7.4.8, the Fourier transform of the FRANKLIN MOTHER WAVELET is
which may be computed by substituting for rp and mcp from equations ( 7.61 ) and ( 7.62) into this equation. We determine 'IjJ by inversion and numerical techniques. Alternately, rewrite equation ( 7.61 ) as
By equation
( 7.53)
and the translation property, it follows that
cp ( t) =
l: bn h (t + n + 1 ) .
n EZ
We determined the coefficients bn b y expanding
( 1 - 2 . 2 w ) - 1 / 2 = V(3"2 ( 3" sm "2
)
W - 1/2
1 + cos "2
in powers of cos w/2 = (eiw/ 2 + e - iw/ 2 ) /2. We mention that the Fourier coefficients bn are bounded by e - a1 n l ( bn = O ( e - a1 n l )) for some a > o (Hernandez and Weiss 1996, Walter 1997) . 0
From the nature of the hat function h ( t ) = ( 1 - I t - 1 J) 1[o, 2] ( t ) used in Example 7.8.2, the Franklin wavelets should produce effective approxi mations of functions with slowly changing derivatives (described nicely by straight line approximations) . A large number of basic properties of