Wavelet Theory: An Elementary Approach with Applications

This page intentionally left blank

WAVELET THEORY

This page intentionally left blank

WAVELET THEORY An Elementary Approach With Applications

David K. Ruch Metropolitan State College of Denver

Patrick J. Van Fleet University of St. Thomas

WILEY A JOHN WILEY & SONS, INC., PUBLICATION

The image in Figure 7.6(a) appears courtesy of David Kubes. The photographs on the front cover, Figure 4.4, and Figure 4.11 appear courtesy of Radka Tezaur. Copyright © 2009 by John Wiley & Sons, Inc. All rights reserved. Published by John Wiley & Sons, Inc., Hoboken, New Jersey. Published simultaneously in Canada. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., I l l River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permission. Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic format. For information about Wiley products, visit our web site at www.wiley.com. Library of Congress Cataloging-in-Publication Data: Ruch, David K., 1959Wavelet theory: an elementary approach with applications / David K. Ruch, Patrick J. Van Fleet p. cm. Includes bibliographical references and index. ISBN 978-0-470-38840-2 (cloth) 1. Wavelets (Mathematics) 2. Transformations (Mathematics) 3. Digital Images— Mathematics. I. Van Fleet, Patrick J., 1962- II. Title. QA403.3.V375 2009 515\2433—dc22 2009017249 Printed in the United States of America. 10

9 8 7 6 5 4 3 2 1

To Pete and Laurel for a lifetime of encouragement (DKR) To Verena, Sam, Matt, and Rachel for your unfailing support (PVF)

This page intentionally left blank

CONTENTS

Preface

xi

Acknowledgments 1

The Complex Plane and the Space L 2 (R) 1.1 1.2 1.3 1.4

2

Complex Numbers and Basic Operations Problems The Space L 2 (R) Problems Inner Products Problems Bases and Projections Problems

xix 1 1 5 7 16 18 25 26 28

Fourier Series and Fourier Transformations

31

2.1

32 36 37

2.2

Euler's Formula and the Complex Exponential Function Problems Fourier Series

vii

Vi i i

CONTENTS

2.3 2.4

3

Haar Spaces 3.1 3.2 3.3 3.4 3.5 3.6

4

85 86 93 93 107 108 119 120 133 134 140 141 145

4.1

146 159 163 171 172

4.3

The One-Dimensional Transform Problems The Two-Dimensional Transform Problems Edge Detection and Naive Image Compression

Multiresolution Analysis

179

5.1

Multiresolution Analysis Problems The View from the Transform Domain Problems Examples of Multiresolution Analyses Problems Summary

180 196 200 212 216 224 225

Daubechies Scaling Functions and Wavelets

233

6.1

234

5.2 5.3 5.4 6

The Haar Space Vo Problems The General Haar Space Vj Problems The Haar Wavelet Space W0 Problems The General Haar Wavelet Space Wj Problems Decomposition and Reconstruction Problems Summary

49 53 66 72 82

The Discrete Haar Wavelet Transform and Applications

4.2

5

Problems The Fourier Transform Problems Convolution and 5-Splines Problems

Constructing the Daubechies Scaling Functions

CONTENTS

6.2 6.3

7

277

7.1

The Discrete Daubechies Wavelet Transform Problems Projections and Signal and Image Compression Problems Naive Image Segmentation Problems

278 290 293 310 314 322

Biorthogonal Scaling Functions and Wavelets

325

8.1

326 333 334 350 353 368 370 375 375 388 390 397

7.3

8.2 8.3 8.4 8.5 8.6

9

246 251 265 268 276

The Discrete Daubechies Transformation and Applications

7.2

8

Problems The Cascade Algorithm Problems Orthogonal Translates, Coding, and Projections Problems

IX

A Biorthogonal Example and Duality Problems Biorthogonality Conditions for Symbols and Wavelet Spaces Problems Biorthogonal Spline Filter Pairs and the CDF97 Filter Pair Problems Decomposition and Reconstruction Problems The Discrete Biorthogonal Wavelet Transform Problems Riesz Basis Theory Problems

Wavelet Packets

399

9.1

400 413 414 424 424 439 440

9.2 9.3 9.4

Constructing Wavelet Packet Functions Problems Wavelet Packet Spaces Problems The Discrete Packet Transform and Best Basis Algorithm Problems The FBI Fingerprint Compression Standard

Appendix A: Huffman Coding

455

X

CONTENTS

Problems

462

References

465

Topic Index

469

Author Index

479

Preface

This book presents some of the most current ideas in mathematics. Most of the theory was developed in the past twenty years, and even more recently, wavelets have found an important niche in a variety of applications. The filter pair we present in Chapter 8 is used by JPEG2000 [59] and the Federal Bureau of Investigation [8] to perform image and fingerprint compression, respectively. Wavelets are also used in many other areas of image processing as well as in applications such as signal denoising, detection of the onset of epileptic seizures [2], modeling of distant galaxies [3], and seismic data analysis [34, 35]. The development and advancement of the theory of wavelets came through the efforts of mathematicians with a variety of backgrounds and specialties, and of engineers and scientists with an eye for better solutions and models in their applications. For this reason, our goal was to write a book that provides an introduction to the essential ideas of wavelet theory at a level accessible to undergraduates and at the same time to provide a detailed look at how wavelets are used in "real-world" applications. Too often, books are heavy on theory and pay little attention to the details of application. For example, the discrete wavelet transform is but one piece of an image compression algorithm, and to understand this application, some attention must be given to quantization and coding methods. Alternatively, books might provide a detailed description of an application that leaves the reader curious about the theoretical foundations of some of the mathematical concepts used in the model. With this XI

Xi i

PREFACE

book, we have attempted to balance these two competing yet related tenets, and it is ultimately up to the reader to determine if we have succeeded in this endeavor. To the Student If you are reading this book, then you are probably either taking a course on wavelets or are working on your own to understand wavelets. Very often students are naturally curious about a topic and wish to understand quickly their use in applications. Wavelets provide this opportunity— the discrete Haar wavelet transformation is easy to understand and use in applications such as image compression. Unfortunately, the discrete Haar wavelet transformation is not the best transformation to use in many applications. But it does provide us with a concrete example to which we can refer as we learn about more sophisticated wavelets and their uses in applications. For this reason, you should study carefully the ideas in Chapters 3 and 4. They provide a framework for all that follows. It is also imperative that you develop a good working knowledge of the Fourier series and transformations introduced in Chapter 2. These ideas are very important in many areas of mathematics and are the basic tools we use to construct the wavelet filters used in many applications. If you are a mathematics major, you will learn to write proofs. This is quite a change from lower-level mathematics courses where computation was the main objective. Proof-writing is sometimes a formidable task and the best way to learn is to practice. An indirect benefit of a course based on this book is the opportunity to hone your proof-writing skills. The proofs of most of the ideas in this book are straightforward and constructive. You will learn about proof by induction and contraposition. We have provided numerous problems that ask you to complete the details of a portion of a proof or mimic the ideas of one case in a proof to complete another. We strongly encourage you to tackle as many of these problems as possible. This course should provide a good transition from the proofs you see in a sophomore linear algebra course to the more technical proofs you might see in a real analysis course. Of course, the book also contains many computational problems as well as problems that require the use of a computer algebra system (CAS). It is important that you learn how to use a CAS — both to solve problems and to investigate new concepts. It is amazing what you can learn by taking examples from the book and using a CAS to understand them or even change them somewhat to see the effects. We strongly encourage you to install the software packages described below and to visit the course Web site and work through the many labs and projects that we have provided. To the Instructor In this book we focus on bridging the gap often left between discrete wavelet transformations and the traditional multiresolution analysis-based development of wavelet theory. We provide the instructor with an opportunity to balance and integrate these ideas, but one should be wary of getting bogged down in the finer details of either

PREFACE

XI I i

topic. For example, the material on Fourier series and transforms is a place where instructors should use caution. These topics can be explored for entire semesters, and deservedly so, but in this course they need to be treated as tools rather than the thrust of the course. The heart of wavelet theory is covered in Chapters 3,5, and 6 in a comprehensive approach. Extensive details and examples are given or outlined via problems, so students should be able to gain a full understanding of the theory without handwaving at difficult material. Having said that, some proofs are omitted to keep a nice flow to the book. This is not an introductory analysis book, nor is the level of rigor up to that of a graduate text. For example, the technical proofs of the completeness and separation properties of multiresolution analyses are left to future courses. The order of infinite series and integration are occasionally swapped with comment but not rigorous justification. We choose not to develop fully the theory of Riesz bases and how they lead to true dual multiresolutions of L 2 (R), for this would leave too little time for the very real applications of biorthogonal filters. We hope students will whet their appetites for future courses from the taste of theory they are given here! We feel that the discrete wavelet transform material is essential to the spirit of the book, and based on our experience, students willfindthe applications quite gratifying. It may be tempting to expand on our introduction to these ideas after the Haar spaces are built, but we hope sufficient time is left for the development of multiresolution analyses and the Daubechies wavelets, which can take considerable time. We also hope that instructors will take the time for thorough treatment of the connections between standard wavelet theory and discrete wavelet transforms. Our experience, both personally and with teaching other faculty at workshops, is that these connections are very rewarding but are not obvious to most beginners in the field. Some interesing problems crop up as we move between L 2 (R) and finite-dimensional approximations.

Text Topics In Chapter 1, we provide a quick introduction to the complex plane and L 2 (R), with no prior experience assumed, emphasizing only the properties that will be needed for wavelet development. We believe that the fundamentals of wavelets can be studied in depth without getting into the intricacies of measure theory or the Lesbesgue integral, so we discuss briefly the measure of a set and convergence in norm versus pointwise convergence, but we do not dwell heavily on these ideas. In Chapter 2 we present Fourier series and the Fourier transform in a limited and focused fashion. These ideas and their properties are developed only as tools to the extent that we need them for wavelet analysis. Our goal here is to prepare quickly for the study of wavelets in the transform domain. For example, the transform rules on translation and dilation are given, since these are critical for manipulating scaling function symbols in the transform domain. 5-splines are introduced in this chapter as an important family of functions that will be used throughout the book, especially in Chapter 8.

XIV

PREFACE

In Chapter 3 we begin our study of wavelets in earnest with a comprehensive examination of Haar spaces. All the major ideas of multiresolution analysis are here, cast in the accessible Haar setting. The properties and standard notations of approximation spaces Vj and detail spaces Wj are developed in detail with numerous examples. Students may be ready for some applications after the long Haar space analysis, and we present some classics in Chapter 4. The ideas behind filters and the discrete Haar wavelet transform are introduced first. The basics of processing signals and images are developed in Sections 4.1 and 4.2, with sufficient detail so that students can carry out the calculations and fully understand what software is doing while processing large images. The attractive and very accessible topics of image compression and edge detection are introduced as applications in Section 4.3. In Chapter 5 we generalize the Haar space concepts to a general multiresolution analysis, beginning with the main properties in the time domain. Section 5.2 begins the development of critical multiresolution properties in the transform domain. In Section 5.3 we present some concrete examples of functions satisfying multiresolution properties. In addition to Haar, the Shannon wavelet and I?-splines are discussed, each of which has some desirable properties but is missing others. This also provides some motivation for the formidable challenge of developing Daubechies wavelets. We return to 5-splines in Chapter 8. Chapter 6 centers on the Daubechies construction of continuous, compactly supported scaling functions. After a detailed development of the ideas, a clear algorithm is given for the construction. The next two sections are devoted to the cascade algorithm, which we delay presenting until after the Daubechies construction, with the motivation of plotting these amazing scaling functions with only a dilation equation to guide us. The cascade algorithm is introduced in the time domain, where examples make it intuitively clear, and is then discussed in the transform domain. Finally, we study the practical issue of coding the algorithm with discrete vectors. After the rather heavy theory of Chapters 5 and 6, an investigation of the discrete Daubechies wavelet transform and applications in Chapter 7 provides a nice change of pace. An important concept in this chapter is that of handling the difficulties encountered when the decomposition and reconstruction formula are truncated, which are investigated in Section 7.2. Our efforts are rewarded with applications to image compression, noise reduction and image segmentation in Section 7.3. In Chapter 8 we introduce scaling functions and wavelets in the biorthogonal setting. This is a generalization of an orthogonal multiresolution analysis with a single scaling function to a dual multiresolution analysis with a pair of biorthogonal scaling functions. We begin by introducing several new ideas via an example from 5-splines, with an eye toward creating symmetric filters to be used in later applications. The main structural framework for dual multiresolution analyses and biorthogonal wavelets is developed in Section 8.2. We then move to constructing a family of biorthogonal filters based on J3-splines using the methods due to Ingrid Daubechies in Section 8.3. The Cohen-Daubechies-Feauveau CDF97 filter pair is used in the JPEG2000 and FBI fingerprint compression standards, so it is natural to include them in the book. The method of building biorthogonal spline filters can be adjusted fairly easily to

PREFACE

XV

create the CDF97 filter pair, and this construction is part of Section 8.3. The pyramid algorithm can be generalized for the biorthogonal setting and is presented in Section 8.4. The discrete biorthogonal wavelet transform is discussed in Section 8.5. An advantage of biorthogonal filter pairs is that they can be made symmetric, and this desirable property affords a method, also presented in Section 8.5, of dealing with edge conditions in signals or digital images. A fundamental theoretical underpinning of dual multiresolution analyses is the concept of a Riesz basis, which is a generalization of orthogonal bases. The very formidable specifics of Riesz bases have been suppressed throughout most of this chapter in an effort to provide a balance between theory and applications. As a final and optional topic in this chapter, a brief examination of Riesz bases is provided in Section 8.6. Wavelet packets, the topic of Chapter 9, provide an alternative wavelet decomposition method but are more computationally complex since the decomposition includes splitting the detail vectors as well as the approximations. We introduce wavelet packet functions in Section 9.1 and wavelet packet spaces in Section 9.2. The discrete wavelet packet transform is presented in Section 9.3 along with the best basis algorithm. The wavelet packet decomposition allows for redundant representations of the input vector or matrix, and the best basis algorithm chooses the "best" representation. This is a desirable feature of the transformation as this algorithm can be made application-dependent. The FBI fingerprint compression standard uses the CDF97 biorthogonal filter pair in conjunction with a wavelet packet transformation, and we outline this standard in Section 9.4.

Prerequisites The minimal requirements for students taking this course are two semesters of calculus and a course in sophomore linear algebra. We use the ideas of bases, linear independence, and projection throughout the book so students need to be comfortable with these ideas before proceeding. The linear algebra prerequisite also provides the necessary background on matrix manipulations that appear primarily in sections dealing with discrete transformations. Students with additional background in Fourier series or proof-oriented courses will be able to move through the material at a much faster pace than will students with the minimum requirements. Most proofs in the book are of a direct and constructive nature, and some utilize the concept of mathematical induction. The level of sophistication assumed increases steadily, consistent with how students should be growing in the course. We feel that reading and writing proofs should be a theme throughout the undergraduate curriculum, and we suggest that the level of rigor in the book is accessible by advanced juniors or senior mathematics students. The constant connection to concrete applications that appears throughout the book should give students a good understanding of why the theory is important and how it is implemented. Some algorithms are given and experience with CAS software is very helpful in the course, but significant programming experience is not required.

XVI

PREFACE

Possible Courses for this Book The book can serve as a stand-alone introduction to wavelet theory and applications for students with no previous exposure to wavelets. If a brisk pace is kept in line with the prerequisites discussed above, the course could include the first six chapters plus the discrete Daubechies transform and a sample of its applications. While considerable time can be spent on applied projects, we strongly recommend that any course syllabus include Chapter 6, on Daubechies wavelets. The construction of these wavelets is a remarkable mathematical achievement accomplished during our lifetime (if not those of our students) and should be covered if at all possible. Some instructors may prefer to first cover Chapters 3 and 4 on Haar spaces before introducing the Fourier material of Chapter 2. This approach will work well since aside from a small discussion of the Fourier series associated with the Haar filter, no ideas from Fourier analysis are used in Chapters 3 and 4. A very different course can be taught if students have already completed a course using Van Fleet's book Discrete Wavelet Transformations: An Elementary Approach with Applications [60]. Our book can be viewed as a companion text, with consistent notation, themes, and software packages. Students with this experience can move quickly through the applications, focusing on the traditional theory and its connections to discrete transformations. Students completing the discrete course should have a good sense of where the material is headed, as well as motivation to see the theoretical development of the various discrete transform filters. In this case, some sections of the text can be omitted and the entire book could be covered in one semester. A third option exists for students who have a strong background in Fourier analysis. In this case, the instructor could concentrate heavily on the theoretical ideas in Chapters 5, 6, 8, and 9 and develop a real appreciation for how Fourier methods can be used to drive the theory of multiresolution analysis and filter design.

Problem Sets, Software Package, and Web Site Problem solving is an essential part of learning mathematics, and we have tried to provide ample opportunities for the student to do so. After each section there are problem sets with a variety of exercises. Many allow students to fill in gaps in proofs from the text narrative, as well as to provide proofs similar to those given in the text. Others are fairly routine paper-pencil exercises to ensure that students understand examples, theorem statements, or algorithms. Many require computer work, as discussed in the next paragraph. We have provided 430 problems in the book to facilitate student comprehension material covered. Problems marked with a * should be assigned and address ideas that are used later in the text. Many concepts in the book are better understood with the aid of computer visualization and computation. For these reasons, we have built the software package ContinuousWavelets to enhance student learning. This package is modeled after the DiscreteWavelets package that accompanies Van Fleet's book [60]. These packages are available for use with the computer algebra systems (CAS)

PREFACE

XVÜ

Mathematica®, Matlab®, and Maple . This new package is used in the text to investigate a number of topics and to explore applications. Both packages contain modules for producing all the filters introduced in the course as well as discrete transformations and their inverses for use in applications. Visualization tools are also provided to help the reader better understand the results of transformations. Modules are provided for applications such as data compression, signal/image denoising, and image segmentation. The Cont inuousWavelet s package includes routines for constructing scaling functions (via the cascade algorithm) and wavelet functions. Finally, there are routines to easily implement the ideas from Chapter 3 — students can easily construct piecewise constant functions and produce nice graphs of projections into the various Vj and Wj spaces. The course Web site is http ://www.stthomas.edu/wavelets

On this site, visitors willfindthe software packages described above, several computer labs and projects of varying difficulty, instructor notes on teaching from the text, and some solutions to problems. D A V I D K. R U C H

PATRICK J. VAN FLEET Denver, Colorado USA St. Paul, Minnesota USA March 2009

This page intentionally left blank

Acknowledgments

We are grateful to several people who helped us with the manuscript. Caroline Haddad from SUNY Geneseo, Laurel Rogers, and University of St. Thomas mathematicians Doug Dokken, Eric Rawdon, Melissa Shepart-Loe, and Magdalena Stolarska read versions of the first four chapters. They caught several errors and made many suggestions for improving the presentation. We gratefully acknowledge the National Science Foundation for their support through a grant (DUE-0717662) for the development of the book and the computer software. We wish to thank our editor Susanne Steitz-Filler, for her help on the project. Radka Tezaur and David Kubes provided digital images that were essential in the presentation. We also wish to salute our colleagues Peter Massopust, Wasin So, and Jianzhang Wang, with whom we began our journey in this field in the 1990s. Dave Ruch would like to thank his partner, Tia, for her support and continual reminders of the importance of applications, and their son, Alex, who worked through some of the Haar material for a school project. Patrick Van Fleet would like to express his deep gratitude to his wife, Verena, and their three children, Sam, Matt, and Rachel. They allowed him time to work on the book and provided support in a multitude of ways. The project would not have been possible without their support and sacrifice. D.K.R. and P. V.F. XIX

This page intentionally left blank

CHAPTER 1

THE COMPLEX PLANE AND THE SPACE L2(R)

We make extensive use of complex numbers throughout the book. Thus for the purposes of making the book self-contained, this chapter begins with a review of the complex plane and basic operations with complex numbers. To build wavelet functions, we need to define the proper space of functions in which to perform our constructions. The space L 2 (R) lends itself well to this task, and we introduce this space in Section 1.2. We discuss the inner product in L 2 (R) in Section 1.3, as well as vector spaces and subspaces. In Section 1.4 we talk about bases for L 2 (R). The construction of wavelet functions requires the decomposition of L 2 (R) into nested subspaces. We frequently need to approximate a function f(t) G L 2 (R) in these subspaces. The tool we use to form the approximation is the projection operator. We discuss (orthogonal) projections in Section 1.4. 1.1

COMPLEX NUMBERS AND BASIC OPERATIONS

Any discussion of the complex plane starts with the definition of the imaginary unit:

i = \TI\ Wavelet Theory: An Elementary Approach with Applications. By D. K. Ruch and P. J. Van Fleet Copyright © 2009 John Wiley & Sons, Inc.

1

2

THE COMPLEX PLANE AND THE SPACE L 2 (R)

We immediately see that i2 = ( v ^ ) 2 = - 1 ,

i 3 = i2 · i = -i,

i 4 = (-1) · (-1) = 1

In Problem 1.1 you will compute in for any integer n. A complex number is any number of the form z — a + bi where a, b G M. The number a is called the real part oiz and b is called the imaginary part oiz. The set of complex numbers will be denoted by C. It is easy to see that R c C since real numbers are those complex numbers with the imaginary part equal zero. We can use the complex plane to envision complex numbers. The complex plane is a two-dimensional plane where the horizontal axis is used for the real part of complex numbers and the vertical axis is used for the imaginary part of complex numbers. To plot the number z — a + bi, we simply plot the ordered pair (a, b). In Figure 1.1 we plot some complex numbers. • -1 +2i

2 Imaginary

1

Real -2

• -2-i

1

-1

2

-1

_2«► - 2 i

Figure 1.1

Some complex numbers in the complex plane.

Complex Addition and Multiplication Addition and subtraction of complex numbers is a straightforward process. Addition of two complex numbers u = a + bi and v = c + di is defined as y = u + v = (a + c) + (b + d)z. Subtraction is similar: z = t¿ — ^ = (a — c) + (6 — d)z. To multiply the complex numbers u — a-\-bi and v = c + di, we proceed just as we would if a + 6z and c + di were binomials: i¿ · i; = (a + 6z)(c + di) = ac + adi -f bei + frdz2 = (ac — bd) + (ad + bc)i Example 1.1 (Complex Arithmetic) Let u = 2 + i, v = — 1 — i, y = 2z, awJ 2 = 3 + 2i. Compute u + v, z — v, u · y, and v · 2.

1.1 COMPLEX NUMBERS AND BASIC OPERATIONS

3

Solution u + v = (2-l) + (l-l)i = l = (Z- (_i)) + (2 - (_1))¿ = 4 + 3z Z-V u · y = (2 + z) · 2i = 4z + 2i 2 = - 2 + 4z v · 2 = ( - 1 - i) - (3 + 2i) = ( 3 ( - l ) - (-1)2) + ( 3 ( - l ) + 2(-l))z - - 1 - hi

Complex Conjugation One of the most important operations used to work with complex numbers is conjugation. Definition 1.1 (Conjugate of a Complex Numbers) Let z — a + bi e C. The conjugate ofz, denoted by~z,is defined by ~z = a — bi

Conjugation is used to divide two complex numbers and also has a natural relation to the length of a complex number. To plot z = a + bi, we plot the ordered pair (a, b) in the complex plane. For the conjugate ~z — a — bi, we plot the ordered pair (a, —b). So geometrically speaking, the conjugate ~z of z is simply the reflection ofz over the real axis. In Figure 1.2 we have plotted several complex numbers and their conjugates. •

2

Imaginary

u = - l +2i

•

►v = i

l·

z=2 w=w=1

teal

Î

-1

2 5=2

~1i,..., VN)T, elements of our space will be functions. We can view a digital image as a function of two variables where the function value is the gray-level intensity, and we can view audio signals as functions of time where the function values are the frequencies of the signal. Since audio signals and digital images can have abrupt changes, we will not require functions in our space to necessarily be continuous. Since audio signals are constructed of sines and cosines and these functions are defined over all real numbers, we want to allow our space to hold functions that are supported (the notion of support is formally provided in Definition 1.5) on R. Since rows or columns of digital images usually are of finite dimension and audio signals taper off, we want to make sure that the functions f{t) in our space decay sufficiently fast as t —> ±00. The rate of decay must be fast enough to ensure that the energy of the signal is finite. (We will soon make precise what we mean by the energy of a function.) Finally, it is desirable from a mathematical standpoint to use a space where the inner product of a function with itself is related to the size (norm) of the function. For this reason, we will work in the space L 2 (R). We define it now. L2(R)

Defined

Definition 1.3 (The Space L 2 (R)) We define the space L 2 (R) to be the set LZ(R) = {f:R-^C\

/ \f(t)\zdt 0, we can find an interval that contains a and has measure less than e (the interval (a — e/4, a + e/4) with measure e/2 works). We can generalize this argument to claim that a finite set of points has measure 0 as well. The general definition of sets of measure 0 is typically covered in an analysis text (see Rudin [48], for example). The previous discussion leads us to the notion of equivalentfunctions. Two functions f(t) and g {t) are said to be equivalent if fit) = g(t) except on a set of measure 0. We state the following proposition without proof. Proposition 1.2 (Functions for Which ||f(t)|| = 0) Suppose that f(i) Then \\f(t) || = 0 if and only if' f{t) = 0 except on a set of measure 0.

G L 2 (R). ■

1.2 THE SPACE L2 (R)

9

Examples of Functions in L2(R) Our first example of elements of L2 (R) introduces functions that are used throughout the book. Example 1.2 (The Box n(t), Triangle A(t), and Sine Functions) We define the box function Π(ί)

the triangle function

tt

0ooJ_L

L-+ooJ0

Both of these integrals diverge — in particular lim /

L-^OOJQ -^°° Jo

t2ndt=

lim —

L-^OO 2n

1

t2ndt

„ L 2 n + 1 -> oo

4-1

Thus we see that no monomials are elements ofL2(R). We could generalize part (a) to easily show that polynomials are not elements ofL2(M). Since Π(ί/4) is 0 whenever t £ [0,4), we can write f2(t) as

« 0. Then there exists a real number L > 0 such that —L

/

/ΌΟ

\f(t)\2dt

+

-oo

/»

\f(t)\2dt

JL

=

J\t\>L

\f(t)\2dt<e

Before we prove Proposition 1.3, let's understand what it is saying. Consider the L 2 (R) function f4(t) from Example 1.3(d). Let's pick e = 10~ 16 . Then Proposition 1.3 says that for some L > 0, the "tail" / I/4OOI2 at of the integral / |/ 4 (¿)| 2 at L

R

will satisfy

L

\àt
0 and let's first consider the integral J0°° \f(t)\2 dt. Since f(t) G L 2 (R), we know that this integral converges to some value s. Now write the integral as a limit. For TV G N we write

/ Jo

| / ( í ) | 2 d í = lim / ^ ^ ° ° Jo

|/(í)|2dí

= lim [1\f(t)\2dt+ N^cvJo

N ^

-fe+1 /»AC+1

lim V Σ //

iV->oo

k=0 k=0Jk N

lim

y^ak

k=0

f2\f(t)\2dt

Jl

\f(t)\2 dt 1/wi ¿

+ ---+ Γ

JN-1

\f(t)\2dt

THE COMPLEX PLANE AND THE SPACE L 2 (R)

14

fc+1

where α^ =

J \f(t)\2 dt > 0. Thus we can view the integral as the sum of the

k oo

series ^ α^ and we know that the series converges to s. k=0

Recall from calculus that a series converges to s if its sequence of partial sums N

SN = Σ

a

k converges to s. The formal definition of a convergent sequence says

fc=0

that for all e > 0, there exists an TVo G N such that whenever TV > iVo, we have \SN — s\ < e. We use this formal definition with e/2. That is, for § > 0, there exists an 7V0 G N such that N > N0 implies that \sN - s\ < §. In particular, for N — iV0 we have kiVo -S\

(1.12)




\sNo-s\

U

/»OO

= \J2ak1

fc=o No

= Σ/ ■

= oo

N0

0

0

1

70

\f(t)\2d

-

/»fe+1

2

l/WI ^-/

pN0

roo

l/(*)l2di

/»oo

|/(í)l2dí- / Jo

/ Jo

|/(í)|

Now / |/(¿)| 2 dt > / |/(¿)| 2 dt, so that we can drop the absolute-value symbols and rewrite the last identity as

|%o -

pl\0

\f(t)\2dt-

/ Jo

s

\

=

pOO

pOO

Jo

pN0

\f(t)\2dt-

Jo

\f(t)\2dt \f(t)\2dt

Jo

•»OO

= /

2

l/(*)l d*

JNo

(1.13)

Combining (1.12) and (1.13) gives poo l2dl / l/(*)l

JNn

In a similar manner (see Problem 1.19), we can find Ni > 0 such that -ΛΓι

l/(*)f di -oo


N0, —L < —Ni, and from this we can write /»OO pOO (1.14) / l/WI 2 di< / | / ( í ) | 2 d í < and

Δ

'No JNQ

JL

N

>

p — iVi

—L·

/ l/WI 2 di< Combining (1.14) and (1.15) gives -oc

/

(1.15) 2

J — oo

|/(í)| dí
0 such that whenever n > L, we have \\fn(t) — f(t)\\ < e. m If a sequence of functions fn(t), n— 1,2,... converges in L 2 (R) to /(£), we also say that the sequence of functions converges in norm to f(t). Other than the use of the norm, the definition should look similar to that of the formal definition of the limit of a sequence, which is covered in many calculus books (see Stewart [55], for example). The idea is that no matter how small a distance e we pick, we are guaranteed that we can find an TV > 0 so that n > N ensures that the distance between fn(t) and f(t) is smaller than e. Example 1.5 (Convergence in L 2 (R)) Let fn(t) = tn Π (t). Show that fn(i) converges (in an L 2 (R) sense) to 0. Solution Let e > 0. We note that Π(£)2 = Π(ί) and compute

\\fn(t)-o\\ = (

ft2nn(t)dt

*■/

*

THE COMPLEX PLANE AND THE SPACE L2 (R)

16

Now, if we can say how to choose L so that 1/ y/2n + 1 < e whenever n > L, we are done. The natural thing to do is to solve the inequality 1/ y/2n + 1 < efor n. We obtain Now the right-hand side is negative when e > 1, and in this case we can choose any L > 0. Then n > L gives / 2 1 , 1 < 1 < e. When e < 1, we take L — ^ ^ — l) to complete the proof. m In a real analysis class, we study pointwise convergence of sequences of functions. Convergence in norm is quite different, and in Problem 1.20 you will investigate the pointwise convergence properties of the sequence of functions from Example 1.5. PROBLEMS 1.11

Suppose that / f(t) dt < oo. Show that: R

(a) / f(t - a) at = J f(t) dt for any a £ R R

R

(b) / f(mt + b) dt = — / f(t) dt for m, b G R, with m / 0 m

R

R

1.12 Use the definition (1.6) of sinc(t) in Example 1.2 and L'H ôpital's rule to show that f(t) = sinc2(t) is continuous for t G R and then show that the maximum value of/(t)isl. 1.13 Determine whether or not the following functions are elements of L 2 (R). For those functions that are in L 2 (R), compute their norms. (a) f(t) = e-l'l

( c ) ^ ) = (l+t2)-1/2 (d) h(t) = n(t) (cos(2^) + ism(2nt)) 1.14

Give examples of L 2 (R) functions / such that ||/(t)|| = 0 and f(t) Φ 0 at

(a) a single point (b) five points (c) an infinite number of points *1.15

Show that the function f(t) =

— G L 2 (R).

PROBLEMS

* 1.16

17

Consider the function

ί \£ - | ί + 3, - 2 < ¿ < 1 /(*) = ^ 2ί - 4, [ 0,

1< ί < 2 otherwise

Show that/(t) eL 2 (M). 1.17

Show that the function g(i) from Example 1.4(d) is in L 2(

1.18 Find the support of each of the following functions. For those functions / that are compactly supported, identify supp(/). (a) fi(t) = e~t2

(b) f2(t) =

n(2t-k),keZ

(c) f3(t) =

n(2h-k),j,keZ

(d) f4(t) = sinc(i) ( e ) h{t) = Π ( ¿ ) sinc(7rf), where n is a positive integer (f) /β(ί) = Σ Cfc«(í - 2k), where cfc ^ 0, k e Z, fe=0

= { * - „otherwise and n is a positive integer 1.19 Complete the proof of Proposition 1.3. That is, show that if f(t) G L: and e > 0, there exists an iVi > 0 such that -JVi

/.

/ ( i ) |2| 2 d i < e / 2

1.20 In Example 1.5 we showed that fn{t) = tn\l(t) converges in norm to f(t) — 0. Does fn (t) converge pointwise to 0 for all t G M? 1.21

Consider the sequence of functions m - / *n' ^ n l ] ~ \ 0,

~1 < * < 1 otherwise

(a) Show that g n (£) converges in norm to 0. (b) Show that lim gn(a) = 0 for a φ ± 1 . n—>oo

(c) Compute lim gn(l) and lim gn(—1).

THE COMPLEX PLANE AND THE SPACE L2 (R)

18

1.3

INNER PRODUCTS

Recall that for vectors u, v G R ^ w i t h u = (i¿i,i¿2, · · ·, i¿jv) T andv = ( v i , ^ , . . . , VN)T, we define the inner product as TV U v

U V = ^2 fc=l

kk

Inner Products Defined We can also define an inner product for functions in L 2 (R). In some sense it can be viewed as an extension of the definition above (see Van Fleet [60]). We state the formal definition of the inner product of two functions f(t),g(t) G L 2 (R) at this time: Definition 1.8 (The Inner Product in L 2 (R)) Let f(t), g(t) G L 2 (R). Then we define the inner product off(t) and g(t) as (f(t),g(t))=

[ f(t)g(t) dt JR

(1.16)

Here are some examples of the inner products: Example 1.6 (Computing L 2 (R) Inner Products) Compute the following innerproducts: (a) The triangle function f(t) Example 1.2)

t

W/(*) = { o,

= A(t) and the box function g(t) — \l(t) (see

andg{t)z

0,

/\ £(4\ A/. , i\ T (,\ Í sin(27rt), (c)f{t) = A(t + l)andg(t)=\Q^

'

t = | ¿; J j j ;

(1.28) ■

Suppose that W is a subspace of L (R) and {ek(t)}ke z is an orthonormal basis for W\ Since {ek(t)}ke z is a basis for W, we can write any f(t) G W as 2

/(í) = X)afcefc(í) fcez

(1.29)

where α^ G C. It would be quite useful to know more about the coefficients α&, k G Z. We compute the inner product of both sides of (1.29) with ej(t) to obtain

^(*)>=( Σα^(ί),ε,-(ί)) fcez fcGZ

= ^2ak kez

JR

ek{t)ej{t)àb

1.4 BASES AND PROJECTIONS

27

Since {ek(t)}ke z is an orthonormal basis, the integral in each term of the last identity is 0 unless j =fc.In the case where j =fc,the integral is 1 and the right-hand side of the last identity reduces to the single term: aj = {f(t),ej(t))=

Í

JR

f(t)ej(t)dt

(1.30)

Projections An orthonormal basis gives us a nice representation of the coefficients α&, k G Z. Now suppose that g(t) is an arbitrary function in L 2 (R). The representation (1.30) suggests a way that we can approximate g (t) in the subspace W. We begin by defining a, projection. Definition 1.12 (Projection) Let W be a subspace of L2(R). ThenP: L 2 (R) -» W is a projection/row L 2 (R) into W if for all f(t) G W, we W e P ( / ( i ) ) = /(t). ■ Thus a projection is any linear transformation from L 2 (R) to subspace W that is an identity operator for f(t) G W. If you have taken a multivariable calculus class, you probably learned how to project vectors from R 2 into the subspace W = {ca | c G R} where a is some nonzero vector in R 2 (see Stewart [55]). This is an example of a projection (using the vector space R 2 instead of L 2 (R)) with P(v) = ί π ^ - j a. A useful way to project g(t) G L2 (R) into a subspace W is to take an orthonormal basis {ek{i)}kez and write P(g(t)) = "£(g(t),ek(t))ek(t)

(1.31)

kez

We need to show that (1.31) is a projection from L 2 (R) into W. To do so, we need the following auxiliary results. The proofs of both results are outlined as exercises. Proposition 1.11 (The Norm of P(g(t))) Let W be a subspace ofL2(R) with orthonormal basis {ek{t)}kez- For g{t) G L 2 (R), the function P{g(t)) defined in (1.31) satisfies

\\P(9(t))\\2 = T,\(9(t)^k(t))\2

(1.32)

keZ

■

Proof: Problem 1.34.

Proposition 1.12 (An Upper Bound for ||P(g(t))||) LetW be a subspace ofX2(R) with orthonormal basis {ek(t)}kez- Then \\P(g(t))\\, where P(g(t)) is defined in (1.31), satisfies

\\P{g{t))\\ < llflWII

(1-33)

28

THE COMPLEX PLANE AND THE SPACE L2 (R)

Proof: Problem 1.35.

■

Proposition 1.12 tells us that if g(t) G L 2 (R), then so is P{g(t)). required to establish the following result.

This fact is

Proposition 1.13 (A Projection from L 2 (R) into W) LetWbeasubspaceofL2(R) with orthonormal basis {ek(t)}kezThen the function P(g(t)) defined by (1.31) is a projection from L 2 (R) into W. m Proof: From Proposition 1.12, we know that P(g(t)) G L 2 (R) whenever g(t) G L 2 (R). We need to show that for any f(t) G W, we have P(f(t)) = f(t). Since f(t) G W, we can write it as a linear combination of basis functions: f(t) =

Y/ajej(t)

Then P(f(t))

=

kez

J2{f(t),ek(t))ek(t)

= X^(5Z%-ej(í),efc(í)> ek{t) kez jez kezjez Since {ek(t)}ke z is an orthonormal basis for W, the inner product (ej(t), e^(t)) is nonzero only when j = k. In this case the inner product is 1. Thus the double sum in the last identity reduces to a single sum with j replaced by k. We have P(f(t))

= Y/akek(t) kez

= f(t)

and the proof is complete.

■

PROBLEMS 1.32

Suppose that {β&(£)} is an orthonormal basis for L 2 (R). For L G Z, L > 0,

let/i,(i)=

L

X] akek(t).

Show that

fc=-L

II/L(*)II 2 = Σ «i fe=-L

1.33 Suppose that P is a projection from vector space V into subspace W. Show that P 2 = P. That is, show that for all v G V, we have P 2 v = P v .

PROBLEMS

1.34

29

Expand the right-hand side of \\P(g(t))\\2 =

(P(9(t)),P(9(t)))

=( £>(*).ek w> efc ^' Σ tew.e* (*)) e*(*)) to provide a proof of Proposition 1.11. 1.35 In this problem you will prove Proposition 1.12. The following steps will help you organize the proof. Let g(t) G L 2 (R) and suppose that {ek(t)}kez is an orthonormal basis for subspace W. (a) LetgL(t)

L

= £

k=-L

(g(t),ek(t))ek(t).

(g(t),gL(t))

Showthat

= (gL(t),g(t))=

¿

k=-L

\(g(t),ek(t)}\2

(b) Use part (a) to show that \\9(t)-gL(t)\\2

= \\9(t)\\2-l

Σ

\(9(t),ek(t))\2

+ \\9L(t)\\2

k=-L L

(c) Show that

||C/L(¿)|| 2

=

Σ

k=-L

K#M> efcW)l2· This is a special case of Prob-

lern 1.32. (d) Use parts (b) and (c) to write \\g(t)-9L(t)\\2+

Σ

k=-L

\(9(t),ek(t))\2

= \\g(t)\\2

and thus infer that £

Kí?(í),e fc (í))| 2 oo

\\sn(u) - f(u)\\ -► 0

as

n -► oo

The definition says nothing about the pointwise convergence of the sequence {s η(ω)}. As we learned in Example 1.5 and Problem 1.20, it is entirely possibly for s η(ω) to converge in norm to f(u), but not converge pointwise to f(u) for certain values of ω. Of course, there are conditions that we can impose on f(u) to guarantee that sn(u) converges pointwise to /(ω). The interested reader is referred to Kammler [38] or Boggess and Narcowich [4]. The reader with an advanced background in analysis should see Körner [40]. Fourier Coefficients The Ck for k G Z are called the Fourier coefficients for /(a;). For applications we will need a formula for the c^. The following proposition tells us how to write the Fourier coefficients in terms of f(u) and complex exponential functions. 3

The proof of this result is beyond the scope of this book.

2.2 FOURIER SERIES

39

Proposition 2.1 (Fourier Coefficients) Suppose that f (ω) G L 2 ([—π,π]) has the Fourier series given by (2.17). Then the Fourier coefficients ck can be expressed as Ck =

1

Γ

2~ /

(2.18)

/M^M^

Proof: We rewrite (2.17) using a counter index j instead of k and then multiply both sides of (2.17) by ek(uj) to obtain oo

f{u)ek{u))

=

oo lJUJ

Σ

Cje

ek(uj)=

^

j— — oo

cjej(u)e^)

j = —oo

Next we integrate the above identity over the interval — [π,π]. Some analysis is required to justify passing the integral through the infinite summation, but the step is valid (see Rudin [48]) and we obtain π

/

ρπ

f(üü)ek(u)áu;

°°

= /

^

Cjej(u)ek(u;) άω J

J = -oo

~*

= 2π^ The 2πβ^ appears in the last line by virtue of the fact that the inner product (ej(üü), ek(uj)) is 0 for j φ k. When j = k, which happens exactly once in the entire sum, the inner product is 2π. Solving the last equality for ck gives the desired result. ■ Examples of Fourier Series Let's look at some examples of Fourier series for functions in L 2 ( [—π, π]). Example 2.1 (Fourier Series for a Piecewise Constant Function) Consider thefunction

Í

-i

'Ί π

/

2

π

0, otherwise The function is plotted in Figure 2.2. Find the Fourier series for f(oj). Solution We need to find the Fourier coefficients ckfor f(co). Thus we consider the integral

40

FOURIER SERIES AND FOURIER TRANSFORMATIONS

nil

-nfl

Figure 2.2

Ck

A piecewise constant function on [—π, π)

1 Γ 2¿ I

f(u)ek(u )άω ikuj

áüü

2πΛ π/2 ( l

—2π

í

W 2

/ \Ι-π/2

cos(A:c

sin(A:cc;) do; J-K/2 J

Now the integrand of the second integral above is an odd function and since we are integrating it over an interval symmetric about 0, we see that the integral value is 0. The integrand of the first integral is an even function so we can rewrite this integral as r^/2

I Ck

— ·2 / ί7Γ 2π Jo 1

i

ηπ/2

cos{kuü) άω = — / π Jo 7Γ

cos (fco;) άω

1

Ifk = 0, we obtain CQ = — · — = - . If k ^ 0, we see that π 2 2 π/2

Ck = -— sin(/ccc;) kn 0 1 . /fe?r = -— sin -— kn V 2

The value ofck is Ofor k φ 0 eve«, so the Fourier series for /(ω) is Akuj

(2.19)

kodd ^

can further simplify the series above. We split the summation into two sums:

/M = \ + - E 7 sin f Ϊ ) e ^ + Í Σ 7 sin ( ^ kodd k0

2.2 FOURIER SERIES

41

In the sum over the negative odd integers, replace k with —k and use the fact that sin (~ψ) = - sin ( ^ ) to write ikijû x

7

1 ν^ 1 . fkw\

ikul

fcodd

x

7

fcodd

k>0

1

kodd

v

7

k>0

1 v ^1 .

7

/fc7r\

fcodd

fc>0 fc>0

5 + ;ΣΪ-(τ)(^+«-*) v

fc odd

7

fe>0

2 + ^Efc Sln (^Tj cos (M 1

2 ^ 1 .

/ΛτΛ

fc odd

x

fc>0

., ,

7

. f(2k-l)n\

1 2 ^ 1 Z 1 fc=l

\

//ΛΙ

_

λ

/

Note that we used (2.8) to write 2 cos(küü) = eikuj + e~ikuJ. Now sin ( ( 2 % 1 ) π ) ( _ l ) / c + 1 ^ o r /c = 1,2,..., 50 iAe senes becomes

l k+1 / H = 52 +π5^Σ 2fc^r-^() - 1 1

2

^

1

cos 2

(( ^ - iM

fc=l

fc=l

Figure 2.3 contains plots of the partial sum 1

2^(-l)fc

cos (2/c 1)w /"M = 2-^^èfc3T ( - ) fc=l ^or various values

ofn.

It should not be a surprise that the Fourier series for f(uj) is reduced to a series in terms of cosine. Since f{uS) is an even function, we do not expect any contributions to the series from the "odd" parts sin (fco;) of the complex exponential functions e /~(ω). In Problems 2.11 and 2.12 you will investigate further the symmetry properties of Fourier series and coefficients. Example 2.1 allows us to better understand the convergence properties of Fourier series. The sequence of functions fn(u) converges in norm to f{uj) and converges pointwise to /(a;) everywhere but ω = ± f. Indeed, if you look at the plots of fn(uj) for n = 10, 20,100 in Figure 2.3, you will notice "overshoots" and "undershoots" forming at ω = ± ξ . This is the famous Gibbsphenomenon [29, 30]. For a nice

42

FOURIER SERIES AND FOURIER TRANSFORMATIONS

(b) Μω)

(a) / i H

(c) A M

-irH Figure 2.3

j/yVvvv^^v^^ —0

1

A

°°

= « + -5- V 2 7Γ2 ^

1

7771 ττ cos((2/c - l)a/) u ; ; (2fc - l ) 2

(2.21)

Figure 2.5 contains plots of the partial sum 9η{ω) = \ ~ Ι Σ

^

Υ cos((2A: - 1)ω)

k=l

for various values ofn.

m

Examples 2.1 and 2.2 showed how to construct Fourier series by using Proposition 2.1 to compute the Fourier coefficients. In the next example we see that this direct computation is sometimes unnecessary.

2.2 FOURIER SERIES

(a) 91 (ω)

(b) 02 M

(C) 03 ( ω )

(d) ριο(ω)

45

Figure 2.5 Plots of the partial sum gn(u) from Example 2.2 for n = 1,2, 3,10. Example 2.3 (Expanding to Find the Fourier Series) Find the Fourier series for the function h{uo) = sin 3 (a;) (1 + cosa;). Solution We observe that h{uo) is an odd function so we could use Problem 2.12(a) to realize that the Fourier coefficients are purely imaginary andean be computed as Ck =

i

Γ /

h(uS) sin(A;a;) άω

π Jo

—

/

sin3(a;) (1 + cosa;) sin(ÄXJ) cL;

π Jo

The above integral is quite tedious to compute, so we use an alternative means of finding the Fourier series for h(u). We use (2.8) and (2.9) to expand cos ω and sin ω, respectively, in terms of complex exponentials. We have sin3(u;) (1 + coscc?)

2i

1 +

eluJ + e"

Cubing thefirstfactor gives -3ίω

2i

+

3i

3i

■ ew +

3ίω

-e

(2.22)

46

FOURIER SERIES AND FOURIER TRANSFORMATIONS

Multiplying (2.22) by ( 1 + e ^ + 2 e

v ;

16

8 β3ίω

+τ ^

J and expanding gives

+i

8 e4

8

8

8

(2·23)

"

o lo can read the Fourier coefficients from (2.23). They are listed in Table 2.1. Table 2.1

The Fourier coefficients for the function from Example 2.3.

k

hk A

i

k

hk

A

i

-4

"le

4

Ï6

-3

-Î 8

3

* 8

8 3z ¥ |fc| > 4 ,fc= 0

8 3i ~16

0

^ caw easily verify that as indicated by Problem 2.12(a), the Fourier coefficients are pure imaginary numbers and satisfy Ck — —C-k (Problem 2.12(b)). We can also write (2.23) as a series in terms ofsm(küü) as suggested by Problem 2.12(c). Grouping conjugate terms in (2.23) together and using (2.9) gives h{u>) = i ( ε 4 ί ω - ε - 4 ί ω ) + I (β3ίω - e~3i") 16 8

+ ? (e¿w - e - i u 0

- l- (e2i" 8

e~2i")

o

= — · 2% sin(4u;) — - - 2i sin(3a;) — - · 2% sin(2cc;) + — · 2% sin(a;) 16 8 8 8 1 1 1 3 = — - sin(4(¿>) + - sin(3a;) + - sin(2u;) — - sin(a;) v / g 4 4 4

Computing One Fourier Series from Another The computation of Fourier coefficients can be quite difficult so, as was the case in Example 2.3, we look for alternative methods for computing Fourier coefficients. In many cases, we can use a known Fourier series for one function to help us construct the Fourier series for another function. Here is an example.

2.2 FOURIER SERIES

47

Example 2.4 (Fourier Series of a Piecewise Constant Function) Consider the expandedbox function ρ(ω) G L 2 ( [—π,π]) given by

0,

0 < ω < π otherwise

The function ρ{ω) is plotted in Figure 2.6.

Figure 2.6

A piecewise constant function on [—π, π).

Find the Fourier series for ρ(ω). Solution Rather than use Proposition 2.1 to compute the Fourier coefficients for ρ(ω) directly, we observe ώαίρ(ω) can be obtained from f((J) in Example 2.1 by translation. That is, ρ{ω) = f (ω -

|J

We can use (2.19) to find the Fourier series for p{uo):

ρ{ω) = f (ω 1

|)

1 ^

1

π ^ k V 2 koaa fcodd 11 v ^ 1 1 ikn 2 ikuj e- / e + + 7 sin ( — I 2 ñ / fcodd 1 1 - y ^ - sin ( — "\ ¿ f c e ^ 2 2 y fcodd 2

Jfë ca« re). (Hint: Problem 1.9 will be useful.) 2.20

Prove Proposition 2.3.

2.21

Suppose f(u),g(uj)

E L2( [—π, π]) have the Fourier series representation oo

/ H =

£

oo

cfceifa"

fc—— o o

Prove the following: (a) lfg(u>) = /(-o;),thend f e = c_fc. (b) If g(u) = / ( - ω ) , then dfc = c¡T.

L. If G(CJ) = £ e^e*^, where gk = (-l) fc /i L _ fc , write G(u) in fc=0

terms of i2"(u;). *2.27 Suppose that TV = 2M is a positive integer and let Η(ω) — cos^ (^). Find the Fourier series for Η(ω). {Hint: Use (2.8) to write cos (~ ) in terms of complex exponentials and then show that Η(ω) = —^β~ιΜω (elUJ + l) theorem.)

. Use the binomial

*2.28 We have developed the notion of Fourier series for functions f(u) G L 2 ( [—π, π]), but we could easily develop the theory for f(u) G L2 ( [—L, L]), where L > 0 is a real number. (a) Show that the complex exponential function β%(ω) = periodic function and that e%(u) G L 2 ( [—L, L]).

27Vlhuj 2L

e

^

is a 2L-

2.3 THE FOURIER TRANSFORM

53

(b) Show that for j , k G Z, we have

<e»,e£H> = 0). That is,

fit) = Σ cke27rikt/i2L)

(2.28)

k£Z

To motivate the Fourier transformation for functions in L 2 (R), we will use t as our independent variable in Fourier series representations instead of the ω that was used as the independent variable in Section 2.2.

4

54

FOURIER SERIES AND FOURIER TRANSFORMATIONS

where Ck

= W7 I f(t)e-27rikt/(2L) 2£ J-L

dt

(— [L f{u)e-2*iku'^

du) e2-ífc*/(2L)

(2.29)

and/(t)GL2([-L,L]). It is quite inviting to let L —> oo and see what happens to the series representation of/(£). There are several steps in what follows that need justification, but formally inserting (2.29) into (2.28) and letting L —► oo gives f(t) = lim Y = lim y

\ [

f(u)é*ik(t-u)/L

du

= 1 ι ™Σ(έ/ 1 Ι /(»κ"-" ( * , *)ΐ

). The proofs of each rule are straightforward. In most cases we can start with the definition of the Fourier transform (2.35) and manipulate it to obtain the desired result. Our first rule says that the Fourier transform is linear. Proposition 2.5 (Linearity Rule for Fourier Transforms) Suppose that j \ (t) and ¡2{t) have Fourier transforms f\{oo) and j'2(ω), respectively, and let g {t) — a\f\(t) +

60

FOURIER SERIES AND FOURIER TRANSFORMATIONS

0'2Í2{t)y where ai, Ü2 G C. Then g(uj) = αι/ι(α;) + α 2 Α(ω)

Proof: Using (2.35) we have

V ¿π

JR

= -J= / ( a 1 / 1 ( í ) + a 2 / 2 ( í ) ) e - i a ' í d í V ¿ JR

=

a i

- L / ' / 1 ( í ) e - - i d í + a 2 4 = Í Í2(t)e-iut dt

y/Ζπ JR = CLihfa) + a2Í2{w)

V2π

JR

Our next proposition says that if you translate a function, then you modulate the Fourier transform by a complex exponential. Proposition 2.6 (Translation Rule for Fourier Transforms) Assume that f(uj) is the Fourier transform of f(t). Let a G M and suppose that g(t) = f(t — a). Then g(u) =

e-^fioj)

Proof: We start with the defining relation (2.35) and write 9{ω) = ^=

f g{t)e-^

di = - L

y ¿IT JR

γζπ

[ f(t - a

JR

^

dt

Now we make the it-substitution u = t — a so that di¿ = di and t — u + a. Note that the limits of integration are unchanged, so that

¿M = - L

y/Ζπ

2π

[

JR

f(t-a)e-^dt

I f(u)e~iuj(u+a)

JR

·>*-}= [

VZ7T JR

"7M

du

f(u)e-^udu

2.3 THE FOURIER TRANSFORM

61

The next proposition tells us how to compute the Fourier transform of f (at). Proposition 2.7 (Dilation Rule for Fourier Transforms) Assume that f{uj) is the Fourier transform off(t). Suppose that a G R, a > 0 and let g[t) = f(at). Then a

\aJ

In Problem 2.32 you will show how to extend this rule to the case where a Φ 0. Proof: We begin with (2.35) and write 9(ω)

= 4 =

/ 9(t)e-^

di = 4 =

/ / M e " * * " ¿*

We now make the ^-substitution u = at or £ = - . The differential is di¿ = a at α

i

or at = ¿ di¿. It is also important to note that since a > 0, the limits of integration are unchanged. We have 3 H = - = / /(aí)e-íwtdí v 2π J R =

/ f(u)e~iuju/a

= --£=

[

a

du

f{u)e-i(u/a)uau

\aJ

Since the formulas for f(u) (2.35) and f(t) (2.36) are similar, it is natural to expect that a rule exists for computing g(uj) when g(t) is the Fourier transform of f(t). Proposition 2.8 (Inversion Rule for Fourier Transforms) Suppose that f(t) has Fourier transform f(oS) and assume that g (t) = f(t). Then g(uo) = f(—uo). m Proof: We use (2.35) to write 9(ω) = 4 = / 9ity-^ y ζπ

JR

d* = 4 = / / ( ί ) β ΐ ( " ω ) ί di y ¿π

JR

Using (2.36), we see that the right-hand side of the above identity is simply /(—ω). m The next rule shows us how to find the Fourier transform of a function f(t) scaled by t. The proof is a bit different from the ones we have seen thus far.

62

FOURIER SERIES AND FOURIER TRANSFORMATIONS

Proposition 2.9 (Power Rule for Fourier Transforms) Suppose that g {t) = t-f(t) e L 2 (R) and let f(uj) be the Fourier transform of'fit). Then g(uj) = i f'(u). m Proof: We begin by writing the Fourier transform of g (t). g{w) = - - L / g{t)e~™ at = - ¿ = /' t ■ f(t)e-^

dt

(2.44)

We now compute

This is a little tricky since we need to pass the derivative through the improper integral. These steps can be justified and made rigorous, but the analysis necessary to do so is beyond the scope of this text. The interested reader can consult Rudin [48] to learn more about justifying the exchange of the derivative and integral. We formally compute f'{u) and obtain

""»-¿UJÍ"««-"* ν2π

-^-e-iutdt αω

JR

e~iujt dt iuJt

át

If we compare the right-hand side of this last identity to (2.44), we see that ί'(ω) = -ig{uj)

or

g(uj) = if'(u)

The last three rules deal with reflections, modulations, and derivatives of functions, respectively. The proofs of these propositions are left as problems. Proposition 2.10 (Reflection Rule for Fourier Transforms) Suppose that f(u) is the Fourier transform of f (t) and let g(t) — f(—t). Then g(ω) = /(—ω). m Proof: Problem 2.33.

■

Proposition 2.11 (Modulation Rule for Fourier Transforms) Assume that f(ui) is the Fourier transform of f(t). Suppose that a G R and let git) — eltaf(t). Then g{u) = f(u - a). m Proof: Problem 2.34.

■

2.3 THE FOURIER TRANSFORM

63

Proposition 2.12 (Derivative Rule for Fourier Transforms) Suppose that f'(t) G L 2 (R) satisfies the Dirichlet conditions given in Proposition 2.4 and assume that ¡(ω) is the Fourier transform of f(t). Ifg(t) = f'(t), then g(u) = ίω/(ω). m Proof: Problem 2.35.

■

Computing Fourier Transforms Using Rules Now that we have stated all these rules for computing Fourier transforms, let's look at some examples implementing them. We use the two Fourier transform pairs from Example 2.5 and the transform pair from Problem 2.39. Example 2.6 (Using Rules to Compute Fourier Transforms) Compute the Fourier transforms of the following functions: (a) /i(*) = cos(47rt)n(i)

±oo

This follows from the fact that f(t) G L 2 (E).) 2.36 Suppose that the nth derivative / ( n ) ( ¿ ) G L 2 (R) satisfies the Dirichlet conditions given in Proposition 2.4. Generalize Proposition 2.12 and show that if then g(uj) = (ίω)η/(ω). (Hint: Use induction on n in conjuncg(t) = f(n\t), tion with Proposition 2.12.) 2.37 In this problem you will generalize Proposition 2.9. Suppose that n is a positive integer and assume g(t) = tnf(t) G L 2 (R). Let f(u) be the Fourier transform of f(t). Show that g(uj) = inf'(uj). (Hint: Use induction on n in conjunction with Proposition 2.9.) 2.38

This problem requires some knowledge of multivariable calculus. Show that e"' 2 dt = ypH

(2.50)

This result is necessary to complete Problem 2.39. The following steps will help organize your work. (a) First observe that the left-hand side of (2.50) is simply y 2π/(0), where f(t) = e~l , and show that 2π/(0)2=/"

/

e-^+^dtds

(2.51)

68

FOURIER SERIES AND FOURIER TRANSFORMATIONS

(b) Convert (2.51) to polar coordinates and thereby show that /»OO

2TT/(0) 2 = 2TT

/ Jo

e~r\àr

(c) Use the ^-substitution u = — r 2 in part (b) to obtain the desired result. 2.39 In this problem you will compute the Fourier transform of f(t) = e ~l . The following steps will help you organize your work. (a) Differentiate f(t) to obtain f'(t) + 2t · f(t) = 0. (b) Take Fourier transforms of both sides of the identity from part (a). You will use the derivative rule (Proposition 2.12) and the power rule (Proposition 2.9) to show that

/M (c) Integrate both sides of the previous identity from 0 to u and simplify to obtain / ( « ) = /(0)e-» 2 / 4 (d) Use part (c) along with Problem 2.38 to show that

*2.40 We need to exercise care when utilizing several Fourier transform rules. Suppose that we know the Fourier transform f(uS) of function f(t). Let a,b £ R with a > 0 and define g(t) = f(at — b). It should be clear that we need both the dilation rule and the translation rule to find g{u). (a) Show that (/(í) = / ( a ( í - ¿ ) ) · (b) Define h(t) = f(at) and find h(u). (c) Write g(t) in terms of/i(t) and use part (b) to show that §(CJ) = - e ~ i w 6 / a / Í - V (d) Use the result you obtained in part (c) to find §(ω) if g(t) — e _I2:E+3I. 2.41 Show that if we change a > 0 to a φ 0 in the hypothesis of the dilation rule (Proposition 2.7), then we can show that g(u) = —-f (— ) . \a\ \aJ 2.42 The Gaussian or normal distribution is an important distribution in statistics and its probability distribution function f(t) is defined by 1

(*-μ) 2

f(t) = - = - e—5=r\/2πσ

(2.52)

PROBLEMS

69

where μ and σ are real numbers with σ > 0. Find f(u). (Hint: Problems 2.39 and 2.40 will be helpful.) *2.43 We will have need for the Fourier transform of the triangle function A(t) defined by (1.5). In this problem we will show that Α(ω) = - ^ = e~iuj s i n c V / 2 ) ν2π

(2.53)

We could compute the Fourier transformation directly via Definition 2.4, but the calculations are tedious. In this problem we use the rules for computing Fourier transformations to find Λ(α;). Two possible ways to use Fourier transformation rules are given below. (a) Show that Λ'(ί) = Π(ί) - Π(ί - 1) (except at the points = 0,1,2). Now use the derivative rule and the translation rule to find Α(ω). (b) Show that A(t) = t Π (t) - (2 - t) Π (t - 1) and use the power rule and the translation rules to find Α(ω). (Hint: The computations here are particularly tedious. It might be wise to use a computer algebra system such as Mathematica to check your work.) In Section 2.4 we will find a much easier way to compute Α(ω). 2.44 Compute the Fourier transforms of the following functions. You will need the transform pairs from Example 2.5, Problem 2.39, and Problem 2.43. (a) /i(t) = n ( f ) (b) h(t) = Π(ί - i ) (c) f3(t) = n(3t - 4) (d) / 4 (t) = sin(t)e-l*l

(e) h(t)

t, 1, 4-i, 0,

0< t 0, andfor \l{u) we need to consider cases for u < 0, 0 < u < 1, andu > 1. And while we are considering the various pieces, we must remember that t varies over all reals! Let's first consider fit — u). Since f{u) = f(—u), we only need to translate by t in order to plot f(t — u). Let's start with t < 0. Both f(t — u) andV\{u) are plotted in Figure 2.12(a). Notice that we have labeled each piece off(u). When u < t, then t — u > 0 and —\t — u\ = —{t — u) — —t + u. In this case f(u) = e~t+u. In a similar manner we see that when u>t, we have f(u) = et~u.

t 0 1 (a) f(t - u) and n(u) for t < 0

0 t 1 (b) f(t - u) and n(it) for 0 < t < 1 3

01

1

2

t

(c) f{t - u) and n(u) for t > 1

Figure 2.12 (/*n)(t).

2

4

(d) (/*n)(t)

The functions f{t — u) and F\(u) for t > 0 and the convolution product

When t < Owe have ( / * Π ) ( ί ) = / et-udu Jo

= et [ e~u du = et(l-e-1) Jo

-i\ = _ ê J> -

e „t-\

76

FOURIER SERIES AND FOURIER TRANSFORMATIONS

When 0 < t < 1 we have two integrals that contribute to the convolution value:

( / * n ) ( f ) = í e- í + n chx+ / el~u du = e-t(et-l)+et(e-*-e-1)

Finally, when t > 1 we have , . _ _t ( / * Π ) ( ί ) = / . , - t + w jdt¿ = e (e - 1) - e 1

^k have plotted (f * Π)(ί) z« Figure 2.12(d). Notice that although F\(t) is discontinuous at t = 0,1, it appears that (/ * Π)(ί) ¿s a continuous function for all reals. In Problem 2.50 you will show that (/ * Π)(ί) ¿s indeed a continuous function. Although this is only one example, we will see that the convolution operator is a smoothing operator. That is, the resultingfunction is at least as smooth as the two convolved functions. For the convolution product (Π * Π) (t), we must reflect i~l(u) about the u = 0 axis and then translate it by t units. This process is plotted in Figure 2.13(a) for t < 0. As we move t to the right, we note that the product ofr\(t — u) and \~\(u) is 0 up

t-i

1

t

(a) t < 0

u

t-1

0

t

1

u

(b) 0 < t < 1

1

2

(d) (Π * Π)(ί) = Λ(ί)

Figure 2.13 The convolution product (Π * Π)(ί) = Λ(ί). to t = 0. From t = 0 to t = 1, the convolution product is simply the area of the rectangle with height 1 and width t (see Figure 2.13(b)). At t = 1, ¿/ze foxes a/zg« so z7z¿zí z7ze area (and thus the convolution product at t — I) is 1. For 1 < t < 2, the boxes still overlap (see Figure 2.13(c)) and the overlap produces a rectangle of height 1 and width 1 — (t — 1) = 2 — t. Once t moves past 2, the boxes no longer overlap so that the convolution product is 0. Thus we have the piecewise formula for

2.4 CONVOLUTION AND B-SPLINES

{

(Π*Π)(ί):

t,

77

0< t ) = [ f(u) i-^= [ g(t - u)e-^-^ JuER \λ/2π JteR

dt) β~ίωηau /

(2.58)

We now make the substitution s = t — u in the inner integral of (2.58). Note that as = at and the limits of integration are unchanged. The inner integral in (2.58) becomes

78

FOURIER SERIES AND FOURIER TRANSFORMATIONS

- L

/

g(t-

u)e-^-^

at = -±= f

g{s)e-^

as

We can insert this result into (2.58) to complete the proof: h(w)=

f(u)g(w)e-iuuau

f

f(u)e-^uau

= g(w)[

= g(u)V2¿-}=[ νζπ Juem. = y/tog(u)f(u)

f(u)e-*"udu

Let's look at some examples. Example 2.8 (Examples Using the Convolution Theorem) Find the Fourier transforms of the convolution products obtained in Example 2.7. Solution In Example 2.7 we computed the convolution product off(t) = e ~'*Ι and\l(t):

Í

é-é-1,

tj,k(t) }kez is an orthonormal basis for Vj. m Now that we have introduced the functions φ¿? & (t), let's look at an example about how the indices j andfetell us about the support properties of the function. Example 3.4 (Graphing φ^ k (t)) Describe the support and plot of each of the following functions. (a) ¿3,7(i) (b) 0-2,i (t) (c) 0_ 4> _ 6 (i) fó> 5,ls(¿)

98

HAAR SPACES

Solution All the functions are plotted in Figure 3.8. The best way to analyze the support of each function is to write it as a translate ofφ(2H). We know that φ(2Η) is compactly supported with supp(0(2-?£)) = [θ, ψ\. For part (a), we have φ3 3, k e Z, by k k + 1 ] k k+ fc ^. (3.10) te 3,k 23 23 23 J 2i ' 2·? Â

ΨΗ

We will refer to the first index j as the level of the interval.

m

The adjective dyadic describes objects made up of two units. It is natural to use the word here since the the length of each i ^ k is 2·7. Let's look at some examples of different intervals J¿ &.

100

HAAR SPACES

Example 3.5 (Examples of Intervals Ij5k) Sketch each of the intervals Ij^ given below. (a) I 3 , - i (b)

(c)

1-2,4

I0,k,keZ

Solution For part (a) we have ^3,-1 = [— ¿-, " ^ 1 ) = [~§?θ)· We can write the interval in part (b) as 1-2,4 = \_ΦΊ, f=r) = [16,20). Finally, we have from part (c) io,fc = m [ψ, η^τ) = [k,k + I). Each of the intervals is plotted in Figure 3.9.

0 16

17

(a)/3,-i

18

19

20

(b)/_

k+1

(c)

IQ,

k

Figure 3.9 The intervals h,-i, 1-2,4, and Io,kWe are particularly interested in the intersection of two dyadic intervals. For instance, in Example 3.5(c) we learned that J0> k = [k, k + 1), where k G Z. These intervals are Jo, * = . . . , [-2, - 1 ) , [-1,0), [0,1), [1,2), . . . (3.11) and note that none of these intervals intersect. For the intervals I\,k,k h,k = [§, n j r ) = [f, § + | ) and these intervals are h,k = · · ·, - 1 , - 2

4,0




2'

fc+1/2 2·/'

jfe+j 2-'

Figure 3.10 The point m/2£, located in the left half of i^fc. We can write the numerator of the upper bound of (3.15) as 2e~3k + 2i~j~1. Since I > j , we know that 2£~J > 2 and 2i~J~1 > 1 and both are integers. So 2£-i (k + l/2)eZ and thus 2£~i (k + l/2)/2* must be an endpoint for some dyadic interval at level I. Thus the right endpoint ^φ^- of jT^m must be less than or equal to (k + l/2)/2· 7 . This implies that Í£ ;rn resides in the left half of Ij^. The case where +2/ < ψ < ^- is similar and is left to the reader. ■

Projecting Functions From L2(R) Into Vj3 We can use the basis functions <j>j,k(t), k G Z and Proposition 1.13 to create projections of functions g(t) G L 2 (R) onto Vj. The projection is given by

Pgj(t) =

Y,(j,k(t),g(t)^k(t) ke:

(3.16)

3.2 THE GENERAL HAAR SPACE Vj

103

We can use (3.9) to rewrite (3.16) as P9j(t) = Σ{ν/2φ{2Η kez =

kez

- k),g(t))2^^(2h

- k)

2ΐΣ(φ(2Η-Ιί),9(ί))φ(Μ-ΐ!)

and use it to find the projections in the following example. Example 3.6 (Projections into Vj) Consider the function g(t) = e - ^ from Example 3.2. Compute and plot the projections Pg¿ (t) and P9}-i(t). Solution For thefirstprojection, we have P9,2(t) = 4 £ > ( 4 t - k),g(t)){U - k) Using Problem 3.11 we know that supp(02, k) = [f ? ^4^] can be written as (

so

that the inner product

(fc+l)/4 fc/4

(