Numerical Analysis of Spectral Methods : Theory and Applications

CBMS-NSF REGIONAL CONFERENCE SERIES IN APPLIED MATHEMATICS A series of lectures on topics of current research interest in applied mathematics under the direction of the Conference Board of the Mathematical Sciences, supported by the National Science Foundation and published by SI AM. GARRETT BIRKHOFF, The Numerical Solution of Elliptic Equations D. V. LINDLEY, Bayesian Statistics, A Review R. S. VARGA, Functional Analysis and Approximation Theory in Numerical Analysis R. R. BAHADUR, Some Limit Theorems in Statistics PATRICK BILLINGSLEY, Weak Convergence of Measures: Applications in Probability J. L. LIONS, Some Aspects of the Optimal Control of Distributed Parameter Systems ROGER PENROSE, Techniques of Differential Topology in Relativity HERMAN CHERNOFF, Sequential Analysis and Optimal Design J. DURBIN, Distribution Theory for Tests Based on the Sample Distribution Function SOL I. RUBINOW, Mathematical Problems in the Biological Sciences P. D. LAX, Hyperbolic Systems of Conservation Laws and the Mathematical Theory of Shock Waves I. J. SCHOENBERG, Cardinal Spline Interpolation IVAN SINGER, The Theory of Best Approximation and Functional Analysis WERNER C. RHEINBOLDT, Methods of Solving Systems of Nonlinear Equations HANS F. WEINBERGER, Variational Methods for Eigenvalue Approximation R. TYRRELL ROCKAFELLAR, Conjugate Duality and Optimization SIR JAMES LIGHTHILL, Mathematical Biofluiddynamics GERARD SALTON, Theory of Indexing CATHLEEN S. MORAWETZ, Notes on Time Decay and Scattering for Some Hyperbolic Problems F. HOPPENSTEADT, Mathematical Theories of Populations: Demographics, Genetics and Epidemics RICHARD ASKEY, Orthogonal Polynomials and Special Functions L. E. PAYNE, Improperly Posed Problems in Partial Differential Equations S. ROSEN, Lectures on the Measurement and Evaluation of the Performance of Computing Systems HERBERT B. KELLER, Numerical Solution of Two Point Boundary Value Problems J. P. LASALLE, The Stability of Dynamical Systems - Z. ARTSTEIN, Appendix A: Limiting Equations and Stability of Nonautonomous Ordinary Differential Equations D. GOTTLIEB AND S. A. ORSZAG, Numerical Analysis of Spectral Methods: Theory and Applications PETER J. HUBER, Robust Statistical Procedures HERBERT SOLOMON, Geometric Probability FRED S. ROBERTS, Graph Theory and Its Applications to Problems of Society

(continued on inside back cover)

David Gottlieb

Tel-Aviv University and

Steven A. Orszag Massachusetts Institute of Technology

Numerical Analysis of Spectral Methods:

Theory and Applications

SOCIETY FOR INDUSTRIAL AND APPLIED MATHEMATICS PHILADELPHIA, PENNSYLVANIA

1977

All rights reserved. No part of this book may be reproduced, stored, or transmitted in any manner without the written permission of the Publisher. For information, write the Society for Industrial and Applied Mathematics, 3600 University City Science Center, Philadelphia, Pennsylvania 19104-2688. Printed by Capital City Press, Montpelier, Vermont, U.S.A. Copyright 1977 by the Society for Industrial and Applied Mathematics. Second printing 1981. Third printing 1983. Fourth printing 1986. Fifth printing 1989. Sixth printing 1993. is a registered trademark.

Contents Preface Section 1 INTRODUCTION Section 2 SPECTRAL METHODS Section 3 SURVEY OF APPROXIMATION THEORY Section 4 REVIEW OF CONVERGENCE THEORY Section 5 ALGEBRAIC STABILITY Section 6 SPECTRAL METHODS USING FOURIER SERIES Section 7 APPLICATIONS OF ALGEBRAIC-STABILITY ANALYSIS Section 8 CONSTANT COEFFICIENT HYPERBOLIC EQUATIONS Section 9 TIME DIFFERENCING Section 10 EFFICIENT IMPLEMENTATION OF SPECTRAL METHODS . . . . Section 11 NUMERICAL RESULTS FOR HYPERBOLIC PROBLEMS Section 12 ADVECTION-DIFFUSION EQUATION Section 13 MODELS OF INCOMPRESSIBLE FLUID DYNAMICS Section 14 MISCELLANEOUS APPLICATIONS OF SPECTRAL METHODS Section 15 SURVEY OF SPECTRAL METHODS AND APPLICATIONS . . . . Appendix PROPERTIES OF CHEBYSHEV POLYNOMIAL EXPANSIONS . . PROPERTIES OF LEGENDRE POLYNOMIAL EXPANSIONS . . . References Bibliography Index

v 1 7 21 47 55 61 79 89 103 117 121 139 143 149 155 159 162 163 164 168

This page intentionally left blank

Preface Spectral methods involve seeking the solution to a differential equation in terms of a series of known, smooth functions. They have recently emerged as a viable alternative to finite difference and finite element methods for the numerical solution of partial differential equations. The key recent advance was the development of transform methods for the efficient implementation of spectral equations. Spectral methods have proved particularly useful in numerical fluid dynamics where large spectral hydrodynamics codes are now regularly used to study turbulence and transition, numerical weather prediction, and ocean dynamics. In this monograph, we discuss the formulation and analysis of spectral methods. It turns out that several features of this analysis involve interesting extensions of the classical numerical analysis of initial value problems. This monograph is based on part of a series of lectures presented by one of us (S.A.O.) at the NSF-CBMS Regional Conference held at Old Dominion University from August 2-6, 1976. This conference was supported by the National Science Foundation. We should like to thank our colleagues M. Deville, M. Dubiner, M. Gunzburger, B. Gustaffson, D. Haidvogel, M. Israeli, and J. Ortega for helpful discussions. We are grateful to E. Cohen, A. Patera, and K. Pitman for their assistance in preparing graphs and tables. Some calculations were performed at the Computing Facility of the National Center for Atmospheric Research which is supported by the National Science Foundation. One of us (D.G.) would like to acknowledge support by the National Aeronautics and Space Administration while in residence at ICASE, NASA Langley Research Center, Hampton, Virginia. Both authors would like to acknowledge support by the Fluid Dynamics Branch of the Office of Naval Research and the Atmospheric Sciences Section of the National Science Foundation. Hampton, Virginia Cambridge, Massachusetts September 1977

V

This page intentionally left blank

SECTION 1

Introduction In this monograph we give a mathematical analysis of spectral methods for mixed initial-boundary value problems. Spectral methods have become increasingly popular in recent years, especially since the development of fast transform methods (see § 10), with applications in numerical weather prediction, numerical simulations of turbulent flows, and other problems where high accuracy is desired for complicated solutions. We do not discuss the sophisticated applications of spectral methods here. A survey of some applications is given in § 15. Instead, we concentrate on the development of a mathematical theory that explains why spectral methods work and how well they work. Before presenting the theory, we begin by giving some simple examples of the kinds of behavior that we wish to explain. Spectral methods involve representing the solution to a problem as a truncated series of known functions of the independent variables. We shall make this idea precise in § 2, but we can illustrate it here by the standard separation of variables solution to the mixed initial-boundary value problem for the heat equation. EXAMPLE 1.1. Fourier sine series solution of the heat equation. Consider the mixed initial-boundary value problem

The solution to (1.1) is

are the coefficients of the Fourier sine series expansion of /(*). Recall that any function in 1/2(0, TT) has a Fourier sine series that converges to it in L2(0, TT); the Fourier sine series of any piecewise continuous function f(x) which has bounded variation on (0, TT) converges to \[f(x + )+/(* -)] throughout (0, TT) (see § 3). 1

2

SECTION 1

A spectral approximation is gotten by simply truncating (1.2) to

and replacing (1.3) by the evolution equation

with the initial conditions an(Q)-fn (n = 1, • • • , N). The spectral approximation (1.5)-(1.6) to (1.1) is an exceedingly good approximation for any t > 0 as AT-> oo. In fact, the error u(x, t)-uN(x, f)goes to zero more rapidly than e~N ' as N-+ oo for any t > 0. In contrast, a finite difference approximation to the heat equation using N grid points in x but leaving t as a continuous variable (a "semi-discrete" approximation) leads to errors that decay only algebraically with N as N -> oo. (Of course, if we solve (1.6) by finite differences in t the error of the spectral method would go to zero algebraically with the time step Af. However, we shall neglect all time differencing errors for now and study only the convergence of semi-discrete approximations. Time-differencing methods are discussed in § 9.) EXAMPLE 1.2. Fourier sine series solution of an inhomogeneous heat equation. Not all spectral methods work as well as the trivial one just outlined in Example 1.1. Consider for example the solution to the problem

with the same initial and boundary conditions as before. The Fourier sine coefficients of the exact solution are now

where en = 0 if n is even and en = 1 if n is odd. Spectral approximations are now given by (1.5) with (1.6) replaced by

the solution of which is (1.7) for « = ! , • • -,7V. Now the truncation error u(x, t)-ux(x, t) no longer decays exponentially as N-*oo; the error is of order AT3 as N->oo for fixed x, 00. In other words, the results to be anticipated from this spectral method behave asymptotically as 7V-» oo in the same way as those obtained by a third-order finite-difference scheme (in which the error goes to zero like A*3 = (Tr/N)3).). For this problem, straightforward solution by finite differences may be more efficient and accurate than solution by Fourier series.

INTRODUCTION

3

The last example may be disturbing but even more serious difficulties confront the unwary user of spectral methods, as the next example should make amply clear. EXAMPLE 1.3. Fourier sine series solution of the one-dimensional wave equation. Consider the mixed initial-boundary value problem for the one-dimensional wave equation

The exact solution to this well posed problem is u (x, t) = xt. This solution can also be found by Fourier sine series expansion of u(x, t). To do this, we substitute (1.2) into (1.8) and re-expand all terms in sine series. The Fourier expansion of du/dx is

where integration by parts gives

Also, the Fourier sine coefficients of x are 2/n(-l)" +1 and the Fourier sine coefficients of t are (4t/(irn))en, where en = 0 if n is even and en = 1 if n is odd Equating coefficients of sin nx in (1.8a), we obtain

The Fourier sine coefficients of the exact solution u(x, t) = xt are

It is easy to verify by direct substitution that these coefficients exactly satisfy (1.11); in particular, the sum in (1.11) converges for all t. Now suppose we employ a spectral method based on Fourier sine series to solve this problem. We seek a solution to (1.8) in the form of the truncated sine serie (1.5). If the exact coefficients an(t) are used in (1.5) then u(x, t)-uN(x, f)-*0 as

4

SECTION 1

N -* oo; f or each fixed x, 0 < x < IT, and t > 0 the error is of order l/NasN-+<x> (see §3). However, it is not reasonable to assume that the expansion coefficients an (t) are known exactly in this case because of the complicated couplings between various n in the system (1.11). It is more reasonable to determine them by numerical solution of an approximation to (1.11). Galerkin approximation (see § 2) gives the truncated system of equations

The truncation of the infinite system (1.11) to the finite system (1.12) is a standard way to approximate infinite coupled systems. Unfortunately, it need not work. In Figs. 1.1-1.2 we show plots of the approximations UN(X, t) at / = 5 given by (1.5) for N = 50, 75. These plots are obtained by numerical solution of (1.12) with an(0) = 0; the time steps used in the numerical solution of (1.12) are

FlG. 1.1. A plot of the Galerkin approximation UN(X, t) to (1.8) for N = 50 at t — 5. This solution is obtained by numerical integration of (1.12). Time differencing errors are negligible. The exact solution u = xt at t = 5 is also shown.

INTRODUCTION

5

so small that time differencing errors are negligible. It is apparent that the approximate solutions with N finite do not converge to the exact solution as N increases! The divergence of this spectral method will be explained in § 6.

FIG. 1.2. A plot of the Galerkin approximation UN(X, t) to (1.8) for N =75 att = 5. This solution is obtained by numerical integration of (1.12). Time differencing errors are negligible. The exact solution u = xt at t = 5 is also shown.

Not all spectral methods give such poor results. A properly formulated and implemented spectral method gives results of striking accuracy with efficient use of computer resources. The choice of an appropriate spectral method is governed by two main considerations: (i) Accuracy. In order to be useful a spectral method should be designed to give results of greater accuracy than can be obtained by more conventional difference methods using similar spatial resolution or degrees of freedom. The choice of appropriate spectral representation depends on the kind of boundary conditions involved in the problem. (ii) Efficiency. In order to be useful the spectral method should be as efficient as difference methods with comparable numbers of degrees of freedom. For similar

6

SECTION 1

work, spectral methods should produce more accurate results than conventional methods. In § 15, we present a catalog of different spectral methods and indicate the kinds of problems to which they can be most usefully applied. Many examples of efficient and accurate spectral methods will be given later.

SECTION 2

Spectral Methods The problems to be studied here are mixed initial-boundary value problems of the form

where D is a spatial domain with boundary 3D, L(JC, t) is a linear (spatial) differential operator and B(x) is a linear (time-independent) boundary operator. Here we write (2.1)-(2.3) for a single dependent variable « and a single space coordinate x with the understanding that much of the following analysis generalizes to systems of equations in higher space dimensions. Also, attention is restricted to problems with homogeneous boundary conditions because the solution to any problem involving inhomogeneous boundary conditions is the sum of an arbitrary function having the imposed boundary values and a solution to a problem of the form (2.1)-(2.3). The extension to nonlinear problems will be indicated in Examples 2.9-2.10 at the end of this section. Before discussing spectral methods for solution of (2.1)-(2.3) let us set up the mathematical framework for our later analysis. It is assumed that, for each t, u(x, t) is an element of a Hilbert space W with inner product ( • , • ) and norm || • ||. For each t > 0, the solution1 u(t) belongs to the subspace 38 of % consisting of all functions u € ffl satisfying Bu = 0 on 3D. We do not require that u (x, 0) = g(x) e 3 but only that u(x, 0)e 3C. The operator L is typically an unbounded differential operator whose domain is dense in, but smaller than, %C. For example, if L = d/dx and ffl = £2(0,1), the domain of L can be chosen as the dense set of all absolutely continuous functions on O^x ^ 1. If the problem (2.1)-(2.3) is well posed, the evolution operator is a bounded linear operator from %€ to $. Boundedness implies that the domain of the evolution operator can be extended in a standard way from the domain of L to the whole space $C (Richtmyer and Morton (1967, p. 34)). For notational convenience we shall assume henceforth that L is time independent so that the evolution operator is exp (Lt). In this case the formal solution of (2.1)-(2.3) is

1

We will often denote u(x, t) by u(t) when discussing u as a function of t. 1

8

SECTION 2

This formal solution is justified under the conditions that /(?), Lf(t\ and L2f(t) exist and are continuous functions of / in the norm || • || for all / ^ 0 (see Richtmyer and Morton (1967)). The semi-discrete approximations to (2.1) to be studied here are of the form

where, for each t, UN(X, t) belongs to an AT-dimensional subspace 38N of 55, and L/v is a linear operator from 9C to $N of the form

Here PN is a projection operator of 3€ onto 28N and fN = PNf. We shall assume that 0. Since e may be chosen small we may replace sin \t by \t with a maximum error of O(e3). Also since f(x — t) is piecewise continuous, we may assume that f(x-t) is continuous for 0 < t ^ e and —e ^ t < 0 with at worst a jump discontinuity at t = 0. Therefore we may replace f(x -1) by f(x -) for t > 0 and f(x + ) for t < 0 giving

SURVEY OF APPROXIMATION THEORY

23

Since

for any fixed E>0, we obtain

proving (3.3). In the neighborhood of a point of discontinuity of /(*) [or x = 0 and x = 2n if /(0 + )^/(277--)] the convergence of g/cOO to its limit (3.3) as K-*oo is not uniform. To investigate the detailed approach of g^C*) to g(x) near a point of discontinuity x0 of /(*), we use the asymptotic integral representation (3.13) to obtain

24

SECTION 3

FIG. 3.2 Aplot of the sine integral Si(z)defined in(3.14b) for

for every fixed z. Since e is assumed small we can approximate f(x for 0 < s < e and by f(x0 - ) f o r - e < s < 0 . Therefore, for each fixed z and e,

SURVEY OF APPROXIMATION THEORY

25

FIG. 3.3. A plot ofthe N-term partial sums ofthe Fourier sine series expansion of the function x/ IT for N = 10, 20, 40. The function x/tr is also plotted. Observe the Gibbs phenomenon near x = IT. Also observe that the rate of convergence of the Fourier series is like l/Nfor x in the interior of the interval 0 < x < IT.

for any fixed z. Here the sine integral Si (z) is defined

A plot of Si (z) is given in Fig. 3.2. The result (3.14) shows that, if x-x0= O(l/K) ) as AT->oo, then gK(x)-2[f(xo + )+/(*o —)] = O(\). This shows the nonuniformity of convergence of gic(x) to /(*) in the neighborhood of the discontinuity x0. This nonuniform behavior of the limit gje(*)-*/(*) as K-+OO is called the Gibbs phenomenon. To illustrate the Gibbs phenomenon in an actual Fourier series, we plot in Fig. 3.3 the partial sums of the Fourier sine series expansion of the function The extended function fs(x] is discontinuous at x = IT leading to the Gibbs phenomenon there.

26

SECTION 3

As K -* oo, the maximum error of the partial sums of a Fourier (complex or sine or cosine) series in the neighborhood of a point of discontinuity occurs at the maximum of Si (z). Since Si'(z) = 0 when z = mr for n = ± l , ±2, • • • , the maximum error must occur at one of these points. It is easy to argue that the maximum of Si (z) actually occurs at z = TT where

Thus, the maximum overshoot of the partial sums of the Fourier series near a discontinuity occurs near x = x0 + oo. However, a further integration by parts in (3.31) shows that if the Sturm-Liouville problem is nonsingular and if h(a) or h(b)^Q, then an behaves like l/n3 as «-»oo. In general, unless f(x) satisfies an infinite number of very special conditions at jc = a and x = b, then an decays algebraically as n -* oo. These results on algebraic decay of errors in expansions based on nonsingular second-order eigenvalue problems generalize to higher-order eigenvalue problems. For example, as H-»OO, the expansion coefficients in an in /(*) = Z"=o